Skip to content

Fix memory_zone cleanup on exception#13932

Merged
honnibal merged 1 commit intoexplosion:masterfrom
kvr06-ai:fix/13924-memory-zone-exception-cleanup
Mar 15, 2026
Merged

Fix memory_zone cleanup on exception#13932
honnibal merged 1 commit intoexplosion:masterfrom
kvr06-ai:fix/13924-memory-zone-exception-cleanup

Conversation

@kvr06-ai
Copy link
Contributor

@kvr06-ai kvr06-ai commented Mar 7, 2026

Summary

When an exception propagates out of nlp.memory_zone(), the cleanup code in StringStore.memory_zone() and Vocab.memory_zone() never runs. This leaves the vocab in a broken state (transient strings/lexemes not cleared, self.mem still pointing to a temporary pool), causing an AssertionError in Vocab.get() on the next call to nlp().

Fixes #13924

Root cause

Both StringStore.memory_zone() and Vocab.memory_zone() use @contextmanager with cleanup code placed after yield but without a try/finally wrapper. In a @contextmanager, code after yield only runs on normal exit. If an exception is thrown at the yield point, it propagates without executing the cleanup.

Changes

  • spacy/strings.pyx: Wrap yield and cleanup (clearing transient keys, restoring self.mem) in try/finally
  • spacy/vocab.pyx: Same pattern for _clear_transient_orths() and self.mem restore
  • spacy/tests/vocab_vectors/test_memory_zone.py: Add regression test that raises inside a memory zone and verifies the vocab is usable afterward

Testing

  • All existing test_memory_zone tests pass (no change to happy-path behavior)
  • New test_memory_zone_exception_cleanup verifies:
    • vocab.in_memory_zone is False after exception
    • Pre-existing words still accessible
    • Transient words from the failed zone are cleaned up
    • Vocab accepts new words without errors

Wrap yield in try/finally in StringStore.memory_zone and
Vocab.memory_zone so transient state is always cleaned up,
even when an exception propagates through the context manager.
@kvr06-ai
Copy link
Contributor Author

kvr06-ai commented Mar 7, 2026

Hi @honnibal — this is a small fix for the memory_zone context managers in StringStore and Vocab. The cleanup code after yield doesn't run when an exception propagates through a @contextmanager (needs try/finally). The result is a corrupted vocab state that triggers an AssertionError in Vocab.get() on subsequent calls.

The fix wraps the existing cleanup in try/finally in both files. Happy-path behavior is unchanged. Would appreciate your review when you get a chance.

@honnibal
Copy link
Member

Thanks!

@honnibal honnibal merged commit cfa1d3a into explosion:master Mar 15, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Exception in memory_zone does not clear vocab

2 participants