fix: share a single BM25 model between indexing and search - #127
Merged
Merged
Conversation
The server lifespan constructed its own BM25SparseProvider and stored it in
server/state.py for the search tools, while IndexPipeline resolved the separate
module singleton in server/embeddings/bm25.py. Two Bm25("Qdrant/bm25") models
were loaded per process, and close_sparse_embedding_provider() only released
the module one.
Drop the state.py holder and route both callers through
get_sparse_embedding_provider(), mirroring how the dense provider is already
shared. The lifespan now just warms that singleton so the first query still
does not pay the model-load cost.
Sharing one model across indexing and search is safe: passage_embed and
query_embed are pure over the loaded vocabulary and already run in an executor
thread, so no additional locking is needed.
Closes #70
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
release-please bumps pyproject.toml only; the lockfile's own project version entry was still 1.2.2. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Closes #70
Problem
Two
Bm25("Qdrant/bm25")models were loaded per process:server/main.pyconstructed aBM25SparseProvider()in the lifespan and stored it inserver/state.py, where the search tools picked it up viaget_sparse_provider().IndexPipeline.__init__resolved the separate module singleton inserver/embeddings/bm25.pyviaget_sparse_embedding_provider().On top of the doubled memory,
close_sparse_embedding_provider()only cleared the module singleton — the copy held instate.pylived for the whole process.Change
Drop the
state.pyholder entirely and route both callers throughget_sparse_embedding_provider(), mirroring how the dense provider is already shared (nostate.pyentry; every caller goes through the factory singleton). The lifespan now just calls the getter to warm the model at startup, so the first query still doesn't pay the load cost, and teardown releases the only instance.Sharing one model across indexing and search needs no extra locking:
passage_embed/query_embedare pure over the loaded vocabulary and already run in an executor thread.server/state.py— removed_sparse_provider,get_sparse_provider(),set_sparse_provider()server/tools/search.py— both call sites use the shared getterserver/main.py— warm the singleton instead of building a second onetests/tools/test_search.py— patch targets repointed (mechanical)tests/embeddings/test_bm25_singleton.py— new guard test: stubsBm25so nothing is downloaded, asserts the pipeline and search modules resolve to the same instance, plus singleton and close behaviourA second commit syncs
uv.lock's project version to the1.3.1release, which release-please left at1.2.2.Testing
uv run pytest— 365 passed. Ruff is not installed in this environment, so no lint run.🤖 Generated with Claude Code