Skip to content

fix(retrieval): #882 bound the query context to the model's window and stop citing what it never read - #893

Merged
jasonssdev merged 1 commit into
mainfrom
fix/query-context-budget-882
Aug 27, 2026
Merged

fix(retrieval): #882 bound the query context to the model's window and stop citing what it never read#893
jasonssdev merged 1 commit into
mainfrom
fix/query-context-budget-882

Conversation

@jasonssdev

Copy link
Copy Markdown
Owner

The defect

openkos query built its prompt from the full body of every retrieved document with no size bound and no disclosure. Ollama does not raise on an oversized prompt — it discards the overflow and answers normally — so --save filed every retrieved citation as provenance verbatim, including documents the model had never read. That is a false provenance claim on disk, and no surface (query, lint, status) could reveal it.

#888 chunked the embedding side, but VectorStore.query collapses to one hit per document, so the retrieval unit stayed the whole document and this defect survived it intact.

Measured live, not asserted

Same corpus, same num_ctx: 12288, real qwen3:8b:

arm prompt prompt_eval_count
no bound (pre-fix) 208,222 chars 6,146
bounded (post-fix) 8,410 chars 2,449

6146 is the exact truncation constant #866 recorded. At the ratio the bounded arm establishes (3.434 chars/token) the unbounded prompt should evaluate ~60,634 tokens, so 54,488 of them — 89.9% — were discarded in silence. The bounded arm's chars and tokens agree, so the whole prompt is read.

What changed

Review

Receipt review-dc71e9c4ba1743da, four lenses, APPROVED with zero corrections consumed. Three earlier candidates were fixed pre-capture:

  • a fair_shares invariant violation (CRITICAL) — fair_shares([0, 100], budget=1) returned [1, 0], spending the only available char on an empty body and starving the larger block;
  • an order-dependence that contradicted its own docstring — [7, 9, 100] at budget 11 gave [4, 4, 3], the same multiset reversed gave [3, 4, 4]. Fixed in the code rather than weakened in the docs;
  • a vacuous test of mine — reworded the notice in an earlier round and left the test grepping for a word it no longer contained, so the assertion passed unconditionally.

Declared, not fixed

The excerpt search is O(n²) in windows: 0.3 ms at 55K chars, 27 ms at 500K, 434 ms at 2M. Left as is — the fit is not strictly monotonic in window count (adjacency changes the join cost), so a binary search could return a smaller excerpt than one that fits. Two lenses raised it; the measurement is in the docstring.

Verification

5747 passed, 1 skipped · coverage 96.79% (gate 90) · ruff check, ruff format --check, mypy . clean over 275 files · 36/36 harness self-tests.

Closes #882

…d stop citing what it never read

`query` built its prompt from the full body of every retrieved document
with no size bound and no disclosure. Ollama does not raise on an
oversized prompt -- it discards the overflow and answers normally -- so
`--save` filed every retrieved citation as provenance verbatim,
including documents the model had never read. That is a false provenance
claim on disk, and no surface (`query`, `lint`, `status`) could reveal it.

Measured live against qwen3:8b at num_ctx 12288, same corpus both arms:

  no bound   208,222 chars -> prompt_eval_count 6146
  bounded      8,410 chars -> prompt_eval_count 2449

At the ratio the bounded arm establishes (3.434 chars/token) the unbounded
prompt should evaluate ~60,634 tokens, so 54,488 of them -- 89.9% -- were
being discarded in silence. The bounded arm's chars and tokens agree, so
the whole prompt is read.

#888 chunked the embedding side but `VectorStore.query` collapses to one
hit per document, so the retrieval unit stayed the whole document and
this defect survived it intact.

- New leaf `prompt_budget` owns the bound: planning window, char budget,
  even-coverage excerpt, and a water-filling split of one window across
  competing blocks. #866's `_bounded_prompt_source` becomes a thin wrapper
  over the same renderer so ingest and retrieval cannot drift apart.
- `Citation.excerpted` and `AnswerResult.excerpted_titles` record what was
  SENT, not what was cited: the attribution filter would otherwise erase
  the disclosure exactly when the answer cited nothing.
- A block left no room at all is dropped from the prompt AND the citation
  list, and reported separately -- a bare label is still a numbered block
  the model can cite. The disclosure travels on the no-match return too,
  or an entirely dropped context reads as "the bundle has nothing".
- `query` names the affected documents on stderr, marks partial citations
  `[partial]`, and discloses both cases in the `--save` plan before the gate.

Review receipt review-dc71e9c4ba1743da, four lenses, APPROVED with zero
corrections consumed. Three earlier candidates were fixed pre-capture:
a `fair_shares` invariant violation where an empty body took the only
available char and starved a larger block; an order-dependence that
contradicted its own docstring; and a vacuous test of mine that grepped
for a word the reworded notice no longer contained.

Known and declared: the excerpt search is O(n^2) in windows -- 0.3 ms at
55K chars, 434 ms at 2M. Left as is because the fit is not strictly
monotonic in window count, so a binary search could return a smaller
excerpt than one that fits.

Closes #882
@jasonssdev
jasonssdev merged commit bd95870 into main Aug 27, 2026
7 checks passed
@jasonssdev
jasonssdev deleted the fix/query-context-budget-882 branch August 27, 2026 04:16
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

query sends unbounded context and files provenance the model never read

1 participant