fix(ingestion): #884 extraction_notice records every condition, not the strongest - #892
Merged
jasonssdev merged 1 commit intoAug 27, 2026
Merged
Conversation
…he strongest
extraction_notice held ONE value, so when a run tripped several disclosure
conditions the strongest token overwrote the weaker at write time. lint --
the audit surface -- then rendered "Unevidenced objects:" and
"Staging-dropped candidates:" as independent sections while the second
silently masked the first.
lint could not paper over that from its own side: the displaced token is
destroyed at ingest, so by the time lint reads the document there is nothing
left to disclose. Recording every condition is the only place the fix can
live. (The issue proposed "make lint disclose the masking" as the cheapest
remedy; implementing it showed it is not implementable, and collapses into
this one.)
The masking was PERVASIVE, not the rare case the issue described. Making the
field plural turned ten existing ingest tests red with the shape
['judge-selection-unavailable', 'objects-without-evidence'] == 'judge-...'
-- those fixtures had been tripping two conditions and losing one all along,
invisible because the tests only asserted the winner.
Also fixed, per the same issue: an in-batch slug collision no longer counts
as a staging loss. When two candidates of one run slugify alike the FIRST
was already staged, so the content is on disk and the bundle represents the
source -- the identical reasoning the code already applied to the create-only
skip. Counting it also left debt no command could clear, since the redo that
marker prescribes (--re-extract) reproduces the same collision.
Precedence did not disappear; it moved to where it belongs. It was a rule
that DESTROYED data in the frontmatter, and is now a presentation rule in the
batch summary, which counts a file once however many conditions it carries.
On disk a single condition stays a bare scalar, byte-identical to earlier
releases; only a genuine co-occurrence widens to a list, and readers accept
both shapes so existing Sources keep their disclosure.
Two false claims already written in the code were corrected:
- lint.py stated "disjoint by construction: a Source carries at most one
token" -- true of the TOKENS, false of the CONDITIONS. That sentence was
the justification for the broken design.
- A comment added in this change promised a single condition would write the
bytes it always did while the code wrote a list. The code was changed to
match the comment.
Four tests named "..._outranks_..." were renamed and strengthened: after the
change they passed while asserting only membership, so their names claimed
more than their bodies checked.
The headline lint test uses TWO Sources with the tokens in OPPOSITE order.
One alone does not discriminate -- whichever token sits at index 0 is still
found by a reader inspecting only the first element. Mutating either lint
reader to doc.extraction_notices[:1] now fails; with a single fixture,
neither did.
Specs updated: ingestion's precedence requirement rewritten, its two
precedence scenarios replaced, two scenarios added (collision without a
marker, scalar-on-disk for one condition).
Full suite 5659 passed, 1 skipped; ruff clean; mypy clean over 272 files.
Closes #884
jasonssdev
deleted the
fix/extraction-notice-carries-every-condition-884
branch
August 27, 2026 01:42
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Closes #884
The defect
extraction_noticeheld one value, so when a run tripped several disclosure conditions the strongest token overwrote the weaker at write time.lint— the audit surface — then renderedUnevidenced objects:andStaging-dropped candidates:as independent sections while the second silently masked the first.The issue's cheapest proposed remedy was "make
lintdisclose the masking". Implementing it showed that is not possible: the displaced token is destroyed at ingest, so by the timelintreads the document there is nothing left to disclose. That remedy collapses into this one.The masking was pervasive, not rare
Making the field plural turned ten existing ingest tests red with this shape:
Those fixtures had been tripping two conditions and losing one all along — invisible, because the tests only asserted the winner. This was not a quirk of the E2E corpus.
Also fixed: the in-batch collision is no longer a staging loss
When two candidates of one run slugify alike, the first was already staged, so the content is on disk and the bundle represents the source — the identical reasoning the code already applied to the create-only skip. Counting it also left debt no command could clear, since the redo that marker prescribes (
--re-extract) reproduces the same collision deterministically.Precedence moved rather than disappeared
It was a rule that destroyed data in the frontmatter. It is now a presentation rule in the batch summary, which counts a file once however many conditions it carries.
lintenumerates them all.On disk, a single condition stays a bare scalar — byte-identical to earlier releases. Only a genuine co-occurrence widens to a list, and readers accept both shapes, so existing Sources keep their disclosure.
Two false claims already in the code, corrected
lint.pystated "disjoint by construction: a Source carries at most one token" — true of the tokens, false of the conditions. That sentence was the justification for the broken design.Tests
Four tests named
..._outranks_...were renamed and strengthened: after the change they passed while asserting only membership, so their names claimed more than their bodies checked.The headline
linttest uses two Sources with the tokens in opposite order. One alone does not discriminate — whichever token sits at index 0 is still found by a reader inspecting only the first element. Mutating eitherlintreader todoc.extraction_notices[:1]now fails; with a single fixture, neither did.Specs
ingestion/spec.md's precedence requirement rewritten, its two precedence scenarios replaced, and two scenarios added: an in-batch collision without a marker, and scalar-on-disk for a single condition.Full suite 5659 passed, 1 skipped.
ruffclean.mypyclean over 272 files.Found during the 0.2.10 manual E2E.
🤖 Generated with Claude Code
https://claude.ai/code/session_01MZz1zEMqg4X5mDEwwfC6dj