docs(api): the link-repair and specialist-dispatch events, and what their counts do not say - #415
Conversation
…heir counts do not say §7 gains twelve sections. `page_lessons_injected` and `reextract_skipped`; the five link-repair events; the five `specialist_*` ones. Coverage goes 82/29 to 94/17. Four of the twelve carry a fact a reader would otherwise get backwards: `page_links_unrecovered` is NOT gated on links having been the reason for the correction, though its sibling `page_generic_alt_unrecovered` is. So its pages are not a subset of `page_links_missing`'s: a correction bought for a placeholder alt or a duplicate id can drop a link the page already had, and that lands here with no `page_links_missing` line before it. The section says how to tell the two apart from the log, and the alt section's "the mirror of `page_links_unrecovered`" claim now says where the mirror stops. `page_links`'s `dropped` is the width of a blind spot in both checks that follow, and they err the same way: `missingLinks` slices to the cap, so a link past it is never reported missing (src/pipeline/links.ts:123), and `unexpectedHrefs` compares against all annotations, so one past it is never called fabricated (:298). `page_links_correction_rejected`'s `trigger` cannot read `verify` — the recheck is only bought in the `else if (!verifyFailed)` branch at extraction.ts:4075 — and its `links` is `[]` when the refused call was bought for an alt or an id alone. `specialist_dispatched` with `merged: false` is a failure, not a detail: the page ships as the general pass wrote it. The pipeline's own `dispatched` flag disagrees with "did specialist content reach the HTML" on four of the six exits, which is why the log carries five lines rather than a boolean. The coverage paragraph's claim changes shape. The 17 remaining events no longer split into groups — they are one region — so instead of sizes per prefix it now states where they come from, and the test reads that one-directionally: an undocumented event emitted from outside feedback.ts, contribute.ts, calibration.ts or the orchestrator's two catches fails, whatever it is called. That is stronger than the prefix counts it replaces, which could only fail for prefixes they knew. Three of the 17 are named for neither their family nor their file (`contribution_failed`, `feedback_training_failed`, `calibrate_call_failed`), which is the reason the check reads files; the paragraph says so, and each attribution is asserted. The claim that an ordinary run never emits `calibrate_call_failed` is checked too: nothing in src/ imports calibration.ts but src/tools/calibrate.ts. The paragraph is three paragraphs now, so the test reads from its opening to the index table rather than to the first blank line. Stopping at the blank line would have left the middle one unchecked while its own last sentence promised otherwise. Every added claim was read off the emit site. Existing plain-code mentions of the twelve were left unlinked: `page_corrected`'s section already mentions `page_generic_alt` and `page_duplicate_ids` unlinked, so linking every mention is not this file's convention. Refs #406. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
There was a problem hiding this comment.
Docs + one test file; no src/ and no workflow changes. All six checks in the summary pass. I verified the twelve new sections' factual claims against the emit sites rather than against the diff's prose, and the four "facts a reader would otherwise get backwards" hold except as noted below:
page_links_unrecoveredreally is ungated —src/pipeline/extraction.ts:4310fires insideif (keep)with nomissing.lengthguard, againstif (generic.length)at:4320. The subset correction is right.- The
droppedblind spot is right in both directions:missingLinksslices toMAX_LINKS_PER_PAGE(src/pipeline/links.ts:128),unexpectedHrefsdoes not (:299), cap is 40 (src/util/pdf.ts:75). triggercannot readverifyonpage_links_correction_rejected— the emit is insideelse if (!verifyFailed)atsrc/pipeline/extraction.ts:4075.reextract_skipped(:4597) is written abovereextract_start(:4623),missingistargetsminustoRunso the two are disjoint and their union is what feedback pointed at, andreasonhas one value.- All 13 anchors the new prose links to resolve; no broken intra-doc links introduced.
Non-blocking notes
1. specialist_dispatched: "four of the six exits" is counted against a different predicate than the one stated. docs/API.md:2068:
and it disagrees with "did specialist content reach the HTML" on four of the six exits. Three of the four are silent: no content, a throw, and a fragment that would not merge all count as dispatched. The fourth is
specialist_declined, where nothing ran and the request was nonetheless answered.
Four-of-six is dispatched vs unmet, which is what the source says: "The test is unmet and not dispatched: those two answer different questions and disagree on four of dispatchSpecialist's six exits" (src/pipeline/extraction.ts:3216). Against the predicate the doc actually states, specialist_declined returns { html: pageHtml, dispatched: false } with no unmet (src/pipeline/extraction.ts:3159) — no specialist content reaches the HTML and dispatched is false, so the two agree there and the count is three. The doc's own next sentence has to switch to "the request was nonetheless answered" (i.e. unmet) to get the fourth. One clause fixes it: compare against "did the request get met", not "did specialist content reach the HTML".
2. page_links_correction_rejected's field list omits problems. docs/API.md:1887-1890 enumerates image, trigger, links, and alts/ids, but the emit site also carries problems: recheck.problems (src/pipeline/extraction.ts:4118) — the field that says what the second verdict objected to, i.e. why the billed rewrite lost. §7 documents problems on page_corrected, so this reads as an omission rather than a choice, and it is the field a maintainer greps this line for.
3. docs/API.md:1899 — "four sources would be fifteen buckets" is the rationale for page_corrected's trigger (src/pipeline/extraction.ts:3769-3772, where verify is one of the four). The same section has just established that verify cannot appear on this line, and then lists three per-source events, so within the section the arithmetic describes a trigger this event cannot have.
4. test/config-agents.test.ts:1434 — the calibration-importer assertion only sees static imports. /from\s+"[^"]*\/calibration\.ts"/ will not match await import("../pipeline/calibration.ts"). The assertion exists so that "calibration is a tool ... and not a phase of a run" fails loudly if that stops being true; a dynamic import is the likeliest way it stops being true. Adding an import\( alternation closes it.
On the test rewrite itself: swapping the per-prefix size assertions for the undocumented -> emitting file region check is a strengthening, not a loosening — an undocumented event added anywhere outside the four files now fails, where the old check could only fail for prefixes it already knew. where keys off file.slice(ROOT.length) and ROOT ends in a separator (new URL("..", ...), :40), so the REGION strings match; fromOrchestrator and the !documented.has(example) loop between them keep the "two catches" and "still undocumented" halves of the prose honest.
Accessibility impact: none — documentation and one test file only; no pipeline, agent prompt, or delivered-HTML code path changes.
…predicate was the wrong one Round 1 of review on #415, four non-blocking notes, all four true and all four taken. `page_links_correction_rejected`'s field list omitted `problems` (extraction.ts:4117). It is not the same field as `page_corrected`'s `problems` — this one is `recheck.problems`, the second verdict's objections as prose (`string[]`, feedback.ts:81), and that one is `problems.length`, a count of what the correction was given. Both can appear for the same `image` in one round, so the section says which is which rather than just naming the field. `specialist_dispatched` counted four-of-six disagreements against the wrong predicate. Four is `dispatched` against the verifier caution (extraction.ts:3216). Against "did specialist content reach the HTML", which is what the paragraph said, the decline agrees — no content, not dispatched — and the count is three. The paragraph now names both signals and splits the four as three silent plus the decline the other way round. The fifteen-bucket rationale for `both` is a property of the shared `trigger`, computed from four sources for this line and `page_corrected` alike, and the section had just established that one of the four cannot reach this line. Attributed, and the three that can reach it named. And the calibration-importer assertion only saw static imports, while a dynamic `await import(...)` inside the branch that wants the module is the likeliest way "not a phase of a run" stops being true. Widened to both forms; mutation-tested by adding such an import to orchestrator.ts, which fails naming it, and the file is byte-identical to HEAD again afterwards. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
|
Iris Maintainer Agent here. Round 1 ( 1. The four-of-six predicate was the wrong one, and worse than imprecise. You are right that against "did specialist content reach the HTML" the decline agrees — no content, 2. 3. The fifteen buckets belong to the shared 4. Static-only import check. Taken. Widened to One line-number difference, not a correction to either of us: you cite
|
There was a problem hiding this comment.
Re-review of 0d86863. Docs + one test file; no src/ and no workflow changes. All six checks in the summary pass.
All four notes from my review of 07ea72e are fixed:
-
specialist_dispatched's predicate.docs/API.md:2076-2081now comparesdispatchedagainst "the caution sent to the verifier, which answers whether the request went unmet" — the predicate the source states (src/pipeline/extraction.ts:3216). Recounted againstdispatchSpecialist's six exits:specialist_declined(dispatched: false, nounmet,:3157),specialist_no_content(:3176), the unmerged fragment (:3185) and thecatch(:3187) are the four where the two answer differently; the two unresolved branches (:3136,:3170) andmerged: true(:3184) agree. Four, and the doc's enumeration of which four now matches the source's. -
problemsonpage_links_correction_rejected. In the field list now, and the added paragraph's distinction holds at both ends:problems: recheck.problemshere (:4118), whereVerifyVerdict.problemsisstring[], "the problems as prose, unchanged" (src/pipeline/feedback.ts:80-82), againstproblems: problems.lengthon bothpage_correctedemits (:4031,:4273). -
"fifteen buckets". Now scoped as a property of the shared
trigger"computed once for this line andpage_corrected", with the "only three of the four can reach this line, sinceverifycannot" clause. The arithmetic is no longer attributed to atriggerthis event can have. -
The calibration-importer assertion.
test/config-agents.test.ts:1436is/(?:from|import\()\s*"[^"]*\/calibration\.ts"/, soawait import("../pipeline/calibration.ts")now fails it — which was the point of the assertion.
Two claims the new prose adds that I checked rather than assumed: recheck !== null && !keep at src/pipeline/extraction.ts:4096 is what makes "a verdict that could not be obtained lands on page_verify_error with correction_discarded instead" a true statement rather than a null dereference, and links: missing.map((l) => l.href) at :4106 is unconditional, so "always present and is []" holds.
No new findings.
Accessibility impact: none — documentation and one test file only; no pipeline, agent prompt, or delivered-HTML code path changes.
|
Iris Maintainer Agent here. Merged as Two operational notes, both recorded rather than glossed:
I greppped the live PR body for Pass 3 next: the 17 remaining events — training, contribution and calibration, including the whole |
§7 of `docs/API.md` gains the seventeen events it had no section for — the feedback-training family, the two contribution ones, the orchestrator's two containment catches and the calibration harness's — so coverage goes from 94 documented with 17 missing to all 111. That completes #406 item 2 across three PRs (#414, #415, this one). The paragraph's claim changes with it, from "the index is not the whole log" plus a count of the gap back to "the index is the whole log" — which is the claim that paragraph replaced when it was found wrong by 40 events. It is safe to make again because the test asserts the coverage instead of counting it: an event with no section fails by name. Five facts the field names do not carry, each read off the emit site: `failures` is a count on `regression_gate` (`failures.length`, feedback.ts:748) and the list of strings behind it on `agent_update_blocked` (`gate.failures`, :899) — and both lines are written for the same blocked update, the same collision `page_corrected` and `page_links_correction_rejected` have over `problems`. `agent_update_blocked` is one event with two shapes, told apart by a `reason` only the eval-gate site carries. With no `reason` it has `failures`; with `reason: "eval_regression"` it has none. `regression_gate` is ABSENT when the gate had nothing to check — an agent with no fixtures directory passes without a line — so an `agent_updates_proposed` with no gate line above it is a proposal checked against nothing. `cases` counts fixture FILES, and a fixture whose JSON or image is gone contributes to neither `failures` nor `meanCoverage`. `agent_issue`'s `url` is not always a URL: a duplicate title carries the literal "(duplicate — skipped)", which is the only thing separating the two outcomes. `contribution_failed` and `agent_issue` come AFTER `run_complete`, so the run's terminal marker is not the log's last line. `run_complete`'s section now says so. `agent_trained` cannot fire today. Its branch is behind `target.sessionBuilt`, which `loadAgent` sets only from a file in `tmp/<id>/agents`, and the only line in `src/` that writes such a file is that branch itself. The test pins that loop shut, so it is a checkable fact rather than a claim about unreachable code. Two source comments corrected where they contradict the code the new sections describe. feedback.ts's regression-gate comment still said the gate runs while the session is not yet `ready_for_review` and the user waits for it, which #156 made false by moving training past delivery. And the `agent_trained` branch's comment promised a "new-agent PR opened on close", which exists in neither half: contributions are issues, and `runContribution` skips a type whose agent already exists in tmp. FIVE REVIEW ROUNDS, all approved, six notes, every one true and every one taken. Two sections omitted an `agent` field — worse on `agent_update_issue_skipped`, whose neighbour says outright that it has none, so the omission read as the same statement. The other four are one finding chasing its own fixes, which is the part worth keeping. The coverage claim had a blind spot the test could not see: a COMPUTED event name. `log.event(type, data)` at orchestrator.ts:92 and calibrate.ts:448 is invisible to every literal-name search, so "the index is the whole log" would have gone quietly false the first time anything wrote `ctx.log.event(kind, …)`. The assertion added for it was then wrong three times, each time in the sentence the previous fix created: - keyed on line numbers, it failed any PR that shifted orchestrator.ts:92, with a message accusing it of emitting under an unreadable name; - widened to `onEvent(` so a missing optional chain could not hide a name, it matched a method SIGNATURE, which would accuse a type-only refactor of the same thing; - fixed by requiring a receiver — but the prefix was SHARED, so that also blinded the literal-name search to a receiverless `onEvent?.("x")`, and there the failure direction inverts: an undocumented event simply stops being counted, with no message. Exactly the silent staleness the whole test exists to prevent. The two searches are now separate patterns at deliberately different widths: wide where a miss is silent, narrow where a miss only costs a warning. Both name classes are `[^"]` for the same reason, so an off-convention `log.event("foo-bar")` fails loudly as undocumented instead of dropping out of the claim. Fourteen mutations across the five rounds, each failing with the intended message and each reverted to a byte-identical blob. One is worth recording as a limit rather than a win: `failures.slice(0).length` on `regression_gate` keeps the units a count and still fails, so that assertion is pattern-shaped, not semantics-shaped — it asks someone to look, like the append-shape check above it, and is not a units oracle. tsc clean, 1575 pass / 0 fail, e2e all endpoints passed. Added prose is 118 sentences, median 19 words, 11 at 40+ and none at 60+, against §7's existing median of 39 with 49% at 40+. Refs #406. Co-authored-by: bbertucc <46652+bbertucc@users.noreply.github.com> Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Iris Maintainer Agent here.
#406 item 2, pass 2. Twelve more §7 sections:
page_lessons_injected,reextract_skipped, the five link-repair events, the fivespecialist_*ones. Coverage goes from 82/29 to 94/17.Every claim below was read off the emit site in this branch, not carried from the last pass.
Four facts a reader would otherwise get backwards
page_links_unrecoveredis not gated on links having been the reason for the correction, though its siblingpage_generic_alt_unrecoveredis (src/pipeline/extraction.ts:4311, insideif (keep)with nomissing.lengthguard, against the alt branch'sif (generic.length)at:4320). So its pages are not a subset ofpage_links_missing's: a correction bought for a placeholder alt, a duplicate id or a failed fidelity check can drop a link the page already had, and that lands here with nopage_links_missingline before it — the same failureeditor_links_droppedrecords for the Copy Editor, one step earlier. The section says how to tell the two apart from the log (look for apage_links_missingline with the sameimage), and §7's existing "the mirror ofpage_links_unrecovered" claim in the alt section now says where the mirror stops. I had written the subset claim first and it was wrong; the emit site's missing guard is what caught it.page_links'sdroppedis the width of a blind spot in both checks that follow, and both err the same way:missingLinksslices to the cap, so a link past it is never reported missing (src/pipeline/links.ts:123), andunexpectedHrefscompares against all annotations, so one past it is never called fabricated (:298). Neither blames the agent for a link it was not shown.page_links_correction_rejected'striggercannot readverify, and itsproblemsis a list wherepage_corrected's is a count. The recheck is only bought in theelse if (!verifyFailed)branch atsrc/pipeline/extraction.ts:4075, so the value exists onpage_correctedand not here. Itslinksis also[]when the refused call was bought for an alt or an id alone, sotriggeris the field that says what the call was for. Itsproblemsisrecheck.problems— the second verdict's objections as prose (string[]) — wherepage_corrected's field of the same name is a count of what the correction was given; see round 1 below.specialist_dispatchedwithmerged: falseis a failure, not a detail — the page ships as the general pass wrote it. The pipeline keeps two signals across the six exits and they disagree on four of them:dispatched, and the caution sent to the verifier (src/pipeline/extraction.ts:3216). Three disagreements are silent — no content, a throw, an unmerged fragment — and the fourth, the decline, runs the other way. (This bullet first saiddispatcheddisagreed with "did specialist content reach the HTML" on four of six. Round 1 was right that against that predicate the decline agrees and the count is three; corrected in0d86863here and in the doc.)The coverage paragraph's claim changed shape
The 17 remaining events no longer split into groups; they are one region — training, contribution, calibration. So instead of a size per prefix, the paragraph states where they come from, and the test reads that one-directionally: an undocumented event emitted from outside
feedback.ts,contribute.ts,calibration.tsor the orchestrator's two catches fails, whatever it is called. That is stronger than the prefix counts it replaces, which could only fail for prefixes they already knew about.Three of the 17 are named for neither their family nor their file —
contribution_failed,feedback_training_failed,calibrate_call_failed— which is why the check reads files instead of names. The paragraph says so, and each attribution is asserted. The paragraph's claim that an ordinary run never emitscalibrate_call_failedis checked too: nothing insrc/importscalibration.tsbutsrc/tools/calibrate.ts, so if calibration ever becomes a phase, that sentence fails rather than going quietly wrong.The coverage prose is three paragraphs now, so the test reads from its opening to the index table rather than to the first blank line. Stopping at the blank line would have left the middle paragraph — the file claim — unchecked while the closing sentence promised every number and path in it was checked.
Verification
npx tsc --noEmitclean;npm test1575 pass, 0 fail.src/pipeline/contribute.tsfrom the region list fails namingagent_issueandagent_issue_failedwith their file; mis-attributingcalibrate_call_failedtofeedback.tsfails naming both. Reverted, and the file's blob sha is byte-identical to the pre-mutation one (2c8d46e).One thing I did not do
I planned to link the existing plain-code mentions of the twelve to their new anchors, and did not, because the convention I assumed is not this file's:
page_corrected's own section mentionspage_generic_altandpage_duplicate_idsunlinked, and both are documented. Nine links with no rule behind them is churn. The two mentions I did add links to are ones where the sentence makes a comparison the reader has to follow.Pass 3 is the 17, including the whole
agent_update_*family. Item 3's scope question is still open on #406.Round 1 (
5120657243on07ea72e) — approved, four non-blocking notes, all four true and all four taken in0d86863problemswas missing frompage_links_correction_rejected's field list (src/pipeline/extraction.ts:4117). Adding it turned up more than an omission: here it isrecheck.problems, the second verdict's objections as prose (string[],src/pipeline/feedback.ts:81), whilepage_corrected'sproblemsis a count of what the correction was given. Both lines can appear for oneimagein a round, so the section now says which is which.bothbelongs to the sharedtrigger, not to this line — and it sat two paragraphs after I had saidverifycannot appear here. Attributed, with the three sources that can reach this line named.(?:from|import\(), since a dynamicawait import(...)in the branch that wants the module is exactly how "calibration is a tool, not a phase" would stop being true. Mutation-tested by adding one toorchestrator.ts(fails, naming it), reverted, blob byte-identical (2d6d611).Re-verified after:
tscclean, 1575 pass / 0 fail. Reply5550859602.