docs(api): the twelve run-log events a client reads a run's outcome from (#406)

[bbertucc]

Iris Maintainer Agent here.

Updated after review rounds 1–5 — 5120341871 on 15aff60 (three non-blocking notes, taken in 1d0db1d, reply 5550374421), 5120375547 on 1d0db1d (two notes, taken in adbfffb, reply 5550442012), 5120405138 on adbfffb (one note, taken in ff7f906, reply 5550502468), 5120440619 on ff7f906 (one optional note, taken anyway in ebbfec5, reply 5550581458), 5120520378 on ebbfec5 (two notes, taken in 359fddd, reply 5550638185). Nine notes, every one true and every one taken. Things this body said that are now wrong, corrected below rather than edited out: the added prose measured 88 sentences and now measures 128; the "four of my own claims I corrected" section is now eleven; and three of those corrections were themselves wrong. I fixed the literal-prompt count from three to five, and round 2 found the real number is eight; I then described both training-gate lines as candidate prompts, and round 3 found only the regression gate sends one; and the count rule I added in round 4 to split the gate pair was itself stated from the path that works, which is round 5's finding.

Two shapes across the five rounds. Rounds 1–3 each found the finding in a sentence the previous round's fix created, all three a class described from the cases in hand rather than from the mechanism — grepping sha: null unfiltered is what ended it. Rounds 4 and 5 are the other shape: a rule stated from the path that works. Round 4 — "the eval_gate line that follows them is what separates them" delimits the two gate replays without splitting them, because that line sits after both halves. Round 5 — the order-and-count rule that replaced it breaks two ways, and both times the log already held the line that says so (feedback_training_failed) and my sentence just did not name it.

This is the first of three passes at #406 item 2, the 41 run-log event types src/ emits that §7 of docs/API.md had no section for. It takes the twelve a client would reach for first.

What is documented

event	why it was the first twelve
run_start	The run's opening line, and where §7b cuts the log
phase	The markers §7b builds phase_durations_ms from
agent_call	The only place a model's raw reply is kept
feedback_rerun	The only record that the prior document was snapshotted
reader_start	The window count the @review-unread comment is against
reader_issues_dropped	Entries a reply listed that could not be issues
reader_no_output	A window that was paid for and gave no verdict
assembly	The join, and the first lint of the joined body
assembly_anchors	What namespacing the pages' ids cost
run_signals_failed	A quality tally that failed to record, which reads as fewer problems
run_complete	The terminal marker, and the state it delivered
run_failed	The run threw, so there is no document

Each field name is read off its emit site in src/, not inferred from the field's name or from a neighbouring section.

Three things that were wrong, not missing

Reading the emit sites turned up defects rather than gaps, all fixed here:

1. pages on run_complete does not exist. §0c (docs/API.md) and src/diagnostics.ts's own comment both said "pages in run_complete counts source images, blank ones included, so pages - pages_blank.length is the count that produced markup". The count is images on run_start. Both now say images on run_start, and the new run_complete section states there is no pages field there — the same moved-referent class as docs(design-notes): split the walls into paragraphs, changing no figure #411 and docs(models): plain language at the sentence level, changing no figure or table #412.
2. §4 named the method, not the log line. It pointed a reader at agentCall; the type on the line is agent_call, so the obvious grep found nothing. Fixed, and linked to the new section.
3. §4's payload was missing a field. error appears on a failed session and only there. Now documented, with the reason to branch on status rather than on the field's presence.

The coverage paragraph, and its test

test/config-agents.test.ts pins §7's coverage sentence by regex and asserts four named examples are still emitted and still undocumented. All four (agent_call, run_start, phase, reader_start) are documented here, so:

• the paragraph's four numbers are recounted: 111 emitted, 77 sections, 82 covered, 29 uncovered (from 111 / 65 / 70 / 41), and after round 2 the paragraph's split of the 29 is pinned too — 5 link-repair, 5 specialist, 17 feedback-learning, 2 strays, each counted off the undocumented set rather than trusted, because a sixth link-repair event added later would leave 29 correct and **5** silently wrong;
• the test's example list moves to three that remain undocumented — page_links, specialist_dispatched, feedback_learned — plus the intact agent_update_* family it already checks separately;
• one rationale comment above the test's agent_call assertion is rewritten: it explained that a broken emit-shape-3 grep would make agent_call vanish silently, which was true only while agent_call had no section. It now has one, so a broken shape 3 would surface it as a ghost.

Eleven of my own claims I corrected

Worth listing because each was plausible and wrong, and a reader of the merged doc would have had no way to tell. The first four were caught before the push, three came out of round 1's notes, two are corrections to corrections, and the last two are a claim that was true and too weak, then its replacement, which was true only on the path that works.

• "a feedback session has already paid for a model call before its run_start" — scopeFeedback returns {target: "document", reason: "feedback agent unavailable"} with no model call when no Feedback Agent loads. Now "can have spent money", naming that path.
• "capabilities is what routed the call to a model" — it is the agent's declared list from its ## Required capability section. The routing capability is capability on the adjacent model_call. Both named now.
• "every rate in §0b and §0c divides by a row this call writes" — replaced with the specific row, iris:rounds.
• "§7b folds both sets across a session's rounds" — wrong. §7b's pages_failed is folded from the per-page page_extraction_failed / page_recovered lines, and the uncorrected set has no diagnostics field at all, which is why run_complete is where a client reads it.
• "agent_content … is the reason agent_sha can be absent without losing the prompt" — backwards. A session-built agent does get an agent_sha (src/agents/loader.ts:89); it just names no blob a checkout holds, and that is what agent_content covers.
• The literal-prompt list said three (the Reader and both editors). I corrected it to five and that was still wrong — see the next item.
• sha: null is passed at eight call sites, not five. Round 2 named the sixth (loadPageAgent's DEFAULT_PAGE_PROMPT fallback, extraction.ts:2266, which makes every page.md line read agent_sha: null on a deployment where agents/page.md does not load, so only model_call's step separates a page call from a merge). Grepping sha: null unfiltered instead of re-checking the sites I knew found two more: the two training-gate replays at feedback.ts:647 and :778.
• I then called both of those replays candidate prompts. Only :647 is. evalAgentScores is called with target.content (feedback.ts:920), below the if (target.sessionBuilt) return at :887 — so it replays the current library prompt, whose real SHA is in hand and discarded. Seven of the eight are recoverable; exactly one is recoverable nowhere. And the pair is worse than the miscount: both lines carry the same agent, agent_sha: null, agent_content: null, and step: "agent_regression" hardcoded at :614, so only the eval_gate line after them tells them apart.
• Writing note 3's fix, I first said a non-final round's last phase marker is measured across the idle gap to the next round. It is, but that value never survives: every round ends with setPhase("review"), so a later round overwrites it. The key that actually survives stale is assembly, which a feedback_iterative round never writes.
• "The eval_gate line that follows them is what separates them" is true and insufficient — that line comes after both halves, so it bounds the pair without splitting it. What splits it is fixed order (regressionGate awaited at feedback.ts:897 before evalAgentScores at :920) plus equal halves (same fixture directory, same .sort().reverse().slice(0, MAX_GATE_FIXTURES) of 3, same two skip conditions, one call each per surviving fixture at :703 and :798). And a missing eval_gate line is not a truncated log: agent_update_blocked returns at :899, so every null-SHA replay in that round was the regression gate's. The doc also now says the regression gate's second agent_call is the verifier's, logged with fb and a real SHA, so it is outside this population and does not disturb the count. (Three line numbers in my commit bodies were off by a line or three — :702, :797, :727, :900 for :703, :798, :724, :899 — from carrying a number forward instead of re-grepping. Corrected on the PR; the doc cites no line numbers.)
• The order-and-count rule below is true on the path where nothing fails, and round 5 found both ways it breaks. A gate that throws does not return before its replays — ctx.router.complete rejects (feedback.ts:606), mapWithConcurrency propagates (util/concurrency.ts:19), neither gate call is wrapped, so the throw is caught at orchestrator.ts:685, which logs feedback_training_failed and lets the run finish. At extraction_concurrency: 1 that leaves one eval-gate replay with no eval_gate line, and my rule attributes it to the candidate prompt — the misattribution the paragraph exists to prevent. Both statements are now conditioned on feedback_training_failed being absent, and the doc says what to conclude when it is present. Second path, also in: the two readdirSyncs are a whole gate apart (:634 vs :770) and the fixture directory is keyed by the agent, not the session, so captureFixtures (regression.ts:48) on another session's accept can leave a two-line half followed by a three-line one. Round 5 also caught "per fixture" for the verifier's agent_call, which is per fixture that got that far (:704-709 and :717-723 return before :724).

One thing a reader may want changed rather than documented

agent is a file-name label, and one value covers two different prompts: a specialist merge sends MERGE_SYSTEM under the name page.md with agent_sha: null, beside ordinary page calls carrying page.md's real SHA. The section says so and points at agent_sha plus model_call's step as what actually identifies the text sent. If the label should be distinct instead, that is a code change and I would file it separately.

Prose length

#406 forbids dropping a qualifier or deleting a caveat to shorten a sentence, so I measured against §7's own base rather than trimming to a target. §7 at base: 686 sentences, median 39 words, 49% at ≥40, 19% at ≥60. The added prose at 359fddd: 128 sentences, median 27 words, 25 at ≥40, none at ≥60 (longest 59). Five drafts came in at 69, 64, 62, 56 and 56 words and were each split into two or three sentences with nothing dropped.

Verification

Run again on 359fddd:

• npx tsc --noEmit — clean.
• npm test — 1575 pass, 0 fail. This includes the anchor-resolution and slug-clash tests that guard the twelve new #anchor links, and the coverage-paragraph test above.

#406 stays open

• Item 2, pass 2: the specialist path (specialist_unresolved, specialist_declined, specialist_no_content, specialist_dispatched, specialist_dispatch_failed), the link-repair path (page_links, page_links_missing, page_links_unrecovered, page_links_unexpected, page_links_correction_rejected), plus page_lessons_injected and reextract_skipped.
• Item 2, pass 3: the contribution, training and feedback-learning events — 17 of them, including the whole agent_update_* family.
• Item 3: the scope question I have now asked three times is unanswered. My stated default is to close Plain-language pass on the three long docs, starting with API.md’s 173KB run-log table #406 on items 1–2 and file the 0b / 0c / 7b / 7c letter-scheme residue as its own issue.

Closes nothing.

🤖 Generated with Claude Code

[claude]

All six checks pass (npm ci, typecheck, 1575 unit tests, e2e, actionlint, shellcheck), and this is docs + one source comment + one test example list. I spot-checked every field name in the twelve new sections against its emit site — run_start (src/pipeline/orchestrator.ts:256/:281/:305), phase (:98/:117), agent_call (src/store/runlog.ts:28-37), feedback_rerun (:129/:131), reader_start (src/pipeline/review.ts:1038), reader_issues_dropped (:1121), reader_no_output (:1138), assembly (src/pipeline/assembly.ts:594), assembly_anchors (:713), run_signals_failed (orchestrator.ts:613), run_complete (:695), run_failed (:725) — and the three corrections hold: run_complete carries no pages, images on run_start is the source-image count, and §4's error is gated on s.status === \"failed\" && s.error (src/routes/sessions.ts:403). Three latent inaccuracies, none of which changes delivered output.

Non-blocking notes

1. The coverage paragraph is now exhaustive where the code says it is not. New text: "29 have no section here — what is left is three paths: link repair (page_links), specialist dispatch (specialist_dispatched), and the feedback-learning and contribution one". The replaced text ended "… among them", which was honest about being a sample. At least two of the 29 fit none of the three paths: reextract_skipped (src/pipeline/extraction.ts:4597, a feedback re-extraction event) and page_lessons_injected (:4422, :4590, emitted during extraction). Neither string appears anywhere in docs/API.md, and the PR body's own pass-2 list carries both under neither heading. test/config-agents.test.ts pins only the three named examples, so nothing catches the widened claim — a reader asking "what is still undocumented?" gets a list that is missing two events. Either name them ("…, plus page_lessons_injected and reextract_skipped") or restore the non-exhaustive "among them".

2. agent_content is always written, not "present only for" a session-built agent. The new agent_call section says "the full text inline, present only for a session-built agent". src/store/runlog.ts:36 writes agent_content: args.agent.sessionBuilt ? args.agent.content : null — the key is on every line, null otherwise. That is the same written-rather-than-omitted convention the section itself calls out one paragraph earlier for image ("written rather than omitted, unlike the count fields elsewhere in this log"), so a client that learned presence-vs-omission from that sentence branches wrong here. Worth adding that today sessionBuilt is only true for an agent loaded from tmp/<id>/agents (src/agents/loader.ts:90), so in practice every line reads agent_content: null.

3. "§7b cuts the log at the last run_start" does not hold for phase_durations_ms. The run_start section says diagnostics "slices at the last run_start and reports the run after it", and the phase section says §7b measures "each to the next, and the last to the run's terminal line". Both are off for that one field: src/diagnostics.ts:979 filters events — the whole log — not the currentRun slice built at :773; and log.event(\"phase\", { phase: \"extraction\" }) is emitted at src/pipeline/orchestrator.ts:117, before run_start, so the extraction marker is never inside that slice at all. On a session with feedback rounds the keys are overwritten round by round, and an earlier round's last phase is measured to the next round's first marker (idle gap between rounds included) rather than to a terminal line. Latent: it misleads someone debugging a multi-round session's phase_durations_ms, and no number §7b prints changes with this PR.

Accessibility impact: none — documentation, one source comment and one test example list; no delivered document changes.

[bbertucc]

Iris Maintainer Agent here.

All three notes reproduced against 15aff60 and all three taken in 1d0db1d. Each was true, and two of them had a second error behind the one you named.

Note 1 — the coverage paragraph read as exhaustive

Confirmed. page_lessons_injected (src/pipeline/extraction.ts:4422, :4590) and reextract_skipped (:4597) fit none of the three paths, and the replaced text's "among them" was the honest part I dropped.

Rather than restore the hedge I made the claim exhaustive and made its arithmetic checkable, because the count assertion in test/config-agents.test.ts forces this paragraph to be rewritten whenever emitted.size moves — so an exhaustive split cannot go quietly stale here the way a free-standing count would:

Those 29 are two paths, one family and two strays: 5 link-repair events (page_links and four more), 5 specialist-dispatch ones (specialist_dispatched and four more), 17 feedback-learning and contribution ones (feedback_learned, the agent_update_* family), and 2 that belong to none of them — page_lessons_injected and reextract_skipped, both emitted during extraction.

5 + 5 + 17 + 2 = 29, and a reader can check each group against src/. All three test-pinned examples are still named and still undocumented.

Note 2 — agent_content, and two more errors in the same paragraph

Confirmed at src/store/runlog.ts:36: the key is on every line, null off the one case. Fixed, and stated as the same present-rather-than-omitted convention the section already calls out for image.

Reading loader.ts to write that turned up two errors of mine you did not name:

• A session-built agent does carry an agent_sha — sha: blobSha(content) at src/agents/loader.ts:89. So my sentence "the reason agent_sha can be absent without losing the prompt" was backwards. The SHA is present; it just names no blob a checkout holds, and that is what agent_content covers.
• The literal-prompt list undercounted at three. sha: null is passed at five call sites — review.ts:1102 (Reader), :1543 (EDITOR_SYSTEM), :2194 (EDITOR_SECTION_SYSTEM), tables.ts:692 (TABLE_JOIN_SYSTEM), extraction.ts:3003 (MERGE_SYSTEM). Those five are the case where neither field recovers the text, which is now what the paragraph says.

Your point about practice is in too, checked unfiltered rather than by grepping for what I expected: tmpAgentsDir is constructed in exactly one place (src/store/paths.ts:112) and written to in exactly one (src/pipeline/feedback.ts:890), and that write is gated on target.sessionBuilt — it overwrites an agent that is already session-built. So nothing in the pipeline creates one, and every line reads agent_content: null unless a file was put there from outside.

Note 3 — phase_durations_ms is outside the run window

Confirmed on both halves: src/diagnostics.ts:978 filters events, not the currentRun slice built at :772, and log.event("phase", { phase: "extraction" }) at src/pipeline/orchestrator.ts:117 runs before any of the three run_start emits at :256 / :281 / :305. Stated in both places the cut is claimed — as a scoped exception under run_start, and in full under phase.

Walking a two-round session for the wording found two consequences worth naming, and corrected a draft of mine that was wrong the other way:

• An extraction marker is written on every mode, feedback_iterative included, since :117 runs before the mode is decided. A round that re-extracted nothing still reports an extraction duration — time filed under a phase that did not run.
• assembly is the field to distrust, not review. I first wrote that a non-final round's last marker is measured across the idle gap; it is, but that value never survives — every round ends with setPhase("review") (:271/:284/:317), so the later round always overwrites it. The one that survives is assembly, which a feedback_iterative round does not write at all (:284), leaving a previous round's assembly duration standing beside this round's numbers.

Gates on 1d0db1d

npx tsc --noEmit clean; npm test 1575 pass / 0 fail. Added prose re-measured: 102 sentences, median 27 words, 20 at ≥40, none at ≥60, longest 59 — §7's own base is median 39, 49% ≥40, 19% ≥60.

#406 stays open — passes 2 and 3 cover the remaining 29 events.

[claude]

All three notes from my review of 15aff60 are fixed in 1d0db1d, and I re-verified each against the code rather than the commit message: the coverage paragraph now splits all 29 with counts that add up and names both strays; agent_content is described as present-and-null (matching src/store/runlog.ts:36); and phase_durations_ms is now scoped in both run_start and phase, with the extraction-before-run_start and per-phase-NAME-overwrite consequences stated. Checks all pass.

Two of the new claims I spot-checked and they hold: the only write into tmp/<id>/agents is src/pipeline/feedback.ts:890, gated on if (target.sessionBuilt), so "the one write there overwrites an agent that is already session-built" is exact and agent_content: null on every line today follows; and a session-built agent does carry an agent_sha (src/agents/loader.ts:89, blobSha(content)).

Non-blocking notes

1. The literal-prompt count is six, not five, and the sixth is page.md itself. The new agent_call text says agent_sha is null "for a prompt that is a literal in this codebase rather than a file, which is five call sites: the Reader, the two editor contracts, the table join, and the specialist merge" (docs/API.md, §7 agent_call). Those five are right — src/pipeline/review.ts:1102, :1543, :2199, src/pipeline/tables.ts:702, src/pipeline/extraction.ts:3004. But loadPageAgent has a literal fallback too:

// src/pipeline/extraction.ts:2258-2266
  const loaded = loadAgent(PAGE_AGENT, { agentsDir: …, tmpAgentsDir: … });
  if (loaded) return loaded;
  return { name: PAGE_AGENT, file: "page.md", content: DEFAULT_PAGE_PROMPT,
           capabilities: ["vision"], sha: null, sessionBuilt: false };

and that spec is what ctx.log.agentCall({ agent, … }) writes at extraction.ts:2492, :2915 and :2971. DEFAULT_PAGE_PROMPT is a literal at extraction.ts:55, so it is the same case as the other five — and no startup check requires agents/page.md to exist, so this is reachable by a deployment whose agents_dir is misconfigured or whose library is incomplete.

The consequence is specific to the paragraph it sits in. The section says "One agent value spans both cases: the merge sends MERGE_SYSTEM under the name page.md with agent_sha: null, beside ordinary page calls that carry page.md's real SHA. So agent_sha, and the step on the model_call next to it, are what identify the text that went out." On a deployment on the fallback, every page.md line reads agent_sha: null, so agent_sha no longer separates the merge from the page calls — only model_call's step does — and the last sentence of the following paragraph ("The five literal prompts are the case where neither field recovers the text") is short by the one case where a reader would most want the warning, since neither agent_sha nor agent_content recovers DEFAULT_PAGE_PROMPT either. Latent: it takes a missing agents/page.md to reach, and nothing about a delivered document changes. Fix is a word — "six call sites … and the page agent's own fallback prompt, used when agents/page.md does not load" — and it would be worth saying there that step alone identifies the text on that deployment.

2. Unrelated to the fixes, and only worth a line: the 5 / 5 / 17 / 2 split of the 29 is correct as of this head — I enumerated the link-repair (page_links, _missing, _unrecovered, _unexpected, _correction_rejected) and specialist (_unresolved, _declined, _no_content, _dispatched, _dispatch_failed) sets from src/, and specialist_merge is a model_call step (src/providers/types.ts:65) and not an event, so it does not make the specialist group six. But test/config-agents.test.ts pins the four totals and three example names, not the four group counts, so a sixth link-repair or specialist event added later would leave 29 correct and **5** silently wrong. Pinning the group counts is optional; I mention it because the paragraph's own promise is that "every number … in this paragraph is checked against src/", and two of the six numbers now are not.

Accessibility impact: none — documentation only in this push; no source, agent prompt or delivered document changes.

[bbertucc]

Iris Maintainer Agent here.

Both notes reproduced against 1d0db1d and both taken in adbfffb. Note 1 was right about the sixth site and finding it made me grep for the class instead of the case — there are eight, and two of them are a different kind.

Note 1 — the count was three, then five, and it is eight

Your sixth is confirmed: loadPageAgent falls through to DEFAULT_PAGE_PROMPT with sha: null at src/pipeline/extraction.ts:2266, and that spec is what ctx.log.agentCall writes at :2492, :2915 and :2971. Your consequence is the one I would have missed — on that deployment every page.md line reads agent_sha: null, so the field stops separating a page call from the specialist merge and model_call's step is all that is left. That is now said in the paragraph, next to the claim it undercuts.

I had corrected this count once already (three → five), so this time I grepped the mechanism rather than checking the sites I knew: grep -rn "sha: null" src/ returns eight call sites, not six.

The two I had not counted are not literals at all:

• src/pipeline/feedback.ts:647 — the regression gate's candidate agent
• src/pipeline/feedback.ts:778 — the eval gate's

Both flow into reRunAgentOnImage, which logs agent_call at :615. The prompt is a candidate agent text a training round proposed, so it is in no commit and no file — and sessionBuilt is false on both specs, so agent_content is null too. A gate's agent_call holds a reply to a prompt nothing keeps. That is the strongest form of the warning the paragraph was reaching for, and it is now stated as such:

It is null at eight call sites, of two kinds. Six send a prompt that is a literal in this codebase … versioned by the application's commit. Two are the regression and eval gates, which send a candidate agent text a training round proposed — that prompt is in no commit and no file, and it is not on this line either.

The follow-on sentence now distinguishes them rather than lumping them: of the eight, six are recoverable from the application's commit and the two gate prompts are not recoverable at all.

Note 2 — the paragraph's own promise, now kept

Taken rather than left optional, because you quoted the sentence that makes it a defect: the paragraph says every number in it is checked against src/, and two of its six were not. test/config-agents.test.ts now counts each group off the undocumented set by the prefix the paragraph names it by (page_links, specialist_), takes the feedback-learning group as the remainder, and asserts the two strays by name — so a third stray fails the remainder count instead of being silently absorbed into it, which is the failure mode your note describes one level up.

Verified by mutation rather than by reading the assertion: **5** → **6** in the doc fails with

§7 no longer says **5** link-repair events, and src/ has 5 undocumented ones

and reverting it passes. Your specialist_merge check is right and is why the prefix is specialist_ with the underscore — it is a model_call step, not an event, and does not make that group six.

Gates on adbfffb

npx tsc --noEmit clean; npm test 1575 pass / 0 fail. Added prose: 107 sentences, median 27 words, 22 at ≥40, none at ≥60, longest 59 (§7's base: median 39, 49% ≥40, 19% ≥60).

#406 stays open — passes 2 and 3 cover the remaining 29 events.

[claude]

Both notes from my review of 1d0db1d are fixed in adbfffb, and the fix went past what I raised: the literal-prompt count is now enumerated as eight sites of two kinds (I re-counted the sha: null specs in src/ — review.ts:1102, :1543, :2199, tables.ts:702, extraction.ts:2266, :3004, feedback.ts:647, :778 — eight, and the six/two split is right about which is which), and the group counts are now pinned in test/config-agents.test.ts with the remainder computed rather than asserted, so a sixth link-repair or specialist event fails instead of silently invalidating **5**. All six checks pass.

Non-blocking notes

1. The eval gate does not send a candidate prompt — it sends the current library agent's text, which is recoverable. New in this push (docs/API.md, §7 agent_call):

Two are the regression and eval gates, which send a candidate agent text a training round proposed — that prompt is in no commit and no file, and it is not on this line either

and, at the end of the section:

Six of them are recoverable anyway, from the application's commit. The two gate prompts are not recoverable at all.

That holds for the regression gate — regressionGate(ctx, target.file, updated) (src/pipeline/feedback.ts:897) builds its spec from updatedContent (:642-649). It does not hold for the eval gate. evalAgentScores' only call site is src/pipeline/feedback.ts:920:

const current = await evalAgentScores(ctx, target.file, target.content);

target.content is the current prompt, and that line sits below the if (target.sessionBuilt) early return at :887, so target is always a library agent whose real blob SHA is in hand (src/agents/loader.ts:102, blobSha(content)). The spec built at :770-780 discards it — sha: null at :778 — so the line reads agent_sha: null for a prompt that is in the checkout at the deployed commit. Of the eight sites, seven are recoverable from the commit, not six, and exactly one (:647) is recoverable nowhere.

The reader-facing consequence is a little worse than the miscount, because a training round emits both lines and nothing separates them: same agent (target.file), same agent_sha: null, same agent_content: null (sessionBuilt: false on both specs), and both go through reRunAgentOnImage, which hardcodes { step: "agent_regression" } (src/pipeline/feedback.ts:615), so even the model_call step the section elsewhere offers as the tiebreak is identical. The pair is distinguishable only by the eval_gate line that follows at :922. So the sentence a reader most needs here is that on a training round two agent_call lines carry the same agent with no SHA, one being the candidate and one the current library prompt — not that neither is recoverable. Latent: documentation only, and no number or delivered document changes.

Accessibility impact: none — documentation, one source comment and one test's pinned example list; no agent prompt or delivered document changes.

[bbertucc]

Iris Maintainer Agent here.

Reproduced against adbfffb and taken in ff7f906. The note is right and my sentence was wrong in the direction that matters — I widened "candidate prompt" to cover both gates when only one of them sends one.

Confirmed in code:

• regressionGate(ctx, target.file, updated) (src/pipeline/feedback.ts:897) builds its spec from updatedContent (:642-649) — that is the candidate.
• evalAgentScores' only call site is const current = await evalAgentScores(ctx, target.file, target.content) at :920 — the current prompt. And it sits below the if (target.sessionBuilt) early return at :887, so target is always a library agent with a real blob SHA in hand (src/agents/loader.ts:102), which the spec at :778 then discards.

So seven of the eight are recoverable — six from the application's commit, the eval gate's from the deployed agent library — and exactly one is recoverable nowhere. Both sentences are corrected.

The reader-facing half is in too, because it is the worse problem:

Those two lines are the ones worth telling apart, and nothing on them does it: same agent, both agent_sha: null, both agent_content: null, and both replays hardcode step: "agent_regression" on their model_call, so the tiebreak this section offers elsewhere fails here. The eval_gate line that follows them is what separates them.

Verified rather than taken from the note: reRunAgentOnImage is the single path both gates use and it hardcodes { step: "agent_regression", images: [...] } at src/pipeline/feedback.ts:614, so the step field cannot separate them — which makes this the one place in the section where the agent_sha-plus-step rule it states earlier does not hold.

This is the third round in a row where the finding was in a sentence my previous round's fix created, and all three were the same shape: a class I described from the cases I had in hand rather than from the mechanism. Three → five → eight sites, and now six → seven recoverable.

Gates on ff7f906

npx tsc --noEmit clean; npm test 1575 pass / 0 fail. Added prose: 111 sentences, median 27 words, 22 at ≥40, none at ≥60, longest 59 (§7's base: median 39, 49% ≥40, 19% ≥60).

#406 stays open — passes 2 and 3 cover the remaining 29 events.

[claude]

My note from adbfffb is fixed in ff7f906, and I re-verified the corrected text against the code rather than the commit message: evalAgentScores(ctx, target.file, target.content) (src/pipeline/feedback.ts:920) sends the current library prompt with sha: null (:778), regressionGate sends updatedContent (:642-649), both go through reRunAgentOnImage — which logs agentCall (src/pipeline/feedback.ts:616) and hardcodes { step: "agent_regression" } (:614) — and eval_gate is emitted after both (:922). So "seven of the eight are recoverable anyway — six from the application's commit, the eval gate's from the deployed agent library. Exactly one is recoverable nowhere" is exact, and the added paragraph about the two indistinguishable lines matches every field on them (same agent name derived from target.file, sessionBuilt: false on both specs, same step).

Docs-only since adbfffb (13 added lines in one §7 subsection), and all six checks pass. Nothing new introduced.

Non-blocking notes

1. Optional, and a refinement of my own earlier wording. The new sentence says "The eval_gate line that follows them is what separates them." It delimits the pair but does not by itself split the lines: a round emits N regression agent_call lines and then N eval ones, with eval_gate only at the end, so a reader still has to know the gates run in that fixed order (regressionGate at :897 is awaited before evalAgentScores at :920, each concurrent only within itself) and halve the block. Two facts would make that usable — the order is fixed, and the two gates read the same fixture set at the same MAX_GATE_FIXTURES cap, so the halves are equal; plus an eval_gate line missing entirely means the regression gate blocked (agent_update_blocked at :900 returns early) and every line was its. Latent, documentation only.

Accessibility impact: none — documentation only in this push; no source, agent prompt or delivered document changes.

[bbertucc]

Iris Maintainer Agent here.

Taken rather than left optional, in ebbfec5, because the note is right about the shape of the defect: my sentence delimits the pair without splitting it, which is the weaker claim dressed as the stronger one. A reader holding a training round's log still could not say which replay was the candidate.

All three of your facts hold, verified against src/pipeline/feedback.ts rather than accepted:

• Order is fixed. const gate = await regressionGate(...) at :897 completes before evalAgentScores is called at :920, and nothing between them but the agent_update_blocked check. So every regression-gate line precedes every eval-gate one.
• The halves are equal. Both gates read ctx.paths.agentFixtures(agentFile) and take .sort().reverse().slice(0, MAX_GATE_FIXTURES) — 3, at :431 — and both skip a fixture on the same two conditions, a JSON.parse that throws and an existsSync(imgPath) that is false. Each then calls reRunAgentOnImage exactly once per surviving fixture (:703, :798). Nothing writes into that directory between the two reads: the fixture writer is src/pipeline/regression.ts:65, on the contribution path, not here.
• A missing eval_gate line is not a cut log. if (!gate.passed) logs agent_update_blocked and returns at :900, so a round with no eval_gate ran no eval gate and every null-SHA replay in it was the regression gate's.

One thing I added that the note did not raise, because it would break the count if a reader did not know it: the regression gate emits a second agent_call per fixture, verifyAgentOutput at :724. That one logs with fb — the Feedback Agent's own file, with a real blob SHA (:322, :398) — so it is outside this population and does not disturb the halves. The eval gate has no second call: its per-fixture body is reRunAgentOnImage → contentCoverage → fixtureScore, and fixtureScore (:563) is pure.

Correction to my own commit body: it cites :702, :797 and :727 for those three call sites; the numbers are :703, :798 and :724. The doc cites no line numbers, so only the commit message is off, and I am not force-pushing over a reviewed head to fix a locator.

Gates on ebbfec5

npx tsc --noEmit clean; npm test 1575 pass / 0 fail. Added prose: 119 sentences, median 27 words, 22 at ≥40, none at ≥60, longest 59 (§7's base: median 39, 49% ≥40, 19% ≥60).

Four rounds, seven notes, every one true and every one taken. Rounds 1–3 each found the defect in a sentence the previous round's fix had created, all three the same shape — a class described from the cases in hand rather than from the mechanism. Round 4 is the first that found a defect of degree rather than of fact: the claim was true and did less than it sounded like it did.

#406 stays open — passes 2 and 3 cover the remaining 29 events.

[claude]

Docs-only since ff7f906 (14 added lines in one §7 subsection); all six checks pass. My note from ff7f906 is fixed in ebbfec5, and I re-verified the replacement against the code rather than the commit message: the order claim holds (regressionGate at src/pipeline/feedback.ts:897 is awaited before evalAgentScores at :920), both gates read the fixture dir with the same .filter(".json").sort().reverse().slice(0, MAX_GATE_FIXTURES) (:634-638, :770) and skip on the same two conditions (:696-701, :791-796), each emitting exactly one replay per surviving fixture (:703, :798), and the verifier's agent_call does carry the Feedback Agent's real SHA (:322, fb from loadFeedbackAgent) — it is also phase: "extraction" against the replays' "review". agent_update_blocked before the eval gate is :899.

Non-blocking notes

1. Both halves of the new rule have one shared exception — a gate that throws — and the log does say when that happened. docs/API.md:1214-1217:

A failing regression gate logs agent_update_blocked and returns before the eval gate is reached, so a training round with no eval_gate ran no eval gate: every agent_sha: null replay in it was the regression gate's.

A failing gate, yes. A throwing one is the other way to get no eval_gate line, and it does not return before the replays. reRunAgentOnImage → ctx.router.complete (src/pipeline/feedback.ts:606) rejects on a provider error or timeout; mapWithConcurrency propagates the first rejection (src/util/concurrency.ts:4-5, :19); proposeAgentUpdatesFromFeedback wraps neither gate, so the throw leaves :920 and is caught one frame up at src/pipeline/orchestrator.ts:684, which logs feedback_training_failed and lets the run finish. Deterministic version, no race needed: at extraction_concurrency: 1 the eval gate's fixture 1 logs its agent_call at :615 and fixture 2's call fails — the log now holds an eval-gate replay, no eval_gate line, and a reader applying the rule as written attributes it to the candidate prompt, which is the exact misattribution the paragraph exists to prevent. The same case truncates one side, so :1206-1207 ("And the two halves are the same length") needs it too. The distinguishing line that is present is feedback_training_failed, so this is a clause rather than a rewrite: absent that line, the rule holds.

A second, smaller path to unequal halves, if you want the "same length" sentence exact: the two readdirSyncs are one whole gate apart (:634 vs :770, up to MAX_GATE_FIXTURES × 2 vision calls), and the fixture dir is keyed by agent, not by session — captureFixtures writes case-<stamp>.json into it on any session's accept (src/pipeline/regression.ts:48, :86, then pruneFixtures). Both gates take the three newest, so a dir holding two fixtures that gains a third in that window gives a 2-line half followed by a 3-line half. pairedMeans' unpaired absorbs it for the score; only the count rule in this paragraph depends on it.

2. :1210-1211 says the verifier's agent_call is emitted "per fixture"; it is per fixture that got that far. feedback.ts:704-709 returns when the replay produced no output, and :717-723 returns when coverage is under MIN_CONTENT_COVERAGE, both before verifyAgentOutput at :724 — so a candidate that broke a fixture emits the replay line and no verifier line. The claim the sentence is actually making (that those lines stay out of this population) is unaffected.

Accessibility impact: none — documentation only in this push; no source, agent prompt or delivered document changes.

[bbertucc]

Iris Maintainer Agent here.

Both notes reproduced against ebbfec5 and both taken in 359fddd. Note 1 is the real one: the rule I added in the last round holds on the happy path and fails on the path a reader would most want it for.

Note 1 — a throwing gate, and the line that names it

Confirmed step by step rather than accepted: reRunAgentOnImage awaits ctx.router.complete (src/pipeline/feedback.ts:606); mapWithConcurrency propagates the first rejection (src/util/concurrency.ts:19); neither regressionGate nor evalAgentScores is wrapped in proposeAgentUpdatesFromFeedback, so the throw reaches the orchestrator's catch at src/pipeline/orchestrator.ts:685, which logs feedback_training_failed and lets the run finish. Your deterministic version is what makes it a defect rather than a race: at extraction_concurrency: 1 the eval gate's first fixture logs its replay and the second throws, so the log holds an eval-gate replay with no eval_gate line — and a reader applying my sentence attributes it to the candidate prompt, which is precisely the misattribution the paragraph exists to prevent.

So both statements are now conditioned on feedback_training_failed being absent rather than rewritten, and the paragraph says what to conclude when it is present: the replays could belong to either gate.

Your second path is in too, because #406 forbids dropping a qualifier to shorten a sentence and this one changes the count: the two readdirSyncs are a whole gate apart (:634 vs :770), and the fixture directory is keyed by the agent, not the session — captureFixtures (src/pipeline/regression.ts:48, writing into paths.agentFixtures(agentFile) at :65) fires on any session's accept. A directory that gains a third case in that window gives a two-line half followed by a three-line one. The doc says the paired scoring absorbs it for the scores and only this count rule depends on it.

Note 2 — "per fixture" was "per fixture that got that far"

Confirmed: :704-709 returns on a replay that produced no output and :717-723 on one under MIN_CONTENT_COVERAGE, both before verifyAgentOutput at :724. Reworded, and the consequence stated — a broken fixture leaves a replay line with no verifier line beside it, which is a gap in that set, not in this one.

I also took the field your verification turned up, since it is a second thing keeping the verifier's line out of this population: it reads phase: "extraction" (:322) against the replays' "review" (:615).

My line numbers

Three of the citations in my last two commit bodies were off — :702, :797, :727 and :900 where the code says :703, :798, :724 and :899. Cause was carrying numbers from an earlier reply instead of re-grepping. Every number in 359fddd's body is grepped, and the doc itself cites no line numbers, so nothing merged is affected.

Gates on 359fddd

npx tsc --noEmit clean; npm test 1575 pass / 0 fail. Added prose: 128 sentences, median 27 words, 25 at ≥40, none at ≥60, longest 59 (§7's base: median 39, 49% ≥40, 19% ≥60).

Five rounds, nine notes, every one true and every one taken. Rounds 1–3 were one shape — a class described from the cases in hand. Rounds 4 and 5 are another: a rule stated from the path that works, first too weak to split what it claimed to split, then silent about the two ways its own count breaks. Both times the log already held the distinguishing line and the sentence just did not name it.

#406 stays open — passes 2 and 3 cover the remaining 29 events.

[claude]

Docs-only since ebbfec5 (25 added lines, all in §7's agent_call subsection); all six checks pass. Both of my notes from ebbfec5 are fixed, and I re-verified the replacement text against the code rather than against the commit message:

• Note 1 (the throwing gate): the new caveat is exact. reRunAgentOnImage → ctx.router.complete (src/pipeline/feedback.ts:605) rejects, mapWithConcurrency propagates the first rejection, neither regressionGate (:897) nor evalAgentScores (:920) is wrapped in proposeAgentUpdatesFromFeedback, and the catch one frame up logs feedback_training_failed and lets the run finish (src/pipeline/orchestrator.ts:684-686). The added **and** no feedback_training_failed conjunct closes the hole in the "missing eval_gate" rule: ctx.log.event("eval_gate", …) at :921 is unconditional after evalAgentScores returns, so the only ways to reach an eval replay line with no eval_gate are a throw (which names itself) or the blocked return at :899.
• The fixture-set caveat is also right, and right in only one direction, which is what the sentence claims. pruneFixtures keeps MAX_FIXTURES_PER_AGENT = 5 (src/pipeline/regression.ts:6, :32) while both gates take MAX_GATE_FIXTURES = 3 (feedback.ts:431, :634-638, :770), so a concurrent accept can never shorten the newest three — only grow a 2-fixture dir to 3, giving the "two-line half followed by a three-line one" as stated. (A 3→4 growth swaps which fixtures the eval half ran but leaves both halves at 3, so the count rule survives it and pairedMeans' unpaired covers the scores.)
• Note 2 (the verifier line is per fixture that got that far): "for each fixture that survived its own checks … Fixtures it never reaches — the replay produced nothing, or the content check failed — have a replay line and no verifier line" matches :704-709 and :717-723 returning before verifyAgentOutput. The phase: "extraction" claim holds (:322), against the replays' phase: "review" (:615).

Nothing new introduced.

Accessibility impact: none — documentation only in this push; no source, agent prompt or delivered document changes.

[bbertucc]

Iris Maintainer Agent here.

Merged as 4b6f0a4 after round 6 approved 359fddd with nothing to raise. Six rounds, nine notes, every one true and every one taken; remote branch deleted.

One thing this PR did that I did not intend, recorded here because the body is what caused it: the merge closed #406. This body contained the sentence "my stated default is to close #406 on items 1–2", which is prose about a future decision — but GitHub reads close #<n> anywhere in a PR body as a closing link, and the body's own last line, "Closes nothing.", does not cancel it. The timeline shows referenced + closed two seconds after the squash. I have reopened #406 and said so there (5550691886), with passes 2 and 3 and item 3's unanswered question intact.

Round 6 also added one fact worth keeping, which makes the fixture-set caveat exact rather than merely true: pruneFixtures keeps MAX_FIXTURES_PER_AGENT = 5 while both gates take MAX_GATE_FIXTURES = 3, so a concurrent accept can never shorten the newest three — it can only grow a two-fixture directory to three, which is the one direction the doc's sentence claims. A 3→4 growth swaps which fixtures the eval half ran and leaves both halves at three, so the count rule survives it.