RT 2026-07-27 - area 8 - sonnet (Sonnet 5) - yield 2

[ms609-agent]

sonnet (Sonnet 5) | effort: unrecorded | 2026-07-27

Migrated record. This round predates the move to Discussions; it is reproduced
verbatim below from dev/red-team/log.md, which is now closed to new entries. It is
posted here so this area has a round record whose createdAt orders correctly against
the other areas' — the backfill ran oldest-first for that reason. All fourteen posting
timestamps are later than the rounds they describe; the real date is in the title and
in the date: field.

---

area: 8 (Test suite health)
reviewed_by: sonnet finder (a9d2c730) + haiku verifier (adfdc309) + orchestrator adjudication (opus, Opus 5)
date: 2026-07-27
tier: sonnet (Sonnet 5) — rotation (7 mod 13)+1 = 8; last visit (2026-06-16) yielded at sonnet ⇒ same rung, fresh agent. The rung also moved 4.6 → Sonnet 5 since that visit, which reinforces the same-rung choice rather than changing it; no fresh-angle-revisit block was used (that is for dry seams).
yield: 2 confirmed and filed (T-362 P3, T-363 P3) out of 3 candidates; 1 REFUTED. 0 inline fixes applied by design — see BRIEF DEVIATION below. Read at HEAD 5f8d0557, unchanged for the whole round (finder re-checked at exit).
notes: THE HEADLINE IS A POSITIVE RESULT, AND IT COST A REBUILD TO GET. The round's most valuable output is not a bug: the finder mutation-tested the T-330 regression file (test-ts-t330-collapse-hsj-xform.R) rather than reasoning about it — built a detached scratch worktree, disabled both ds.scoring_mode == HSJ || XFORM guards in src/ts_collapsed.cpp (:24-25, :140-144), R CMD INSTALL-ed into an isolated .mut-lib, and re-ran the file against the mutated build. 5 of 5 non-trivial assertions failed exactly as the fix's design predicts (the ts_collapse_pool clade-survival check, both HSJ Nnode/polytomy checks, the HSJ n_topologies >= 2 MPT-dedup check, the XFORM Nnode check). That file is doing real work. Main tree untouched, worktree removed, git status/rev-parse confirmed clean before and after (orchestrator re-verified: only untracked scratchpad/ remains). This is the technique area 8 should default to — the area's one proven yield class is the tautological regression test (T-322), and "does the assertion fail if the fix is reverted?" answers that directly, where reading the assertion does not.

FILED (2 — both P3, both the same defect shape: practice moved, tests/testing-strategy.md didn't). (1) T-362 — test-MaximizeParsimony-features.R is Tier 1 by the doc's own rule (non-ts- prefix, :26-28) and so runs on CRAN, but measures ≈16.6 s against the documented "Fast (< ~2 s per file)" criterion (:12-13, restated :100). The two Tier-1 files measured for scale came in at 0.41 s and 0.43 s, so the budget is achievable and this file is the outlier. The sharp part: T-351 (940e4af5, filed and fixed earlier TODAY) is the finding that this budget was breached — it moved a 20.2 s block and a 4.08 s block out to test-ts-driven.R, 40.8 s → 16.6 s, and its own closure note says "verified both files still pass". Pass/fail, not timing. So the fix verified the wrong property and the finding it was filed for survived its own closure, same day. Worst remaining block is :498-520 (two full MaximizeParsimony() runs, maxSeconds = 3 each, ≈4.0 s); maxSeconds is wall-clock, so on a slower CRAN machine the file can only get worse, and the doc itself files timing-sensitive tests under Tier 3 as "fragile on shared CI runners" (:70). Independently, the file carries a per-test skip_on_cran() at :263, which contradicts Tier 1's "no skip_on_cran()" clause outright (test-SearchControl.R:53,:261 likewise) — mixed-tier files are undocumented. (2) T-363 — test-ParsSim.R, test-MaddisonSlatkin.R, test-recode-hierarchy.R each open with the verbatim # Tier 2: skipped on CRAN boilerplate + file-level skip_on_cran(), but the doc's two file rules are jointly exhaustive and mutually exclusive by prefix (Tier 1 = all non-ts-, :26-28; Tier 2 = "All remaining test-ts-*.R", :40), and none of the three is in the Tier-2 list. The harm is not the inconsistency: Tier 1 exists to "catch breaking changes caused by edits to dependency packages", these three test exactly that R-level surface, and gated as they are a dependency break in them is invisible on CRAN — the one place Tier 1 was meant to catch it. Not a speed trade-off (test-ParsSim.R = 50/50 pass in ≈3.98 s). Likely accidental: all three guards land in the same bulk commit 78b74147, which applied the boilerplate without amending the doc.

REFUTED (1). RT8-3 — test-ts-hsj.R:617-639's comment claims "the driven pipeline and the test bridge must compute the same absent_state" while the test calls TreeLength(), not the driven search. The finder had already ruled out the tautology reading itself (both sides do their own argument assembly; TreeLength.phyDat vs .list diverge in a RenumberTips/.Recompress prelude the helper skips) and filed it only as a comment-overclaims-coverage note, on the stated condition "refute if some other test covers that call site." Verifier found test-ts-hsj.R:352 ("MaximizeParsimony runs with inapplicable='hsj'") does reach R/MaximizeParsimony.R:1217. Condition met ⇒ dropped. Residual, not filed: :352 is a smoke test, so it would catch a crash at :1217 but plausibly not a wrong absent_state that still scores. Cheap to settle next round; too thin to file.

ORCHESTRATOR DID NOT TAKE THE VERIFIER'S WORD ON THE CRUX. T-363 turns entirely on whether the doc's Tier-1 rule is a hard rule (⇒ bug) or a rule-of-thumb with tacit exceptions (⇒ doc-gap note), and that is exactly the judgement a haiku verifier is least reliable on. Read testing-strategy.md directly and re-derived it: haiku's reading is right, and the doc is in fact stricter than haiku said — :12-13 makes "no skip_on_cran()" part of the definition of Tier 1, not just the < 2 s figure, which is what promotes T-362's :263 guard from cosmetic to a second breach of the same clause. Also independently confirmed T-362's (a)-(d) sub-claims and traced the three T-363 guards to their originating commit (78b74147) to distinguish accidental from deliberate.

RESOLVED BEFORE DISPATCH, NOT HANDED TO THE FINDER (both would have burned budget): tests/testthat/_problems/ — 28 tracked files, looked like a never-reviewed live seam; added 78b74147, deleted by cf9f19b35 (ms609#251, Morphy→native kernel), absent at HEAD. And test-ts_collapse_pool.R does exist and its underscore puts it outside area 8's own test-ts-*.R glob, so no round had ever owned it — folded into the brief explicitly; finder confirms the re-added form (4b833e7f, after delete 796c01035) is the sane one, two focused tests, no churn residue.

BRIEF DEVIATION (deliberate): the finder was told NOT to apply inline fixes, against this area's normal economy ("reliably yields inline fixes"; the 2026-06-16 round applied two). Reason: tests/testthat/ is the live surface for concurrent sessions right now — three fix branches merged in the hours before this round and HEAD had already moved twice during the preceding tidy. The finder reported proposed fixes with exact text instead. Nothing needed applying in the end (both findings are maintainer-judgement calls between two legitimate remedies, not mechanical fixes), so this cost nothing this round. Revert to allowing inline fixes once tests/ is quiet.

RULED OUT WITH EVIDENCE — do not re-check. T-322's fix (test-ts-wagner.R:223-247) is confirmed non-tautological, and a suite-wide grep for the same signature (min_steps = integer(0) alongside a real concavity) found no other occurrence — that whole bug shape appears closed. test-ts-resample.R:153's result$converged || result$sa_iterations == 10L looks like a vacuous OR but is a genuine 3-way invariant: src/ts_resample.cpp:172,219 has two break paths (check_interrupt(), empty pool) hitting neither arm — same shape as the 2026-06-16 refute of its sibling at test-ts-tbr-search.R:91, so that pattern has now been refuted twice on the merits; a third round should not file it a third time. The XPIWE/NA-IW "x4 batch vs scalar" port guards (test-ts-tbr-dirty-rescore.R:123-185) compare two independently implemented paths (their comments record one catching a real nx_cs/active_mask bug) — not a same-formula tautology even though neither side is an external oracle. T-351's destination in test-ts-driven.R:355-435 is clean — each moved block redefines its own dataset/ds locally, so the move broke nothing silently. No missing-TreeSearch:::-prefix bugs and no un-seeded sample()/stochastic calls anywhere in the scoped corpus (the 2026-05-26 set.seed class has not reappeared).

FOR THE NEXT AREA-8 ROUND. (1) test-ts-simplify.R:145-163 ("IW scores are consistent across simplifiable datasets") passes a degenerate min_steps = autap_ds$weight * 0L — all zeros, not a real MinimumLength() — and asserts only is.finite/>= 0. That is T-322's shape with weak assertions on top; it was left as a note rather than filed because it is a coverage-weakness not a wrong-answer, but it is the strongest remaining lead and a mutation test would settle it. (2) Extend the mutation-testing technique to the rest of the per-finding regression corpus — test-ts-t306-accept-guard.R and test-ts-tbr-dirty-rescore.R next; budget ~1 rebuild each, scratch worktree only. (3) Settle the RT8-3 residual above. (4) Methodology gotchas that cost the finder real time — read these first: a skip_on_cran()-gated file runs zero tests unless NOT_CRAN=true, and testthat reports that as a clean pass, so always check the count of tests actually run; and dev/build-fast.R hot-swaps the DLL only — the R-level code in that library stays whatever was last R CMD INSTALL-ed, which produced 8 phantom failures (T-340/341/342 checks) from a lagging local lib, a fresh instance of [[stale-local-treesearch-lib]]. A full R CMD INSTALL --library=... cleared them.
Seam status: STILL YIELDING (2 confirmed at sonnet, and the strongest remaining lead is explicitly named above) → next area-8 visit stays sonnet, with mutation testing as the default method rather than an experiment.

---