RT 2026-08-05 - area 1 - opus (Opus 5) - yield 8 #170
ms609-agent
announced in
01 · Fitch scoring correctness
Replies: 0 comments
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Uh oh!
There was an error while loading. Please reload this page.
opus (Opus 5) | effort: unrecorded | 2026-08-05
Migrated record. This round predates the move to Discussions; it is reproduced
verbatim below from
dev/red-team/log.md, which is now closed to new entries. It isposted here so this area has a round record whose
createdAtorders correctly againstthe other areas' — the backfill ran oldest-first for that reason. All fourteen posting
timestamps are later than the rounds they describe; the real date is in the title and
in the
date:field.area: 1 (Fitch scoring correctness) — version-bump revisit
reviewed_by: opus finder a681ba267668c44d8 + haiku verifier a269ed170472c58bf (RT1-A1-01..08 batch) + sonnet verifier a1355d889b272f745 (RT1-A1-08 re-verify, overturning a flawed haiku refutation)
date: 2026-08-05
tier: opus (Opus 5)
yield: 8 filed (#76-#83: sev:med #77 (A1-02), #83 (A1-08, upgraded from finder's
low); sev:low #76,#78,#79,#80,#81,#82) + 1 doc fix + 1 new regression test (65 assertions) applied inline; 0 refutednotes: Rotation
(13 mod 13)+1 = 1. Legend reconciled at round start: no rung has moved since 2026-07-27, so the routing was purely the standing version-bump-revisit record for this area (dry at opus-4.8, 2026-07-24; re-targeted to opus-5 with a fresh angle, NOT fable — version bump precedes rung bump). Brief explicitly chartered the ONE residual the 2026-07-24 round's relativeTS_L3B_ORACLE-style checks structurally cannot see: a bug shared by BOTH the incremental and full-recompute paths, invisible to any check that only compares the two against each other. Pasted the prior round's full derivation as claims to break, per the version-bump-revisit protocol.THE CHARTERED QUESTION WAS ANSWERED, AND THE ANSWER IS NEGATIVE — recorded as a genuine result, not an absence of one. The finder built two independent reference implementations sharing no code with
src/ts_fitch*— a from-scratch Sankoff DP for EW, and a from-scratch brute-force applicability-enumeration + region DP for BGS inapplicable scoring — and cross-checked the absolute output of the realscore_tree()against them across ~3000 datasets (multi-state, ambiguous,{-,X}/{-}/?-dense inapplicable, degenerate, deep-pectinate and shallow-balanced). 0 mismatches wherever the criterion is tie-break-free; the one-sided never-below-minimum bound held 1000/1000 including 500{-,X}-dense cases; rooting-invariance held on every edge across 150+1500+576 test configurations; block-boundary additivity held across nChar ∈ {1,5,63,64,65,70,130}; IW/profile C++-vs-R agreement held 0/900. The shared-machinery correctness question for area 1 is now closed at this tier — a future revisit needs either a genuinely new angle (the finder's suggestion: cross-checkvroot_cache/below_actives_cachecandidate-scoring values against an independent rescore of the applied move, rather than the reconstruction layer already covered here) or an escalation to fable if a new delta lands that this round's coverage doesn't reach.Incidental product of the chartered work: the BGS criterion this engine implements is now pinned down in writing for the first time in this repo.
minover applicability reconstructions that are Fitch-optimal for the binary applicability character, ofsum(within-region Fitch length) + (tip-bearing regions - 1), applicable-preferred tie-break on{-,X}tips. Two consequences that will look like bugs to a future reviewer and are not, now both encoded as assertions in the newtests/testthat/test-ts-na-oracle.R: (1) the engine can exceed the unconstrained minimum-cost reconstruction (17/500 definite-applicability cases, 30/500 with{-,X}) because Fitch minimises applicability changes, not regions — exactly what Goloboff et al. 2021 critique and whatinapplicable = "xform"exists to address; (2) two applicable tips with the same state separated by inapplicable tips score 1, not 0.Eight findings filed, none touching the shared-scoring correctness question itself — all either perf, latent-unreachable, or a documentation/invariant gap. RT-A1-02 (#77, med): the EW dirty-buffer allocation was
static thread_local-optimised, then reverted by the emutls/MinGW fix (d6fa51293) for correctness, and the cost was never recovered on the NA hot path (exact_verify_sweep, ~97.7% of native-NA wall per this project's Mission-B profiling) — same class exists in 4 sites acrossts_fitch.cpp/ts_fitch_na_dirty.h/ts_fitch_na_incr.h. RT-A1-08 (#83, upgraded to med):TreeLength()defaults to extended IW (XPIWE) butPrepareData()/EdgeListScore()— the documented defaultTreeScorerforTreeSearch()/Ratchet()/Jackknife()— always scores plain IW with no way to opt into XPIWE, so the two nominally-identical "score this tree at concavity k" paths silently disagree by default (600/600 measured mismatches, resolved by explicitextended_iw=FALSE). RT-A1-01 (#76, low): a detached-tip OOB guard present infitch_na_score(added after a real ASAN abort) is missing from both incremental NA sibling loops — latent, currently unreachable. RT-A1-03/04/05/06/07 (#78-#82, low): residual per-call allocations in two "allocation-free"-labelled functions; a negative-delta path inprecompute_profile_deltawhose bounded-scorer-bail soundness rests on an unstated, unasserted invariant; an unproven (7500+ fuzz cases, 0 counterexamples, no proof) equivalence between two differentss_appderivations for EW vs IW/profile NA scoring; a diagnostic-only export whose own internal sum doesn't reconcile because it omitsprecomputed_steps; and two harmless dead/unreachable code fragments.ONE HAIKU REFUTATION OVERTURNED ON RE-VERIFICATION — record the pattern, not just the outcome. RT-A1-08 was initially routed to the haiku batch (all 8 candidates were low/med severity, per this skill's severity-matched-verification rule — no
sev:highthis round to trigger automatic peer-tier routing). The haiku verifier refuted it by arguingextended_iwis a legitimate, documented feature — true, and not the claim. The actual claim was a default mismatch between two commonly-used paths for the same nominal computation. Per [[redteam-reverify-flawed-refutes]] and this project's standing note to route library-fact verdicts above haiku, re-verified with a sonnet pass, which traced the full default chain (R/tree_length.R→R/PrepareData.R→src/ts_rcpp.cpp'sxpiwe=falseC++ default) and confirmed REAL, overturning the refutation. Generalisable point for this severity-routing rule: "not sev:high" does not mean "safe to haiku" when the claim is a library/default-semantics fact rather than a straightforward code-pattern check — this round's median finding (allocation exists, guard missing) was well within haiku's competence, but the one finding requiring cross-file default-chain reasoning was not, independent of its filed severity.Fixed inline, verified before committing. Corrected a stale comment in
fitch_incremental_uppass(src/ts_fitch.cpp:244-251) that still described thestatic thread_localoptimisation as current; it was reverted byd6fa51293and the comment was never updated (this round's finder caught it while investigating RT-A1-02). Addedtests/testthat/test-ts-na-oracle.R(new, 65 assertions, 2.3s), the independent-reference NA oracle the chartered work produced as a byproduct — built via the tarball recipe into.agent-rt1, ran withNOT_CRAN=true, confirmed 65/65 pass before committing; build artifacts (.agent-rt1/,src/*.o,src/*.dll) cleaned up after.Seam status: shared-correctness question closed at this tier; perf/hygiene seam still yielding (8 filed, 0 refuted) → a future area-1 visit should target the candidate-scoring layer (
vroot_cache/below_actives_cachevs an independent rescore of the applied move) per the finder's own suggested next angle, not another reconstruction-layer sweep.All reactions