RT 2026-08-05 - area 14 - opus (Opus 5) - yield 36 #171
ms609-agent
announced in
14 · Statistics & support metrics
Replies: 0 comments
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Uh oh!
There was an error while loading. Please reload this page.
opus (Opus 5) | effort: unrecorded | 2026-08-05
Migrated record. This round predates the move to Discussions; it is reproduced
verbatim below from
dev/red-team/log.md, which is now closed to new entries. It isposted here so this area has a round record whose
createdAtorders correctly againstthe other areas' — the backfill ran oldest-first for that reason. All fourteen posting
timestamps are later than the rounds they describe; the real date is in the title and
in the
date:field.area: 14 (Statistics & support-metrics cluster) — first-ever review, area enacted this round
reviewed_by: opus finder a01961467a9937203 + opus verifier a13226f4cad264aea (7 high-sev/memory-safety candidates) + haiku verifier a02207b4e4057b65e (30 low/med batch) + orchestrator direct code-read (A14-11, A14-12 — omitted by the haiku batch's return)
date: 2026-08-05
tier: opus (Opus 5)
yield: 36 filed (#85-#120): sev:high 4 (A14-01,13,19,20), sev:med 12 (A14-02,03,07,14,15,22,24,25,26,28,33,35), sev:low 20 (A14-04,05,06,08,09,10,11,12,16,17,18,23,27,29,30,31,32,34,36,37) + 4 doc/dead-code fixes applied inline (1 typo, 3 unused-variable removals); 1 refuted (A14-21) from 37 candidates
notes: Dispatched directly at the user's explicit request (
/red-team area 14), which named an area that did not yet exist as a scope row —area:14/area:15labels had been pre-created by the 2026-08-04 area-12 round's restructuring proposal, but the row itself was left unenacted. Enacted this round, before dispatching: added area 14 (statistics & support-metrics cluster,src/MaddisonSlatkin.cpp+expected_mi.cpp+quartet_concordance.cpp,R/Concordance.R+Consistency.R+PresentContra.R+TaxonInfluence.R+ScoreSpectrum.R+QuartetResolution.R+WhenFirstHit.R+RandomTreeScore.R+ClusterStrings.R+WideSample.R+ParsSim.R+Bootstrap.R) tofocus-areas.mdatstart_tier: opusper the area-12 proposal's rationale (the one file anyone had looked at,MaddisonSlatkin.cpp, already carried a real platform-specific crash — PR ms609#272 — while unowned).Nin the rotation-formula note updated 13→14. Area 15 (legacy pure-R search API) deliberately left un-enacted — out of scope for this explicit area-14 request. [Superseded at merge: PR #57 landed first and enacted BOTH rows independently, so the scope row,start_tier: sonnet(a deliberate maintainer decision) andN = 15are #57's, not this round's.R/PresentContra.R/R/ClusterStrings.Rwere merged into #57's row;R/Bootstrap.Rsits in area 15, so A14-36 was filed across the boundary. The round itself ran atopusand the yield stands.]HIGHEST YIELD ON RECORD FOR THIS ROTATION — 36 confirmed findings from a single first-ever pass, more than double the previous record (area 10's 8-in-a-round). The area-12 proposal's own justification (an "unowned but already bitten" cluster) proved conservative, not alarmist. Four sev:high findings, one of which (A14-01:
MaddisonSlatkin(steps < 0)writes through a nulldata()pointer) reproduces a segfault with a one-line user-facing repro and zero input validation on the exported entry point. The other three high-severity findings are all silent-wrong-number bugs in commonly-reachable R-layer code: A14-13 (ClusteringConcordance()recycles a shorter split-index vector against a longer character-index vector whenever the tree carries tips absent from the dataset — silently wrong, not just wrong-with-a-warning, wheneverlength(keep)dividesNTip(tree)), and a matched pair inConsistency.R's support-statistic machinery — A14-19 (.CharLengthCache's key omits the tree, so scoring two different trees against the same dataset in one session silently returns the wrong tree's cached median) and A14-20 (.SortTokens()rewrites a partial-ambiguity character as full ambiguity whenever the dataset's contrast has more than one ambiguous level) — both of which corruptrhi, a headline published statistic (Steell 2025), with a plausible-looking wrong number rather than an error.One high-severity candidate REFUTED, and the refutation is itself informative: the finder's library was stale. A14-21 claimed
CharacterLength()'s non-binary-tree guard (.CheckTreeCharLen()) was missing, unlikeTreeLength()'s, and that the resulting memory-unsound C++ read produced non-deterministic output (0,0,0 in 1 of 6 fresh-process runs, 1,1,1 in the other 5) on aCollapseNode()-polytomy input — the same class as issue #16/T-400. The opus verifier traced the guard toR/tree_length.R:454-457and found it already present, added in7c8c3ab06("fix: reject non-binary trees at the TreeState boundary") — an ancestor of current HEAD — and reproduced the claimed repro erroring cleanly in 6/6 fresh processes via all three ofCharacterLength(),Consistency()andTreeLength(). The finder almost certainly built against a pre-7c8c3ab06install. Do not re-file this claim without first confirming which commit introduced the guard predates the build under test — this is exactly the failure mode [[redteam-verify-against-current-tip]] exists to catch, now observed inside a single round rather than across rounds.Verification gap, self-corrected: the haiku batch silently dropped 2 of 30 candidates. The haiku verifier's returned table had only 28 rows against 30 candidates sent (missing A14-11, A14-12) with no note explaining the omission. The orchestrator read both cited code sites directly rather than re-dispatching a third verifier pass: A14-11 (
quartet_concordance.cpp'sn0[state]/n1[state]indexing has no lower-bound guard against a negativestate— confirmed unguarded, safe today only by the R caller's contract) and A14-12 (a{0,-}-style contrast level is misclassifiedisGroupingbyConcordance.R's ambiguity check because the check'srowSumsexcludes the\"-\"column — confirmed by hand-tracing a constructed{0,-}contrast row throughisAmbig/groupingCols) — both REAL. Process note for future rounds: count the verifier's returned rows against the candidate count sent; a silent drop is not the same as a REFUTED verdict and needs the same anti-duplication care as any other gap.Structural pattern across the round, useful for scoping the next visit: the cluster is systematically weak on R's array-dimension-drop behaviour.
ConcordanceTable()(A14-14),ClusteringConcordance(return=)(A14-15) andConsistency()(A14-22) all error identically whenever an array axis (splits, patterns, or characters) has length exactly 1 and a subsetting operation is missingdrop = FALSE.ClusterStrings()(A14-25) has the same shape on a singleton cluster. Four independent instances of one missing idiom — a future pass targeting exactly this pattern across the rest of the R codebase (not just this area) would likely be efficient.High-severity signal for escalation, NOT resolved by A14-21's refutation. The finder's underlying suspicion — that the C++ char-steps kernel (
ts_char_steps, area unclear — outside every current scope row's explicit file list, adjacent to area 1'sts_fitch*and area 5'sR/tree_length.R) may be memory-unsound on multifurcating trees — is now moot for the specific reachability path claimed (the R-layer guard blocks it), but the finder's claim of process-to-process non-determinism on a guard-bypassing direct C++ call was never independently tested by either verifier, since the guard made the R-level repro impossible to run in the first place. Not filed, not ruled out — a future round with an actual C++-level (not R-level) test harness could still check whetherts_char_stepsitself is sound when fed a raw non-binary edge matrix, independent of whether any R entry point can currently reach it. Low priority: the R-layer guard covers every known call site.Fixed inline, verified before committing:
src/MaddisonSlatkin.cpp:1587typo in a dead-code boolean expression (!(p > NEG_INF) || !(p > NEG_INF)→!(b > NEG_INF) || ..., inside the unreachableSolverclass — see A14-04);R/Concordance.Rdeadp <- num / denand deaddiscarded = 0removed;R/Consistency.RunusedtipLabelremoved;R/ClusterStrings.RunusednMethodsChecked/methInc/nK/kIncremoved. All four are inside dead/unreachable code paths or clearly-inert local bindings — no behaviour change, no test run required.Ruled out this round, do not re-hunt: the
MaddisonSlatkinDP recursion itself (brute-forced against exhaustive tree enumeration across 4 ambiguous-token configurations, exact match to the last tree — the bugs are all in the plumbing around it, never the DP);ExpectedLength's de-duplication of random relabellings (looks like a bias, isn't — uniform sampling over a distinct set is still unbiased for the population median);weighted.mean()'s zero-weight guard inQuartetConcordance;TreeDist::ClusteringInfoDistance's tip-pruning behaviour (confirmsTaxonInfluence()'s reference-vs-reduced comparison is methodologically sound, downgrading the finder's initial A14-32 suspicion to a narrower latent-fragility finding);quartet_concordance.cpp's concordant/decisive combinatorics (algebraically correct);.pars_sim_init_char's claimed achievable score bound (genuinely achieved).Not yet examined, flagged as the obvious next seam:
R/PresentContra.Rwas read but not exercised against a forest whose trees have tips absent from the reference (callsKeepTipfirst, should be safe, not proven). Area 15 (legacy pure-R search API,R/CustomSearch.Retc.) remains proposed-not-enacted — itsarea:15label exists but no scope row; still urgent per the area-12 round's original note (#16/T-400'sEdgeListScore()exposure).Seam status: still yielding, heavily (36 filed from 37 candidates, only 1 refuted and that refutation was a stale-library artifact, not a genuine clean result) → next area-14 visit should stay opus with a fresh agent, starting from the array-dimension-drop pattern and the not-yet-examined
PresentContra.Rforest-mismatch angle.All reactions