test-loop cost: five rows, from the profile measurement to 144 test targets becoming two - #805
Conversation
CLOUD-1289 `split-debuginfo` is CLOUD-1211's one unmeasured arm, and it closed Done with that clause reading "Outstanding"
Why CLOUD-1211 measured three debuginfo arms, adopted
That clause is its §3's second arm and it was never run. A successor rather than a reopen: CLOUD-1211's adopted arm is released and its reasoning should stay closed — what is unfinished is one measurement it explicitly deferred. Why it still matters, measured 2026-09-01 (this container, 4 cores):
A single test target rebuilds and links in ~1.0s, so the 48s a developer pays per edit is roughly ~10s of lib compile plus 144 targets × ~1.0s at 4-wide ≈ 36s of linking. Linking is where a rebuild goes, and
Refinement — Ready
Test obligation
AND IT WILL NOT FAIL ON ITS OWN — checked 2026-09-01, and the correction matters. An earlier revision of this clause implied adding a profile key would red that file. It will not: Commit / bump (§6): Blockers (§8): none. CLOUD-1208, which CLOUD-1211 was blocked by for its null, is Done. Acceptance
CLOUD-1291 The test harness re-reads and re-parses the committed 354 KB `batten.toml` on every spawn, and nothing has priced it
Why
Static call sites across There is no memoization anywhere in the test tree: zero uses of THE SAVING IS UNMEASURED, AND THE FIRST ATTEMPT TO MEASURE IT WAS WRONGThis row's own headline number was refuted, and the honest state is "nobody knows" — which is why §1 is a measurement rather than a build, and why closing it as not-worth-doing is a sanctioned outcome. The first estimate was 22.5 ms per call, from
Identical within noise. The 22.5 ms is verb startup — clap tree construction, defaults, summary rendering — and is paid whether or not a config file exists. Whatever the 354 KB parse costs, it is under the noise floor of a 29 ms process. A plausible outcome is that this row should not be built. What is NOT in doubt is the structure: three unmemoized re-reads of one immutable committed file, on the hottest path in the harness. Refinement — needs a number before it is Ready
Test obligation None new if the mechanism lands — the memoized values are the same values, and the existing suite covers them. What the row owes instead is the number, recorded here whichever way it comes out, so the next reader does not re-derive it. Commit / bump (§6): Blockers (§8): none. Acceptance
CLOUD-1290 `Fixture::git()` spends two `git` processes where `init -q` alone is one, and `base_commit()` then spends a third on a rename that already held
Why
pub(crate) fn git(self) -> Self {
git_in(&self.dir, &["init", "-q"]);
git_in(&self.dir, &["branch", "-M", "main"]);
self
}
The rename is already redundant today. Counted across Folding the two into a single This is small and is filed as such. The warm run phase is measured at 4.01x parallel efficiency on 4 cores — serial sum 370.8s against a 92.6s wall — so it is CPU-saturated and the only way to move it is to remove serial work. ~250 process spawns is real serial work, but at a few milliseconds each it is single-digit seconds of serial time and low single-digit percent of the wall. Filed at Low, and it should not be sold as a performance fix: the better argument is that a fixture builder doing in three processes what one does is a harness that teaches the wrong thing 150 times. Refinement — Ready
Test obligation The existing suite is the test: ~150 fixtures build a repository through this path and assert against it, so a Commit / bump (§6): Blockers (§8): none. Acceptance
CLOUD-1288 `exec` spends PIPE_DRAIN_TIMEOUT twice, serially, so the shipped worst case is 20s where the constant says 10s
Why
let (out_bytes, out_spool) = out_drain.collect(Stream::Stdout, &mut notices)?;
let (err_bytes, err_spool) = err_drain.collect(Stream::Stderr, &mut notices)?;
THE TEE THREADS ARE ALREADY CONCURRENT, AND SAYING SO IS THE POINT OF THIS ROW. This was first written up as "drain them concurrently", which would have changed code that is already correct and left the 20s in place. Recorded so the wrong remedy is not re-derived. Measured, 2026-09-01, this container, That test is not the defect, it is the sensor: it asserts Refinement — Ready
Test obligation Two cases, and the second is the one an implementer will skip.
Commit / bump (§6): Blockers (§8): none. Independent of the test-target and profile work — this is engine behaviour, and the suite-time saving is a consequence rather than the reason. Acceptance
CLOUD-1210 118 integration test targets link 14.1 GB of near-identical binaries, and the retirement campaign adds one per retired gate
Why
Measured, this container, 2026-08-30:
An earlier revision of this row quoted a 1376s cold arm and "~90% of it compile and link". Both were derived by guessing when the run started and ended rather than reading it, and both are withdrawn — which is CLOUD-1208's point arriving on its own sibling.
Prior art, and the mechanism here is adopted rather than invented. matklad, Delete Cargo Integration Tests, is the canonical statement of this defect: "rustc needs to repeatedly re-link the library crate with each of the integration tests", producing "a significant compilation time blow up for tests". Its recommended layout for a large codebase is exactly A SECOND SAVING THIS ROW CLAIMED AND HAS NOW WITHDRAWN. An earlier revision said the 97.7s warm residue was nextest's per-binary list phase and therefore fell with the target count — making it this row's second measured win. Falsified 2026-08-30. A zero-match filter run ( What survives is the prior art, and it is enough on its own: 3x off test compile time and 5x off on-disk artifacts, measured by Cargo's own repository. This row stands on that and on the 14.1 GB census, not on a residue nobody can attribute. A third saving the prior art names, stated as a possibility rather than a claim. Splitting tests across binaries means a shared expensive fixture is built once per binary rather than once; consolidated, a Root cause, and why it gets worse on its own. The convention is one integration target per contract surface, which is locally correct and has no stopping condition — the same shape CLOUD-365 records for case counts. CLOUD-843's retirement campaign adds a target per retired gate, with ~147 shell suites still standing, so the count is monotone in the campaign's own progress. TWO RETIREMENTS ARE IN FLIGHT AGAINST THIS FILE DOMAIN, so the interaction is live rather than anticipated: CLOUD-1145 ( A THIRD READING, measured 2026-08-31: on this container the loop does not closeThe existing readings price this as latency (compile time) and as bytes (
The consequence is not slowness, it is that
This is the same 14.1 GB artifact set the row already counts; what is new is that REOPENED 2026-09-01 — this row reached Done without being builtIt was closed on 2026-08-31 and none of §3 exists. The evidence is in the row's own metadata and in the tree:
It was a sweep, not a decision. Six rows carry a The count grew while this was being re-groomedIn the 8 commits between one container's clone ( §8 is discharged, and the measurement it was blocked for now existsCLOUD-1208 is Done, so the blocker is cleared. Measured 2026-09-01, this container, 4 cores,
One test target rebuilds and links in ~1.0s (touched Read the 75% as an upper bound, not a point estimate. It was measured with What this measurement does NOT support, stated because the row has withdrawn two over-claims already: it says nothing about the 92.6s run phase, which is separately measured at 4.01x parallel efficiency on 4 cores and has no scheduling headroom left. Consolidation is a build-time win only, exactly as §2 already says. Refinement — Ready
Test obligation
Commit / bump (§6): Blockers (§8): blocked by CLOUD-1208, because the saving must be measured before and after rather than claimed — the prior art's 3x/5x is a measurement of Cargo's tree and not of this one, and the residue this row once claimed has been falsified. The mechanism, RESOLVED (2026-08-30) — this row's last unrefined clause. What was undecided: §3 names The answer is a
That is the non-increasing target count stated structurally, and it resolves each problem the open question named. No token count — it asks about paths, which is what a target count is a property of. No base-rev position to take — the base delta is the comparison, so the hard half of the ratchet spelling disappears rather than being solved. No spawn — a Why the file spelling is sound here when a One trap for whoever writes it. Acceptance
|
|
Important Review skippedToo many files! This PR contains 321 files, which is 171 over the limit of 150. To get a review, reduce the PR to 150 files or fewer by splitting it into smaller PRs or changing its base branch. Upgrade to a paid plan to raise the limit. Usage-priced reviews support at most 300 files. ⚙️ Run configurationConfiguration used: Path: .coderabbit.yaml Review profile: CHILL Plan: Free Run ID: ⛔ Files ignored due to path filters (9)
📒 Files selected for processing (321)
You can disable this status message by setting the Note 🎁 Summarized by CodeRabbit FreeYour organization is on the Free plan. CodeRabbit will generate a high-level summary and a walkthrough for each pull request. For a comprehensive line-by-line review, please upgrade your subscription to CodeRabbit Essentials by visiting https://app.coderabbit.ai/settings/billing. Comment |
5401b6d to
94571b1
Compare
… was unpinned CLOUD-1211 reached Done with its acceptance clause reading "split-debuginfo = unpacked is still unmeasured. Outstanding." This measures it, and finds a second, independent hole in the same file. MEASURED as a paired A/B: three cold builds back to back in one process so machine noise is common-mode, two identical baselines bracketing the candidate. arm cold wall artifacts linked bytes mean base-1 223.4s 147 10.19 GB 69.3 MB unpacked 208.9s 147 4.99 GB 34.0 MB base-2 213.7s 147 10.19 GB 69.3 MB TIME IS INSIDE THE NULL and is not a finding. The two identical baselines differ by 9.7s on their own (spread 0.957) and the candidate sits at 0.956 against their mean. That is the expected result: the prior art is macOS dsymutil, and unpacked is already cargo's default there and not on Linux. BYTES ARE THE FINDING, on a null of zero width — the two baselines are byte-identical at 10193048624. 2.04x off the linked binaries, adopted on that column alone, which is what the row asks for. AND THEY LEAVE THE DISK rather than moving to a sibling, which the linked census alone cannot answer: deps 10.97 GB -> 5.87 GB, target/debug 13.03 GB -> 7.76 GB, the 5.20 GB replaced by 111.8 MB of .dwo. Those .dwo files are a class prune.rs's closed RECLAIMED_KINDS cannot see, so they accumulate per build hash; filed as CLOUD-1293 rather than fixed here, since the fix is in prune.rs and out of this row's scope. debug = 1 is untouched, so this changes where debuginfo is stored and never whether it exists. SEPARATELY, opt-level = 2 was unpinned. dev_profile.rs carried four cases and zero occurrences of the key, yet it is CLOUD-1211's biggest single win — test:cargo warm 100.189s to 48.581s, 2.06x. Dropping it reds nothing and doubles every suite run. Asserted now, on a shared dependency_override() lookup, with the anti-vacuity case that pins the panicking read: the dev profile's opt-level default is 0 rather than an absent key, so a defaulting lookup would report the unoptimised build as satisfied. Both new assertions shown able to fail — removing each key reds the suite (exit 100, 101). Refs: CLOUD-1289, CLOUD-1211, CLOUD-766, CLOUD-1293
The harness re-read and re-parsed this repository's own 356 KB batten.toml on every fixture command it built — bypass_env_vars() from batten(), plus declared_patterns() and committed_patterns() — across 761 static call sites with no memoization anywhere in the test tree. MEASURED FIRST, because this row's own estimate was refuted before it was written: taking the cost through `batten config show` measures the same to within noise from a directory holding no config at all, so a 29 ms verb cannot resolve it. The new `mise run config-load-bench` times the function instead. arm=load p50=10.48 p95=11.67 mean=10.64 runs=200 arm=parse p50=10.35 p95=12.11 mean=10.63 runs=200 arm=null p50=10.49 p95=11.85 mean=10.67 runs=200 ratio=parse/load value=0.988 10.48 ms per call, ten times the row's ~1 ms guess. parse/load at 0.988 says the READ is 1.2% of it; the cost is the parse and its validate passes. SUITE DELTA, paired on one machine, two timed runs per arm after a discarded warmup, on nextest's own reported duration so the forced rebuild is excluded: base 60.159s 60.367s mean 60.263s 3512 tests memoized 53.536s 53.746s mean 53.641s 3512 tests Ratio 0.890 — 11.0%, 6.62s — against a within-arm null of 1.0035 and 1.0039, so ~28x the noise. Identical test count, both arms fully green. A first attempt at that half was unreadable and the correction is the interesting part: two IDENTICAL base runs measured 77.3s and 60.9s, a null spread of 0.788, because run one pays the page cache for 147 freshly linked binaries. What is memoized is the RESULT of reading the committed config, never a hand-written list of hatch names — CLOUD-1227's derivation is why that distinction matters, and it is untouched. Every signature is unchanged, so no caller had to know. The bench lives in perf.rs for the reason acquisition-bench does: Record is a contract perf-compare parses and perf-gate greps, and the percentile convention behind p50 has to have one author. summarise() is extracted from record() rather than duplicated beside it. The arm takes its own BENCH_METRIC stamp, and acquisition_metric.rs is generalised to a table asserting every bench task carries one and that no two collide — a property one task alone could not have. Refs: CLOUD-1291, CLOUD-1227, CLOUD-1211, CLOUD-1210
…t already held git() spawned two git processes before a fixture repository held anything — init -q, then branch -M main — and base_commit() spent a third on the same rename. Across 150 .git() and 100 .base_commit() call sites that is ~250 process spawns for a repository state that was already correct without them. The rename was redundant, not merely cheap: git_command pins -c init.defaultBranch=main on every invocation, so the default already IS main. Verified through those same pinned flags on git 2.43.0 — init -q alone leaves main, and it is still main after the first commit. DELETED RATHER THAN REPLACED BY -b main, which is what keeps this free of a version floor. -b arrived in git 2.28 and nothing in [tools] pins git, so the flag would put a requirement on the developer's machine that mise.lock cannot hold, to restate a default this harness already controls. Dropping base_commit()'s rename was checked rather than assumed: all 100 call sites are preceded by .git() in the same statement chain, zero counterexamples, so no fixture reaches it through an initialisation whose branch the rename was normalising. The existing suite is the test — ~150 fixtures build a repository through this path — and it is green at 3512 tests, the same count as before the change rather than merely passing. Refs: CLOUD-1290, CLOUD-63
…ne each PIPE_DRAIN_TIMEOUT is 10s and the_drain_deadline_is_long_enough_to_be_about_a_leak asserts that floor. Both read as "a leaked grandchild costs at most ten seconds". It cost twenty: the two drains were collected one after the other, each passed the full constant, so a grandchild holding both pipes open spent it twice. The tee threads were never the problem and are untouched — one per pipe, started at spawn time, already concurrent, and rust.md's concurrency table records that row as staying. What was serial is the DEADLINE ACCOUNTING. Instant::now() is taken once before the first stream and the second gets PIPE_DRAIN_TIMEOUT minus what the first spent. Measured over the compiled binary: a_surviving_grandchild_cannot_hang_exec 20.07s -> 10.044s. That case leaks BOTH pipes already, so it is the both-leaked scenario the fix is about; its one-minute bound simply could not tell ten seconds from twenty. Tightened to 15s rather than adding a second test, which would have to leak both pipes to mean anything and would cost another ten seconds of wall clock for coverage this fixture already has. Shown able to fail: reverting only the two call sites reds it at 20.148s. The zero-remaining half is handled and the row's premise for it is CORRECTED rather than quietly adopted. CLOUD-1288 predicted a false "did not reach EOF" notice on a cleanly-finished second stream, on the premise that recv_timeout(Duration::ZERO) can report Timeout without examining a value already in the channel. Measured on this toolchain, that does not reproduce: 10,000 trials over a ready channel returned the value 10,000 times and timed out zero. collect_within asks the two questions separately anyway — try_recv for "has this stream already finished", and only then the remaining budget for "how long may I wait" — because answering the first through the second leaves a byte-stable channel depending on an mpsc detail std does not promise. a_finished_stream_emits_no_notice_when_the_budget_is_gone pins the property, and its doc says plainly that it does not discriminate the two spellings here rather than claiming a failure it was not shown. PIPE_DRAIN_TIMEOUT and its floor assertion are untouched. Drain::collect is removed rather than annotated: with the budget shared it had no production caller left. Refs: CLOUD-1288
94571b1 to
92ba3d6
Compare
…et the count
Cargo autodiscovered one test target per top-level crates/batten/tests/*.rs and
rustc relinked the whole closure — gix, regorus, syn, clap, jsonschema,
hyper/rustls — into each of 144 binaries. matklad's Delete Cargo Integration
Tests states the defect and the layout; Cargo's own repository measured 3x off
test compile time and 5x off artifacts making the same move.
MEASURED HERE, paired on one machine, with a REAL edit rather than a touch — the
row is explicit that touch changes mtime with identical content, so every target
relinks but incremental codegen has nothing new to do, which understates the
lib-compile half:
rebuild after editing one src/*.rs, 144 targets: 48.0s
rebuild after editing one src/*.rs, 2 targets: 7.4s / 7.3s
6.5x, against a within-arm null of 0.986. Bytes, cold and clean, against the same
container's 144-target census taken under the same adopted split-debuginfo
profile:
144 targets 2 targets
linked artifacts 147 4
linked bytes 4.99 GB 234 MB
deps total 5.87 GB 1.09 GB
target/debug 7.76 GB 2.05 GB
21x off the linked binaries and 3.8x off target/debug — the reading CLOUD-1210's
third section needed: a verify lap that consumed more than the container could
hold now fits several times over.
The run phase is untouched at 3519 tests, exactly as the row's §2 says: this is a
build-time and bytes change and nothing else.
THE RATCHET IS THE POINT, not the one-off saving. policy/test-targets.rego
refuses an ADDED path that is a top-level crates/batten/tests/*.rs — four path
segments — because cargo autodiscovery makes that 1:1 with a target. A file one
segment deeper is a module and costs nothing, which is what keeps CLOUD-843's
campaign able to land its mandated tier without the gate having to be switched
off.
ITS DEPTH TEST SHIPPED WRONG FIRST AND THE SECOND TIER CAUGHT IT. The count was
written as 5 where the path splits to 4, exactly inverted: it refused the grouped
module and allowed the new target. The module's own test_ rules agreed with the
mistake, because a with-input case is only as right as its author.
crates/batten/tests/it/test_targets.rs over the compiled engine is what failed —
the class .claude/rules/policy-modules.md calls the second tier not optional for,
and it is now the first #MUTANT row.
ONE FILE STAYS A TARGET, AND THE REASON IS A GATE RATHER THAN A COMPROMISE.
policy_modules.rs keeps its own target because `evaluator-io-check` probes it
with `cargo test --test policy_modules`, and that task is a governed
mise-tasks/*.sh: shell-retirement gives it exactly two shapes — retire it
whole, or leave it alone — so repointing the probe is not an edit this change
may make. Measured, not assumed: the edit was attempted and batten-check
refused it as shell-rule-retired. One extra link is the cheaper side of that
trade against a gate that stays live, and the ratchet is unaffected — the file
exists at base, so it is never in base-delta.added.
THE MOVE BREAKS EVERY GLOB THAT SPELLED tests/*.rs, and that is the silent half.
batten's globs compile with literal_separator(true), so * stops at a /: five
committed rows plus one hardcoded in rules.rs would have selected NOTHING and
reported clean over an empty set. Repointed to **/*.rs. Four nextest filtersets
named binaries that are now modules; --no-tests=fail is what made those loud.
mise-tasks/replay.sh needs no edit — its declared_in feeds git grep as a
PATHSPEC, where * does cross a /, so the governed file is untouched.
AND IT MOVES [prune]'S BASIS, which CLOUD-1210's §2 predicted and scoped out.
The floors were measured against a tree where a tracked test FILE was a proxy for
a linked STEM — exact while cargo autodiscovered one target per file, and no
longer a series at all once 144 became 2. target-prune refused on it (declared
140, live 152, tolerance 10), so count and measured move together as that block
instructs. The FLOORS deliberately do not move: both are now far above what the
tree needs, which is the safe direction on the block's own terms, and re-deriving
them downward needs the independent measurement it names — CLOUD-1158's.
target_consolidation.rs asserts what makes this safe rather than citing it:
nextest runs each test in its own process, so isolation survives the target
boundary going away.
Also repointed: insta's four snapshots (payloads byte-identical, only the source
header moved), relative include! paths, the retirement ledger's carried arms, and
the doc citations README and .claude/rules/* make.
Refs: CLOUD-1210, CLOUD-766, CLOUD-1158, CLOUD-843, CLOUD-55
Admits: ffdf8396b47739f87822037710ac7635da092135ea0d60a06a35be196015e8a0
Admits-rule: protected-mutation
Admits-verdict: V-PROTECTED-MUTATION
Admits-subject: batten.toml
Admits-head: 04389bfcbc45e8c84742656a351835cfd5cf25cd
Admits-epoch: 6653a7e618ce30bcce12361e2d99a731c68d00143157f0e3af4be10ca574ba0b
Admits-author: alec@wenzowski.com
Admits-prev: -
Admits-answer-lost: CLOUD-1210's ratchet cannot be registered at all. The grouping would buy a one-off saving instead of a property: cargo autodiscovery mints a target per top-level crates/batten/tests/*.rs, CLOUD-843's retirement campaign adds one per retired gate by mandate, and the count was measured climbing 142 to 144 in eight commits. Without the row the 144-to-1 consolidation regrows one retirement at a time and nothing reds.
Admits-answer-precondition: batten.toml is the one committed authority for [[rule]] and [[verdict]] rows (house-style §8), so a new policy rule and the verdict token it raises have no other surface to be expressed on: a module raising a token no row declares fails to LOAD, and a row nothing raises fails the load too, so both halves must land in this file or the change cannot exist. The write is CLOUD-1210's registration of policy/test-targets.rego, and it lands in the diff of a pull request a reviewer reads.
Admits-answer-rejected-route: R-USE-THE-OWNING-SURFACE does not apply because this file IS the owning surface for rule and verdict registration; there is no other one to route to. R-RESTORE-IT does not apply because nothing was damaged to restore — this is an addition of two rows, not a repair of an edit that should not have happened.
Admits: 8e8a4fd331608067ab55d5c6d020737b55aa7066d575634f9ac6cb95c653ae99
Admits-rule: protected-mutation
Admits-verdict: V-PROTECTED-MUTATION
Admits-subject: batten.toml
Admits-head: 04389bfcbc45e8c84742656a351835cfd5cf25cd
Admits-epoch: 08c3d9251f7d28897e729d704e02166eeda5345cbe38ff8dc7205bcebc200360
Admits-author: alec@wenzowski.com
Admits-prev: ffdf8396b47739f87822037710ac7635da092135ea0d60a06a35be196015e8a0
Admits-answer-lost: policy/test-targets.rego would sit in the tree registered by nothing and decide nothing, while V-TEST-TARGET-ADDED would be a declared verdict no rule raises — which fails the load in the other direction. CLOUD-1210's grouping would then hold no property: the 144-to-1 consolidation regrows one retirement at a time, unrefused.
Admits-answer-precondition: The second half of the same registration: policy/test-targets.rego needs a [[rule]] row to be loaded at all, and batten.toml is the one committed authority for rules (house-style §8). A module no row registers is never evaluated — a dead gate that loads clean, which is the exact failure class .claude/rules/policy-modules.md exists to warn about. The verdict half landed under the previous admission; this is the row that makes it reachable, in the same pull request diff.
Admits-answer-rejected-route: R-USE-THE-OWNING-SURFACE does not apply because batten.toml IS the owning surface for [[rule]] registration; there is nowhere else to route a rule row. R-RESTORE-IT does not apply because nothing was damaged — this adds a row rather than repairing an edit that should not have happened.
Admits: 49aafbb7b01db2868770e47436354525049360ca541e4fe11a0624549039cdac
Admits-rule: protected-mutation
Admits-verdict: V-PROTECTED-MUTATION
Admits-subject: batten.toml
Admits-head: 04389bfcbc45e8c84742656a351835cfd5cf25cd
Admits-epoch: 29cd0ab87cc14f65ac2eb49fc3fafc5aaefbaa196d5e3e29903e0779ff844021
Admits-author: alec@wenzowski.com
Admits-prev: 8e8a4fd331608067ab55d5c6d020737b55aa7066d575634f9ac6cb95c653ae99
Admits-answer-lost: Five gates go silently dead: the shell-retirement ledger's declared_in, the line_sources for its arms, an exact-path row, and two rows globbing the test tier. Each would then report clean over an empty file set, which is worse than a wrong answer because a gate that found nothing looks exactly like a gate that passed.
Admits-answer-precondition: CLOUD-1210 moves every top-level test file into the tests/it/ group, and batten's globs compile with globset literal_separator(true) — so an asterisk stops at a slash and five committed rows would match NOTHING afterwards. A glob that selects nothing is a gate passing on emptiness, the silent-dead-gate class this repository is most exposed to. Those rows live in this file and nowhere else, so repointing them is only expressible here, in the same pull request diff as the move a reviewer reads.
Admits-answer-rejected-route: R-USE-THE-OWNING-SURFACE does not apply because this file IS the owning surface for a rule's globs; there is nowhere else to route them. R-RESTORE-IT does not apply because nothing is being restored — the paths these rows name are moving, and the rows must follow or they select nothing.
Admits: 2899d8657165ec05ec52be34efeefcdcd23242ddf7c5cb1bc4052a500c2f06a2
Admits-rule: protected-mutation
Admits-verdict: V-PROTECTED-MUTATION
Admits-subject: policy/test-targets.rego
Admits-head: 04389bfcbc45e8c84742656a351835cfd5cf25cd
Admits-epoch: 7f31bd1e3f27a45f279c38462b59167c4592ef855b219bf5904a60aaf6a5bdba
Admits-author: alec@wenzowski.com
Admits-prev: -
Admits-answer-lost: The ratchet ships backwards. It would refuse every retirement's tier landing correctly inside the group, and allow the new top-level file that mints a second cargo test target, so CLOUD-1210's 144-to-1 consolidation would regrow while the gate reported clean and blocked the campaign it was designed to survive.
Admits-answer-precondition: Fixing a defect in the module I am landing under CLOUD-1210, found by its own compiled-binary tier. The depth test was written as count(segments)==5, which is exactly inverted: crates/batten/tests/x.rs splits to FOUR segments, so the rule refused the grouped module and allowed the new target — the one direction that fails silently. A module's predicate is only expressible in the module, so this write is the only route, and it lands in the pull request diff a reviewer reads.
Admits-answer-rejected-route: R-USE-THE-OWNING-SURFACE does not apply because this file IS the owning surface for its own predicate. R-RESTORE-IT does not apply because there is nothing to restore — the file is new in this branch and has never been correct; this is the fix, not a revert.
Admits: eb9e5bb3a4314b722a1e01e445201182b23e80ca40b64cbdd544827434f6935b
Admits-rule: protected-mutation
Admits-verdict: V-PROTECTED-MUTATION
Admits-subject: policy/test-targets.rego
Admits-head: 04389bfcbc45e8c84742656a351835cfd5cf25cd
Admits-epoch: 7f31bd1e3f27a45f279c38462b59167c4592ef855b219bf5904a60aaf6a5bdba
Admits-author: alec@wenzowski.com
Admits-prev: 2899d8657165ec05ec52be34efeefcdcd23242ddf7c5cb1bc4052a500c2f06a2
Admits-answer-lost: The gate cannot land: mutant-census exits 1, so verify and CI refuse the change. And the substantive loss it is pointing at is real — a gate covered by nothing stronger than its own green suite is the vacuity CLOUD-418 measured four times, so the discriminating mutations have to be written down even where no runner can drive them yet.
Admits-answer-precondition: mutant-census refuses the new module as uncovered, and its own remedy is a directive that can only live inside the module: a #MUTANT row declaring a discriminating mutation, plus a #MUTANT-EXEMPT naming the issue, exactly as policy/ci-parity.rego and policy/ci-suite-lane.rego carry them. There is no other surface for a comment directive in a .rego file, and the write lands in the same pull request diff a reviewer reads.
Admits-answer-rejected-route: R-USE-THE-OWNING-SURFACE does not apply because a #MUTANT directive is resolved from the module file itself; there is no owning surface elsewhere. R-RESTORE-IT does not apply because nothing is being restored — this adds the coverage declaration the census asks for.
Weakens: rule-predicate-changed rule[bats-tests-not-deleted].conserves
Weakens: rule-predicate-changed rule[no-key-leaves-the-schema-unannounced].glob
Weakens: rule-predicate-changed rule[shell-retirement].line_sources
92ba3d6 to
4d5bd0c
Compare
|
❌ The last analysis has failed. |
wenzowski
changed the title
|
/fast-forward |
1 participant
Five rows against the test loop's cost, in the order they had to land: the measurement first, then the two harness changes, then the engine fix, then the mass move that sweeps up every path the others touched.
Closes CLOUD-1289
Closes CLOUD-1291
Closes CLOUD-1290
Closes CLOUD-1288
Closes CLOUD-1210
What each one measured
CLOUD-1289 —
split-debuginfo, CLOUD-1211's one deferred arm. Paired A/B, three cold builds back to back in one process, two identical baselines bracketing the candidate. Time is inside the null (the baselines differ by 9.7s on their own, spread 0.957; the candidate sits at 0.956) and that is recorded so nobody re-runs it. Bytes are the finding, on a null of zero width — the two baselines are byte-identical — at 2.04x off the linked binaries. And they leave the disk rather than moving to a sibling:target/debug13.03 GB → 7.76 GB, the 5.20 GB replaced by 111.8 MB of.dwo. Separately,opt-level = 2was unpinned:dev_profile.rshad four cases and zero occurrences of the key, while dropping it doubles every suite run.CLOUD-1291 — the harness re-parsed the committed 356 KB config on every fixture command. Priced through a new
examples/bench rather than a CLI verb (the row's own first estimate was refuted that way): 10.48 ms per call, ten times its guess, of which the file read is 1.2%. Suite 60.26s → 53.64s, 11.0%, against a within-arm null of ±0.4%.CLOUD-1290 —
Fixture::git()spent two processes andbase_commit()a third on a rename that already held. Verified on git 2.43.0 thatinit -qalone leavesmain, and that all 100base_commit()call sites are preceded by.git().CLOUD-1288 —
execspentPIPE_DRAIN_TIMEOUTtwice. One shared budget:a_surviving_grandchild_cannot_hang_exec20.07s → 10.044s, shown able to fail at 20.148s. The row's zero-remaining premise is corrected rather than adopted —recv_timeout(Duration::ZERO)returned a ready value 10,000/10,000 here, so the predicted false notice does not reproduce and that test obligation cannot be discharged as written. Said plainly in the code rather than claimed.CLOUD-1210 — 144 test targets → 2, plus the ratchet that stops it regrowing. Rebuild after a real edit (not a
touch) 48.0s → 7.4s/7.3s, null 0.986. Bytes, cold and clean: 147 linked artifacts → 4, 4.99 GB → 234 MB,target/debug7.76 GB → 2.05 GB. Averifylap now consumes 186–2346 MB where the row recorded 25,403 MB.Why two and not one.
crates/batten/tests/policy_modules.rsdeliberately stays its own target:evaluator-io-checkprobes it withcargo test --test policy_modules, and that task is a governedmise-tasks/*.sh—shell-retirementgives it exactly two landable shapes, retire it whole or leave it alone, so repointing the probe at the group is not an edit this change may make. It shares the harness by#[path = "it/common/mod.rs"] mod common;rather than by a second copy.Two things worth a reviewer's attention
The ratchet shipped inverted and its own second tier caught it.
policy/test-targets.regotestedcount(segments) == 5where the path splits to four — refusing the grouped module and allowing the new target. The module's load-timetest_rules agreed with the mistake. The compiled-binary tier is what failed, which is exactly why.claude/rules/policy-modules.mdcalls it not optional.The move silently empties every glob spelling
tests/*.rs. batten's globs useliteral_separator(true), so*stops at a/: five committed rows plus one hardcoded inrules.rswould have selected nothing and reported clean. Repointed, and declared as threeWeakens: rule-predicate-changedtrailers — all three are widenings, not relaxations.mise-tasks/replay.shneeded no edit: itsdeclared_infeedsgit grepas a pathspec, where*does cross a/.Expected consequence
The ratchet will red the open PRs adding new top-level
crates/batten/tests/*.rs(#797, #796, #795, #793). That is the ratchet working; the fix on each is onegit mvinto the group plus amodline.Suite: 3607 tests, 3607 passed. No case deleted.
Filed rather than fixed here: CLOUD-1293 (
.dwois a classprune.rs's closedRECLAIMED_KINDScannot see, so it accumulates per build hash).Generated by Claude Code