ci: fmt-gate the workspace-excluded tooling crates - #426
Merged
Conversation
… tier `cargo fmt --all` formats workspace members, and every crate under `tooling/` is a `[workspace].exclude` entry, so this crate has never been fmt-gated. Its four Rust files had drifted: `println!`/`format!`/`assert!` argument wraps in gaps.rs, probe.rs and real_corpus.rs, and one `read_to_string` wrap in lib.rs. The two SHA-256 tables in lib.rs get `#[rustfmt::skip]` instead. Every element is 11 chars, one over rustfmt's `short_array_element_width_threshold`, so rustfmt would break the 64 round constants and the 8 initial hash values one per line — 72 lines of noise in place of the spec's own rows. Skipping them keeps the diff at +27/-7 rather than +92/-17. Pure formatting: no behaviour changes.
`cargo fmt --all` reaches workspace members, and `members = ["crates/*"]`, so no crate under `tooling/` has ever been fmt-gated — the drift the previous commit cleared had accumulated unnoticed because nothing was looking. This is wider than the compile gap `check-dng-real` closes. That one is unique to `gamut-dng-real-conformance`, the only excluded crate nothing depends on; the other fourteen oracles are compiled by `clippy --workspace --all-targets` through their path dev-dependencies. rustfmt has no such reach: a dependency edge does nothing for `fmt --all`, so it is blind to all fifteen. `fmt-tooling`/`fmt-tooling-check` iterate the manifests and hang off `fmt`/`fmt-check` via `depends_post`, so the hooks (hk.pkl) and CI's existing `mise run fmt-check` step inherit the gate with no second call site to keep in sync. The check variant deliberately does not stop at the first drifting crate, so one run names all of them. rustfmt compiles nothing, so the added cost is seconds and the `fast` CI job still builds no native code. Also corrects the sccache rationale comment in ci.yml, which claimed the vendored oracles "are workspace members under `tooling/`". They are the opposite — `[workspace].exclude` entries reached as path dev-dependencies. The comment's actual point survives: rust-cache drops them for being local path crates, which is why sccache earns its place.
A build script that cannot be run — as distinct from one that fails to compile — is repeatedly misread as a code problem and "fixed" in the build script. It is neither. The binaries under `target/debug/build/` lose the owner-execute bit and land at `-rw-rwx---`, where the group keeps `x`, so `ls` output looks unremarkable while cargo, which runs as the owner, cannot execute them. Documented next to the `tooling/build-env` paragraph because it is the adjacent trap: both are environmental failures that present as native-build breakage and invite a spurious change to the build. States the symptom and the recovery only — the mechanism that clears the bit is outside this repo and not yet established, so the note does not name a cause it cannot support.
the reformat here touches, adding code that has never been fmt-gated. Merging master puts that code under the new gate so CI proves the composition rather than leaving it to merge time.
`cargo fmt --all` means "all packages, and also their local path-based dependencies", so running it per excluded manifest followed those edges out of `tooling/` and back into the workspace: 459 file-visits across 212 files, 181 of them under `crates/` and already covered by the `fmt --all` the task runs first. It also reported one drifting file once per dependent crate that could see it — mangling `libtiff-oracle/src/lib.rs` named it three times over. Dropping `--all` leaves each manifest formatting its own package, and the loop already iterates all fifteen: 31 file-visits over the same 31 `tooling/` files, none duplicated, none under `crates/`. Coverage is unchanged — verified the gate still catches drift in lib, examples and tests, the last two being where the drift this branch fixes actually was.
The four figures in the comment above `fmt-tooling` were produced by parsing
rustfmt's verbose output, which lists only a subset of the files it visits, so
every one of them undercounted. Re-measured by ground truth — dirty every `.rs`
file in the tree, then read back which paths the gate reports:
file-visits distinct files of which under `crates/`
with --all 1131 518 483
without 35 35 0
The argument the comment makes is unchanged and stronger. What `--all` drags in
is 483 of 518 files, not 181 of 212, and the duplicate reporting it causes peaks
at 18 repetitions of a single file (`crates/gamut-jxl/tests/common/mod.rs`), not
the three a narrower probe had suggested.
Also records what the numbers alone do not: the `tooling/` coverage is
set-identical between the two forms — the same 35 files across all 15 excluded
crates, each visited exactly once — so dropping the flag costs no reach.
Comment only; no task body, glob, or `depends_post` wiring changes.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Follow-up to #415, which closed the compile half of this gap and named the formatting half as
worth deciding on. This is that decision.
The gap
cargo fmt --allformats workspace members, andmembers = ["crates/*"]. Every one of thefifteen crates under
tooling/is a[workspace].excludeentry, so none of them has ever beenfmt-gated — not by CI, not by the hooks.
This is wider than the compile gap
check-dng-realcloses. That one is unique togamut-dng-real-conformance, the only excluded crate nothing depends on; the other fourteenoracles are compiled by
clippy --workspace --all-targetsthrough their path dev-dependencies.rustfmt has no equivalent reach — a dependency edge does nothing for
fmt --all— so it is blindto all fifteen.
Measured across every excluded manifest, only one crate had actually drifted:
gamut-dng-real-conformanceThe other fourteen are clean by discipline, not by enforcement. That is the thing worth fixing.
The reformat
git blameputs the driftingprobe.rshunks on5b75cc9c, so this is pre-existing and notsomething #415 introduced — its own added code is fmt-clean.
The two SHA-256 tables in
src/lib.rsget#[rustfmt::skip]rather than rustfmt's output. Everyelement is 11 chars, one over rustfmt's
short_array_element_width_thresholdof 10, so rustfmtbreaks the 64 round constants and the 8 initial hash values one per line — 72 lines of noise
replacing the FIPS 180-4 rows.
Skipping a table means the formatter no longer guards it, so the guarantee has to come from
somewhere else. It does:
src/lib.rscarriessha256_matches_the_known_vectors, the two canonicalNIST vectors, and it passes (below). That is a stronger claim than rustfmt could make — the
tables are correct, not merely formatted — and it is what makes the opt-out safe rather than
merely convenient.
The gate
fmt-tooling/fmt-tooling-checkiteratetooling/*/Cargo.tomland hang offfmt/fmt-checkvia
depends_post, sohk.pkl's hooks and CI's existingmise run fmt-checkstep inherit the gatewith no second call site to keep in sync. The check variant deliberately does not stop at the first
drifting crate, so one run names all of them. rustfmt compiles nothing, so the
fastCI job stillbuilds no native code.
Coverage, measured rather than assumed. Dirtying every
.rsfile undertooling/and readingback which paths the gate reports: it visits all 35 of them, across all 15 excluded crates, each
exactly once, and exits 1. There is no tooling source the gate cannot see.
Why the per-manifest loop is not
--allcargo fmt --allmeans "all packages, and also their local path-based dependencies". Becausegamut-dng-real-conformancepath-depends on../../crates/*,--allon that one manifest followsthose edges and drags the whole workspace back in. Measured by the same method, on a tree where
every
.rsfile was made dirty so nothing could hide:crates/--allThe 483
crates/files are already covered by thecargo fmt --allthatfmt-checkruns first.The
tooling/coverage is set-identical between the two forms —commover the reported setsis empty in both directions. What
--allbought was cost and noise: it names one drifting file onceper dependent crate that can see it, up to 18 times for a single file
(
crates/gamut-jxl/tests/common/mod.rs), in the one output a contributor reads to find the drift.The comment above the tasks records this, so the absent flag does not read as an omission and get
"fixed" back in.
ci.yml's sccache rationale comment also claimed the vendored oracles "are workspace members undertooling/". They are the opposite. The comment's point survives — rust-cache drops them for beinglocal path crates — so only the false clause changed.
Interaction with #415
#415 landed first (
0851e173), so the ordering this PR originally reasoned about is settled andinverted. Its rewrite of
examples/probe.rs(+86/−2) is code the fmt gate had never seen. Thisbranch merges
origin/master(4319849) to bring that in and prove the gate against it ratherthan against a base that no longer exists. The merge is conflict-free and
mise run fmt-checkpasses on the merged tree.
The
error[E0609]: no field 'exif_extra' on type 'DecodedDng'that this PR previously carried as apre-existing failure was #415's to repair, and it is cleared:
cargo check --manifest-path tooling/gamut-dng-real-conformance/Cargo.toml --all-targetspasses on the merged tree.The
AGENTS.mdnote0109d3fadds seven lines toAGENTS.md: a build script cargo cannot run — as distinct from onethat fails to compile — is usually a lost owner-execute bit under
target/debug/build/, not a codeproblem, and is recovered with
chmod -R u+x.It is not part of the fmt gate. It rides along because it was hit while validating this branch,
and it belongs next to the
tooling/build-envparagraph it sits under — the adjacent trap, wherean environmental failure presents as native-build breakage and invites a spurious edit to the build
script. Flagging it explicitly because it is the one changed path that maps to neither this PR's
description nor a linked issue; say so and I will split it onto its own PR.
Validation
Every command below completed against the working tree at
ccb6c7b.mise run fmt-checkfmt-tooling-checkobserved executing viadepends_postmise run lint--workspace --all-targets --all-features, 0 diagnosticsmise run check-commitscargo check --manifest-path tooling/gamut-dng-real-conformance/Cargo.toml --all-targetsmise run fetch-dng-samplesc6f8e3a4, 170 MiBmise run test-dng-realThe corpus run is the one that matters most here, because it is the only thing that executes the
reformatted
tests/real_corpus.rsrather than merely compiling it:Proven non-vacuous, at every target kind. A gate that passes because it inspected nothing is the
failure mode here. In a clean-room copy of the merged tree: unmodified,
fmt-tooling-checkexits 0;with drift injected into a tooling crate's
src/,examples/andtests/file, it exits 1 andnames each exactly once; and the fix variant
fmt-toolingclears all three, after which thecheck exits 0 again.
examples/andtests/matter specifically — that is where the drift this PRfixes actually was, so a gate reaching only
libwould have been a silent regression.Glob edge cases, exercised rather than argued. A
tooling/directory with noCargo.tomlisskipped correctly (loop exits 0). A glob that matches nothing fails loudly in every candidate
shell —
shandbashpass the literal pattern to cargo, which errors and setsfail=1;zshaborts on the unmatched glob. There is no silent-pass path.
Behaviour preservation.
git diff -wagainst master is empty forgaps.rs,probe.rsandreal_corpus.rs.src/lib.rs's whitespace-stripped delta is +237 characters, accounted forentirely by three comment lines and the two
#[rustfmt::skip]attributes.Risks and rollout
No workspace crate is touched, so release-plz derives no version bump while #412 is in flight, and
the tier this reformats is dev-only and excluded from every published package.
What genuinely remains, stated rather than rounded away:
test-dng-realis not a per-PR gate. I ran it by hand for this change and it passes, but theper-PR lane only runs
check-dng-real, the compile half. A futuregamut-dngchange can stillbreak the corpus assertions with every per-PR check green, and master's extended run is where it
surfaces. That is pre-existing and not this PR's to fix, but this PR does not fix it either.
AGENTS.mdis outside this PR's stated scope (see above). Small and documentation-only, butwhether it belongs here is a reviewer's call, not mine.
#[rustfmt::skip]tables are guarded by their known-answer test, not by the formatter. Avalue error is caught; a purely cosmetic edit inside those two arrays is not. That is the intended
trade, and it is worth knowing it is a trade.