A missing cmp or python3 said the engine was broken, not the tool absent - #42
Conversation
Line 10 states the contract: this suite "never treats a missing prerequisite as a pass — it says SKIP, loudly." The curl guard says the same thing in more words, and for the same reason. Neither `cmp` nor the `python3` behind range_server.py was held to it. Measured on a fresh MSYS2 UCRT64 install, which is the documented Windows build path (sqliteai#36) and ships neither diffutils nor python3: FAIL expert cache changes results FAIL read-ahead changes results FAIL router lookahead changes results FAIL purgeable slots change results FAIL range server did not start (resume, state-file skip not run) ... 8 in total from cmp, 4 more from python3 Every one of those names the engine. "expert cache changes results" is the report of a bit-identity violation in the cache — the thing this suite exists to catch — and it fired because /usr/bin/cmp was not installed. The verdict was not merely unhelpful, it was the wrong verdict about the wrong component, and it is indistinguishable in a log from the real defect. So `cmp` is guarded once and reached through same(), which returns 0, 1 or 2 for identical / differ / no-tool. Callers that could only say PASS or FAIL now have somewhere to put the third case. cmp's own exit 2 ("could not read a file") folds into 1, which is what the old `if cmp -s` did with it, so no existing verdict moves. The download section already had the right shape for this — NO_CURL skipped all five of its checks — so that flag becomes DL_MISS and carries the reason instead of a bare 1. python3 joins curl in setting it, and the fixture that python3 builds is guarded behind it too. The SIMD check is left alone deliberately: with no cmp it falls through to the fp-noise bound, which is a real verdict on its own and needs no cmp. It loses the bit-identity claim and nothing else. Verified on Windows 11 / Zen 2 / UCRT64 / gcc 16.2.0, by hiding each tool in turn and running the suite: cmp hidden before: 8 FAIL after: 8 SKIP, 0 failed python3 hidden before: 4 FAIL after: 5 SKIP, 0 failed both present 44 passed, 0 failed, 13 skipped (unchanged, exit 0) Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01B5eVMiauR4Lkt8Dhc67MNb
|
Merged. Reproduced here first, because the whole point of this change is the Both tools hidden by a PATH holding symlinks to everything else, so the two
The eight on main, verbatim:
Your reading of why this matters more than a label is right and is why it went The part you could not observe, observedYou wrote:
Leaving out a failure you had not seen was the right call. It also means the Same defect, different block: the fp-noise comparators and the That is not a criticism of this PR, which does what it says and does it On the two you set aside
53 passed / 0 failed / 6 skipped here with everything installed, unchanged from |
…cope 1. The VQ4P cache check called cmp -s directly, undoing sqliteai#42 one commit earlier at the same call site by name: on a PATH without diffutils (fresh MSYS2 UCRT64) it false-FAILed 'VQ4P expert cache changes results' as the only failure on an otherwise clean board. Now the same() helper with the 0/1/* case, identical to the VQ3R cache check directly above. 2. SIMD-vs-CPU tolerance 1e-3 -> 1e-5. The review's mutation table puts the intact kernel at 9.54e-07 and every single-edit break (j3 shift, T1/T2 swap, one LUT entry +1) at 0.002-0.003: 1e-5 keeps 10x headroom over noise and 300x under the break signal. 3. The block comment claimed 'agreement is exact rather than within fp noise', but the arm lands in the fp branch on every host measured (what reaches the logits also went through the other dispatched kernels). Rewritten to say what the check actually asserts. 4. Scope sentence added: the container is 4 layers / 3 MoE, so the arm bounds gross kernel errors, not the depth-amplified discontinuity mode (real index_bits 6 container: max diff 0.58 at 27 layers with the kernel correct). A green run must not be over-trusted. Verified on macOS arm64 (NEON path): 48 passed, 0 failed, 13 skipped; the VQ4P SIMD check passes through the tightened 1e-5 branch.
Almost none of this is the engine. It is Windows, real containers, and the four feasibility gates that were still open, and most of it was found by people running hardware this project does not own. GATES.md has no open gate for the first time, and two of the three that closed this cycle closed against the change they were proposing. No ABI move: src/waste.h changed only in a comment this cycle, unlike 0.6.8. waste_kernels gained vq_rows_p6 but that lives in src/waste_backend.h and is not public. Added: the VQ4P apply behind a dispatch slot with an AVX-512 VBMI kernel beside it (#41), bit-identical on ARM against a real index_bits 6 container and still never executed on any machine here; WASTE_DUMP_SCORES (#45) and WASTE_CCR_LAMBDA (#46); the working set printed by `waste plan`, which was quoted by the line below it and never shown; and a test for the fp8 block-scale mapping whose only stub had replaced it with the identity (#40). Fixed: the five gaps of #36 and the two that survived the first attempt — VirtualLock bounded by the minimum working set rather than the maximum, so WASTE_MLOCK wired nothing on Windows twice over; diskbench truncating its offset through a 32-bit off_t at its own default file size, where the quiet half read the wrong place successfully and kept the working set inside an SSD's SLC cache; container JSON through Python text mode, which made the same conversion produce byte-different containers on Windows; a missing cmp reporting `expert cache changes results` on a clean checkout (#42); plus #35, #30, and the CLI help that said one thread per core when it is one per logical CPU (#44). Recorded as measured and not adopted, which is the half of this file that is easy to drop: the budget resolver's quantum stands after Gate 7 (§63); cache-conditional routing stays a knob because it clears README's KL bar and not its continuation bar; the thread default did not move despite 16 threads measuring 1.6x below the plateau; no budget policy fixes two K3 opens on 64 GB (§64, #31 closed, #49 opened for the residue); and VQ4P is not a throughput upgrade over VQ3R on a GB10 despite complete and exact CUDA coverage (§62). 53 passed, 0 failed, 6 skipped against Kimi-Linear and K3.
2 participants
tests/run.sh:10states the contract:and the
curlguard says it again, in more words, for exactly this reason:cmpand thepython3behindrange_server.pywere not held to that. This brings them into line.Evidence
Captured on Windows 11 / Ryzen 7 3700X (Zen 2) / MSYS2 UCRT64 / gcc 16.2.0. The before side is the merge-base
7f1fbbain a sibling worktree, the after side is this branch; each tool was hidden by moving its binary aside for the duration of both runs, so the two sides differ only in the commit.LC_ALL=C,TERMandCOLUMNSpinned, identicalgrepfilter applied to both.cmp(diffutils) not installed — before, eight checks report FAIL and name the engine.expert cache changes resultsis the report of a bit-identity violation in the expert cache, and it fired because/usr/bin/cmpwas absent. After, the same eight report SKIP and name the missing tool. The pass count is unchanged at 36; only the eight false failures move.python3not installed — before, four checks reportrange server did not start, blaming the download script for a missing interpreter. Note thatconvert.pyandserve, which are guarded, correctly SKIP in the very same frame: the inconsistency is visible without leaving the image. After, all five download-script checks SKIP and name the tool, consistent with the checks beside them.Why this matters more than a cosmetic label
Every one of those FAIL lines names the engine or the download script.
expert cache changes resultsis exactly what this suite is for — and in a log it is indistinguishable from the real defect.mla_use_nope: false was read as NoPE and skipped the rotationnames the precise bug #27 fixed.A first-time Windows contributor's first run of this suite currently accuses the expert cache of returning different logits, on a clean checkout, because diffutils is not installed.
UCRT64 is the documented Windows build path (#36) and ships neither
diffutilsnorpython3, so this is the default first experience there, not an exotic configuration.The change
cmpis guarded once and reached throughsame(), which returns0/1/2for identical / differ / no-tool. Call sites that could only say PASS or FAIL now have somewhere to put the third case.cmp's own exit 2 ("could not read a file") folds into 1 — which is whatif cmp -salready did with it — so no existing verdict moves.The download section already had the right shape:
NO_CURLskipped all five of its checks. That flag becomesDL_MISSand carries the reason rather than a bare1;python3joinscurlin setting it, and the fixturepython3builds is guarded behind it too.The SIMD check is deliberately left alone. With no
cmpit falls through to the fp-noise bound, which is a real verdict on its own and needs nocmp. It loses the bit-identity claim and nothing else — so it keeps running rather than skipping.Full results
cmphiddenpython3hiddenbash -nclean. No file outsidetests/run.shis touched.Two things noted and deliberately not included
Separate concerns; happy to send either as its own PR.
.gitignorehas no*.exeor*.dll. It coverswaste,libwaste.a,*.so,*.dylib,sweep— so a native Windows build leaveswaste.exe,sweep.exe,libwaste.dllandlibwastevq.dlluntracked.python3calls inside the engine and rotary blocks are still unguarded (the fp-noise comparators,make_test_container.py). Same class of bug — but I did not observe them failing, because by the time this machine had a container-capable runpython3was installed. This project measures rather than asserts, so I have left out a failure I did not actually see.Context
Found while standing up a native Windows x86 measurement box (128 GB, Zen 2, PCIe 4.0 NVMe) to work on Gate 7 and the AVX2 side of #32/#38. This was the first thing in the way. Related to #36, which is where the "gaps no CI job can reach" framing comes from.
🤖 Generated with Claude Code
https://claude.ai/code/session_01B5eVMiauR4Lkt8Dhc67MNb