A container the suite can reach at index_bits 6, and a VQ4P arm in run.sh

[mfethe1]

Requested by @Ultron09 in #41: teach make_test_container.py --index-bits 6 and wire the VQ4P engine check into run.sh. This is that PR; the kernel it covers is vq_rows_p6, which #41's AVX-512 work joins.

Why

CLAUDE.md names the trap this closes:

the same applies to --index-bits 6, which exercises a different kernel and a different record fmt

running the suite on one is how the Q4G load path stayed broken through green runs

Until now vq_rows_p6 had no container make check could reach: the synthetic one is always index_bits 8, and a real --index-bits 6 conversion needs the source weights and hours. My own review on #41 said I had tried to close this and could not — this is the closing half, and #41's kernel gets the same checks for free once it lands.

What

--index-bits 6 writes the one combination the engine accepts at 6 (src/model.c refuses anything but 4 stages × 64 entries): records carry the WQ_VQ4P fmt byte, indices packed 4×6 bits into 3 bytes with the same little-endian bit order and [M/B][pos][row] blocking as convert.py's block_indices_packed, and the manifest's expert_quant carries index_bits/fmt/bits_per_weight exactly as convert.py writes them.

The default path emits byte-identical containers: manifest.json gains no key (the engine reads absence as 8), so the rotary fixture's container_sha256 gate is untouched. Verified per-file by sha256 at seed 0, plain and --rope, before and after.

run.sh gains a VQ4P engine (index_bits 6) section running the same four self-comparisons the engine block runs on VQ3R: records through the C structs (test_container now accepts the WQ_VQ4P fmt byte — same size and layout as VQ3R, only the payload packing differs), chunked == token-at-a-time, SIMD vs the CPU baseline, and expert-cache bit-identity. A few MB, milliseconds to build, so it runs on every host CI does.

Evidence

Same tree (2fa0796 on main at 6430585), two platforms:

host	board	VQ4P arm
macOS arm64, clang 17	48 passed, 0 failed, 13 skipped (baseline 44/0/13)	all 4 PASS — NEON vq_rows_p6
Windows 11, Ryzen 7 3700X (Zen 2, no AVX-512), MSYS2 UCRT64 gcc 16.2.0	46 passed, 0 failed, 14 skipped (baseline 42/0/14)	all 4 PASS — scalar vq_rows_p6

Both boards are exactly baseline + 4: no prior PASS moved. The SIMD-vs-baseline check lands in the suite's existing "within fp noise" branch on both hosts, same as the VQ3R arm; the cache check is bit-identical on both. Also: the same-seed VQ4P container is byte-identical across Windows and macOS (6/6 file sha256s equal — the newline="\n" discipline #47 landed, working), and the 4×6 packing round-trips against the engine's P6_J0..P6_J3 macros on 1,000,000 random rows with 0 mismatches.

What this does not claim: quality of the noise codebooks (there is none — it is noise), and nothing about the AVX-512 kernel itself, which stays #41's to land; when it does, this arm covers it on Cascade Lake / Skylake-SP CI runners without further work.

[mfethe1]

Following up on the #41 measurement (dispatched vs WASTE_BACKEND=cpu: max diff 0.41, 162233/163840 over 1e-3 on NEON; @GTSUlteer's 1.23 / 163297 on AVX2): the same comparison on the tiny synthetic container here is max diff 9.5e-07, 0/256 over 1e-3 (mac-arm64, NEON vs CPU). So the arm in this PR passes — but for the reason you predicted, not the reason the check implies: the synthetic weights never land on the discontinuous side of the int8 LUT, so 'within fp noise' is data-luck, not a property of the kernel.

That makes the arm weaker than it looks on real containers, and I'd rather say so on the PR than let a green run carry it. The check that would actually pin it is the one you named in #41 — bit-identity of the int8 table (or of the routes) against the scalar path — since a broken kernel and a single entry rounding the other way are indistinguishable at logits level on 6-bit indices. The LUT is a pure function of the codebook, so it can be checked exhaustively in milliseconds with no container at all: build the table scalar, build it through the kernel's lane path, cmp the 256 (or 4096, per stage) bytes.

If that sounds right I'll send it as a follow-up — test_vq_lut.c, container-free, runs on every platform including the CI-forced synthetic path.

[mfethe1]

Traced this further after PR #41 merged — the dispatch table now has three real vq_rows_p6 implementations (model.c:2129 portable, kda_neon.c:100 NEON, simd_avx512.c:129 AVX-512 VBMI), all consuming the same int8_t* lut8 built once by vq_quant_range (model.c:2054, not itself in the dispatch table — same code path regardless of WASTE_BACKEND).

That rules out the LUT-building step as the source: given the same float activations, vq_quant_range produces the same int8 table and scale on every backend. Given the same table, all three consumers do pure integer accumulation with no float reordering to disagree on — the code's own comment at model.c:2166 says as much. So the divergence has to come from upstream of quantization: WASTE_BACKEND=cpu doesn't only swap vq_rows_p6, it swaps every dispatched kernel in the forward pass, so the float activations feeding that token's vq_quant_range call differ minutely (ordinary SIMD-reduction-order noise, ~1e-6 relative) between a normal run and a CPU-forced one — and vq_quant_range's own scale is max_abs/127, so that ordinary noise gets amplified into a 1-LSB flip for any table entry sitting near a k+0.5 boundary. There's already a documented instance of exactly this amplification in this file (model.c:2365: "a 1e-8 difference... moves every entry near a rounding boundary by one LSB. That measured as 0.68 on a logit"). Real weights and real activations are generic enough that some block, somewhere across 163,840 outputs, always sits near enough to a boundary for this to show — which is also the honest answer to why the synthetic-noise container in this PR doesn't hit it: not enough boundary-adjacent structure in that data, not a property of the kernel being more correct there.

That confirms the fix precisely: the test needs to hold the int8 table fixed (not derived from a live, backend-dependent forward pass) and compare only the integer-accumulation step across backends. Scoped the harness assuming a test_vq_lut.c in the shape already proposed above:

• Build a handful of fixed vqp_arg cases directly (lut8, packed 4×6-bit idx via the P6_J0..P6_J3 macros in simd.h, lscale, unit scale) — including entries deliberately placed at k±0.5 boundaries, since that's the case a live-activation test would only hit by chance.
• Call waste_vq_rows_p6 directly (already extern, declared in kda.c) as the reference, and waste_k.vq_rows_p6 after waste_backend_init(WASTE_BE_AUTO) vs WASTE_BE_FORCE_CPU for whichever accelerated kernel the host has.
• Diff the integer accumulator output exactly — no tolerance, since there's no float step left once the table is fixed.

I don't have AVX-512 VBMI or ARM hardware here to build and run the SIMD arm myself, so I stopped short of writing it rather than ship an untested harness — happy to build the portable-reference half now if useful, or defer entirely if this is already in progress on your end.