bench-gate: false-positive FAIL on linux-glibc-x86_64 that the contention controls did not flag

[orieg]

Benchmark Gate failed on the weekly schedule for linux-glibc-x86_64 on 2026-08-31 and passed a week later on the identical commit with an identical toolchain. The two flagged cells returned to baseline with no source change, so the FAIL was a false positive.

The concern is not the two cells. It is that the run's own contention controls read clean while it happened, so this class of artifact currently reaches a FAIL verdict unfiltered.

Runs

Date	Run	Commit	Verdict
2026-08-24	32695086626	4dd3abe	PASS
2026-08-31	33387655960	99701b4	FAIL
2026-09-07	34110549368	99701b4	PASS

Flagged cells — axis timing.s_over_c

Ratio of the vendored libJudy patches against the pristine-static arm. Per-cell floor 18.5%.

Cell	Baseline	08-31 (FAIL)	drift	09-07 (PASS)	drift
api.setop.diff.bitset	0.72785	0.89565	+23.05% [22.74, 23.54]	0.73536	+1.03%
api.setop.xor.bitset	0.71376	0.89040	+24.75% [24.10, 26.90]	0.72338	+1.35%
core.string_to_mixed_adaptive.iter	0.89027	0.62586	−29.70%	0.88938	−0.10%

Ruled out

Identical across the FAIL and the subsequent PASS:

• Vendored libJudy source — same tree hash 7c9f86ea0df9ff277729297e0f23981a0e3e1a86.
• Compiler — cc (Ubuntu 13.3.0-6ubuntu2~24.04.1) 13.3.0.
• PHP — 8.4.25 in both. (The 08-24 PASS ran 8.4.24, so the patch bump was a candidate; the 09-07 PASS on 8.4.25 refutes it.)
• Kernel / VM image — 6.17.0-1022-azure #22-Ubuntu.
• Extension and libjudy/ sources — the only commits between the 08-24 PASS and the 08-31 FAIL are docs(readme): refresh Expanse engine install (apt/rpm + libjudy-compat); add Expanse's native PHP packages #203 (README) and fix(ci): make nightly Valgrind job load the extension; guard bench tests under memcheck #204 (CI workflow plus bench .phpt guards). No C source changed.

Why the existing guards did not catch it

On the failing run, attempt 2:

• Host hygiene ok at all four checkpoints: load1=1, cpus=4, foreign=0.0%.
• control php-only: −0.65%, scatter 3.19% over 44 rows.
• control C-vs-C: +0.04%, scatter 1.98% over 51 cells.
• baseline-stability guard: stable.

Both controls and the stability guard passed while three cells moved 23–30%.

Two further properties constrain the cause:

1. It is not uniform. api.setop.intersect.bitset and api.setop.union.bitset sit on the same axis and did not trip, which does not match broad LLC or bandwidth contention.
2. It survived an in-run retry. Both attempts in run 33387655960, roughly six minutes apart on the same VM, flagged the same two cells. A confirmation re-run inside the job would not have suppressed it; the artifact appears to have a per-VM lifetime rather than a per-trial one.

Direction is also mixed — two cells worse, one substantially better — which fits a code-layout or per-host memory-topology effect more than a systematic slowdown.

Follow-up

Decide which of these the gate should adopt. No source fix is implied; the extension is not regressed.

• Extend the controls so whatever moved here is observable. The current pair sees interpreter drift and LLC/bandwidth contention, and this passed both.
• Consider whether the bitset setop cells warrant per-cell floors above the 18.5% axis floor. Their recorded cross-run drift is low (diff 2.49%, xor 1.12%), so they carry the tightest floors on the axis and trip first on any host-level artifact.
• If a re-run policy is added, it must cross VM boundaries — an in-job retry is already ineffective per the evidence above.

Artifacts carry the per-cell JSON and are retained 90 days from each run.