fix(os): make the release gate report only what it actually ran (#1064) - #1127
Conversation
Two halves of one defect: a gate that reports success without having run. `--phase all` executed five of eight phases while docs/dev/appliance-release.md told a maintainer that step 1 of a cut ran everything. So every cut that followed the checklist silently skipped the three mid-write power cuts, the mid-commit cut, the corrupt-bundle refusal, the factory reset, the wedged-/data recovery and the whole media channel. The arm now runs all eight, and the doc names them and says to record the dated per-phase count for the tip actually being cut — an old green standing in for a new one is the same defect wearing a different hat. verify-image's two strongest checks — built-from-the-expected-commit, and the artifact-versus-tree comparison that unpacks the baked container to compare wizard.py — are conditional, and their skips printed nothing while the summary said "0 failed". Those are precisely the checks that exist because an image once shipped a dashboard two commits stale and passed everything else. Skips are now counted, printed, and fatal: a run that declined to compare the artifact against the tree does not get to call the image verified. And `_build_image` now exports PITHEAD_EXPECT_COMMIT, so the guard is on in the one caller that is not a human typing a command — it was off in every battery run, which is where it would have mattered most. Tier-1 2636/0. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
|
Bench verification status at handoff: still running, so this should not merge yet. The image build on the KVM bench has been re-run twice for reasons worth recording, neither of them about the fix:
Both are notes about reproducing What still has to be observed before merging, from a repo-root checkout of this branch on the bench:
Build logs are on the bench at |
The guard #1064 switched on could never pass. `_build_image` handed over `git rev-parse --short HEAD` while `os/build-image.sh` stamps the full sha into /opt/pithead/BUILD_COMMIT, and verify-image compared the two for equality — so every image the harness built would have failed its own verification, and the battery would have died at the first `_build_image` call. A gate that refuses everything is the same defect as one that refuses nothing, pointed the other way. The harness now hands over the full sha, and verify-image matches on a prefix so the short sha an operator copies out of `git log --oneline` verifies too. The "-dirty" suffix is left to the clean-tree check, which is the one that means it; the hand-rolled dirty suffix in the harness is gone, since it used `git status --porcelain` (untracked files count) where build-image.sh uses `git diff --quiet` (they do not), and the two answers disagreed. Bench-proven on the KVM image built from a97d5bc, all five scenarios: unset, repo root -> 2 skipped, exit 1 unset, outside the repo root -> 3 skipped, exit 1 short sha (a97d5bc) -> 92 passed, 0 failed, exit 0 full sha -> 92 passed, 0 failed, exit 0 a sha it was not built from -> 1 failed, exit 1 Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
|
Bench verification is complete, and it found a defect in this branch. Fixed in d8a239d — please The three scenarios the previous comment asked forRun from a repo-root checkout of this branch on the KVM bench, against the image built from
Both skip scenarios behave exactly as intended, including the third skipped line ("the artifact What the third scenario found
Two smaller things fell out of it. The harness's hand-rolled The fix, and its bench proofThe harness hands over the full sha;
The last row matters as much as the first two: the check still fails on the thing it exists to Tier-1 gains three assertions pinning the shape agreement, with the mutations named in the test |
# Conflicts: # tests/stack/run.sh
Closes #1064. Both halves are the same defect: a gate that reports success without having run.
--phase allran five of eightThe
all)arm called boot, update, install, provision and rig.docs/dev/appliance-release.mdsaid "
--phase allruns everything", and step 1 of Cutting a release prescribes exactly thatcommand — so a maintainer following the checklist believed the three mid-write power cuts, the
mid-commit cut, the corrupt-bundle refusal, the factory reset, the wedged-
/datarecovery and thewhole media channel had run. None of them had.
The arm now runs all eight. The doc names them, warns that it is an evening rather than a coffee
break, and says to record the dated per-phase count for the tip actually being cut — an old green
standing in for a new one is the same defect wearing a different hat.
verify-image reported "0 failed" for checks it declined to run
Two blocks are conditional: the built-from-expected-commit pair (when
PITHEAD_EXPECT_COMMITisunset) and the whole artifact-vs-tree comparison (when not run from the repo root) — the one that
unpacks the baked container archive to compare
wizard.py. Neither skip printed anything.Those are precisely the checks that exist because an image once shipped a dashboard two commits
stale and passed everything else. Skips are now counted, printed as
(SKIPPED: reason), andfatal: a run that declined to compare the artifact against the tree does not get to call the
image verified.
_build_imagenow exportsPITHEAD_EXPECT_COMMIT, computed the same wayBUILD_COMMITisrecorded, so the stale-artifact guard is on in the one caller that is not a human typing a command
— it was off in every battery run, which is where it mattered most.
Coverage
The harness only runs on the KVM bench, so the wiring is asserted in tier-1: all eight
phase_*calls present in the
allarm, thePITHEAD_EXPECT_COMMITprefix on the verify call, andverify-image's skip counting and refusal. Dropping a phase from the arm, or the env prefix, turns
the matching assertion red. Tier-1 2636 passed / 0 failed.
The end-to-end proof is running on the bench as this opens: a fresh image build, then
verify-image in three shapes — no
PITHEAD_EXPECT_COMMIT, run from outside the repo, and fullypinned — to confirm the first two now exit non-zero and the third still passes. I will post the
result here; this should not merge before it does.