Skip to content

Write down the checks that pass without proving anything - #29

Merged
yujiezhang-ops merged 1 commit into
mainfrom
docs/verification-checklist
Aug 13, 2026
Merged

yujiezhang-ops merged 1 commit into
mainfrom
docs/verification-checklist

Conversation

@yujiezhang-ops

Copy link
Copy Markdown
Collaborator

A checklist of the traps this repository actually sprang during the BootAgent
rename, each paired with the check that catches it. Every entry earned its place
by catching something real, or by failing to and letting something through.

Kept specific to this repository on purpose. A checklist of generic advice does
not get run.

The two that matter most

An unauthenticated e2e run hides its most relevant coverage. test:e2e
reports a green 198 passed instead of 213 once the release feed hits the
unauthenticated rate limit: site.spec.ts:371/:470 skip themselves when no
release is visible, which is correct for a fork with no releases and
indistinguishable from a spent rate limit. The result is green with the
download-page coverage missing — exactly where a change to that path does its
damage. CI injects the token, so this only bites locally, which is where the
decision to push gets made.

A passing visual snapshot does not mean the baseline is current. The rename
changed the brand name rendered inside ActivationConsole, well within the
captured region, and all six baselines still passed: the glyph delta is smaller
than maxDiffPixels: 1_500, a tolerance sized for the rounded window edge. Left
alone they would have kept showing the old product name while reporting success.
Related and easy to miss: --update-snapshots only rewrites baselines for
failing tests, so on a passing suite it is a no-op that looks like it worked.
--update-snapshots=all is what rebuilds.

Because the tolerance hides small text changes, a refresh here cannot be justified
by "the test failed, so I updated it". The checklist records how the rename's
refresh was actually checked — per-pixel, confirming zero residual below y=40.

Also covered

  • Confirming a failure predates your change before attributing it to your work.
    site.spec.ts:1213 fails intermittently under parallel load for reasons
    unrelated to anything you are likely to be changing.
  • Bulk find-and-replace hitting the file you exempted. This is in here because it
    happened: the rename's own audit named src/lib/downloads.ts:48 as
    must-not-change, and the sweep changed it anyway, breaking the header's GitHub
    link. The e2e suite caught it; the sweep did not. Exempt by path in the file
    list, not by intention.
  • Verifying rendered output rather than source, since a source grep cannot
    distinguish a deliberate survivor from an oversight.
  • Not editing vendored data, including the screenshot provenance records that
    describe what is visible inside the images.
  • Parking a question when a tool contradicts itself, rather than picking the
    convenient answer and editing a vendored file on it.
  • Checking a style class exists before using it, and confirming a git mv of a
    binary did not re-encode it.

README

Also corrects the README, which said the product repository still carried the old
name. It has since been renamed to
MaimoryLab/BootAgent, so that
sentence had become false rather than merely stale, and the link now points at the
current slug. The remaining old-name references in this repository are deliberate;
the README now says where that is recorded instead of implying they are pending
cleanup.

Verified with pnpm run build — 47 pages through astro check and
validate-build.mjs. Docs and README only, no source or vendored data touched.

Every entry is a trap this repository actually sprang during the BootAgent
rename, paired with the check that catches it. Kept concrete on purpose —
a checklist of generic advice does not get run.

The two worth knowing before touching anything here:

  - An unauthenticated e2e run reports a green 198 instead of 213 because
    the download-page tests skip themselves once the release feed hits the
    rate limit. Green, with the coverage missing exactly where a change to
    the download path does its damage.
  - The visual baselines did not fail on a rename that changed text inside
    the captured region: the glyph delta fits under maxDiffPixels, which is
    sized for the window edge. They would have kept showing the old product
    name while reporting success. Related, --update-snapshots is a no-op on
    a passing suite; =all is what rebuilds.

Also corrects the README, which claimed the product repository still carried
the old name. It has been renamed to MaimoryLab/BootAgent, so that sentence
had become false rather than merely stale. The remaining old-name references
in this repository are deliberate, and the README now says where that is
recorded instead of implying they are pending cleanup.
@yujiezhang-ops
yujiezhang-ops merged commit e2bb6e8 into main Aug 13, 2026
1 check passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant