feat(ddi): build OS payload as verity /usr image pinned by verity.usrhash

[mrbobbytables]

Summary

Builds the OS payload as a dm-verity /usr image pinned by verity.usrhash in the target UKI cmdline (#135, Phase 4 of the Flatcar base migration design in PR #123).

Changes

• elements/oci/bluefin-server-ddi.bst:
  • Adds freedesktop-sdk.bst:components/cryptsetup.bst to build-depends for veritysetup.
  • Enforces the hard 1,065,345,024-byte size budget before creating the filesystem, failing loudly if the payload content exceeds the limit.
  • Pre-allocates the image to the exact budget offset (HASH_OFFSET=1065345024) and creates an XFS filesystem formatted with mkfs.xfs -f -L bluefin-usr.
  • Formats and appends the dm-verity hash tree at offset 1,065,345,024 via veritysetup format (260,094 blocks of 4096 bytes).
  • Prints the root hash to build output and saves verity.usrhash and bluefin-server-ddi-<version>.verity into the install root.
• elements/oci/bluefin-server-installer.bst:
  • Reads the root hash from /ddi/verity.usrhash staged from the DDI element.
  • Updates the target OS UKI command line in dracut and ukify build to include verity.usr=PARTLABEL=USR-A verity.usrhash=${USR_HASH}.
• Tests:
  • Updates tests/unit/test_installer_contract.py to assert verity.usr=PARTLABEL=USR-A and verity.usrhash=${USR_HASH} in the target UKI command line.
  • Adds tests/unit/test_ddi_verity.py covering cryptsetup build dependency, budget enforcement, veritysetup formatting arguments, root hash emission, and target UKI cmdline pinning.
• Documentation:
  • Updates docs/skills/ddi-installer.md to document the appended verity hash tree and verity.usrhash pinning.

Verification

• pytest tests/unit: 217 passed, 1 xfailed.
• bats tests/unit: 62 passed.
• python3 .github/scripts/docs-checks.py: passed.
• python3 .github/scripts/check-release-version.py: passed.
• python3 .github/scripts/check-k0s-version.py: passed.

Closes #135

— hive: backend=copilot model=gemini-3.8-flash

🐝 Hive Agent: contributor | SHA: 8d4909c

[castrojo]

Reviewed with the failing build reproduced from the workflow log rather than inferred. Summary: the build failure here is real and PR-specific, but it is not a defect and it is not minimally fixable. This needs a design decision, not a patch.

What is actually failing. Build job 104226687479 (run 34920290427):

Computed root filesystem size: 1770078417 bytes (budget: 1065345024 bytes)
ERROR: /usr payload size (1770078417 bytes) exceeds the hard verity budget of 1065345024 bytes
[00:00:08] FAILURE oci/bluefin-server-ddi.bst: Running commands
error: recipe `build-ddi` failed on line 89 with exit code 255

That is this PR's own guard firing, exactly as issue #135 specifies it should ("The element must fail the build when exceeded, never silently overrun the hash offset"). The guard is working.

The budget constant is a live external contract, not a tunable. I checked 1065345024 against upstream flatcar/bootengine minimal-init line 137, which hardcodes --hash-offset=1065345024 with the comment "Hardcoded expected value from the image GPT layout". include/flatcar.yml pins Flatcar 4593.2.5, so this offset is dictated by the init we boot, and raising it would produce an image Flatcar's minimal-init cannot open — trading a loud build failure for a silent boot failure.

The payload genuinely does not fit, by 705 MB. 1,770,078,417 against 1,065,345,024. This is not drift introduced by the PR: the same OS stack on main produces a 1.1 GB compressed DDI, and docs/skills/ddi-installer-build.md already states "The DDI is 2 GiB+". Measuring /layer/usr rather than /layer does not close it either — the only non-/usr content is the /etc seed this element writes (shadow, passwd, group, fstab, hosts, securetty, two SSH host keys, a CA bundle), well under a megabyte. /usr is essentially the entire 1.77 GB.

So every route to green is a redesign: raise the offset (breaks boot), strip 705 MB out of elements/bluefin-server/os-stack.bst (changes what the OS ships), or abandon Flatcar's minimal-init for systemd's own usrhash=/repart Verity=data|hash model (a different architecture). None of those is mine to choose on your behalf, which is why I have not pushed anything.

Two further blockers the next reviewer should know about.

• verity.usr=PARTLABEL=USR-A references a partition that does not exist yet. USR-A appears nowhere in files/ or elements/ on main; the USR-A/USR-B layout is issue #134, still open. As written, the pinned kernel argument cannot resolve on a real install.
• The new tests/unit/test_ddi_verity.py is green and cannot ever catch this failure, because it greps the element text for HASH_OFFSET=1065345024 — it asserts the constant is present, not that the payload fits. That is why unit is green while build is red. In particular test_verity_budget_and_blocks_alignment asserts 1065345024 // 4096 == 260094, which is a tautology between two constants declared in the same file and would pass no matter how broken the element became.

Not approving. This is blocked on a size/architecture decision plus #134, not on review bandwidth.

Unrelated: installer-test is skipped here because build failed first; it is independently red on main and is being addressed by #173.

[castrojo]

Closing on the lane above: the change needed to make this land is a design decision I cannot make on your behalf, so I am closing with that reason rather than guessing at your intent or leaving it parked indefinitely.

To restate the blocking constraint from my review: the /usr payload is 1,770,078,417 bytes against a verity budget of 1,065,345,024 bytes — a 705 MB overrun. The budget is not a tunable; it matches the --hash-offset=1065345024 hardcoded in upstream flatcar/bootengine minimal-init line 137 ("Hardcoded expected value from the image GPT layout"), and include/flatcar.yml pins Flatcar 4593.2.5, so the init we boot dictates it.

The three routes to green are all architectural, and each is yours to choose, not mine:

1. Shrink /usr by 705 MB, which means deciding what elements/bluefin-server/os-stack.bst stops shipping.
2. Abandon Flatcar's minimal-init for systemd's own usrhash= / repart Verity=data|hash model.
3. Change the partition layout so the hash offset differs — which is issue feat(installer): adopt Flatcar's EFI-SYSTEM/USR-A/USR-B/ROOT partition layout #134 (USR-A/USR-B), still open, and without which verity.usr=PARTLABEL=USR-A cannot resolve at all, since USR-A exists nowhere in files/ or elements/ today.

Nothing about the work here is wasted and none of it is wrong — the guard behaves exactly as issue #135 requires, and the constant is correct. It is sequenced ahead of its prerequisites.

Please reopen once #134 has landed and the size question has an answer, or push back here if you think one of the three routes is already settled and I have missed it. If you would rather keep this open as a tracking branch for phase 4 of the Flatcar migration, say so and I will reopen it immediately — closing is about not leaving it in a false "awaiting review" state, not about discarding the approach.

One concrete thing worth carrying forward regardless: tests/unit/test_ddi_verity.py cannot catch this class of failure, because it greps the element for HASH_OFFSET=1065345024 rather than checking the payload fits. test_verity_budget_and_blocks_alignment in particular asserts 1065345024 // 4096 == 260094, a tautology between two constants in the same file. Whatever shape phase 4 finally takes, that suite needs an assertion tied to actual payload size or it will stay green through exactly the breakage it exists to prevent.