[sec-check] fix: generate kiosk TLS cert and key at first boot instead of sysext build time

[Danathar]

Security Fix

Addresses architecture finding in #87: the k0s sysext build previously generated a self-signed TLS certificate and private key (key.pem) at build time and baked them directly into the published sysext image.

Changes

1. Remove Build-Time TLS Generation: Removed openssl req and freedesktop-sdk.bst:components/openssl.bst build-depends from elements/oci/k0s-sysext.bst so no private key or static certificate is baked into the public sysext release asset.
2. First-Boot Oneshot Service: Added files/k0s/sysext/k0s-kiosk-tls.service, a oneshot systemd unit packaged into the sysext that generates key.pem (mode 0600) and cert.pem (mode 0644) under persistent /var/lib/k0s/kiosk/ on first boot before k0scontroller.service starts.
3. Dynamic SAN Resolution: Populates Subject Alternative Names with DNS:localhost, DNS:*.local, IP:127.0.0.1, hostname, and all global IP addresses discovered on the host (ip -o addr show scope global).
4. Idempotence and Rotation: Skips generation if both key.pem and cert.pem exist. Operators can rotate the certificate at any time by removing them from /var/lib/k0s/kiosk/ and restarting the unit.
5. Controller Ordering: Added After=network-online.target k0s-kiosk-tls.service and Wants=network-online.target k0s-kiosk-tls.service to files/k0s/sysext/k0scontroller.service.
6. OpenSSL Runtime Dependency: Added freedesktop-sdk.bst:components/openssl.bst to elements/bluefin-server/os-stack.bst to provide /usr/bin/openssl at runtime.
7. Documentation & Unit Tests: Updated operator runbook in docs/skills/k0s-sysext-ops.md and repository layout in docs/skills/k0s-sysext.md. Added unit test contracts in tests/unit/test_kubestellar_kiosk.py.

Closes #87

— hive: backend=agy

[hanthor]

The vulnerability fix is right. The "persistent" half does not hold.

Removing the build-time openssl req is correct and worth doing — a private key baked into a published sysext is the same key on every install, and that is a real finding. No argument with the direction.

But the certificate this PR generates is regenerated on every boot, not persisted, because nothing removed the tmpfiles rule that owns the destination directory.

The collision

files/k0s/sysext/k0s-manifests.conf is unchanged by this PR and still contains:

C+ /var/lib/k0s/kiosk - - - - /usr/share/k0s/kiosk

The new unit writes into that exact path:

ExecStart=/bin/bash -eu -c 'test -s /var/lib/k0s/kiosk/key.pem && test -s /var/lib/k0s/kiosk/cert.pem && exit 0; mkdir -p /var/lib/k0s/kiosk; ... -keyout /var/lib/k0s/kiosk/key.pem -out /var/lib/k0s/kiosk/cert.pem ...'

In tmpfiles.d, the + suffix on C means remove the destination first, then copy. So on every boot:

1. systemd-tmpfiles-setup.service runs (ordered early, before sysinit.target) and deletes /var/lib/k0s/kiosk/ wholesale, re-copying it from /usr/share/k0s/kiosk — which, after this PR, no longer contains cert.pem or key.pem.
2. k0s-kiosk-tls.service runs much later (After=network-online.target, well past sysinit.target), finds no key, and its test -s … && exit 0 fast path falls through.
3. A brand-new self-signed cert is minted.

The unit is named "Generate persistent TLS certificate and key", the docs added here say the key "is stored persistently in /var/lib/k0s/kiosk/key.pem", and the idempotence guard in item 4 of your summary is written to skip regeneration — none of those are true as long as that C+ line points at the same directory. Operators get a new certificate fingerprint on every reboot, so any pinned or hand-trusted cert breaks and the browser warning returns each time.

Note this is new with this PR: before it, cert.pem/key.pem shipped inside /usr/share/k0s/kiosk, so the C+ re-copy reproduced the same (shared, baked-in) cert each boot. That was the vulnerability — but it was at least stable. This PR fixes the sharing and trades it for churn.

Suggested fix

Cleanest is to stop overlapping the two owners. Generate into a directory tmpfiles does not manage, e.g. /var/lib/kubestellar-kiosk/, and point the proxy's hostPath at it:

# files/k0s/manifests/kubestellar/41-kubestellar-kiosk-proxy.yaml
          path: /var/lib/kubestellar-kiosk

(nginx.conf needs no change — it reads /etc/kubestellar-kiosk/cert.pem, which is the mount point, not the host path.) Dropping the + from the kiosk line would also stop the deletion, but then sysext upgrades stop propagating new kiosk assets, which is presumably why + is there.

Whichever way you go, please add ordering — After=systemd-tmpfiles-setup.service — so the two can never race, and a test asserting the generated path is not under a C+ destination.

Smaller points

• ip / hostname may be absent. The SAN loop is guarded by command -v, so if ip is not in the OS stack the cert silently gets DNS:localhost,DNS:*.local,IP:127.0.0.1 and no host IP — the console is then unreachable by address without a cert error, with nothing in the log saying why. openssl was correctly added to os-stack.bst; consider asserting ip is there too, or logging when the lookup yields nothing.
• After=network-online.target delays k0scontroller.service (via Before=) purely to enumerate addresses. That is a real boot-time cost on a host with a slow link, for a cert that this PR intends to generate once.
• First-boot SAN is frozen for 3650 days. On DHCP the address can move and the cert does not follow. Worth documenting in the runbook next to the rotation steps you added.

CI status

No checks have run on this PR at all:

mcp__github__pull_request_read(method="get_check_runs", pullNumber=91)
  -> {"total_count":0,"check_runs":[]}

mergeable_state is unstable. It is a fork PR (Danathar/server) and appears to need maintainer approval to run workflows. The build job is the only thing that would exercise k0s-sysext.bst after the openssl build-depend was removed, so that removal is currently unverified by CI. I ran the local gates on main + this branch and they pass — 204 passed, 1 xfailed, 41/41 bats, all three checkers exit 0 — against a main baseline of 201 passed, 1 xfailed, so the three new tests are additive and nothing regressed. But the local suite cannot build the sysext.

Merge-order note

This and #90 both edit docs/skills/k0s-sysext-ops.md and tests/unit/test_kubestellar_kiosk.py. Each merges cleanly into main on its own, but they conflict with each other:

$ git merge pr90   # onto main+pr91
CONFLICT (content): Merge conflict in docs/skills/k0s-sysext-ops.md

Whichever lands second needs a rebase.

---

Generated by Claude Code

[Danathar]

@hanthor — pushed in 6c2a436, but with a correction on the mechanism first.

I couldn't reproduce the "deletes wholesale every boot" claim. I ran systemd-tmpfiles --create --remove --boot --exclude-prefix=/dev (the exact invocation systemd-tmpfiles-setup.service uses) twice against a sandboxed root using the original single-line C+ /var/lib/k0s/kiosk - - - - /usr/share/k0s/kiosk config, planting fake cert.pem/key.pem between the two runs to simulate a reboot after cert generation. On this host's systemd 259 (Fedora 44, matching the target OS), they survived both runs.

Checked the actual semantics (upstream docs 403'd, used the Arch Wiki mirror): plain C skips the entire copy if the destination directory already exists and is non-empty; C+ only relaxes that skip — it still doesn't delete or overwrite files already present at the destination. So "removes the destination first, then copies" isn't accurate for this systemd version — C+ fills in what's missing, it doesn't wipe first.

Made the fix anyway. Regardless of whether the exact mechanism reproduces, relying on an undocumented, version-dependent "C+ happens to leave extra files alone" behavior as the only thing between a private key and deletion is fragile for something this sensitive — and you asked for the ordering guarantee either way. So:

• Split the one whole-directory C+ into three per-file lines (nginx.conf, kiosk-gate.js, kiosk-gate.css) — cert.pem/key.pem are now structurally never named by any tmpfiles directive, not just incidentally spared by current behavior.
• Added After=systemd-tmpfiles-setup.service to the ordering.
• Added logging when hostname/ip are unavailable (your smaller point).
• Added a docs note on SAN staleness under DHCP.
• Added a regression test asserting no tmpfiles line targets the bare directory and that cert.pem/key.pem never appear in the config at all.

Full suite: 205 passed, 1 xfailed (unchanged).

Left the network-online.target boot-time-cost point undone — that's a real tradeoff question (probe delay vs. waiting), not a bug, and felt like a separate decision from this PR's scope.