Skip to content

feat(tests): add comprehensive Bluefin Server OS test suite - #804

Closed
castrojo wants to merge 4 commits into
mainfrom
feat/server-tests
Closed

castrojo wants to merge 4 commits into
mainfrom
feat/server-tests

Conversation

@castrojo

@castrojo castrojo commented Sep 9, 2026

Copy link
Copy Markdown
Collaborator

Description

Adds comprehensive test coverage for Bluefin Server OS under tests/server/:

  • test_base_os_rootfs.py: XFS DDI rootfs, uutils-coreutils userspace, bash login shell, /var mount unit
  • test_sysupdate.py: systemd-sysupdate / updatectl contracts, mandatory @v wildcards in MatchPattern, GPG verification
  • test_sysext_k0s.py: k0s sysext activation, extension overlay merge into /usr, declarative manifests seeding
  • test_kubestellar_console.py: KubeStellar kiosk proxy hostPort 8080, /healthz, local dev login options alongside GitHub OAuth
  • test_installer_boot.py: systemd-sysinstall interactive/unattended flows, target ESP /EFI/BOOT/BOOTX64.EFI fallback staging
  • test_ssh_security.py: OpenSSH present for on-demand diagnostics, disabled by default via systemd presets
  • test_server_contracts.bats & test_bats_runner.py: Shell contracts for disk detection and boot cmdline

All 75 tests pass cleanly in 0.31s.

castrojo and others added 4 commits September 9, 2026 12:23
Add 75 test cases covering base OS DDI, sysupdate contracts, k0s sysext activation, KubeStellar console with local login, UEFI fallback bootloader staging, and SSH security defaults.

Assisted-by: Gemini 3.8 Flash via GitHub Copilot

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
…ret refs

Assisted-by: Gemini 3.8 Flash via GitHub Copilot

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Assisted-by: Gemini 3.8 Flash via GitHub Copilot

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Signed-off-by: sec-check[bot] <sec-check[bot]@users.noreply.github.com>
@codecov-commenter

Copy link
Copy Markdown

⚠️ Please install the 'codecov app svg image' to ensure uploads and comments are reliably processed by Codecov.

Codecov Report

❌ Patch coverage is 0% with 762 lines in your changes missing coverage. Please review.

Files with missing lines Patch % Lines
tests/server/test_base_os_rootfs.py 0.00% 163 Missing ⚠️
tests/server/test_sysupdate.py 0.00% 137 Missing ⚠️
tests/server/test_kubestellar_console.py 0.00% 127 Missing ⚠️
tests/server/test_sysext_k0s.py 0.00% 120 Missing ⚠️
tests/server/test_installer_boot.py 0.00% 95 Missing ⚠️
tests/server/test_ssh_security.py 0.00% 61 Missing ⚠️
tests/server/conftest.py 0.00% 43 Missing ⚠️
tests/server/test_bats_runner.py 0.00% 16 Missing ⚠️

📢 Thoughts on this report? Let us know!

@castrojo
castrojo enabled auto-merge September 10, 2026 01:58

@hanthor hanthor left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

"All 75 tests pass cleanly in 0.31s" — they skip

tests/server/conftest.py hardcodes a personal absolute path:

DEFAULT_SERVER_PATH = Path("/home/jorge/src/server")

and server_root calls pytest.skip(...) when neither $BLUEFIN_SERVER_ROOT, that path, nor a sibling ../server checkout exists. I copied tests/server/ into a directory with no sibling server repo and ran it:

75 skipped in 0.15s
SKIPPED [1] .../test_sysupdate.py:217: Bluefin Server repository not found.
  Set BLUEFIN_SERVER_ROOT or place at /home/jorge/src/server

On a GitHub runner there is no projectbluefin/server checkout, so every one of these 75 tests is a skip. The "passes" reported in the description came from a machine that happened to have the sibling repo.

No CI job runs them anyway

$ grep -rn "tests/server\|pytest" .github/workflows/
.github/workflows/unit-tests.yml:46:  coverage run -m pytest -n auto tests/unit/ -v

unit-tests.yml collects only tests/unit/. tests/server/ is never executed by any workflow, so even the skips are invisible and a regression in these assertions would never surface. coverage report --fail-under=75 also ignores them.

What this needs before it can land

  1. Remove /home/jorge/src/server. Resolve the server repo from $BLUEFIN_SERVER_ROOT only, or vendor the small config fixtures the assertions read (repart.d/*.conf, the .bst elements) into this repo so the tests are self-contained.
  2. Wire the suite into CI — either add tests/server/ to unit-tests.yml, or add a job that checks out projectbluefin/server and sets BLUEFIN_SERVER_ROOT. Until then this is 1,437 lines that no gate enforces.
  3. test_bats_runner.py also pytest.skips when bats is not on PATH, which it is not on ubuntu-latest by default — install it in the job or drop the wrapper.

Mechanically the assertions themselves look reasonable (string/INI checks against bluefin-server-ddi.bst, 20-root-a.conf, etc.); the problem is purely that nothing runs them.


Generated by Claude Code

@castrojo

Copy link
Copy Markdown
Collaborator Author

Closing. The work is not wrong, but what it needs before it can land is a design decision I should not make on the author's behalf, and @hanthor's changes-requested review has not been addressed.

Verifying his three points against the current diff:

  1. tests/server/conftest.py hardcodes DEFAULT_SERVER_PATH = Path("/home/jorge/src/server") and server_root calls pytest.skip(...) when that path, $BLUEFIN_SERVER_ROOT and a sibling ../server checkout are all absent. On a GitHub runner none of those exists, so all 75 tests skip. The "All 75 tests pass cleanly in 0.31s" in the description came from a machine that happened to have the sibling repo; @hanthor reproduced 75 skipped in 0.15s.
  2. No workflow runs them. .github/workflows/unit-tests.yml:43 collects tests/unit/ only. tests/server/ is never executed, and coverage report --fail-under=75 ignores it.
  3. test_bats_runner.py skips when bats is absent, which it is on ubuntu-latest.

Point 1 is a small fix I could have made. Points 2 and 3 are not: wiring a new suite into CI means either extending unit-tests.yml's collection scope or adding a job that checks out projectbluefin/server and installs bats. Adding a new directory under tests/ is explicitly a Design gate in docs/skills/meta/human-gates/SKILL.md ("Adding a new test suite (new directory under tests/) — affects the variant matrix, runner image, and coverage docs"), and a new CI job that checks out a second repository is a CI-interface change on top of that. Guessing at which shape you want would be inventing a decision, not fixing a defect.

For the record I did rebase this branch onto current main locally and it is clean — 1609 unit tests pass, ruff is clean, the coverage snapshot regenerates correctly. The blocker is not mechanical.

To land it: open the design issue for the server suite (scope, which repo supplies the fixtures, which job runs it), then reopen or re-file with DEFAULT_SERVER_PATH resolved from $BLUEFIN_SERVER_ROOT only — or with the repart.d/*.conf and .bst fixtures vendored so the suite is self-contained — and the CI job included in the same PR.

@castrojo castrojo closed this Sep 18, 2026
auto-merge was automatically disabled September 18, 2026 16:50

Pull request was closed

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants