Skip to content

test(lifecycle): add settled-deployment barrier and activate bootc pin scenario - #837

Open
mrbobbytables wants to merge 2 commits into
projectbluefin:mainfrom
mrbobbytables:settled-deployment-bootc-pin
Open

mrbobbytables wants to merge 2 commits into
projectbluefin:mainfrom
mrbobbytables:settled-deployment-bootc-pin

Conversation

@mrbobbytables

Copy link
Copy Markdown
Contributor

Summary

Addresses #832:

  • Settled-deployment barrier step: Implemented Deployment is settled (and bootc deployment is settled) in tests/lifecycle/features/steps/steps.py. It polls sudo bootc status --format=json over SSH with a bounded 120s deadline (5s poll interval) until a valid booted deployment is present and no staged deployment is in progress (status.staged is None), providing clear diagnostic failure messages on timeout or invalid status.
  • Activate scenario: Inserted the barrier before sudo bootc pin in tests/lifecycle/features/bootc.feature and activated the scenario by dropping @future.
  • Unit test coverage: Added unit test suites TestIsDeploymentSettled and TestDeploymentIsSettled in tests/unit/test_lifecycle_steps.py testing pure parsing logic, transient retry handling, and timeout behavior.
  • Skill update: Documented the settled-deployment barrier pattern in docs/skills/test-authoring/bootc/SKILL.md.
  • Coverage snapshot & review docs: Updated scripts/update_coverage_snapshot.py, regenerated docs/skills/test-authoring/suite-map/SKILL.md (lifecycle active: 30, pending/future: 3), and synchronized docs/qa-review.md.

Validation

  • ruff check tests/ scripts/ --select E,F,W --ignore E501 passed cleanly.
  • behave --dry-run tests/lifecycle/features --name "bootc can pin" verified step resolution.
  • python3 -m pytest tests/unit/ -q (1586 tests passed).
  • python3 scripts/validate_docs.py and python3 scripts/generate_skill_index.py --check passed.
  • python3 scripts/update_coverage_snapshot.py --check passed.
  • Note: Manual workflow dispatch (manual.yml/upgrade-test.yml) via API is blocked for contributor agents in this environment; full local test suite and gate validation executed above.

Fixes #832

— hive: backend=copilot model=gemini-3.8-flash

🐝 Hive Agent: contributor | SHA: e7b05bd4

@codecov-commenter

Copy link
Copy Markdown

⚠️ Please install the 'codecov app svg image' to ensure uploads and comments are reliably processed by Codecov.

Codecov Report

❌ Patch coverage is 98.56115% with 2 lines in your changes missing coverage. Please review.

Files with missing lines Patch % Lines
tests/lifecycle/features/steps/steps.py 95.65% 2 Missing ⚠️

📢 Thoughts on this report? Let us know!

…n scenario

- Add 'Deployment is settled' barrier step definition in steps.py polling
  'sudo bootc status --format=json' over SSH until booted is present and no
  staged deployment is in progress (with bounded 120s deadline).
- Insert barrier before 'sudo bootc pin' in bootc.feature and activate the
  scenario by dropping @future.
- Add comprehensive unit test coverage in test_lifecycle_steps.py.
- Document settled-deployment barrier pattern in docs/skills/test-authoring/bootc/SKILL.md.
- Regenerate coverage snapshot and sync docs/qa-review.md.

Fixes projectbluefin#832

Assisted-by: Gemini 3.8 Flash via GitHub Copilot
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Signed-off-by: mrbobbytables <mrbobbytables@users.noreply.github.com>
@castrojo
castrojo force-pushed the settled-deployment-bootc-pin branch from e7b05bd to 93831bb Compare September 18, 2026 16:46
The rebase regenerated this branch's coverage snapshot to 519/407/112,
but the qa-review prose still quoted the pre-rebase 526/414 figures.
qa-review.md is not written by scripts/update_coverage_snapshot.py, so
it has to be re-derived by hand whenever the generated block moves.

Assisted-by: Claude Opus 4.5 via pi
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

@castrojo castrojo left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Backlog triage pass. I have not approved and have not merged or enqueued anything: .github/copilot-instructions.md says agents never approve, merge, or enqueue, and AGENTS.md:136,163 + docs/skills/meta/human-gates/SKILL.md make merge a human gate. This is a review comment only.

Checked the mechanics against the tree rather than the description, and they hold. tests/lifecycle/features/steps/steps.py:14-24 already imports json, subprocess, from time import sleep, time and run_ssh, so the new barrier's dependencies all resolve; tests/shared/ssh_steps.py:15 confirms run_ssh returns (stdout, returncode), matching the stdout, rc = run_ssh(...) unpack; and subprocess.TimeoutExpired really is re-raised by run_ssh, so that except arm is reachable rather than dead. Polling sudo bootc status --format=json until .status.booted exists and .status.staged is absent is the right shape for the pin race, and dropping @future from that one scenario matches the snapshot delta exactly.

One fix I pushed. After the rebase regenerated the coverage snapshot to 519 scenarios ... 407 active ... 112, docs/qa-review.md still quoted the pre-rebase 526 / 414 / 112. qa-review.md is not written by scripts/update_coverage_snapshot.py (its SUITE_MAP constant targets suite-map/SKILL.md only), so it has to be re-derived by hand whenever the generated block moves. I resynced it in a separate commit; the two now agree.

Two minor smells left for you, neither blocking: the bare except Exception in the poll loop will swallow a genuine JSON-shape change as "not settled yet" until the deadline, and the {timeout:d} variant step is added with no scenario using it.

Sequencing: #836 also edits docs/qa-review.md lines 4-5. Both now carry correct numbers for their own trees, but they are different numbers (#836 is 519/406/113, this is 519/407/112 because it activates one scenario), so whichever lands second must re-derive rather than take either side of the conflict.

What I changed on this branch. It was CONFLICTING. The only conflicted file was docs/skills/test-authoring/suite-map/SKILL.md, and the conflict was entirely inside the <!-- coverage-snapshot:start --> / :end --> markers — the generated block. I rebased onto current main and re-ran python3 scripts/update_coverage_snapshot.py rather than hand-resolving the counts, which is what docs/skills/ci-ops/contributing/SKILL.md:52 asks for. Your own changes are untouched; force-pushed with --force-with-lease against the exact head, and I confirmed no PR head moved during the pass.

Verified locally before pushing and again on the fresh CI run: ruff check tests/ scripts/ --select E,F,W --ignore E501 clean, scripts/update_coverage_snapshot.py --check fresh, scripts/validate_docs.py passing, scripts/generate_skill_index.py --check in sync, and the full tests/unit/ suite green.

Outstanding: two approving reviews, plus the repo-wide ghost-lab outage.

Gate status for this PR

  • Ruleset main — merge queue requires 2 approving reviews (dismiss_stale_reviews_on_push: true, require_last_push_approval: true, empty bypass_actors). Required checks: Lint & syntax, Behave dry-run, pytest.
  • I pushed to this branch, so under require_last_push_approval the two approvals must both come from someone other than me.
  • ghost-lab is red repo-wide and that is not this PR's fault: every lab workflow in the Argo namespace is Failed/Error since 2026-09-15 across testsuite, common, knuckle and bluefin-lts; the last success anywhere was testsuite #790 on 2026-09-15. docs/skills/meta/human-gates/SKILL.md:80-107 still makes it a merge gate, so I am naming it rather than waiving it.
  • On AGENTS.md:146 (matching docs/skills/** update): AGENTS.md calls it a mandatory gate, docs/skills/ci-ops/contributing/SKILL.md:78 calls it a review expectation that no CI job enforces. I filed #861 for that contradiction rather than guessing which reading binds.

@kubestellar-hive kubestellar-hive Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Reviewed — no findings from this perspective.

— hive: agent=reviewer backend=copilot model=claude-fable-5 copilot=1.0.78

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

test(lifecycle): add settled-deployment barrier and activate the bootc pin @future scenario

3 participants