Skip to content

[quality] docs-checks.py main() has zero executed coverage — the docs CI gate's orchestration is untested #105

Description

@kubestellar-hive

Finding

.github/scripts/docs-checks.py is the script .github/workflows/docs-checks.yml runs. Measured coverage over tests/unit/ (coverage.py, --source=.github/scripts):

.github/scripts/check-k0s-version.py     83   1   99%
.github/scripts/check-release-version.py 28   1   96%
.github/scripts/docs-checks.py          136  24   82%   155-182, 186

Lines 155-182 are the whole of main(). tests/unit/test_docs_checks.py drives the helpers directly — load_fm, validate_frontmatter, check_budget, check_stale_flags, check_internal_links, validate_skill — and never runs the orchestration that wires them together.

The orchestration is what makes the gate a gate. Untested today:

  • which documents are budget-checked and with which limits (AGENTS.md 200/150, README.md 200/150, CONTRIBUTING.md 100/80, docs/MVP_1_0_READINESS.md 250/200, docs/DOCUMENTATION_OVERHAUL_PLAN.md 3000/2500)
  • that every file under docs/skills/ is validated and link-checked, not just the first
  • that the top-level docs are stale-flag and internal-link checked, and that DOCUMENTATION_OVERHAUL_PLAN.md is deliberately outside both of those lists
  • the process contract itself: sys.exit(1) plus a FAIL line when anything errored, exit 0 with WARN lines when only warnings accumulated
  • errors/warnings are module-level globals, so nothing asserts they are not inherited across a run

Failure mode: a regression that drops a budget target, stops iterating the skills glob after the first entry, or loses the sys.exit(1) leaves the docs-checks workflow green while checking less. Nothing in the suite would notice.

Recommendation

Add tests/unit/test_docs_checks_main.py driving main() end to end against the isolated tmp tree that tests/unit/conftest.py already builds (the docs_checks fixture repoints ROOT/DOCS_DIR/SKILLS_DIR and clears the globals, so no new fixture machinery is needed). Assert the exit code and stdout contract, parametrise over the five budget targets at limit+5 / limit / warn+1 / warn boundaries, and pin the stale-flag and link-check scopes.

A hold-gated PR implementing exactly this is attached below. It adds one new file and takes docs-checks.py from 82% to 99% (the remaining line is the if __name__ == "__main__" guard). It does not touch tests/unit/test_docs_checks.py, tests/unit/test_k0s_manifests.py, or tests/unit/test_kubestellar_kiosk.py.

Priority

  • Impact: medium — the gate is advisory-quality until its entry point is exercised; the risk is silent scope loss, not a live failure
  • Effort: low — one new test file, no production change, existing fixtures suffice

Filed by quality agent (hold-gated mode)

🐝 Hive Agent: quality | Instance: hosted-projectbluefin-knuckle-gjvq | SHA: b751215

— hive: agent=quality backend=copilot model=claude-opus-5 copilot=1.0.78

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    agent/qualityFiled or owned by the quality agent.hive/hosted-projectbluefin-knuckle-gjvqRouted by the hosted Project Bluefin Hive deployment.qualityCode quality or test-coverage work.testingTest authoring or test infrastructure.

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions