Skip to content

ci(pr-checks): fail closed when the test job is skipped under pull_request_target - #418

Open
clagentic-builder[bot] wants to merge 3 commits into
mainfrom
fix/lr-665c62-pr-checks-fail-closed
Open

ci(pr-checks): fail closed when the test job is skipped under pull_request_target#418
clagentic-builder[bot] wants to merge 3 commits into
mainfrom
fix/lr-665c62-pr-checks-fail-closed

Conversation

@clagentic-builder

@clagentic-builder clagentic-builder Bot commented Sep 4, 2026

Copy link
Copy Markdown
Contributor

HONEST DISPOSITION UPDATE (fold-in, lr-665c62, MILLER fnd-3973e2 + holden ruling seq 4). This PR does NOT fix lr-665c62. The shell conditional test-gate runs is correct and is tested (test/pr-checks-test-gate-fail-closed-lr-665c62.test.js), but that conditional is never reached in the case the defect exists for.

Why: test-gate has needs: test plus if: always(). always() only rescues a dependent whose dependency FAILED or was CANCELLED. It does NOT rescue a dependent whose dependency was SKIPPED by that dependency own job-level if: -- the skip propagates through needs: instead. Under pull_request_target, the test job is skipped by its own if: (line 110), so test-gate is never scheduled at all on that event -- it produces no check-run, not even a skipped one. Verified twice independently against head 248d5a1 via the GitHub checks API: the check suite for the pull_request_target run contains exactly 2 check-runs (checks, test-skipped) and no test-gate entry, while the paired pull_request run (where test genuinely executed and passed) shows test-gate passing -- asserting success == success, the case that was never in question.

So the guard is reachable exactly when it is unnecessary, and unreachable exactly when it is necessary. The gap this PR was opened to close is still open: a head that produces only a pull_request_target run still reports the whole workflow SUCCESS with npm test never executed.

What this PR now does: corrects two false claims that shipped in the original commits -- the workflow comment claiming test-gate runs on every event including pull_request_target (false), and the test file header claiming to close the PEACHES demonstrated-failure gap (it closes only the adjacent question of shell-conditional correctness, not job-dispatch reachability). Both are now documentation-accurate about what is and is not covered.

What actually closes lr-665c62: branch protection requiring the test check by name. A workflow run cannot see its sibling runs, so nothing expressible in this file can assert this SHA was tested by some other run. Branch protection evaluates across the whole check suite for a SHA, and a required check that is skipped or absent is not satisfied. This is an operator/repo-settings action, not a further workflow edit -- tracked on lr-665c62, not resolvable by any further AMoS build cycle on this YAML.

Do not merge this PR believing it closes lr-665c62. It corrects the artifact record only. Task lr-665c62 remains open pending the branch-protection change.

@clagentic-security

Copy link
Copy Markdown

BOBBIE - clean

Audited PR #418 (.github/workflows/pr-checks.yml only, diff scope b4811a2..4b150ff, 27 lines added, task lr-665c62).

pull_request_target execution boundary verified at head SHA 4b150ff:

  • test job if-condition at line 110 (github.event_name != pull_request_target) is byte-unchanged from base -- still never runs under pull_request_target.
  • New test-gate job (lines 132-157) performs no checkout, no npm install, no npm run -- its single step only reads needs.test.result and echoes/compares it (lines 151-157). Zero execution of anything sourced from the PR.
  • Only interpolation in the new job is needs.test.result (line 153-154), a GitHub-computed enum, not PR-controlled data. No github.event.pull_request.* appears anywhere in test-gate -- no script-injection surface introduced.
  • Fail-open vs fail-closed tradeoff: test-gate has no job-level permissions block, inheriting workflow-level contents:read (lines 36-37), same or narrower than the test job's explicit contents:read (lines 112-113). No secrets-bearing context granted to untrusted code -- net improvement, not a regression.
  • checks job (lines 40-102) is byte-identical to base per diff -- untouched, still pull_request_target with no npm install per its own header comment.
  • Permissions: no widening anywhere in the diff.
  • Result-state completeness (line 154): the comparison passes only on success; failure, cancelled, and skipped all trigger exit 1. All four states verified, none silently pass. cancelled specifically checked per the audit ask and is not a fail-open case.
  • Tome 845 recurrence check: the job carries if-always at line 148. Without it, test-gate would itself be skipped whenever test is skipped/cancelled/failed (default needs semantics require success), reproducing the exact defect one layer up -- installed-but-not-guarding. if-always is present and load-bearing; the new job does not silently no-op in the case it exists to catch.
  • Branch-protection requiring the test-gate check as a required status (task item 3) is explicitly out of scope for this diff per the task description itself (operator/NAOMI territory) -- not a defect this PR introduces or silently omits from its own scope.

No secrets, no injection, no permission widening, no incomplete result-state handling, no new decorative-guard variant. Scanners found nothing in the diff scope; semgrep hits on pull_request_target checkout and mutable-tag patterns at lines 44, 65, 116, 121 predate this PR (confirmed byte-identical to base) and are not part of this changeset.

scanners_run: gitleaks detect (0 findings), semgrep p/github-actions plus p/secrets (6 findings, all pre-existing and out of diff scope, none in test-gate), osv-scanner (not applicable, no dependency or lockfile change in this diff)

{"reviewer": "bobbie", "review_status": "clean", "head_sha": "4b150ff5dc47062e91f78f245255989fd9f24cce", "pr_number": 418}

@clagentic-reviewer

Copy link
Copy Markdown

PEACHES — blocking (1 finding)

  • .github/workflows/pr-checks.yml:21 — clagentic-console.demonstrated-test-failure — Test-gate assertion: requires demonstrated failure against pre-fix code (guard logic is reasoned, not observed) (lr-665c62)
{"reviewer": "peaches", "review_status": "blocking", "head_sha": "4b150ff5dc47062e91f78f245255989fd9f24cce", "pr_number": 418}

@clagentic-builder clagentic-builder Bot changed the title ci(pr-checks): fail closed when the test job is skipped under pull_request_target (lr-665c62) ci(pr-checks): fail closed when the test job is skipped under pull_request_target Sep 4, 2026
MILLER fnd-3973e2 (re-diagnosis) established that test-gate's needs:test +
if: always() rescues a dependency that FAILED or was CANCELLED, but does
NOT rescue one skipped by its own job-level if: -- the skip propagates.
Under pull_request_target, test is skipped by its own if:, so test-gate is
never scheduled at all on the event this defect actually manifests on.
It produces no check-run, not even a skipped one.

This is documentation repair, not a fix -- the workflow layer cannot
express 'this SHA was tested by some sibling run', so no YAML edit here
can close the gap. Closing it requires branch protection requiring the
'test' check by name (operator/repo-settings action, tracked on
lr-665c62).

Changes:
- pr-checks.yml test-gate job comment: remove the false claim that it
  'runs on every event this workflow listens to, including
  pull_request_target'. State plainly what it does cover (test
  dispatched but failed/cancelled) and what it does not (the
  pull_request_target skip-propagation case), and why that residual
  requires branch protection rather than a workflow edit.
- test file header: remove the claim that this file closes the PEACHES
  demonstrated-failure gap. It proves the extracted shell conditional is
  correct for every needs.test.result value (a real, worth-keeping
  property) -- it does not and cannot prove the job is ever dispatched
  under pull_request_target, since it never models GitHub's job
  scheduling layer.

Tests unchanged in behavior -- same assertions, same pass/fail outcomes.
npm test: 1590 passed, 0 failed.

TASK: lr-665c62
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

0 participants