ci: add actionlint and a test-environment guard - #48
Merged
Conversation
Two guardrails for two silent failures that both happened here.
actionlint. A duplicate env: key in cmcp's release.yml made GitHub
refuse to load the workflow. It reported that as a "workflow file issue"
on unrelated pushes, the release event ran nothing, and a version sat
unpublished for hours. yaml.safe_load keeps the last value for a
duplicate key without complaining, so local validation passed.
actionlint reports it directly:
key "env" is duplicated in element of "steps" section
Fetched by pinned version and checksum-verified rather than run as a
third-party action, so a check that exists to guard the workflow supply
chain does not add to it. Runs only when .github/workflows changes.
Every repo is clean against it today.
Test-environment guard. pytest puts the source tree on the path, so an
in-process import always finds it. A test that shells out does not get
that: a plain subprocess resolves the distribution normally and, with a
released wheel also installed, finds site-packages. The subprocess then
exercises a published version while the suite reports a pass. That is
how a tutorial test graded against an old schema and looked green.
conftest probes a subprocess once per session and fails loudly if the
package resolves outside the repository. CI installs editable, so it is
a no-op there. A package not importable from a subprocess at all is
fine: that is a path-only setup, not a shadowing install.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01XbDBXDWWvMFa7c2jGgyq9t
tests are run with python -m unittest discover, which never loads a conftest.py, so the guard would have been dead code sitting in the tree looking like protection. actionlint still applies. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01XbDBXDWWvMFa7c2jGgyq9t
imran-siddique
force-pushed
the
ci/guardrails
branch
from
September 7, 2026 00:36
1fa14fc to
b3806e5
Compare
Merged
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Two guardrails, each for a silent failure that actually happened during this security sweep.
1. actionlint
I introduced a duplicate
env:key in cmcp'srelease.ymlwhile hardening tag interpolation. GitHub Actions refuses to load a workflow with a duplicate key, reported it as a "workflow file issue" on unrelated pushes, ran nothing for the release event, and left a version unpublished for hours.It got through because
yaml.safe_loadkeeps the last value for a duplicate key without complaining. actionlint says it plainly:Verified against a reproduction of the exact broken file before adding it.
On how it's installed: fetched by pinned version with a checksum verification, not run as a third-party action. actionlint publishes no official action, and a check whose whole purpose is guarding the workflow supply chain should not add a new dependency to it.
Triggered only on changes under
.github/workflows, so it costs nothing on ordinary PRs. Every repo in the org is clean against it today, so this adds no backlog.2. Test-environment guard
pytest puts the source tree on the path, so an in-process import always finds this tree and looks right. A test that shells out gets no such help:
subprocess.run([sys.executable, ...])resolves the distribution normally, and with a released wheel also installed it finds site-packages.The subprocess then exercises a published version while the suite reports a pass. That is exactly how
trace-spec's tutorial test graded against the previous schema and still looked green, and whycmcp's distribution smoke test reported a version that had nothing to do with the tree.The guard probes a subprocess once per session and fails with an actionable message:
It is a no-op in CI, which installs editable, and verified as such against a correctly configured local venv. A package that is not importable from a subprocess at all is deliberately allowed: that is a path-only setup, not a shadowing install, so nobody gets locked out of running the suite.
Applied to the seven repos that install their own package editable in CI.
examples,integrationsanddemostest against third-party distributions on purpose and get actionlint only.🤖 Generated with Claude Code
https://claude.ai/code/session_01XbDBXDWWvMFa7c2jGgyq9t