fix(init): scaffolded CI key guidance names policy:read - #24
Merged
Merged
Conversation
The scaffolded workflow (evalshift init --ci) defaults to fail-on: policy, which reads the hosted policy-check endpoint (requires policy:read). The scaffold and every doc copy of the CI key guidance named only run:create + run:read, so a key minted per that guidance got a 403 on the policy check and the action silently fell back to fail-on: regression instead of the intended gate. Add run:create + run:read + policy:read to the scaffolded workflow comment, DOCS.md, llms-full.txt, docs/github-action.md, and docs/hosted.md. Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
The CI key that
evalshift init --ciscaffolds, and that the docs describe, wasrun:create+run:read. That's not enough for the default gate:fail-on: policyreads the run's verdict fromGET /runs/{id}/policy-check, which requirespolicy:read(serverapp/policy/service.py).fail-on: regression, printing only a warning.This PR changes the guidance to
run:create+run:read+policy:readin:docs/github-action.mdanddocs/hosted.md.There's also a CHANGELOG
### Fixedentry.project:readis not needed.This change is correct against today's server and action, so it can merge independently of the preflight redesign (evalshift-server / evalshift-action PRs).
Gate:
make ciis green (2352 tests pass). An independent review confirmed the 403 fallback against the server and action code.🤖 Generated with Claude Code