ci: make the gate resilient to transient upstream (apt/registry) flakes — retry infra steps only, never tests - #664
Open
baron-3dl wants to merge 1 commit into
Open
Conversation
…es -- retry infra steps only, never tests Wrap only host package-fetch/setup steps and inline apt fetches with retry-with-backoff + apt Acquire::Retries. No test or build-correctness command is retried, so this cannot mask a real red (Rule 8). Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Why
The conductor kept parking on CI failures with a blanket "GitHub apt-mirror outage, just rerun" read. That read is now disproven (real test reds are mixed in — see the flip triage), but the underlying concern is legitimate: the gate does host-level
apt-getand inline package fetches on the runner, and those genuinely flake on GitHub's mirrors. This makes those infra-only steps self-heal so a transient mirror hiccup never again forces a manual rerun.Hard constraint honored (Rule 8)
Retries wrap only package-fetch / dependency-setup, never a test or build-correctness command. A retry around a real test would mask a genuine red — forbidden. No allowlist bypass added;
tests/integration/test_runtime_target.shuntouched.Exactly what was wrapped — and proof each is infra-only
name:)apt-get update(5×, backoff) +Acquire::Retries=5on installapt-get update/install; no test or compile command in itAcquire::Retries=5Acquire::Retries=5docker run … bash -c "apt-get … && bash /test.sh"Acquire::Retries=5to the two apt-gets onlybash /test.shis still run exactly once, not retriedAcquire::Retries=5to update+installcmake build+run_conformance.shthat follow are untouched and run onceAcquire::Retries=5to update+installrun_corpus.shruns onceFor the three inline
docker run … apt && teststeps I deliberately did not add a loop around the wholebash -c, because that would retry the test too.Acquire::Retriesonly changes apt's internal fetch retry — it cannot re-run or mask the test.Not touched (reported as follow-ups, not fixed here)
actions/checkout@v4,docker/build-push-action@v6,docker/setup-buildx-action@v3,docker/setup-qemu-action@v3— bump to Node24-based action majors in a separate housekeeping PR.distro/Dockerfile.bootable(JSONArgsRecommendedat CMD line ~1076;FromPlatformFlagConstDisallowedat line 51). Cosmetic; separate PR.docker run docker.io/library/alpine:3.20 sh -c '…test…'steps: the image pull is coupled into the same invocation as the test, so a safe fix needs adocker pull-with-retry pre-step, not a wrap of the test. Deferred to avoid touching test steps in this PR.🤖 Generated with Claude Code