Skip to content

CI performance, resilience & prebuilt distribution - #6

Merged
Cro22 merged 2 commits into
masterfrom
feature/ci-perf-improvements
Sep 1, 2026
Merged

CI performance, resilience & prebuilt distribution#6
Cro22 merged 2 commits into
masterfrom
feature/ci-perf-improvements

Conversation

@Cro22

@Cro22 Cro22 commented Sep 1, 2026

Copy link
Copy Markdown
Owner

Five opt-in improvements aimed at CI adoption and lower resource use. None change default behavior. Two commits:

1. CI perf & resilience (b5bf69d)

  • Parallel scenario runneraugur run -concurrency N (alias -j N) runs agent invocations in parallel through the in-process proxy. Per-invocation stdout/stderr is buffered and flushed as a block (no interleaving); the Summary is assembled in deterministic task order. Defaults to 1 (the original sequential path is untouched). Proxy state, trace writer and cassette are already mutex-guarded.
  • Dynamic proxy timeout — replaces the hardcoded 10-minute client timeout with a per-request context deadline (-timeout on run/proxy), overridable per call via the X-Augur-Timeout header for slow reasoning models (o1/o3/r1). Deriving the deadline from the inbound request context keeps client cancellation propagating to the upstream call (no orphaned tokens). -timeout 0 disables it.
  • Opt-in model-alias fallback--normalize-models on aggregate/gate prices a model absent from the snapshot via its longest dash-delimited prefix (gpt-4o-2024-08-06gpt-4o), with a dash-boundary rule so a base name can't swallow an unrelated one. Off by default (an un-priced call is still a hard ErrUnknownModel); every fallback is surfaced in Result.ModelAliases and warned to stderr, so it's never silent.

2. Prebuilt distribution (a7fef9d)

  • GoReleaser cross-compiles linux/darwin/windows × amd64/arm64 (static, CGO off) and publishes augur_<os>_<arch>.tar.gz + checksums.txt to a GitHub release per v* tag. Build metadata is injected via -ldflags.
  • augur version reports the injected build info (from-source builds say dev).
  • Zero-toolchain Actionaction.yml now downloads the release binary matching the pinned tag (github.action_ref) and runner os/arch, sha256-verifies it, and extracts it — so Python/TS/Node agent repos run the gate without a Go toolchain. Transparent source-build fallback for branch/SHA pins, unbuilt platforms, or download failures; new version input forces a tag or source.
  • Release workflow (release.yml) on v* tags; CI gains a goreleaser-check job validating the config on every push.

Not included (from the original proposal)

Deliberately skipped with rationale: context-cancellation propagation was already implemented (proxy.go used NewRequestWithContext(r.Context())); the io.ReadAllLimitReader streaming-hash idea rests on an inaccurate premise (the body must be buffered in full to forward and record it); internal/ restructure and OpenTelemetry ingest are larger, lower-ROI items left for follow-ups.

Verification

  • Full go test ./... green; new tests for runner concurrency (real overlap, ordering, non-interleaved output, abort/continue-on-error), proxy timeout (fires, header override, header stripped), cost.Resolve (exact/fallback/longest-match/dash-boundary), aggregate alias surfacing, and the version command.
  • Action download/checksum/os-arch/version-resolution logic simulated locally (valid + corrupt archive, binary- and two-space checksum formats, tag/branch/source refs). GoReleaser config is schema-checked in CI.

🤖 Generated with Claude Code

Cro22 added 2 commits August 31, 2026 20:12
…lias fallback

Three opt-in improvements aimed at CI adoption, none changing default behavior:

- runner: add Options.Concurrency and `augur run -concurrency/-j N` to run agent
  invocations in parallel. Each invocation's stdout/stderr is buffered and
  flushed as one block so parallel output never interleaves; the Summary is
  assembled in deterministic task order regardless of completion order. Defaults
  to 1 (the original strictly-sequential path is unchanged). The proxy's
  per-(scenario,run) state, the trace writer, and the cassette are already
  mutex-guarded, so parallel invocations record correctly.

- proxy: replace the hardcoded 10m client timeout with a per-request context
  deadline governed by Server.Timeout (flag -timeout on run/proxy) and overridable
  per call via the X-Augur-Timeout header, for slow reasoning models. Deriving the
  deadline from the inbound request context keeps client cancellation propagating
  to the upstream call (no orphaned tokens). -timeout 0 disables the deadline.

- cost: add opt-in Pricing.PrefixFallback + Resolve, matching a model absent from
  the snapshot to its longest dash-delimited prefix (gpt-4o-2024-08-06 -> gpt-4o)
  with a dash-boundary rule so a base name can't swallow an unrelated one.
  aggregate surfaces every fallback in Result.ModelAliases; `-normalize-models`
  on aggregate/gate enables it and warns per normalized model so it is never
  silent. Off by default: an un-priced call is still a hard ErrUnknownModel.

Tests: runner concurrency (real overlap, ordering, non-interleaved output,
abort/continue-on-error), proxy timeout (fires, header override, header stripped),
cost Resolve (exact/fallback/longest-match/dash-boundary), aggregate alias
surfacing. README: new "Tuning for CI" section. Full suite green.
Ship augur as prebuilt, checksum-verified binaries so agent repos (Python/TS/
Node) run the cost gate without a Go toolchain in CI.

- .goreleaser.yaml: cross-compile linux/darwin/windows x amd64/arm64 (static,
  CGO disabled), stable version-free archive names (augur_<os>_<arch>.tar.gz),
  sha256 checksums.txt, and a GitHub release per v* tag. Build metadata is
  injected via -ldflags into main.version/commit/date.
- version_cmd.go: `augur version` reports the injected build info (from-source
  builds honestly say "dev"); it's also the handle the Action uses to confirm a
  downloaded binary runs. Registered in the dispatch table + usage order.
- .github/workflows/release.yml: on a v* tag, run tests then GoReleaser
  (release --clean) to publish the binaries. CI gains a goreleaser-check job so
  the config is validated on every push.
- action.yml: new "Resolve augur binary" step downloads the release asset
  matching the pinned tag (github.action_ref) and runner os/arch, verifies its
  sha256 against checksums.txt (tolerant of coreutils/BSD format variants), and
  extracts it. Falls back to the existing source build for branch/SHA pins,
  unbuilt platforms, or a download failure. New `version` input forces a tag or
  `source`. The from-source path is unchanged, just gated behind need-build.
- README: document prebuilt distribution, the version input, and how to cut a
  release. .gitignore: ignore GoReleaser's /dist/.

The download/checksum/os-arch/version-resolution logic was simulated locally
(valid + corrupt archive, binary- and two-space checksum formats, tag/branch/
source refs). GoReleaser config is schema-checked in CI. Full suite green.
@Cro22
Cro22 merged commit 415051e into master Sep 1, 2026
2 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant