Skip to content

development.md

Latest commit

 

History

History
378 lines (295 loc) · 14.1 KB

development.md

File metadata and controls

378 lines (295 loc) · 14.1 KB
Raw

Developer Build Guide

This guide is for building Codex Warp from source on Linux, macOS, and Windows.

Codex Warp is a Rust 2024 project. It uses reqwest with rustls, so normal builds do not require system OpenSSL development packages. The rustls provider builds AWS-LC from source, so source builds need CMake.

Table Of Contents

Prerequisites

  • Rustup; the exact Rust/Cargo version is selected by rust-toolchain.toml
  • Git
  • A shell for running commands

Install the pinned toolchain and required components from the checked-in file:

rustup show active-toolchain
rustup component list --installed

Check your toolchain:

rustc --version
cargo --version

Linux

Install common build tools.

Debian or Ubuntu:

sudo apt update
sudo apt install -y build-essential cmake pkg-config curl git

Fedora:

sudo dnf install -y gcc gcc-c++ make cmake pkgconf-pkg-config curl git

Arch:

sudo pacman -S --needed base-devel cmake pkgconf curl git

Build and test:

cargo build
cargo test

Release build:

cargo build --release
./target/release/codex-warp --help

macOS

Install Xcode Command Line Tools and CMake:

xcode-select --install
brew install cmake

Install Rust with rustup, then build:

cargo build
cargo test

Release build:

cargo build --release
./target/release/codex-warp --help

Apple Silicon and Intel Macs both build with the default host target. For cross-target builds, install the target explicitly:

rustup target add aarch64-apple-darwin
rustup target add x86_64-apple-darwin

Windows

Recommended setup:

  1. Install Rust from rustup.rs.
  2. Install Visual Studio Build Tools with the Desktop development with C++ workload.
  3. Install CMake, either with the Visual Studio Build Tools CMake component or from cmake.org.
  4. Open PowerShell or Windows Terminal.

Build and test:

cargo build
cargo test

Release build:

cargo build --release
.\target\release\codex-warp.exe --help

If Cargo cannot find a linker, reopen the terminal after installing Visual Studio Build Tools, or run from a Developer PowerShell.

Local Validation

Pull request titles use type(optional-scope)!: concise description. Accepted types are feat, fix, perf, refactor, docs, test, build, ci, chore, and revert. Because the repository squash-merges pull requests, that title becomes the commit subject Release Please evaluates on main. The pull request description becomes the commit body. Use the templates in AGENTS.md so the change lands in the correct changelog section.

Release-policy validation uses the exact Node patch recorded in tools/release-tooling.json. Install Node 24.20.0 before running the complete preflight; the isolated Release Please workspace installs only its committed lockfile with npm ci --ignore-scripts.

Before ordinary local commits, new PR submission, and push that updates a PR, run the full local Linux CI preflight:

bash scripts/ci-preflight.sh

For a PR with a non-main base, use bash scripts/ci-preflight.sh --base origin/<base-branch>. Do not use git commit --no-verify or git push --no-verify to bypass this requirement. Install the durable hook bootstrap once per checkout to run the versioned preflight automatically at commit and push time:

bash scripts/install-git-hooks.sh

The bootstrap remains installed when branches change, but always dispatches to the checked-out branch's versioned hook and preflight scripts. If that branch does not provide the preflight implementation, it fails closed rather than silently skipping the check. Re-run the installer once after updating from an earlier hook installation to migrate its hook path. It also chains the hooks that were active before installation, so existing core.hooksPath and ordinary .git/hooks policies continue to run.

The hooks run for ordinary commits, git am, and branch pushes. Git cannot expose the exact target topology to a preventative hook for a bare non-fast-forward merge; use git merge --no-ff --no-commit <branch>, run the preflight, then commit the result. If you complete a bare merge, run the preflight immediately before pushing. Git also has no preventative hook for bare git cherry-pick or git revert, or for rewritten commits from git rebase / git rebase --continue. Use git cherry-pick --no-commit <commit> or git revert --no-commit <commit>, run the preflight, then commit the result so validation occurs before the commit is recorded. During a conflicted rebase, resolve and stage the conflict, run the preflight, then use git rebase --continue; run it once more after a non-conflicting rebase and before pushing. The pre-push hook remains a backstop for any branch update.

For a non-main PR base, install the hooks with that base so automatic commit and push checks use the same target:

bash scripts/install-git-hooks.sh --base origin/<base-branch>

The preflight runs every Linux CI check explicitly: cargo update --workspace --locked; typos; scripts/source-checks.sh (rustfmt, docs whitespace/prose, language policy, tracked JavaScript syntax, host ESLint for the Web UI pages, chart and analytics harnesses, and crate-wide Clippy); bash scripts/dylint.sh; cargo test --locked; cargo build --locked; RUSTDOCFLAGS='-D warnings' cargo doc --locked --no-deps; CLI --version and --help smoke checks; git diff --check; conditional Rust-diff cargo mutants -o <temporary-dir> --no-shuffle -vV --in-diff ... -- --locked; cargo deny check bans licenses sources; and cargo audit. The Windows job is intentionally excluded. cargo audit runs locally as non-blocking, matching pull-request CI. Weekly and manual advisory runs fail closed.

The chart harness covers chart-math.js policy (ticks, hover identity, keyboard ownership, pointer reclaim only on hit, paint only with a measured CSS width, live-region clear, canvas interactivity attrs, bar paint anchors) and footer-status.js (analytics footer copy when chart-math is missing, boot errors skipping that overlay). It is not a browser canvas stub of app-main.js. The analytics filter and chart-visibility harnesses extract helpers from app-main.js for session filter restore and for hiding pies that the current provider or model filter cannot populate.

For a quick documentation-only feedback loop before the mandatory preflight:

SOURCE_CHECKS_CLIPPY=0 bash scripts/source-checks.sh
git diff --check

Continuous Integration

GitHub Actions classifies each push to main and pull request before starting platform toolchains. Documentation-only changes still run spelling, prose, whitespace, and change-scope regression checks, then report the required Source Checks status without installing Rust. Mixed changes and paths outside the explicit documentation allowlist fail safe by running the full Linux and Windows suites. Pull requests are classified with the script from the immutable base revision, so a classifier change cannot weaken its own scope decision; a missing or invalid base classifier fails closed. Workflow changes remain security-sensitive and require normal branch protection and review.

For changes that require full CI, the Linux job performs:

  • cargo update --workspace --locked so Cargo.lock stays in sync with Cargo.toml
  • typos spell check (_typos.toml)
  • scripts/source-checks.sh (rustfmt, docs whitespace and contraction capitalization, language policy, tracked JavaScript syntax, host ESLint for the Web UI pages, chart and analytics harnesses, crate-wide Clippy with cargo clippy --locked --all-targets --all-features -- -D warnings)
  • cargo test --locked
  • cargo build --locked
  • RUSTDOCFLAGS='-D warnings' cargo doc --locked --no-deps
  • CLI smoke checks for codex-warp --version and codex-warp --help
  • git diff --check

Every pull request reports an incremental mutants status. When the PR has a Rust diff it runs cargo mutants --no-shuffle -vV --in-diff git.diff -- --locked against the PR base SHA; otherwise the job succeeds without installing the mutants toolchain. Surviving mutants on changed lines are a test-quality finding, not a request to add extra unrelated tests.

A separate supply-chain workflow always reports its required cargo-deny status on pull requests, but installs its toolchain and runs cargo-deny plus cargo-audit only when Cargo.toml, Cargo.lock, deny.toml, or the supply-chain workflow changes. Dependency-file pushes to main, weekly schedules, and manual dispatches run the full checks. Advisory failures are non-blocking on pull requests so a new CVE does not freeze unrelated work; scheduled and manual advisory runs fail closed. Do not add _typos.toml-style ignore entries in deny.toml to hide a real license or git-source policy break.

A Windows job runs cargo test --locked, cargo build --locked, and the same CLI smoke checks for full-CI changes so Windows-only build breaks (AWS-LC / linker) show up before a release. Documentation-only changes report Windows as skipped-success without allocating a Windows runner. Cargo caches are written only on main. Keep Source Checks, Dylint, Windows, Incremental (PR diff), and cargo-deny as required status checks on main.

The Dylint job runs only for full-CI changes, same as Windows. It installs cargo-dylint and dylint-link 6.1.0 from source (prebuilt binaries look for the driver sources on the machine that built them) and loads the examples/general lints listed in dylint.toml at tag v6.1.0. Those libraries typecheck with their own nightly, nightly-2026-08-20 at that tag. The project toolchain in rust-toolchain.toml stays the compiler for build and test. Cache ~/.cargo and ~/.dylint_drivers separately from target/dylint, and restore target/dylint only on an exact dylint.toml cache hit so a pin change cannot reuse another revision's .so. Warnings are denied through RUSTFLAGS (-D warnings -A deprecated), not as arguments after cargo dylint --. deprecated stays allowed because the lint nightly is newer than rust-toolchain.toml.

Source Layout

The crate is split by domain so small source changes do not all collide in one file:

File Purpose
src/main.rs Module map and main() entrypoint.
src/server.rs CLI parsing, startup, routes, and top-level handlers.
src/state.rs Shared request state and selected-provider structs.
src/provider.rs Provider selection and provider display names.
src/config.rs TOML config schema and defaults.
src/config_loader.rs Config includes, TOML merging, and provider lookup.
src/models.rs /models catalog fetching, sorting, and metadata shaping.
src/upstream.rs Upstream request dispatch and response plumbing.
src/structured_output.rs Chat Completions JSON Schema compatibility fallback.
src/guardian_compat.rs Guardian auto-review prompt compatibility shim.
src/namespace_helpers.rs Codex namespace-tool expansion for sub-agent helpers.
src/response_codec.rs SSE, chat/responses conversion, and usage normalization.
src/transform.rs Responses-to-chat request/tool/history conversion.
src/transform_morph.rs Configured request morphs and dotted-path edits.
src/tool_policy.rs Optional downstream tool-call approval policy.
src/debug_log.rs Sanitized debug log events and fingerprints.
src/process_log.rs In-memory process log buffer and tracing filter reload.
src/http.rs Shared HTTP headers, endpoint URLs, and proxy errors.
src/ids.rs Generated Responses item/call ids.
src/version.rs Agent name and version reporting.

Testing Layout

Most unit tests live in sibling files named src/<module>_tests.rs, included from the production module with #[cfg(test)] and #[path = "..."]. Keep new tests near the module they exercise instead of adding a new root-level test bundle. Test-quality rules live in AGENTS.md.

Running From Source

Use cargo run during development:

export XIAOMI_TOKEN_PLAN_API_KEY="..."
cargo run -- --config configs/xiaomi-token-plan.toml

Or use a temporary provider destination:

cargo run -- --destination https://provider.example/v1

Useful Environment Variables

  • RUST_LOG=codex_warp=debug: enables debug logging for this crate when debug.tracing_filter is unset. Warp captures this value when tracing starts; changing RUST_LOG later in the same process does not change the live filter. The Web UI Logs tab can override it at runtime through debug.tracing_filter.
  • Provider-specific API keys such as XIAOMI_TOKEN_PLAN_API_KEY.

Example:

RUST_LOG=codex_warp=debug cargo run -- --config configs/xiaomi-token-plan.toml

Release Artifacts

Local release binaries are produced under target/release/.

Typical artifact names:

  • Linux/macOS: target/release/codex-warp
  • Windows: target\release\codex-warp.exe

The runtime config files are not embedded in the binary. Keep codex-warp.toml and any configs/ profiles you want to use next to the working directory where you launch the proxy, or pass explicit --config paths.

Published archives include the executable and the reviewed runtime configuration tree. Official version changes are owned by Release Please; do not manually edit the package version during ordinary development. Nightly builds use a compile-time SemVer prerelease identity without changing Cargo.toml.

Read Releases for the public artifact contract and Release Automation Maintainer Runbook for the protected setup, activation, recovery, and incident procedures.