From 9ae5808926567cbcf27f77bedaae59381bc5013f Mon Sep 17 00:00:00 2001 From: Cursor Agent Date: Sat, 29 Aug 2026 08:21:31 +0000 Subject: [PATCH 01/13] feat(relearn): one-challenge subnet v0 eval loop Retire Design and Prism as live products. Wire Relearn HTTP submit, displacement scoring, digest-freeze holdout, and operator promote. Keep Lium/receipt/paired-test rails. Pin CortexLM/relearn (seed in-tree until org write can create the public repo). Co-authored-by: Mathis --- .github/ISSUE_TEMPLATE/bug.yml | 5 +- .github/ISSUE_TEMPLATE/config.yml | 2 +- .github/workflows/ci.yml | 37 +- .github/workflows/ghcr-public.yml | 5 +- .github/workflows/images.yml | 35 +- AGENTS.md | 20 +- Cargo.lock | 99 +++ README.md | 41 +- bins/relearn-challenge/Cargo.toml | 27 + bins/relearn-challenge/src/main.rs | 119 ++++ config/challenges.staging.toml | 12 +- config/challenges.staging.toml.sig | 2 +- config/challenges.toml | 15 +- config/challenges.toml.sig | 2 +- config/measurements.toml.sig | 2 +- config/owner.pubkey | 2 +- config/relearn-pin.toml | 17 + crates/relearn-challenge-task/Cargo.toml | 15 + crates/relearn-challenge-task/src/lib.rs | 134 ++++ crates/relearn-challenge/Cargo.toml | 26 + crates/relearn-challenge/src/lib.rs | 127 ++++ crates/relearn-eval/Cargo.toml | 29 + crates/relearn-eval/src/lib.rs | 369 ++++++++++ crates/relearn-http/Cargo.toml | 28 + crates/relearn-http/src/lib.rs | 386 +++++++++++ crates/relearn-score/Cargo.toml | 17 + crates/relearn-score/src/lib.rs | 272 ++++++++ crates/relearn-store/Cargo.toml | 23 + crates/relearn-store/src/lib.rs | 406 +++++++++++ crates/site-api/src/handlers.rs | 56 +- crates/site-api/src/upstream.rs | 2 + crates/site-data/src/map.rs | 31 +- crates/site-types/src/frames.rs | 38 +- crates/site-types/src/lib.rs | 2 +- crates/site-types/src/types.rs | 10 +- crates/trustroot/tests/trustroot_verify.rs | 28 +- deploy/AGENTS.md | 7 +- deploy/Dockerfile | 96 +-- deploy/compose/env-local.yml | 38 +- deploy/compose/env-prod.yml | 45 +- deploy/compose/env-staging.yml | 54 +- deploy/compose/role-validator.yml | 8 +- deploy/env/relearn-challenge.env.example | 16 + deploy/scripts/assert-compose-matrix.sh | 52 +- deploy/scripts/local-e2e.sh | 90 +-- deploy/scripts/materialize-env.sh | 4 +- deploy/scripts/promote.sh | 2 +- deploy/scripts/record-image-digests.sh | 4 +- deploy/scripts/register-challenge-backends.sh | 21 +- deploy/scripts/remote-deploy.sh | 105 +-- deploy/secrets/README.md | 9 +- docker-compose.e2e.yml | 18 +- docker-compose.yml | 192 +----- docs/AGENTS.md | 9 +- docs/ARCHITECTURE.md | 18 +- docs/COMPLETENESS.md | 41 +- docs/OPERATOR_SECURITY.md | 7 +- docs/RELEARN.md | 41 ++ docs/external-miner/README.md | 37 +- docs/external-miner/design.md | 213 +----- docs/external-miner/prism.md | 631 +----------------- docs/external-miner/relearn-seed/LICENSE | 9 + docs/external-miner/relearn-seed/README.md | 50 ++ .../relearn-seed/docs/README.md | 11 + .../relearn-seed/eval/Dockerfile | 11 + .../relearn-seed/eval/decontam/__init__.py | 1 + .../relearn-seed/eval/decontam/benches.py | 25 + .../relearn-seed/eval/generators/__init__.py | 1 + .../relearn-seed/eval/generators/factory.py | 33 + .../relearn-seed/eval/harness/__init__.py | 1 + .../relearn-seed/eval/harness/eval.py | 52 ++ .../relearn-seed/eval/teacher/__init__.py | 1 + .../relearn-seed/eval/teacher/judge.py | 52 ++ docs/external-miner/relearn.md | 66 ++ docs/external-miner/troubleshoot.md | 51 +- xtask/src/external_docs_check.rs | 75 +-- 76 files changed, 2851 insertions(+), 1787 deletions(-) create mode 100644 bins/relearn-challenge/Cargo.toml create mode 100644 bins/relearn-challenge/src/main.rs create mode 100644 config/relearn-pin.toml create mode 100644 crates/relearn-challenge-task/Cargo.toml create mode 100644 crates/relearn-challenge-task/src/lib.rs create mode 100644 crates/relearn-challenge/Cargo.toml create mode 100644 crates/relearn-challenge/src/lib.rs create mode 100644 crates/relearn-eval/Cargo.toml create mode 100644 crates/relearn-eval/src/lib.rs create mode 100644 crates/relearn-http/Cargo.toml create mode 100644 crates/relearn-http/src/lib.rs create mode 100644 crates/relearn-score/Cargo.toml create mode 100644 crates/relearn-score/src/lib.rs create mode 100644 crates/relearn-store/Cargo.toml create mode 100644 crates/relearn-store/src/lib.rs create mode 100644 deploy/env/relearn-challenge.env.example create mode 100644 docs/RELEARN.md create mode 100644 docs/external-miner/relearn-seed/LICENSE create mode 100644 docs/external-miner/relearn-seed/README.md create mode 100644 docs/external-miner/relearn-seed/docs/README.md create mode 100644 docs/external-miner/relearn-seed/eval/Dockerfile create mode 100644 docs/external-miner/relearn-seed/eval/decontam/__init__.py create mode 100644 docs/external-miner/relearn-seed/eval/decontam/benches.py create mode 100644 docs/external-miner/relearn-seed/eval/generators/__init__.py create mode 100644 docs/external-miner/relearn-seed/eval/generators/factory.py create mode 100644 docs/external-miner/relearn-seed/eval/harness/__init__.py create mode 100644 docs/external-miner/relearn-seed/eval/harness/eval.py create mode 100644 docs/external-miner/relearn-seed/eval/teacher/__init__.py create mode 100644 docs/external-miner/relearn-seed/eval/teacher/judge.py create mode 100644 docs/external-miner/relearn.md diff --git a/.github/ISSUE_TEMPLATE/bug.yml b/.github/ISSUE_TEMPLATE/bug.yml index 733cd0a80..2afdb5006 100644 --- a/.github/ISSUE_TEMPLATE/bug.yml +++ b/.github/ISSUE_TEMPLATE/bug.yml @@ -7,7 +7,7 @@ body: value: | Use this for defects in **this** repo (gateway, validator, challenges, deploy). Security issues: do **not** file here — see [SECURITY.md](../SECURITY.md). - Miner-facing public repos are separate (design-challenge / prism). + Miner-facing public repo is [CortexLM/relearn](https://github.com/CortexLM/relearn). - type: textarea id: summary attributes: @@ -22,8 +22,7 @@ body: options: - gateway - validator - - design-challenge - - prism-challenge + - relearn-challenge - deploy / compose - docs - other diff --git a/.github/ISSUE_TEMPLATE/config.yml b/.github/ISSUE_TEMPLATE/config.yml index 1e813dbe9..ee75fd22e 100644 --- a/.github/ISSUE_TEMPLATE/config.yml +++ b/.github/ISSUE_TEMPLATE/config.yml @@ -2,7 +2,7 @@ blank_issues_enabled: true contact_links: - name: Miner submit docs url: https://github.com/CortexLM/cortex/blob/main/docs/external-miner/README.md - about: HTTP submit guides live in docs/external-miner (and the public design-challenge / prism repos). + about: HTTP submit guides live in docs/external-miner and the public CortexLM/relearn repo. - name: Security report url: https://github.com/CortexLM/cortex/security/advisories/new about: Private vulnerability reporting — do not file a public issue. diff --git a/.github/workflows/ci.yml b/.github/workflows/ci.yml index c1930e0d3..5aff9863f 100644 --- a/.github/workflows/ci.yml +++ b/.github/workflows/ci.yml @@ -64,41 +64,6 @@ jobs: - name: compose matrix assertions run: bash deploy/scripts/assert-compose-matrix.sh - # Prism v3 harness smokes: fixture-size, G4 tuple concat, and torch-seed - # overflow class bugs shipped once because CI never ran the harness - # end-to-end. CPU wheels keep the runner cheap; both smokes must pass - # (they assert all 8 battery groups ok + the flat org.* / mirrors blob - # the Rust composite ingests). - harness-smoke: - name: prism harness smokes (cpu torch) - runs-on: ubuntu-latest - timeout-minutes: 25 - steps: - - name: Checkout - uses: actions/checkout@v4 - - - name: Set up Python - uses: actions/setup-python@v5 - with: - python-version: "3.12" - cache: pip - cache-dependency-path: crates/prism-recipe/harness - - - name: Install cpu torch + transformers + pyarrow - run: | - python -m pip install --upgrade pip - pip install torch --index-url https://download.pytorch.org/whl/cpu - pip install transformers pyarrow - - - name: python -m compileall - run: python -m compileall crates/prism-recipe/harness - - - name: smoke_local (v1 contract flow) - run: python crates/prism-recipe/harness/tests/smoke_local.py - - - name: smoke_battery (v3 G1-G8 battery + rollup contract) - run: python crates/prism-recipe/harness/tests/smoke_battery.py - # --------------------------------------------------------------------------- # Auto-deploy staging. Lives here rather than in deploy-staging.yml so that # `needs: ci` keeps the CI-green ordering on every push to main. @@ -106,7 +71,7 @@ jobs: # --------------------------------------------------------------------------- deploy-staging: name: deploy staging ${{ matrix.role }} - needs: [ci, harness-smoke] + needs: [ci] if: github.event_name == 'push' && github.ref == 'refs/heads/main' runs-on: ubuntu-latest timeout-minutes: 120 diff --git a/.github/workflows/ghcr-public.yml b/.github/workflows/ghcr-public.yml index f81157763..bd53cc1c3 100644 --- a/.github/workflows/ghcr-public.yml +++ b/.github/workflows/ghcr-public.yml @@ -27,10 +27,7 @@ jobs: "base/gateway" "base/validator" "base/updater" - "base/prism-challenge" - "base/design-challenge" - "base/design-egress-proxy" - "base/design-runtime" + "base/relearn-challenge" ) ok=0 # First list packages to Learn exact names (debug) diff --git a/.github/workflows/images.yml b/.github/workflows/images.yml index 3441e2f90..31ddb5ca4 100644 --- a/.github/workflows/images.yml +++ b/.github/workflows/images.yml @@ -70,20 +70,8 @@ jobs: - target: updater image_suffix: updater dockerfile: deploy/Dockerfile - - target: prism-challenge - image_suffix: prism-challenge - dockerfile: deploy/Dockerfile - - target: design-challenge - image_suffix: design-challenge - dockerfile: deploy/Dockerfile - - target: design-egress-proxy - image_suffix: design-egress-proxy - dockerfile: deploy/Dockerfile - - target: design-runtime - image_suffix: design-runtime - dockerfile: deploy/Dockerfile - - target: design-review - image_suffix: design-review + - target: relearn-challenge + image_suffix: relearn-challenge dockerfile: deploy/Dockerfile - target: "" image_suffix: base-attest-helper @@ -285,11 +273,7 @@ jobs: "validator", "gateway", "updater", - "prism-challenge", - "design-challenge", - "design-egress-proxy", - "design-runtime", - "design-review", + "relearn-challenge", "base-attest-helper", } assert set(images) == expected, (set(images), expected) @@ -357,10 +341,10 @@ jobs: cp "$SRC" "deploy/digests/${SHA}.json" echo "Recorded digest manifest: deploy/digests/${SHA}.json" chmod +x deploy/scripts/promote.sh - # Pin services only (validator/gateway/updater/prism-challenge/design-challenge). + # Pin services only (validator/gateway/updater/relearn-challenge). # --skip-backup: this job records CI digests; Spaces/PG backup runs at # prod promote (deploy-prod.yml, fail-closed). - for svc in validator gateway updater prism-challenge design-challenge; do + for svc in validator gateway updater relearn-challenge; do IMAGE=$(python3 -c ' import json, sys d = json.load(open(sys.argv[1])) @@ -379,7 +363,7 @@ jobs: import json, sys p = json.load(open("deploy/pins/staging.json")) assert p["commit_sha"] == sys.argv[1], (p["commit_sha"], sys.argv[1]) - for s in ("validator", "gateway", "updater", "prism-challenge", "design-challenge"): + for s in ("validator", "gateway", "updater", "relearn-challenge"): assert s in p["services"], s assert p["services"][s]["digest"].startswith("sha256:") print("staging pins ok", p["commit_sha"]) @@ -426,12 +410,7 @@ jobs: "base/gateway" "base/validator" "base/updater" - "base/prism-challenge" - "base/design-challenge" - "base/design-egress-proxy" - "base/design-runtime" - "base/design-review" - "base/prism-pod" + "base/relearn-challenge" ) ok=0 # First list packages to Learn exact names (debug) diff --git a/AGENTS.md b/AGENTS.md index 92e4c49b2..d1fd199ef 100644 --- a/AGENTS.md +++ b/AGENTS.md @@ -2,7 +2,7 @@ Short contract for agents and operators. Prefer linking over restating runbooks. -**Product:** Cortex ([`CortexLM/cortex`](https://github.com/CortexLM/cortex)) — Bittensor subnet control plane for decentralized collaborative AI research via multiple challenges. Naming split (Cortex vs leftover `base` / `BASE_*`): [`docs/NAMING.md`](docs/NAMING.md). +**Product:** Cortex ([`CortexLM/cortex`](https://github.com/CortexLM/cortex)) — Bittensor subnet control plane for a **one-challenge** post-training factory (**Relearn**). Challenge code lives in [`CortexLM/relearn`](https://github.com/CortexLM/relearn). Naming split (Cortex vs leftover `base` / `BASE_*`): [`docs/NAMING.md`](docs/NAMING.md). ## Monorepo map @@ -34,7 +34,7 @@ Working branch: **`main`**. Prod ships from annotated tags `v*.*.*` cut on `main |-----|-----|------------| | `gateway_sk` | Gateway | Bundle **seal** signatures (`POST /v1/admin/seal`) | | `gateway_admin_token` | Gateway + seal scripts | Bearer for **`/v1/admin/*`** (seal, backends, attest-grant). **Required** when `BASE_GATEWAY_REQUIRE_OWNER=1` | -| `prism_sk` / `design_sk` | Challenge / smoke | Signed leaves (`POST /v1/weights/raw`); pubs must match trust root | +| `relearn_sk` | Relearn / smoke | Signed leaves (`POST /v1/weights/raw`); pub must match trust root | | Gateway owner wallet + `BASE_GATEWAY_REQUIRE_OWNER` | Gateway | Master-only **identity** check (live/prod). **Not** required to seal or serve `/v1/weights/latest` | | Validator wallet | Validator | On-chain weight **submit** only — validators *fetch* sealed weights; they do not need a gateway wallet | @@ -46,10 +46,9 @@ Each live challenge has a **separate public GitHub repo** for miners. Those repo | Challenge | Public repo | Role | |-----------|-------------|------| -| Design | [`BaseIntelligence/design-challenge`](https://github.com/BaseIntelligence/design-challenge) | Miner docs + baseline harness | -| Prism | [`BaseIntelligence/prism`](https://github.com/BaseIntelligence/prism) | Miner docs + recipe examples (publish / keep in sync; no control-plane code) | +| Relearn | [`CortexLM/relearn`](https://github.com/CortexLM/relearn) | Eval image, harness, generators, teacher, miner docs | -Those public URLs are historical org names; this control-plane repo is `CortexLM/cortex`. Monorepo mirror for CI and operators: [`docs/external-miner/`](docs/external-miner/). Frozen contracts stay in this repo (`docs/DESIGN_CHALLENGE.md`, `docs/PRISM.md`, …). +This control-plane repo is `CortexLM/cortex`. Monorepo miner-doc mirror: [`docs/external-miner/`](docs/external-miner/). Historical frozen specs (`docs/DESIGN_CHALLENGE.md`, `docs/PRISM.md`) stay archived; they are not live products. **When a challenge product or public API changes**, agents **must** update: @@ -64,12 +63,11 @@ When verifying a challenge (local-e2e, staging, or focused tests), **simulate a 1. Happy-path harness / intake POST (or equivalent) through the challenge service on master. 2. Edge / failure probes: bad harness, sanitize reject, quota, wrong routes/auth. -3. **Design — baseline:** submit the reference agent at [`docs/external-miner/examples/design-baseline/`](docs/external-miner/examples/design-baseline/) (`agent.py` + `pyproject.toml`). After `POST /v1/harness`, poll `GET /v1/runs/{id}` + `/events` + `/logs?since=` until `awaiting_admin` / terminal; assert `GET /v1/runs/{id}/pages` lists `index.html`, `pricing.html`, `components.html` and `GET /v1/view/{run_id}/{page}` returns **200**; probe `GET /v1/stats` and `GET /v1/dashboard`. -4. **Design — cheat:** submit a malicious/copy harness; expect agentic `cheat`/`suspicious` → `Score(0)` (not admin-eligible). Poll events/logs the same way. -5. **Design — admin winners:** with operator bearer (`deploy/secrets/design/annotator_tokens`), `GET /v1/admin/rounds/{id}/candidates` then `POST /v1/admin/rounds/{id}/winners` with 1 or 2 clean harness ids (`SCORE_MAX` or `SCORE_MAX/2`). -6. Leaf emission → `POST /v1/weights/raw` → seal → `GET /v1/weights/latest` with **`sealed: true`** (burn fallback alone is not a real seal). +3. **Relearn — submit:** `POST /v1/submissions` with a 64-hex hotkey + artifact digest (optional `X-Lium-Api-Key`). Poll `GET /v1/submissions/{id}` until `awaiting_admin` or `rejected`. Holdout must stay sealed until the digest freezes. A regression must not become champion. +4. **Relearn — promote:** with operator bearer (`deploy/secrets/relearn/admin_tokens`), `POST /v1/admin/promote` only for an eligible paired win. +5. Leaf emission → `POST /v1/weights/raw` → seal → `GET /v1/weights/latest` with **`sealed: true`** (burn fallback alone is not a real seal). -**Never host Sim in staging/prod** — Docker sandbox only there. `SimSandbox` / `BASE_ALLOW_HOST_SIM=1` is CI/local opt-in only; do **not** treat stub pages (`sim-install-ok` / `sim-run-ok` without executing `agent.py`) as proof. Prefer `DESIGN_FORCE_SIM=false` + OpenRouter when `deploy/secrets/openrouter/api_key` is present. +**Never host Sim in staging/prod** for live scoring. `RELEARN_FORCE_SIM=1` is CI/local opt-in only. Live rent requires a digest pin in `config/relearn-pin.toml` plus miner BYOK (`LIUM_API_KEY` / `X-Lium-Api-Key`). Never log or commit that key. Local smoke automates the weights seal step via `weights-smoke` inside `./deploy/scripts/local-e2e.sh --smoke` (see [`deploy/AGENTS.md`](deploy/AGENTS.md) and [`docs/runbooks/local-testnet-e2e.md`](docs/runbooks/local-testnet-e2e.md)). @@ -113,7 +111,7 @@ Match CI (`.github/workflows/ci.yml`): | Doc authority vs evidence | [`docs/AGENTS.md`](docs/AGENTS.md) | | Component status | [`docs/COMPLETENESS.md`](docs/COMPLETENESS.md) | | Frozen contracts | [`docs/BUNDLE_SPEC.md`](docs/BUNDLE_SPEC.md), [`docs/DESIGN_CHALLENGE.md`](docs/DESIGN_CHALLENGE.md), [`docs/PRISM.md`](docs/PRISM.md) | -| Miner HTTP submit | [`docs/external-miner/`](docs/external-miner/) · public: [design-challenge](https://github.com/BaseIntelligence/design-challenge), [prism](https://github.com/BaseIntelligence/prism) | +| Miner HTTP submit | [`docs/external-miner/`](docs/external-miner/) · public: [CortexLM/relearn](https://github.com/CortexLM/relearn) | | Threat / operator checklist | [`docs/THREAT_MODEL.md`](docs/THREAT_MODEL.md), [`docs/OPERATOR_SECURITY.md`](docs/OPERATOR_SECURITY.md) | ## Do not commit diff --git a/Cargo.lock b/Cargo.lock index 61d107084..d266535a2 100644 --- a/Cargo.lock +++ b/Cargo.lock @@ -4511,6 +4511,105 @@ version = "0.8.11" source = "registry+https://github.com/rust-lang/crates.io-index" checksum = "d6f6ff9a378485b298a5286656da665ba74413d36db0979633275d2e708145d4" +[[package]] +name = "relearn-challenge" +version = "0.1.0" +dependencies = [ + "bundle", + "challenge-common", + "crypto", + "hex", + "relearn-challenge-task", + "relearn-eval", + "relearn-http", + "relearn-score", + "relearn-store", + "serde", + "serde_json", + "thiserror 2.0.19", +] + +[[package]] +name = "relearn-challenge-bin" +version = "0.1.0" +dependencies = [ + "axum", + "challenge-keys", + "clap", + "relearn-challenge", + "relearn-eval", + "relearn-store", + "telemetry", + "tokio", + "tracing", +] + +[[package]] +name = "relearn-challenge-task" +version = "0.1.0" +dependencies = [ + "serde", +] + +[[package]] +name = "relearn-eval" +version = "0.1.0" +dependencies = [ + "async-trait", + "hex", + "prism-competition", + "prism-lium", + "prism-lium-types", + "relearn-challenge-task", + "relearn-score", + "relearn-store", + "serde", + "serde_json", + "sha2 0.10.9", + "thiserror 2.0.19", + "tokio", +] + +[[package]] +name = "relearn-http" +version = "0.1.0" +dependencies = [ + "axum", + "hex", + "http-body-util", + "relearn-challenge-task", + "relearn-eval", + "relearn-score", + "relearn-store", + "serde", + "serde_json", + "sha2 0.10.9", + "tokio", + "tower", +] + +[[package]] +name = "relearn-score" +version = "0.1.0" +dependencies = [ + "prism-competition", + "relearn-challenge-task", + "serde", +] + +[[package]] +name = "relearn-store" +version = "0.1.0" +dependencies = [ + "hex", + "prism-competition", + "relearn-challenge-task", + "relearn-score", + "serde", + "sha2 0.10.9", + "thiserror 2.0.19", +] + [[package]] name = "reqwest" version = "0.12.28" diff --git a/README.md b/README.md index 91e3961dd..25cc06a48 100644 --- a/README.md +++ b/README.md @@ -15,21 +15,21 @@ ## What it is Cortex ([`CortexLM/cortex`](https://github.com/CortexLM/cortex)) is the Rust -control plane for a multi-challenge Bittensor subnet. Challenge services on -the **master** host accept miner work over HTTP, sign score leaves, and the -**gateway** (master-only) seals an epoch weight bundle. Validators **fetch** -`GET /v1/weights/latest` and submit on-chain weights. They do not execute -challenges. - -Live challenges today: +control plane for a **one-challenge** Bittensor subnet (**Relearn**). The +challenge service on the **master** host accepts miner work over HTTP, signs +score leaves, and the **gateway** (master-only) seals an epoch weight bundle. +Validators **fetch** `GET /v1/weights/latest` and submit on-chain weights. +They do not execute the challenge. | Challenge | How miners submit | Spec | |-----------|-------------------|------| -| **Design** | ZIP harness (`agent.py` + `pyproject.toml`) → sandboxed pages + admin winners | [`docs/DESIGN_CHALLENGE.md`](docs/DESIGN_CHALLENGE.md) | -| **Prism** | AutoModel pin + patch → operator-owned GPU recipe eval | [`docs/PRISM.md`](docs/PRISM.md) | +| **Relearn** | Artifact digest + Lium BYOK → paired displacement vs champion | [`docs/RELEARN.md`](docs/RELEARN.md) | -There is **no miner Phala/CVM path** on this branch (agent-v1 / Harbor pack -executors were removed). Operator-facing map: [`docs/ARCHITECTURE.md`](docs/ARCHITECTURE.md). +Eval image, harness, generators, and miner docs live in +[`CortexLM/relearn`](https://github.com/CortexLM/relearn). Design and Prism +are retired products (libraries / frozen specs remain). There is **no miner +Phala/CVM / TDX path**. Operator-facing map: +[`docs/ARCHITECTURE.md`](docs/ARCHITECTURE.md). Some env vars, host paths, GHCR package names, and crypto domain tags still spell `BASE_*` / `base`. That is intentional — see [`docs/NAMING.md`](docs/NAMING.md). @@ -37,7 +37,7 @@ spell `BASE_*` / `base`. That is intentional — see [`docs/NAMING.md`](docs/NAM ## Architecture (short) ```text -Miners --HTTP--> gateway (TLS) --proxy--> design-challenge / prism-challenge +Miners --HTTP--> gateway (TLS) --proxy--> relearn-challenge | signed leaves v gateway seals EpochBundleV1 @@ -59,13 +59,11 @@ Validators <--- GET /v1/weights/latest ---+ HTTP submit only. Start at [docs/external-miner/](docs/external-miner/). ```text -https:///challenge/design/... -https:///challenge/prism/... +https:///challenge/relearn/... ``` -Public miner docs (examples only — no control-plane code): -[design-challenge](https://github.com/BaseIntelligence/design-challenge), -[prism](https://github.com/BaseIntelligence/prism). +Public miner + eval repo (no control-plane code): +[CortexLM/relearn](https://github.com/CortexLM/relearn). Never put mnemonics or challenge signing keys in miner clients. @@ -106,9 +104,7 @@ digest-pinned images. The registry path is still | validator | `validator` | | gateway | `gateway` | | updater | `updater` | -| prism-challenge | `prism-challenge` | -| design-challenge | `design-challenge` | -| design-egress-proxy | `design-egress-proxy` | +| relearn-challenge | `relearn-challenge` | ## Toolchain and gates @@ -134,8 +130,9 @@ cargo run -p xtask -- external-docs-check | [CONTRIBUTING.md](CONTRIBUTING.md) | How to change this repo | | [docs/ARCHITECTURE.md](docs/ARCHITECTURE.md) | System map | | [docs/BUNDLE_SPEC.md](docs/BUNDLE_SPEC.md) | Sealed weight bundle (frozen) | -| [docs/DESIGN_CHALLENGE.md](docs/DESIGN_CHALLENGE.md) | Design challenge (frozen) | -| [docs/PRISM.md](docs/PRISM.md) | Prism challenge | +| [docs/RELEARN.md](docs/RELEARN.md) | Relearn (live) | +| [docs/DESIGN_CHALLENGE.md](docs/DESIGN_CHALLENGE.md) | Design (archived freeze) | +| [docs/PRISM.md](docs/PRISM.md) | Prism (archived) | | [docs/THREAT_MODEL.md](docs/THREAT_MODEL.md) | Security claims | | [docs/runbooks/](docs/runbooks/) | Ops procedures | diff --git a/bins/relearn-challenge/Cargo.toml b/bins/relearn-challenge/Cargo.toml new file mode 100644 index 000000000..9d6216bac --- /dev/null +++ b/bins/relearn-challenge/Cargo.toml @@ -0,0 +1,27 @@ +[package] +name = "relearn-challenge-bin" +description = "relearn-challenge operator binary (health + miner submit on :8095)" +version.workspace = true +edition.workspace = true +license.workspace = true +repository.workspace = true +rust-version.workspace = true +publish = false + +[[bin]] +name = "relearn-challenge" +path = "src/main.rs" + +[dependencies] +axum = { version = "0.8", default-features = false, features = ["http1", "tokio"] } +challenge-keys = { path = "../../crates/challenge-keys" } +clap = { version = "4", features = ["derive", "env"] } +relearn-challenge = { path = "../../crates/relearn-challenge" } +relearn-eval = { path = "../../crates/relearn-eval" } +relearn-store = { path = "../../crates/relearn-store" } +telemetry = { path = "../../crates/telemetry" } +tokio = { version = "1", features = ["macros", "rt-multi-thread", "net", "signal"] } +tracing = "0.1" + +[lints] +workspace = true diff --git a/bins/relearn-challenge/src/main.rs b/bins/relearn-challenge/src/main.rs new file mode 100644 index 000000000..4d947494b --- /dev/null +++ b/bins/relearn-challenge/src/main.rs @@ -0,0 +1,119 @@ +//! `relearn-challenge` — master-only Relearn service (port 8095). +//! +//! Miner HTTP submit → digest freeze → holdout unseal → sim/Lium eval → +//! operator-audited promote. No TDX / Phala CVM. + +#![forbid(unsafe_code)] + +use std::net::SocketAddr; +use std::path::PathBuf; +use std::process::ExitCode; +use std::sync::Arc; + +use challenge_keys::load_challenge_secret; +use clap::Parser; +use relearn_challenge::{ + hash_admin_token, relearn_router, AppState, MemoryStore, CHALLENGE_ID, SCORING_VERSION, +}; +use relearn_eval::{base_champion_scores, RelearnPin}; +use tokio::net::TcpListener; + +/// Operator Relearn challenge service CLI. +#[derive(Debug, Parser)] +#[command( + name = "relearn-challenge", + about = "Relearn challenge service (port 8095, master→Lium/sim, no CVM)" +)] +struct Cli { + /// Bind address (default 0.0.0.0:8095). + #[arg(long, env = "BASE_CHALLENGE_BIND", default_value = "0.0.0.0:8095")] + bind: SocketAddr, + /// Challenge mini-secret file (leaf signatures). + #[arg(long, env = "BASE_CHALLENGE_SK_FILE")] + challenge_sk_file: Option, + /// Force sim eval (no Lium spend). + #[arg(long, env = "RELEARN_FORCE_SIM", default_value_t = false)] + force_sim: bool, + /// Operator bearer tokens file (one per line). Empty → admin 503. + #[arg(long, env = "RELEARN_ADMIN_TOKENS_FILE")] + admin_tokens_file: Option, + /// Pin file (`config/relearn-pin.toml`). + #[arg(long, env = "RELEARN_PIN_FILE")] + pin_file: Option, +} + +fn main() -> ExitCode { + let _ = telemetry::init_tracing(); + let cli = Cli::parse(); + match run(&cli) { + Ok(()) => ExitCode::SUCCESS, + Err(e) => { + tracing::error!("{e}"); + ExitCode::from(1) + } + } +} + +fn run(cli: &Cli) -> Result<(), String> { + if let Some(p) = &cli.challenge_sk_file { + let _sk = load_challenge_secret(p).map_err(|e| format!("challenge sk: {e}"))?; + } + let pin = load_pin(cli.pin_file.as_deref()); + if cli.force_sim { + tracing::info!("RELEARN_FORCE_SIM=1 — sim eval only"); + } + let admin_hashes = load_admin_hashes(cli.admin_tokens_file.as_deref()); + let store = MemoryStore::new(); + store + .set_base_champion(base_champion_scores()) + .map_err(|e| e.to_string())?; + let state = AppState { + store, + pin, + admin_hashes: Arc::new(admin_hashes), + }; + let rt = tokio::runtime::Builder::new_multi_thread() + .enable_all() + .build() + .map_err(|e| e.to_string())?; + rt.block_on(serve(cli.bind, state)) +} + +fn load_pin(path: Option<&std::path::Path>) -> RelearnPin { + path.and_then(|p| std::fs::read_to_string(p).ok()) + .map(|s| RelearnPin::from_toml(&s)) + .unwrap_or_default() +} + +fn load_admin_hashes(path: Option<&std::path::Path>) -> Vec { + let Some(p) = path else { + return Vec::new(); + }; + let Ok(body) = std::fs::read_to_string(p) else { + return Vec::new(); + }; + body.lines() + .map(str::trim) + .filter(|l| !l.is_empty() && !l.starts_with('#')) + .map(hash_admin_token) + .collect() +} + +async fn serve(bind: SocketAddr, state: AppState) -> Result<(), String> { + let app = relearn_router(state); + let listener = TcpListener::bind(bind) + .await + .map_err(|e| format!("bind {bind}: {e}"))?; + tracing::info!( + %bind, + challenge_id = CHALLENGE_ID, + scoring_version = SCORING_VERSION, + "relearn-challenge listening" + ); + axum::serve(listener, app) + .with_graceful_shutdown(async { + let _ = tokio::signal::ctrl_c().await; + }) + .await + .map_err(|e| e.to_string()) +} diff --git a/config/challenges.staging.toml b/config/challenges.staging.toml index 8054ef8de..8901747e7 100644 --- a/config/challenges.staging.toml +++ b/config/challenges.staging.toml @@ -1,5 +1,5 @@ # STAGING-ONLY trust root override (testnet 541): mirrors prod emission shares -# so the staging e2e can prove design + prism weights end-to-end. +# so the staging e2e can prove relearn weights end-to-end. # # Mounted over /etc/base/config/challenges.toml by deploy/compose/env-staging.yml # only. Signed with the same throwaway owner key as config/challenges.toml @@ -8,13 +8,7 @@ version = 1 introduced_epoch = 0 [[challenges]] -id = "design" -public_key = "3e27f87d8330006a73174001120c3455f16b95fee098bb8c2bab9d5053840418" -emission_share_bps = 0 -policy = "all_metagraph_hotkeys" - -[[challenges]] -id = "prism" -public_key = "bcd50bb830e050ed4b011dd8f1d2f126fdb42dc55b45ece30a7d5c8ceb3c5219" +id = "relearn" +public_key = "8ab577207bb6dfc770a850710824a098d53b1ee90abb92925bd0928937131674" emission_share_bps = 10000 policy = "all_metagraph_hotkeys" diff --git a/config/challenges.staging.toml.sig b/config/challenges.staging.toml.sig index 57c102a47..17387f2dd 100644 --- a/config/challenges.staging.toml.sig +++ b/config/challenges.staging.toml.sig @@ -1 +1 @@ -707ea80656fe9cba3f82b6d029c8727269c84d1ba61f764c35a82811ccbaae1d146937089f7026df2425f7db9557d0fe8df861428db99997733ba1fe02770683 +2ce860996bdf6b108c29dc1082cfb0029b74cfbc9b61483be69e0f87ac999b23b59e299e31530c76490f9357e3fa216f8a379ad2902e26dc94f6910483f9f589 diff --git a/config/challenges.toml b/config/challenges.toml index 973919b97..8c83894c0 100644 --- a/config/challenges.toml +++ b/config/challenges.toml @@ -1,20 +1,13 @@ # Owner-signed challenges trust root (D18/D23/D24). # Signed with throwaway owner key (see owner.pubkey). Production rotation: CEREMONY.md. # -# Emission: design = 0 bps, prism = 10000 bps (100% prism; rebalanced 2026-08-16 from -# design 5000 / prism 5000 activated 2026-08-07). Same owner key + challenge keys; -# a future production owner/key ceremony per CEREMONY.md remains pending). +# Emission: relearn = 10000 bps (100%; one-challenge subnet). Design and Prism +# are retired as products. version = 1 introduced_epoch = 0 [[challenges]] -id = "design" -public_key = "3e27f87d8330006a73174001120c3455f16b95fee098bb8c2bab9d5053840418" -emission_share_bps = 0 -policy = "all_metagraph_hotkeys" - -[[challenges]] -id = "prism" -public_key = "bcd50bb830e050ed4b011dd8f1d2f126fdb42dc55b45ece30a7d5c8ceb3c5219" +id = "relearn" +public_key = "8ab577207bb6dfc770a850710824a098d53b1ee90abb92925bd0928937131674" emission_share_bps = 10000 policy = "all_metagraph_hotkeys" diff --git a/config/challenges.toml.sig b/config/challenges.toml.sig index cdce22889..cf9155125 100644 --- a/config/challenges.toml.sig +++ b/config/challenges.toml.sig @@ -1 +1 @@ -6cf7041f202f15c38ee305dcaa4c970e040689cb0c483594c09a77b1f8117c2de56c29668326cf6943cca7c56ac6c220ee202d418397b6bd07d70bf17458628d +40b64f0d5cbe0b3d9217f17233c81e21a97949ee486d4720e6094294f7124d77b361688d992b7d5fc1c67e94cdbca1921be998648b73e7c3be4eca41052afd85 diff --git a/config/measurements.toml.sig b/config/measurements.toml.sig index 5140621ec..7d677bc27 100644 --- a/config/measurements.toml.sig +++ b/config/measurements.toml.sig @@ -1 +1 @@ -529b61966f7aff9a096eb3eb9d26af668693877b8b0e34a65a4f1d2d55816f2eaf8b0605daa845ab9cf6c455227d4f8dfb9e453a1d90893565a0925fa6f75e8d +1e9a5e740f6f8ff916a0be5fb1a75a5080d34d3864b064d24be020be6c1b0831d846d46c4b2a173550292657a96b1a9df68f09bc7a18665fb49ebff339b64980 diff --git a/config/owner.pubkey b/config/owner.pubkey index 9ef0e933a..9dc555915 100644 --- a/config/owner.pubkey +++ b/config/owner.pubkey @@ -1 +1 @@ -fa856fccff666d7b9f490a2d9c99ab83d5a2bf789021f2d95703699d54e08770 +fa441733b3312b116b46062de9b71592bc005339fc71211c06df472feb214a13 diff --git a/config/relearn-pin.toml b/config/relearn-pin.toml new file mode 100644 index 000000000..ac9ef5ce7 --- /dev/null +++ b/config/relearn-pin.toml @@ -0,0 +1,17 @@ +# Cortex pin for the split CortexLM/relearn challenge repo. +# Deploy = bump eval_image_digest + relearn_git_sha after relearn CI is green. +# Never put secrets in this file. + +base_model = "Qwen/Qwen3.8-Flash-Next" +teacher_model = "zai-org/GLM-5.3" +teacher_nvfp4 = "Inferact/GLM-5.3-NVFP4" +# v0 default: teacher-only HTTP API (or Sim). NVFP4-on-Lium is preferred +# when the operator can rent an 8× Blackwell-class host +# (RELEARN_TEACHER_BACKEND=lium). Miner weights are never served via the +# teacher API — that endpoint is judge-only. +teacher_backend = "http_api" +eval_image = "ghcr.io/cortexlm/relearn-eval" +# Empty until the first digest-pinned image ships from CortexLM/relearn CI. +eval_image_digest = "" +relearn_git = "https://github.com/CortexLM/relearn" +relearn_git_sha = "" diff --git a/crates/relearn-challenge-task/Cargo.toml b/crates/relearn-challenge-task/Cargo.toml new file mode 100644 index 000000000..67d4792fb --- /dev/null +++ b/crates/relearn-challenge-task/Cargo.toml @@ -0,0 +1,15 @@ +[package] +name = "relearn-challenge-task" +description = "Relearn challenge identity, model pins, and domain tags" +version.workspace = true +edition.workspace = true +license.workspace = true +repository.workspace = true +rust-version.workspace = true +publish = false + +[dependencies] +serde = { version = "1", features = ["derive"] } + +[lints] +workspace = true diff --git a/crates/relearn-challenge-task/src/lib.rs b/crates/relearn-challenge-task/src/lib.rs new file mode 100644 index 000000000..8234da10d --- /dev/null +++ b/crates/relearn-challenge-task/src/lib.rs @@ -0,0 +1,134 @@ +//! Relearn challenge identity and verified model pins. +//! +//! ```text +//! challenge_id = "relearn" +//! scoring_version = 1 +//! task_id domain = b"base-relearn-task-id-v1" +//! receipt domain = b"base-relearn-receipt-v1" +//! ``` +//! +//! Distinct from `design` / `prism` so leaf digests never collide. +//! No TDX / Phala CVM — master-centralized Lium eval. + +#![forbid(unsafe_code)] +#![allow(clippy::doc_markdown)] + +/// Normative challenge id (trust-root / leaf `challenge_id` string). +pub const CHALLENGE_ID: &str = "relearn"; + +/// UTF-8 bytes of [`CHALLENGE_ID`]. +pub const CHALLENGE_ID_BYTES: &[u8] = b"relearn"; + +/// Live `challenge_scoring_version` (displacement vs champion + gates). +pub const SCORING_VERSION: u16 = 1; + +/// Domain tag for task id digests. +pub const TASK_ID_DOMAIN: &[u8] = b"base-relearn-task-id-v1"; + +/// Domain tag for holdout slice ids. +pub const HOLDOUT_DOMAIN: &[u8] = b"base-relearn-holdout-v1"; + +/// Domain tag for eval-receipt digests. +pub const RECEIPT_DOMAIN: &[u8] = b"base-relearn-receipt-v1"; + +/// Domain tag for promotion attestations. +pub const PROMOTE_DOMAIN: &[u8] = b"base-relearn-promote-v1"; + +/// Integer score lattice max (same scale as other challenges). +pub const SCORE_MAX: u64 = 1_000_000; + +/// Verified Hugging Face id for the base model miners improve. +/// +/// Confirmed 2026-08-29: +pub const BASE_MODEL_ID: &str = "Qwen/Qwen3.8-Flash-Next"; + +/// Verified Hugging Face id for the frozen teacher / judge. +/// +/// Confirmed 2026-08-29: +pub const TEACHER_MODEL_ID: &str = "zai-org/GLM-5.3"; + +/// Community NVFP4 checkpoint for Blackwell serving (not the scored artifact). +pub const TEACHER_NVFP4_ID: &str = "Inferact/GLM-5.3-NVFP4"; + +/// Public miner / eval-image repo. +pub const RELEARN_GIT_URL: &str = "https://github.com/CortexLM/relearn"; + +/// Teacher serving mode. NVFP4-on-Lium is preferred when a Blackwell node +/// can be rented; otherwise a teacher-only HTTP API is the documented fallback. +/// Miner weights are never served through the teacher API. +#[derive(Debug, Clone, Copy, PartialEq, Eq, serde::Serialize, serde::Deserialize)] +#[serde(rename_all = "snake_case")] +pub enum TeacherBackend { + /// Frozen GLM-5.3 NVFP4 on a digest-pinned Lium pod (preferred). + LiumNvfp4, + /// Teacher-only OpenAI-compatible HTTP API (fallback). + HttpApi, + /// Deterministic offline judge (CI / local). + Sim, +} + +impl TeacherBackend { + /// Parse from env (`RELEARN_TEACHER_BACKEND`). + #[must_use] + pub fn from_env() -> Self { + match std::env::var("RELEARN_TEACHER_BACKEND") + .unwrap_or_default() + .to_ascii_lowercase() + .as_str() + { + "lium" | "lium_nvfp4" | "nvfp4" => Self::LiumNvfp4, + "http" | "http_api" | "api" => Self::HttpApi, + _ => Self::Sim, + } + } +} + +/// Default teacher backend for v0: HTTP API (or Sim when no URL). +/// +/// GLM-5.3 is a ~743B MoE. NVFP4 (`Inferact/GLM-5.3-NVFP4`) wants an 8× +/// Blackwell-class host. That is the preferred production path when the +/// operator can rent it; it is not the default for CI or a laptop. +#[must_use] +pub fn default_teacher_backend(api_url_set: bool) -> TeacherBackend { + if api_url_set { + TeacherBackend::HttpApi + } else { + TeacherBackend::Sim + } +} + +#[cfg(test)] +mod tests { + use super::*; + + #[test] + fn challenge_id_is_relearn() { + assert_eq!(CHALLENGE_ID, "relearn"); + assert_eq!(CHALLENGE_ID_BYTES, b"relearn"); + assert_ne!(CHALLENGE_ID, "prism"); + assert_ne!(CHALLENGE_ID, "design"); + } + + #[test] + fn verified_model_ids() { + assert_eq!(BASE_MODEL_ID, "Qwen/Qwen3.8-Flash-Next"); + assert_eq!(TEACHER_MODEL_ID, "zai-org/GLM-5.3"); + assert!(TEACHER_NVFP4_ID.contains("NVFP4")); + } + + #[test] + fn teacher_backend_never_defaults_to_serving_miner_weights() { + assert_eq!(default_teacher_backend(false), TeacherBackend::Sim); + assert_eq!(default_teacher_backend(true), TeacherBackend::HttpApi); + } + + #[test] + fn domain_tags_are_relearn_prefixed() { + assert!(std::str::from_utf8(TASK_ID_DOMAIN) + .unwrap_or("") + .contains("relearn")); + assert!(!std::str::from_utf8(TASK_ID_DOMAIN) + .unwrap_or("") + .contains("prism")); + } +} diff --git a/crates/relearn-challenge/Cargo.toml b/crates/relearn-challenge/Cargo.toml new file mode 100644 index 000000000..0cf908401 --- /dev/null +++ b/crates/relearn-challenge/Cargo.toml @@ -0,0 +1,26 @@ +[package] +name = "relearn-challenge" +description = "Relearn orchestrator: submit, eval, operator promote, D24 emit" +version.workspace = true +edition.workspace = true +license.workspace = true +repository.workspace = true +rust-version.workspace = true +publish = false + +[dependencies] +bundle = { path = "../bundle" } +challenge-common = { path = "../challenge-common" } +crypto = { path = "../crypto" } +hex = "0.4" +relearn-challenge-task = { path = "../relearn-challenge-task" } +relearn-eval = { path = "../relearn-eval" } +relearn-http = { path = "../relearn-http" } +relearn-score = { path = "../relearn-score" } +relearn-store = { path = "../relearn-store" } +serde = { version = "1", features = ["derive"] } +serde_json = "1" +thiserror = "2" + +[lints] +workspace = true diff --git a/crates/relearn-challenge/src/lib.rs b/crates/relearn-challenge/src/lib.rs new file mode 100644 index 000000000..42b629f98 --- /dev/null +++ b/crates/relearn-challenge/src/lib.rs @@ -0,0 +1,127 @@ +//! Relearn orchestrator helpers: D24 leaf plan + crate re-exports. + +#![forbid(unsafe_code)] +#![allow( + clippy::missing_errors_doc, + clippy::doc_markdown, + clippy::must_use_candidate +)] + +use std::collections::{BTreeMap, BTreeSet}; + +use bundle::{NoScoreReasonCode, ScoreOrAbsence}; +use challenge_common::{emit_signed_leaf_set, Hotkey, LeafEmitError}; +use relearn_challenge_task::{CHALLENGE_ID_BYTES, SCORE_MAX}; +use relearn_score::champion_hold_lattice; +use relearn_store::SubmissionState; + +pub use relearn_challenge_task::{ + BASE_MODEL_ID, CHALLENGE_ID, CHALLENGE_ID_BYTES as RELEARN_ID_BYTES, + SCORE_MAX as RELEARN_SCORE_MAX, SCORING_VERSION, TEACHER_MODEL_ID, +}; +pub use relearn_eval::{resolve_teacher_backend, RelearnPin}; +pub use relearn_http::{hash_admin_token, relearn_router, AppState}; +pub use relearn_store::MemoryStore; + +/// Build a D24-complete score map: champion (if any) gets a positive lattice; +/// everyone else is explicit `NoScore` (never silent). +pub fn emission_scores( + expected: &BTreeSet, + champion_hotkey: Option, + champion_lattice: u64, +) -> BTreeMap { + expected + .iter() + .map(|h| { + let s = match champion_hotkey { + Some(c) if c == *h && champion_lattice > 0 => ScoreOrAbsence::Score { + value: champion_lattice.min(SCORE_MAX), + }, + _ => ScoreOrAbsence::NoScore { + reason: NoScoreReasonCode::NotAttempted, + }, + }; + (*h, s) + }) + .collect() +} + +/// Sign the exact-E leaf set for this epoch. +pub fn emit_epoch( + secret: &[u8; 32], + epoch: u64, + expected: &BTreeSet, + champion_hotkey: Option, + champion_lattice: u64, +) -> Result, LeafEmitError> { + let scores = emission_scores(expected, champion_hotkey, champion_lattice); + emit_signed_leaf_set(secret, CHALLENGE_ID_BYTES, epoch, expected, &scores) +} + +/// Lattice for the current store champion, or the hold value when only +/// the base model is live (burn is wrong: the factory still has a champ). +pub fn live_champion_lattice(store: &MemoryStore) -> u64 { + if let Ok(Some(id)) = store.champion_id() { + if let Ok(row) = store.get(&id) { + if row.state == SubmissionState::Champion { + if let Some(v) = row.verdict { + if v.eligible && v.lattice > 0 { + return v.lattice; + } + } + return champion_hold_lattice(); + } + } + } + 0 +} + +/// Parse a 64-hex hotkey. +pub fn parse_hotkey(hex_s: &str) -> Option { + let t = hex_s.trim().trim_start_matches("0x"); + let bytes = hex::decode(t).ok()?; + <[u8; 32]>::try_from(bytes).ok() +} + +#[cfg(test)] +mod tests { + use super::*; + use challenge_common::public_key_from_secret; + use crypto::KEY_LEN; + + fn sk() -> [u8; KEY_LEN] { + let mut s = [7u8; KEY_LEN]; + s[0] = 1; + s + } + + #[test] + fn d24_covers_every_hotkey() { + let a = [1u8; 32]; + let b = [2u8; 32]; + let mut e = BTreeSet::new(); + e.insert(a); + e.insert(b); + let leaves = emit_epoch(&sk(), 9, &e, Some(a), 12_000).expect("emit"); + assert_eq!(leaves.len(), 2); + assert!(matches!( + leaves[&a].score_or_absence, + ScoreOrAbsence::Score { value: 12_000 } + )); + assert!(matches!( + leaves[&b].score_or_absence, + ScoreOrAbsence::NoScore { .. } + )); + let pk = public_key_from_secret(&sk()).expect("pk"); + for leaf in leaves.values() { + challenge_common::verify_leaf_sig(leaf, &pk).expect("sig"); + } + } + + #[test] + fn never_emits_score_on_empty_expected() { + let e = BTreeSet::new(); + let leaves = emit_epoch(&sk(), 1, &e, None, 0).expect("empty E"); + assert!(leaves.is_empty()); + } +} diff --git a/crates/relearn-eval/Cargo.toml b/crates/relearn-eval/Cargo.toml new file mode 100644 index 000000000..52d540fb6 --- /dev/null +++ b/crates/relearn-eval/Cargo.toml @@ -0,0 +1,29 @@ +[package] +name = "relearn-eval" +description = "Relearn eval loop: digest freeze, holdout unseal, Lium/sim, receipts" +version.workspace = true +edition.workspace = true +license.workspace = true +repository.workspace = true +rust-version.workspace = true +publish = false + +[dependencies] +async-trait = "0.1" +hex = "0.4" +prism-competition = { path = "../prism-competition" } +prism-lium = { path = "../prism-lium" } +prism-lium-types = { path = "../prism-lium-types" } +relearn-challenge-task = { path = "../relearn-challenge-task" } +relearn-score = { path = "../relearn-score" } +relearn-store = { path = "../relearn-store" } +serde = { version = "1", features = ["derive"] } +serde_json = "1" +sha2 = "0.10" +thiserror = "2" + +[dev-dependencies] +tokio = { version = "1", features = ["macros", "rt"] } + +[lints] +workspace = true diff --git a/crates/relearn-eval/src/lib.rs b/crates/relearn-eval/src/lib.rs new file mode 100644 index 000000000..133cbcc0c --- /dev/null +++ b/crates/relearn-eval/src/lib.rs @@ -0,0 +1,369 @@ +//! Relearn eval loop: freeze digest → unseal holdout → rent/sim → harvest. +//! +//! Miner pays Lium (`LIUM_API_KEY` / `X-Lium-Api-Key`). The control plane +//! only ever boots a digest-pinned eval image. Teacher HTTP is judge-only +//! and never serves miner weights as the scored artifact. + +#![forbid(unsafe_code)] +#![allow( + clippy::missing_errors_doc, + clippy::doc_markdown, + clippy::cast_precision_loss, + clippy::cast_possible_truncation, + clippy::must_use_candidate, + clippy::significant_drop_tightening +)] + +use prism_competition::ExampleSeries; +use prism_lium::{EvalJobBackend, SimLiumBackend}; +use prism_lium_types::{EvalReceipt, InstanceSpec, NoScoreGate, RemoteExecResult}; +use relearn_challenge_task::{ + default_teacher_backend, TeacherBackend, BASE_MODEL_ID, TEACHER_MODEL_ID, TEACHER_NVFP4_ID, +}; +use relearn_score::SliceScores; +use relearn_store::{unseal_holdout, Holdout}; +use serde::{Deserialize, Serialize}; +use sha2::{Digest, Sha256}; +use thiserror::Error; + +/// Pins Cortex stores for the split `CortexLM/relearn` repo + eval image. +#[derive(Debug, Clone, PartialEq, Eq, Serialize, Deserialize)] +pub struct RelearnPin { + /// `Qwen/Qwen3.8-Flash-Next`. + pub base_model: String, + /// `zai-org/GLM-5.3`. + pub teacher_model: String, + /// Optional NVFP4 id for Lium serving. + pub teacher_nvfp4: String, + /// `lium_nvfp4` | `http_api` | `sim`. + pub teacher_backend: TeacherBackend, + /// Eval image reference (no floating tag in prod). + pub eval_image: String, + /// `sha256:…` digest. Empty until the first green relearn CI image. + pub eval_image_digest: String, + /// `https://github.com/CortexLM/relearn`. + pub relearn_git: String, + /// Pinned git SHA of CortexLM/relearn (empty until first push). + pub relearn_git_sha: String, +} + +impl Default for RelearnPin { + fn default() -> Self { + Self { + base_model: BASE_MODEL_ID.into(), + teacher_model: TEACHER_MODEL_ID.into(), + teacher_nvfp4: TEACHER_NVFP4_ID.into(), + teacher_backend: TeacherBackend::Sim, + eval_image: "ghcr.io/cortexlm/relearn-eval".into(), + eval_image_digest: String::new(), + relearn_git: relearn_challenge_task::RELEARN_GIT_URL.into(), + relearn_git_sha: String::new(), + } + } +} + +impl RelearnPin { + /// Load from `config/relearn-pin.toml` (best-effort key=value / toml-ish). + #[must_use] + pub fn from_toml(body: &str) -> Self { + let mut pin = Self::default(); + for raw in body.lines() { + let line = raw.trim(); + if line.is_empty() || line.starts_with('#') { + continue; + } + let Some((k, v)) = line.split_once('=') else { + continue; + }; + let key = k.trim(); + let val = v.trim().trim_matches('"').to_owned(); + match key { + "base_model" => pin.base_model = val, + "teacher_model" => pin.teacher_model = val, + "teacher_nvfp4" => pin.teacher_nvfp4 = val, + "eval_image" => pin.eval_image = val, + "eval_image_digest" => pin.eval_image_digest = val, + "relearn_git" => pin.relearn_git = val, + "relearn_git_sha" => pin.relearn_git_sha = val, + "teacher_backend" => { + pin.teacher_backend = match val.as_str() { + "lium_nvfp4" => TeacherBackend::LiumNvfp4, + "http_api" => TeacherBackend::HttpApi, + _ => TeacherBackend::Sim, + }; + } + _ => {} + } + } + pin + } + + /// True when a live rent is allowed (real digest pin present). + #[must_use] + pub fn can_rent(&self) -> bool { + self.eval_image_digest.starts_with("sha256:") && self.eval_image_digest.len() >= 71 + } +} + +/// Eval errors. +#[derive(Debug, Error)] +pub enum EvalError { + /// Holdout was requested before the digest freeze. + #[error("holdout still sealed")] + HoldoutSealed, + /// Integrity gate failed. + #[error("integrity: {0}")] + Integrity(String), + /// Lium / backend failure. + #[error("backend: {0}")] + Backend(String), + /// Teacher API is not allowed to receive miner weights. + #[error("teacher API refused miner-weight payload")] + TeacherMinerWeights, +} + +/// One finished eval. +#[derive(Debug, Clone)] +pub struct EvalOutcome { + /// Challenger measurements. + pub scores: SliceScores, + /// Integrity receipt. + pub receipt: EvalReceipt, + /// Holdout after unseal (seed visible only here). + pub holdout: Holdout, +} + +/// Deterministic sim scores from a frozen digest + holdout seed. +#[must_use] +pub fn sim_slice_scores(artifact_digest: &str, holdout_seed: &str) -> SliceScores { + let holdout = series_from("h", artifact_digest, holdout_seed, 120, 0.15); + let public = series_from("p", artifact_digest, holdout_seed, 120, 0.0); + let perturbed = series_from( + "x", + artifact_digest, + &format!("{holdout_seed}-p"), + 120, + -0.02, + ); + let canaries = series_from("c", "canary", holdout_seed, 40, 0.45); + SliceScores { + holdout, + public, + perturbed, + canaries, + agent_trace: 0.85, + } +} + +/// Fixed base-model champion (Qwen3.8-Flash-Next, no miner adapter). +#[must_use] +pub fn base_champion_scores() -> SliceScores { + sim_slice_scores("base-qwen-3.8-flash-next", "base-seed") +} + +fn series_from(prefix: &str, digest: &str, seed: &str, n: usize, bias: f64) -> ExampleSeries { + let mut h = Sha256::new(); + h.update(digest.as_bytes()); + h.update([0xff]); + h.update(seed.as_bytes()); + let root = h.finalize(); + let pairs = (0..n).map(|i| { + let v = f64::from(root[i % 32]) / 255.0; + let score = (0.45 + 0.4 * v + bias).clamp(0.0, 1.0); + (format!("{prefix}{i}"), score) + }); + ExampleSeries::from_pairs(pairs) +} + +/// Unseal holdout only after `frozen_digest` is recorded, then score. +pub fn eval_after_freeze( + pending: &Holdout, + frozen_digest: &str, + artifact_digest: &str, +) -> Result { + let holdout = unseal_holdout(pending, frozen_digest).ok_or(EvalError::HoldoutSealed)?; + if !holdout.unsealed { + return Err(EvalError::HoldoutSealed); + } + let scores = sim_slice_scores(artifact_digest, &holdout.seed_hex); + let metrics = serde_json::to_vec(&serde_json::json!({ + "holdout_n": scores.holdout.len(), + "agent_trace": scores.agent_trace, + })) + .unwrap_or_default(); + let receipt = EvalReceipt { + provider: "sim".into(), + pod_id: format!("sim-{}", &frozen_digest[..8.min(frozen_digest.len())]), + image_digest: String::new(), + submission_hash: frozen_digest.to_owned(), + metrics_hash: EvalReceipt::hash_metrics_bytes(&metrics), + termination_verified: true, + }; + NoScoreGate::check(&receipt, false).map_err(|e| EvalError::Integrity(e.to_string()))?; + Ok(EvalOutcome { + scores, + receipt, + holdout, + }) +} + +/// Teacher request: prompts only. Rejects miner-weight bodies. +#[derive(Debug, Clone, Serialize, Deserialize)] +pub struct TeacherJudgeRequest { + /// Prompt / completion pair to judge. + pub prompt: String, + /// Candidate text. + pub candidate: String, + /// Must be the teacher model id, never a miner artifact. + pub model: String, +} + +/// Refuse any attempt to send miner weights through the teacher API. +pub fn teacher_judge_guard(req: &TeacherJudgeRequest, pin: &RelearnPin) -> Result<(), EvalError> { + if req.model != pin.teacher_model && req.model != TEACHER_MODEL_ID { + return Err(EvalError::TeacherMinerWeights); + } + let lower = req.candidate.to_ascii_lowercase(); + if lower.contains("safetensors") || lower.contains("gguf") || lower.contains("nvfp4") { + return Err(EvalError::TeacherMinerWeights); + } + Ok(()) +} + +/// Resolve the v0 teacher backend. NVFP4-on-Lium is preferred when the +/// operator sets `RELEARN_TEACHER_BACKEND=lium` **and** can rent an 8× +/// Blackwell host; otherwise HTTP API (if `RELEARN_TEACHER_API_URL` is set) +/// or Sim. Miner weights are never the served model. +#[must_use] +pub fn resolve_teacher_backend() -> TeacherBackend { + let env = TeacherBackend::from_env(); + if env == TeacherBackend::LiumNvfp4 { + return TeacherBackend::LiumNvfp4; + } + let api = std::env::var("RELEARN_TEACHER_API_URL") + .ok() + .filter(|s| !s.trim().is_empty()); + if env == TeacherBackend::HttpApi || api.is_some() { + return default_teacher_backend(api.is_some()); + } + TeacherBackend::Sim +} + +/// Rent a digest-pinned eval pod, exec, harvest, terminate. +/// +/// `api_key` is used only to construct the backend the caller already built. +/// This function never logs it. Live rent is skipped when `pin.can_rent()` is +/// false (no published eval digest yet). +pub async fn rent_eval( + backend: &dyn EvalJobBackend, + pin: &RelearnPin, + frozen_digest: &str, + artifact_digest: &str, +) -> Result<(RemoteExecResult, String), EvalError> { + if !pin.can_rent() { + return Err(EvalError::Integrity( + "eval image digest not pinned; refuse live rent".into(), + )); + } + let spec = InstanceSpec { + name: format!("relearn-{}", &frozen_digest[..12.min(frozen_digest.len())]), + max_lifetime_hours: 1.0, + max_price_per_hour: 8.0, + gpu_count: 1, + image_digest: Some(pin.eval_image_digest.clone()), + ssh_public_keys: Vec::new(), + ssh_key_name: None, + preferred_offer_id: None, + template_id: None, + template_name: None, + }; + let inst = backend + .provision(&spec) + .await + .map_err(|e| EvalError::Backend(e.to_string()))?; + let exec = backend + .exec_eval(&inst.id, artifact_digest, frozen_digest, None) + .await + .map_err(|e| EvalError::Backend(e.to_string())); + let term = backend.terminate(&inst.id).await; + let verified = backend.verify_terminated(&inst.id).await.unwrap_or(false); + if let Err(e) = term { + return Err(EvalError::Backend(e.to_string())); + } + if !verified { + return Err(EvalError::Integrity("pod terminate not verified".into())); + } + exec.map(|r| (r, inst.id)) +} + +/// Convenience: sim backend rent that always tears down. +pub async fn sim_rent_roundtrip(digest: &str) -> Result { + let backend = SimLiumBackend::new(); + let pin = RelearnPin { + eval_image_digest: format!("sha256:{}", "ab".repeat(32)), + ..RelearnPin::default() + }; + let (_r, id) = rent_eval(&backend, &pin, digest, digest).await?; + Ok(id) +} + +#[cfg(test)] +mod tests { + use super::*; + use relearn_store::sealed_holdout; + + #[test] + fn unseal_happens_only_after_freeze() { + let pending = sealed_holdout(1, "digest-a"); + assert!(eval_after_freeze(&pending, "", "art").is_err()); + let out = eval_after_freeze(&pending, "digest-a", "art").expect("eval"); + assert!(out.holdout.unsealed); + assert_eq!(out.receipt.submission_hash, "digest-a"); + assert!(out.scores.holdout.len() >= 100); + } + + #[test] + fn teacher_guard_rejects_miner_weight_payload() { + let pin = RelearnPin::default(); + let bad = TeacherJudgeRequest { + prompt: "score".into(), + candidate: "here is a safetensors blob".into(), + model: TEACHER_MODEL_ID.into(), + }; + assert!(teacher_judge_guard(&bad, &pin).is_err()); + let good = TeacherJudgeRequest { + prompt: "score".into(), + candidate: "the capital is paris".into(), + model: TEACHER_MODEL_ID.into(), + }; + assert!(teacher_judge_guard(&good, &pin).is_ok()); + } + + #[test] + fn pin_refuses_rent_without_digest() { + assert!(!RelearnPin::default().can_rent()); + let p = RelearnPin { + eval_image_digest: format!("sha256:{}", "00".repeat(32)), + ..RelearnPin::default() + }; + assert!(p.can_rent()); + } + + #[tokio::test] + async fn sim_rent_tears_down() { + let id = sim_rent_roundtrip("abcdef0123456789").await.expect("rent"); + assert!(id.contains("sim-pod")); + } + + #[test] + fn toml_pin_roundtrip() { + let body = r#" +base_model = "Qwen/Qwen3.8-Flash-Next" +teacher_model = "zai-org/GLM-5.3" +teacher_backend = "http_api" +"#; + let p = RelearnPin::from_toml(body); + assert_eq!(p.base_model, BASE_MODEL_ID); + assert_eq!(p.teacher_backend, TeacherBackend::HttpApi); + } +} diff --git a/crates/relearn-http/Cargo.toml b/crates/relearn-http/Cargo.toml new file mode 100644 index 000000000..298216d5a --- /dev/null +++ b/crates/relearn-http/Cargo.toml @@ -0,0 +1,28 @@ +[package] +name = "relearn-http" +description = "Relearn HTTP surface: submit, status, admin promote" +version.workspace = true +edition.workspace = true +license.workspace = true +repository.workspace = true +rust-version.workspace = true +publish = false + +[dependencies] +axum = { version = "0.8", default-features = false, features = ["http1", "tokio", "json"] } +hex = "0.4" +relearn-challenge-task = { path = "../relearn-challenge-task" } +relearn-eval = { path = "../relearn-eval" } +relearn-score = { path = "../relearn-score" } +relearn-store = { path = "../relearn-store" } +serde = { version = "1", features = ["derive"] } +serde_json = "1" +sha2 = "0.10" +tokio = { version = "1", features = ["macros", "rt", "sync"] } + +[dev-dependencies] +http-body-util = "0.1" +tower = { version = "0.5", features = ["util"] } + +[lints] +workspace = true diff --git a/crates/relearn-http/src/lib.rs b/crates/relearn-http/src/lib.rs new file mode 100644 index 000000000..f0587e2f3 --- /dev/null +++ b/crates/relearn-http/src/lib.rs @@ -0,0 +1,386 @@ +//! Relearn HTTP API (master-only). +//! +//! ```text +//! GET /health +//! GET /v1/status +//! POST /v1/submissions miner submit (digest + optional X-Lium-Api-Key) +//! GET /v1/submissions +//! GET /v1/submissions/{id} +//! POST /v1/admin/promote operator-audited champion flip +//! ``` + +#![forbid(unsafe_code)] +#![allow( + clippy::missing_errors_doc, + clippy::doc_markdown, + clippy::must_use_candidate, + clippy::items_after_statements, + clippy::too_many_lines +)] + +use std::sync::Arc; + +use axum::extract::{Path, State}; +use axum::http::{HeaderMap, StatusCode}; +use axum::response::IntoResponse; +use axum::routing::{get, post}; +use axum::{Json, Router}; +use relearn_challenge_task::{CHALLENGE_ID, SCORE_MAX, SCORING_VERSION}; +use relearn_eval::{base_champion_scores, eval_after_freeze, resolve_teacher_backend, RelearnPin}; +use relearn_score::judge_challenger; +use relearn_store::{ + freeze_submission_digest, public_holdout, sealed_holdout, MemoryStore, Submission, + SubmissionState, +}; +use serde::{Deserialize, Serialize}; +use sha2::{Digest, Sha256}; + +/// Shared HTTP state. +#[derive(Clone)] +pub struct AppState { + /// Submission store. + pub store: MemoryStore, + /// Eval / model pins. + pub pin: RelearnPin, + /// Operator bearer hashes (sha256 hex). Empty → admin 503. + pub admin_hashes: Arc>, +} + +/// Build the router. +pub fn relearn_router(state: AppState) -> Router { + Router::new() + .route("/health", get(health)) + .route("/v1/status", get(status)) + .route("/v1/submissions", post(submit).get(list_subs)) + .route("/v1/submissions/{id}", get(get_sub)) + .route("/v1/admin/promote", post(promote)) + .with_state(state) +} + +async fn health() -> impl IntoResponse { + Json(serde_json::json!({ + "ok": true, + "challenge_id": CHALLENGE_ID, + "scoring_version": SCORING_VERSION, + })) +} + +async fn status(State(st): State) -> impl IntoResponse { + let champ = st.store.champion_id().ok().flatten(); + Json(serde_json::json!({ + "challenge_id": CHALLENGE_ID, + "scoring_version": SCORING_VERSION, + "score_max": SCORE_MAX, + "base_model": st.pin.base_model, + "teacher_model": st.pin.teacher_model, + "teacher_backend": resolve_teacher_backend(), + "eval_image": st.pin.eval_image, + "eval_image_digest": st.pin.eval_image_digest, + "relearn_git": st.pin.relearn_git, + "relearn_git_sha": st.pin.relearn_git_sha, + "champion_id": champ, + "tdx": false, + "phala_cvm": false, + })) +} + +#[derive(Debug, Deserialize)] +struct SubmitBody { + miner_hotkey: String, + artifact_digest: String, + artifact_uri: Option, +} + +#[derive(Debug, Serialize)] +struct SubmitResp { + id: String, + submission_digest: String, + state: SubmissionState, + holdout_unsealed: bool, + eligible: bool, +} + +fn parse_hex64(s: &str, field: &str) -> Result)> { + let t = s.trim().trim_start_matches("0x"); + if t.len() != 64 || !t.chars().all(|c| c.is_ascii_hexdigit()) { + return Err(( + StatusCode::BAD_REQUEST, + Json(serde_json::json!({"error": format!("invalid {field}")})), + )); + } + Ok(t.to_ascii_lowercase()) +} + +fn nonce_from(hotkey: &str, digest: &str) -> String { + let mut h = Sha256::new(); + h.update(b"relearn-nonce-v1"); + h.update(hotkey.as_bytes()); + h.update(digest.as_bytes()); + hex::encode(h.finalize()) +} + +async fn submit( + State(st): State, + headers: HeaderMap, + Json(body): Json, +) -> Result)> { + let hotkey = parse_hex64(&body.miner_hotkey, "miner_hotkey")?; + let artifact = parse_hex64(&body.artifact_digest, "artifact_digest")?; + // Miner BYOK: accepted and never logged. Absence is OK for sim. + let _lium_present = headers + .get("x-lium-api-key") + .and_then(|v| v.to_str().ok()) + .is_some_and(|s| !s.is_empty()); + + let nonce = nonce_from(&hotkey, &artifact); + let submission_digest = freeze_submission_digest(&hotkey, &artifact, &nonce); + let pending = sealed_holdout(0, &submission_digest); + + let row = Submission { + id: String::new(), + miner_hotkey: hotkey, + artifact_digest: artifact.clone(), + artifact_uri: body.artifact_uri, + nonce, + submission_digest: submission_digest.clone(), + state: SubmissionState::Evaluating, + receipt_json: None, + verdict: None, + detail: None, + }; + let row = st + .store + .insert(row) + .map_err(|_| err(StatusCode::INTERNAL_SERVER_ERROR, "store"))?; + + let eval = eval_after_freeze(&pending, &submission_digest, &artifact) + .map_err(|e| err(StatusCode::INTERNAL_SERVER_ERROR, &e.to_string()))?; + let _public = public_holdout(&eval.holdout); + + let champ = st + .store + .champion_scores() + .ok() + .flatten() + .unwrap_or_else(base_champion_scores); + let verdict = judge_challenger(&champ, &eval.scores); + st.store + .record_scores(&row.id, eval.scores.clone()) + .map_err(|_| err(StatusCode::INTERNAL_SERVER_ERROR, "store"))?; + let eligible = verdict.eligible; + let state = if eligible { + SubmissionState::AwaitingAdmin + } else { + SubmissionState::Rejected + }; + let receipt = serde_json::to_string(&eval.receipt).unwrap_or_default(); + let detail = if eligible { + None + } else { + Some(format!("gates={:?}", verdict.failed)) + }; + let row = st + .store + .patch(&row.id, Some(state), Some(receipt), Some(verdict), detail) + .map_err(|_| err(StatusCode::INTERNAL_SERVER_ERROR, "store"))?; + + Ok(( + StatusCode::CREATED, + Json(SubmitResp { + id: row.id, + submission_digest: row.submission_digest, + state: row.state, + holdout_unsealed: eval.holdout.unsealed, + eligible, + }), + )) +} + +async fn list_subs(State(st): State) -> impl IntoResponse { + let rows = st.store.list().unwrap_or_default(); + Json(serde_json::json!({ "items": rows })) +} + +async fn get_sub( + State(st): State, + Path(id): Path, +) -> Result)> { + let row = st + .store + .get(&id) + .map_err(|_| err(StatusCode::NOT_FOUND, "not_found"))?; + Ok(Json(row)) +} + +#[derive(Debug, Deserialize)] +struct PromoteBody { + submission_id: String, +} + +async fn promote( + State(st): State, + headers: HeaderMap, + Json(body): Json, +) -> Result)> { + if st.admin_hashes.is_empty() { + return Err(err(StatusCode::SERVICE_UNAVAILABLE, "auth_unconfigured")); + } + if !admin_ok(&headers, &st.admin_hashes) { + return Err(err(StatusCode::UNAUTHORIZED, "unauthorized")); + } + let row = st.store.promote(&body.submission_id).map_err(|e| { + let code = if e.to_string().contains("unknown") { + StatusCode::NOT_FOUND + } else { + StatusCode::CONFLICT + }; + err(code, &e.to_string()) + })?; + Ok(Json(row)) +} + +fn admin_ok(headers: &HeaderMap, hashes: &[String]) -> bool { + let Some(raw) = headers + .get(axum::http::header::AUTHORIZATION) + .and_then(|v| v.to_str().ok()) + else { + return false; + }; + let token = raw.strip_prefix("Bearer ").unwrap_or(raw).trim(); + if token.is_empty() { + return false; + } + let mut h = Sha256::new(); + h.update(token.as_bytes()); + let got = hex::encode(h.finalize()); + hashes.iter().any(|x| x == &got) +} + +fn err(code: StatusCode, msg: &str) -> (StatusCode, Json) { + (code, Json(serde_json::json!({ "error": msg }))) +} + +/// Hash an admin token the same way the server does. +#[must_use] +pub fn hash_admin_token(token: &str) -> String { + let mut h = Sha256::new(); + h.update(token.as_bytes()); + hex::encode(h.finalize()) +} + +#[cfg(test)] +mod tests { + use super::*; + use axum::body::Body; + use axum::http::Request; + use http_body_util::BodyExt; + use tower::ServiceExt; + + fn digest(label: &str) -> String { + let mut h = Sha256::new(); + h.update(label.as_bytes()); + hex::encode(h.finalize()) + } + + async fn json_req( + app: Router, + method: &str, + uri: &str, + body: serde_json::Value, + auth: Option<&str>, + ) -> (StatusCode, serde_json::Value) { + let mut b = Request::builder().method(method).uri(uri); + if let Some(a) = auth { + b = b.header(axum::http::header::AUTHORIZATION, format!("Bearer {a}")); + } + let req = b + .header("content-type", "application/json") + .body(Body::from(body.to_string())) + .expect("req"); + let resp = app.oneshot(req).await.expect("resp"); + let status = resp.status(); + let bytes = resp.into_body().collect().await.expect("body").to_bytes(); + let v: serde_json::Value = serde_json::from_slice(&bytes).unwrap_or(serde_json::json!({})); + (status, v) + } + + #[tokio::test] + async fn submit_eval_promote_happy_path() { + let token = "op-test-token"; + let store = MemoryStore::new(); + store + .set_base_champion(base_champion_scores()) + .expect("base"); + let app = relearn_router(AppState { + store, + pin: RelearnPin::default(), + admin_hashes: Arc::new(vec![hash_admin_token(token)]), + }); + + let (st, health) = + json_req(app.clone(), "GET", "/health", serde_json::json!({}), None).await; + assert_eq!(st, StatusCode::OK); + assert_eq!(health["challenge_id"], CHALLENGE_ID); + + // High-byte digest tends to beat the base champion in sim. + let artifact = digest("miner-strong-adapter"); + let (st, created) = json_req( + app.clone(), + "POST", + "/v1/submissions", + serde_json::json!({ + "miner_hotkey": digest("miner-hotkey"), + "artifact_digest": artifact, + }), + None, + ) + .await; + assert_eq!(st, StatusCode::CREATED, "{created}"); + assert_eq!( + created["submission_digest"].as_str().unwrap_or("").len(), + 64 + ); + assert!(created["holdout_unsealed"].as_bool().unwrap_or(false)); + + if created["eligible"] == true { + let id = created["id"].as_str().expect("id"); + let (st, promoted) = json_req( + app, + "POST", + "/v1/admin/promote", + serde_json::json!({ "submission_id": id }), + Some(token), + ) + .await; + assert_eq!(st, StatusCode::OK, "{promoted}"); + assert_eq!(promoted["state"], "champion"); + } + } + + #[tokio::test] + async fn promote_requires_bearer() { + let app = relearn_router(AppState { + store: MemoryStore::new(), + pin: RelearnPin::default(), + admin_hashes: Arc::new(vec![hash_admin_token("x")]), + }); + let (st, _) = json_req( + app, + "POST", + "/v1/admin/promote", + serde_json::json!({ "submission_id": "rl_0" }), + None, + ) + .await; + assert_eq!(st, StatusCode::UNAUTHORIZED); + } + + #[test] + fn model_pins_are_verified_ids() { + assert_eq!( + relearn_challenge_task::BASE_MODEL_ID, + "Qwen/Qwen3.8-Flash-Next" + ); + assert_eq!(relearn_challenge_task::TEACHER_MODEL_ID, "zai-org/GLM-5.3"); + } +} diff --git a/crates/relearn-score/Cargo.toml b/crates/relearn-score/Cargo.toml new file mode 100644 index 000000000..928cb62f5 --- /dev/null +++ b/crates/relearn-score/Cargo.toml @@ -0,0 +1,17 @@ +[package] +name = "relearn-score" +description = "Relearn displacement scoring, overfit gates, never-regression promote" +version.workspace = true +edition.workspace = true +license.workspace = true +repository.workspace = true +rust-version.workspace = true +publish = false + +[dependencies] +prism-competition = { path = "../prism-competition" } +relearn-challenge-task = { path = "../relearn-challenge-task" } +serde = { version = "1", features = ["derive"] } + +[lints] +workspace = true diff --git a/crates/relearn-score/src/lib.rs b/crates/relearn-score/src/lib.rs new file mode 100644 index 000000000..4efae90aa --- /dev/null +++ b/crates/relearn-score/src/lib.rs @@ -0,0 +1,272 @@ +//! Displacement scoring for Relearn. +//! +//! Score is challenger vs the previous champion on a shared holdout slice. +//! A regression is never crowned. Promotion additionally requires the +//! operator-audited paired win plus retention / overfit gates. + +#![forbid(unsafe_code)] +#![allow( + clippy::missing_errors_doc, + clippy::doc_markdown, + clippy::cast_precision_loss, + clippy::cast_possible_truncation +)] + +use prism_competition::{ + paired_test, Direction, ExampleSeries, PairedInput, PairedOutcome, PairedRefusal, DEADZONE, +}; +use relearn_challenge_task::SCORE_MAX; +use serde::{Deserialize, Serialize}; + +/// Maximum allowed public-private accuracy gap (absolute). +pub const MAX_PUBLIC_PRIVATE_GAP: f64 = 0.08; + +/// Maximum allowed drop under input perturbation (absolute). +pub const MAX_PERTURB_DROP: f64 = 0.05; + +/// Minimum canary accuracy (known-answer items the base model already solves). +pub const MIN_CANARY_ACCURACY: f64 = 0.95; + +/// Minimum agent-trace score (first-class; 0..1). +pub const MIN_AGENT_TRACE: f64 = 0.5; + +/// Per-example holdout measurements for one submission. +#[derive(Debug, Clone, PartialEq)] +pub struct SliceScores { + /// Holdout items (scored artifact). Higher is better. + pub holdout: ExampleSeries, + /// Public / training-adjacent canary slice (overfit detector). + pub public: ExampleSeries, + /// Same holdout items after a pinned perturbation. + pub perturbed: ExampleSeries, + /// Known-answer canaries (base-model already-correct items). + pub canaries: ExampleSeries, + /// Agent-trace quality in `[0, 1]` (first-class; not a side channel). + pub agent_trace: f64, +} + +impl SliceScores { + /// Mean of a series, or `None` when empty. + #[must_use] + pub fn mean(series: &ExampleSeries) -> Option { + if series.is_empty() { + return None; + } + let n = series.len() as f64; + Some(series.by_cluster.values().sum::() / n) + } +} + +/// Gate that blocked promotion (or would have). +#[derive(Debug, Clone, PartialEq, Eq, Serialize, Deserialize)] +#[serde(rename_all = "snake_case")] +pub enum GateFail { + /// Challenger is not a significant paired win. + NoPairedWin, + /// Challenger lost or tied the champion (never crown a regression). + Regression, + /// Public-private gap too large (memorization / contamination). + PublicPrivateGap, + /// Perturbed holdout collapsed (brittle / overfit). + Perturbation, + /// Canaries failed (catastrophic forgetting of base competence). + Canaries, + /// Agent-trace score below floor. + AgentTrace, + /// Paired test refused (slice mismatch / too thin). + PairedRefusal, +} + +/// Serializable paired-test summary (prism `PairedOutcome` is not serde). +#[derive(Debug, Clone, PartialEq, Eq, Serialize, Deserialize)] +pub struct PairedSummary { + /// Overlapping examples. + pub n_paired: u64, + /// Decided examples (outside the dead zone). + pub n_decided: u64, + /// Bootstrap LCB win-rate (bps). + pub win_rate_lcb_bps: u64, + /// Challenger displaces champion. + pub displaces: bool, +} + +impl PairedSummary { + fn from_outcome(o: &PairedOutcome) -> Self { + Self { + n_paired: u64::try_from(o.n_paired).unwrap_or(u64::MAX), + n_decided: u64::try_from(o.n_decided).unwrap_or(u64::MAX), + win_rate_lcb_bps: o.win_rate_lcb_bps, + displaces: o.displaces, + } + } +} + +/// Full promote / reject verdict. Consensus-critical once leaves are signed. +#[derive(Debug, Clone, PartialEq, Serialize, Deserialize)] +pub struct PromoteVerdict { + /// Whether this submission may become champion after operator audit. + pub eligible: bool, + /// Paired-test outcome when the slices lined up. + pub paired: Option, + /// Gates that failed (empty ⇒ all clear). + pub failed: Vec, + /// Lattice score to emit if this hotkey is the live champion (`0` otherwise). + pub lattice: u64, +} + +/// Judge challenger vs champion. Never returns `eligible` on a regression. +#[must_use] +pub fn judge_challenger(champion: &SliceScores, challenger: &SliceScores) -> PromoteVerdict { + let mut failed = Vec::new(); + + let input = PairedInput { + metric: "relearn.holdout".into(), + direction: Direction::HigherBetter, + slice_id: "holdout".into(), + champion: champion.holdout.clone(), + challenger: challenger.holdout.clone(), + }; + let paired_raw = match paired_test(&input) { + Ok(o) => Some(o), + Err( + PairedRefusal::NotEnoughDecided + | PairedRefusal::NoOverlap + | PairedRefusal::SliceMismatch, + ) => { + failed.push(GateFail::PairedRefusal); + None + } + }; + + match paired_raw { + Some(ref o) if o.displaces => {} + Some(_) => { + failed.push(GateFail::NoPairedWin); + failed.push(GateFail::Regression); + } + None => failed.push(GateFail::NoPairedWin), + } + + if let (Some(pub_m), Some(priv_m)) = ( + SliceScores::mean(&challenger.public), + SliceScores::mean(&challenger.holdout), + ) { + if (pub_m - priv_m).abs() > MAX_PUBLIC_PRIVATE_GAP + DEADZONE { + failed.push(GateFail::PublicPrivateGap); + } + } + + if let (Some(h), Some(p)) = ( + SliceScores::mean(&challenger.holdout), + SliceScores::mean(&challenger.perturbed), + ) { + if h - p > MAX_PERTURB_DROP + DEADZONE { + failed.push(GateFail::Perturbation); + } + } + + if let Some(c) = SliceScores::mean(&challenger.canaries) { + if c + DEADZONE < MIN_CANARY_ACCURACY { + failed.push(GateFail::Canaries); + } + } + + if challenger.agent_trace + DEADZONE < MIN_AGENT_TRACE { + failed.push(GateFail::AgentTrace); + } + + failed.sort_by(|a, b| format!("{a:?}").cmp(&format!("{b:?}"))); + failed.dedup(); + + let eligible = failed.is_empty(); + let lattice = if eligible { + paired_raw + .as_ref() + .map_or(0, |o| lattice_from_win_rate(o.win_rate_lcb_bps)) + } else { + 0 + }; + + PromoteVerdict { + eligible, + paired: paired_raw.as_ref().map(PairedSummary::from_outcome), + failed, + lattice, + } +} + +/// Map bootstrap LCB win-rate (bps) onto the lattice. Champion-hold → 0. +#[must_use] +pub fn lattice_from_win_rate(win_rate_lcb_bps: u64) -> u64 { + let clamped = win_rate_lcb_bps.min(10_000); + u64::from(u32::try_from((u128::from(SCORE_MAX) * u128::from(clamped)) / 10_000).unwrap_or(0)) +} + +/// Champion row always keeps a positive lattice so emission does not burn +/// solely because a challenger was rejected. +#[must_use] +pub fn champion_hold_lattice() -> u64 { + SCORE_MAX / 2 +} + +#[cfg(test)] +mod tests { + use super::*; + + fn series(prefix: &str, n: usize, val: f64) -> ExampleSeries { + ExampleSeries::from_pairs((0..n).map(|i| (format!("{prefix}{i}"), val))) + } + + fn slice(hold: f64, public: f64, pert: f64, canary: f64, trace: f64) -> SliceScores { + SliceScores { + holdout: series("h", 120, hold), + public: series("p", 120, public), + perturbed: series("x", 120, pert), + canaries: series("c", 40, canary), + agent_trace: trace, + } + } + + #[test] + fn never_crowns_regression() { + let champ = slice(0.80, 0.80, 0.79, 0.99, 0.9); + let worse = slice(0.40, 0.40, 0.39, 0.99, 0.9); + let v = judge_challenger(&champ, &worse); + assert!(!v.eligible); + assert!(v.failed.contains(&GateFail::Regression)); + assert_eq!(v.lattice, 0); + } + + #[test] + fn significant_win_plus_gates_is_eligible() { + let champ = slice(0.50, 0.50, 0.49, 0.99, 0.9); + let better = slice(0.80, 0.80, 0.79, 0.99, 0.9); + let v = judge_challenger(&champ, &better); + assert!(v.eligible, "expected eligible, failed={:?}", v.failed); + assert!(v.lattice > 0); + } + + #[test] + fn public_private_gap_blocks() { + let champ = slice(0.50, 0.50, 0.49, 0.99, 0.9); + let leak = slice(0.80, 0.99, 0.79, 0.99, 0.9); + let v = judge_challenger(&champ, &leak); + assert!(v.failed.contains(&GateFail::PublicPrivateGap)); + assert!(!v.eligible); + } + + #[test] + fn canary_collapse_blocks() { + let champ = slice(0.50, 0.50, 0.49, 0.99, 0.9); + let forget = slice(0.80, 0.80, 0.79, 0.20, 0.9); + let v = judge_challenger(&champ, &forget); + assert!(v.failed.contains(&GateFail::Canaries)); + assert!(!v.eligible); + } + + #[test] + fn lattice_is_zero_for_zero_bps() { + assert_eq!(lattice_from_win_rate(0), 0); + assert_eq!(lattice_from_win_rate(10_000), SCORE_MAX); + } +} diff --git a/crates/relearn-store/Cargo.toml b/crates/relearn-store/Cargo.toml new file mode 100644 index 000000000..4f3cd9701 --- /dev/null +++ b/crates/relearn-store/Cargo.toml @@ -0,0 +1,23 @@ +[package] +name = "relearn-store" +description = "In-memory Relearn submissions, champion, sealed holdout" +version.workspace = true +edition.workspace = true +license.workspace = true +repository.workspace = true +rust-version.workspace = true +publish = false + +[dependencies] +hex = "0.4" +relearn-challenge-task = { path = "../relearn-challenge-task" } +relearn-score = { path = "../relearn-score" } +serde = { version = "1", features = ["derive"] } +sha2 = "0.10" +thiserror = "2" + +[dev-dependencies] +prism-competition = { path = "../prism-competition" } + +[lints] +workspace = true diff --git a/crates/relearn-store/src/lib.rs b/crates/relearn-store/src/lib.rs new file mode 100644 index 000000000..207aea1d2 --- /dev/null +++ b/crates/relearn-store/src/lib.rs @@ -0,0 +1,406 @@ +//! In-memory Relearn store: submissions, sealed holdout, champion. +//! +//! Holdout items stay sealed until the submission digest is frozen. + +#![forbid(unsafe_code)] +#![allow( + clippy::missing_errors_doc, + clippy::doc_markdown, + clippy::must_use_candidate +)] + +use std::collections::BTreeMap; +use std::sync::{Arc, Mutex}; + +use relearn_challenge_task::HOLDOUT_DOMAIN; +use relearn_score::{PromoteVerdict, SliceScores}; +use serde::{Deserialize, Serialize}; +use sha2::{Digest, Sha256}; +use thiserror::Error; + +/// Submission lifecycle. +#[derive(Debug, Clone, Copy, PartialEq, Eq, Serialize, Deserialize)] +#[serde(rename_all = "snake_case")] +pub enum SubmissionState { + /// Digest accepted; holdout still sealed. + Accepted, + /// Digest frozen; holdout unsealed; eval running. + Evaluating, + /// Eval finished; waiting operator audit. + AwaitingAdmin, + /// Rejected (regression / gates / integrity). + Rejected, + /// Operator-promoted champion. + Champion, +} + +/// One miner submission. +#[derive(Debug, Clone, Serialize, Deserialize)] +pub struct Submission { + /// Stable id (`rl_` + 16 hex). + pub id: String, + /// 64-hex miner hotkey. + pub miner_hotkey: String, + /// SHA-256 hex of the miner artifact (weights / adapter). Frozen at accept. + pub artifact_digest: String, + /// Optional locator (HF repo, object URL). Never the scored teacher payload. + pub artifact_uri: Option, + /// Digest freeze nonce (hex). + pub nonce: String, + /// `sha256(hotkey || 0xff || artifact || 0xff || nonce)`. + pub submission_digest: String, + /// Lifecycle. + pub state: SubmissionState, + /// Eval receipt JSON (if any). + pub receipt_json: Option, + /// Judge verdict (if any). + pub verdict: Option, + /// Reject / gate reason. + pub detail: Option, +} + +/// Sealed holdout: items hidden until `unseal_after`. +#[derive(Debug, Clone, Serialize, Deserialize)] +pub struct Holdout { + /// Slice id bound into the paired test. + pub slice_id: String, + /// Hex seed. Empty in the public view until unsealed. + pub seed_hex: String, + /// Whether the seed has been revealed for this submission. + pub unsealed: bool, +} + +/// Store errors. +#[derive(Debug, Error)] +pub enum StoreError { + /// Lock poisoned. + #[error("store lock poisoned")] + Poison, + /// Unknown submission. + #[error("unknown submission {0}")] + NotFound(String), + /// Illegal state transition. + #[error("illegal state {0}")] + Illegal(String), +} + +/// In-memory store (v0). Postgres can replace this without changing the HTTP surface. +#[derive(Clone, Default)] +pub struct MemoryStore { + inner: Arc>, +} + +#[derive(Default)] +struct Inner { + next: u64, + submissions: BTreeMap, + champion_id: Option, + /// Per-submission holdout slices (in-memory; not serialized on the HTTP row). + scores: BTreeMap, + /// Live champion slices (promoted miner). Displacement is vs this, not the base. + champion_scores: Option, + /// Baseline champion scores (base model) until a miner is promoted. + base_champion: Option, +} + +impl MemoryStore { + /// Empty store. + #[must_use] + pub fn new() -> Self { + Self::default() + } + + fn lock(&self) -> Result, StoreError> { + self.inner.lock().map_err(|_| StoreError::Poison) + } + + /// Insert a newly accepted submission. + pub fn insert(&self, mut row: Submission) -> Result { + let mut g = self.lock()?; + if row.id.is_empty() { + let n = g.next; + g.next = g.next.saturating_add(1); + row.id = format!("rl_{n:016x}"); + } + g.submissions.insert(row.id.clone(), row.clone()); + Ok(row) + } + + /// Fetch one row. + pub fn get(&self, id: &str) -> Result { + let g = self.lock()?; + g.submissions + .get(id) + .cloned() + .ok_or_else(|| StoreError::NotFound(id.to_owned())) + } + + /// List newest-first. + pub fn list(&self) -> Result, StoreError> { + let g = self.lock()?; + let mut rows: Vec<_> = g.submissions.values().cloned().collect(); + rows.sort_by(|a, b| b.id.cmp(&a.id)); + Ok(rows) + } + + /// Patch state / verdict / receipt. + pub fn patch( + &self, + id: &str, + state: Option, + receipt_json: Option, + verdict: Option, + detail: Option, + ) -> Result { + let mut g = self.lock()?; + let row = g + .submissions + .get_mut(id) + .ok_or_else(|| StoreError::NotFound(id.to_owned()))?; + if let Some(s) = state { + row.state = s; + } + if let Some(r) = receipt_json { + row.receipt_json = Some(r); + } + if let Some(v) = verdict { + row.verdict = Some(v); + } + if let Some(d) = detail { + row.detail = Some(d); + } + Ok(row.clone()) + } + + /// Current champion submission id. + pub fn champion_id(&self) -> Result, StoreError> { + Ok(self.lock()?.champion_id.clone()) + } + + /// Promote `id` and demote the previous champion. + pub fn promote(&self, id: &str) -> Result { + let mut g = self.lock()?; + let prev = g.champion_id.clone(); + { + let row = g + .submissions + .get(id) + .ok_or_else(|| StoreError::NotFound(id.to_owned()))?; + if row.state != SubmissionState::AwaitingAdmin { + return Err(StoreError::Illegal(format!( + "promote requires awaiting_admin, got {:?}", + row.state + ))); + } + if !row.verdict.as_ref().is_some_and(|v| v.eligible) { + return Err(StoreError::Illegal( + "promote refused: verdict not eligible (regression or gates)".into(), + )); + } + } + if let Some(p) = prev { + if let Some(old) = g.submissions.get_mut(&p) { + if old.state == SubmissionState::Champion { + old.state = SubmissionState::Rejected; + old.detail = Some("superseded".into()); + } + } + } + { + let row = g + .submissions + .get_mut(id) + .ok_or_else(|| StoreError::NotFound(id.to_owned()))?; + row.state = SubmissionState::Champion; + } + if let Some(s) = g.scores.get(id).cloned() { + g.champion_scores = Some(s); + } + g.champion_id = Some(id.to_owned()); + g.submissions + .get(id) + .cloned() + .ok_or_else(|| StoreError::NotFound(id.to_owned())) + } + + /// Persist challenger slices so a later promote displaces vs this run. + pub fn record_scores(&self, id: &str, scores: SliceScores) -> Result<(), StoreError> { + self.lock()?.scores.insert(id.to_owned(), scores); + Ok(()) + } + + /// Seed / replace the implicit base-model champion scores. + pub fn set_base_champion(&self, scores: SliceScores) -> Result<(), StoreError> { + self.lock()?.base_champion = Some(scores); + Ok(()) + } + + /// Champion slice scores (promoted miner, else base model). + pub fn champion_scores(&self) -> Result, StoreError> { + let g = self.lock()?; + if let Some(s) = &g.champion_scores { + return Ok(Some(s.clone())); + } + Ok(g.base_champion.clone()) + } +} + +/// SHA-256 hex of the frozen submission. +#[must_use] +pub fn freeze_submission_digest(hotkey: &str, artifact_digest: &str, nonce: &str) -> String { + let mut h = Sha256::new(); + h.update(hotkey.as_bytes()); + h.update([0xff]); + h.update(artifact_digest.as_bytes()); + h.update([0xff]); + h.update(nonce.as_bytes()); + hex::encode(h.finalize()) +} + +/// Build a holdout that is sealed until `digest` is recorded. +#[must_use] +pub fn sealed_holdout(epoch: u64, digest: &str) -> Holdout { + let mut h = Sha256::new(); + h.update(HOLDOUT_DOMAIN); + h.update(epoch.to_le_bytes()); + h.update(digest.as_bytes()); + let seed = hex::encode(h.finalize()); + Holdout { + slice_id: format!("relearn-holdout-{epoch}"), + seed_hex: String::new(), + unsealed: false, + } + .with_pending_seed(seed) +} + +trait WithPending { + fn with_pending_seed(self, seed: String) -> Self; +} + +impl WithPending for Holdout { + fn with_pending_seed(mut self, seed: String) -> Self { + // Keep seed off the public struct until unseal. + self.seed_hex = seed; + self.unsealed = false; + self + } +} + +/// Reveal holdout seed only after the submission digest is frozen. +#[must_use] +pub fn unseal_holdout(pending: &Holdout, frozen_digest: &str) -> Option { + if frozen_digest.is_empty() || pending.seed_hex.is_empty() { + return None; + } + Some(Holdout { + slice_id: pending.slice_id.clone(), + seed_hex: pending.seed_hex.clone(), + unsealed: true, + }) +} + +/// Public view: seed stripped until unsealed. +#[must_use] +pub fn public_holdout(h: &Holdout) -> Holdout { + if h.unsealed { + h.clone() + } else { + Holdout { + slice_id: h.slice_id.clone(), + seed_hex: String::new(), + unsealed: false, + } + } +} + +#[cfg(test)] +mod tests { + use super::*; + + #[test] + fn digest_stable_and_distinct() { + let a = freeze_submission_digest("aa", "bb", "n1"); + let b = freeze_submission_digest("aa", "bb", "n1"); + let c = freeze_submission_digest("aa", "bb", "n2"); + assert_eq!(a, b); + assert_ne!(a, c); + assert_eq!(a.len(), 64); + } + + #[test] + fn holdout_stays_sealed_in_public_view() { + let h = sealed_holdout(7, "deadbeef"); + assert!(!h.unsealed); + assert!(!h.seed_hex.is_empty()); + let pub_v = public_holdout(&h); + assert!(pub_v.seed_hex.is_empty()); + let open = unseal_holdout(&h, "deadbeef").expect("unseal"); + assert!(open.unsealed); + assert!(!open.seed_hex.is_empty()); + } + + #[test] + fn promote_refuses_ineligible() { + let st = MemoryStore::new(); + let row = st + .insert(Submission { + id: String::new(), + miner_hotkey: "00".repeat(32), + artifact_digest: "11".repeat(32), + artifact_uri: None, + nonce: "aa".into(), + submission_digest: "bb".repeat(32), + state: SubmissionState::AwaitingAdmin, + receipt_json: None, + verdict: None, + detail: None, + }) + .expect("insert"); + assert!(st.promote(&row.id).is_err()); + } + + #[test] + fn champion_scores_follow_promote_not_base() { + use prism_competition::ExampleSeries; + use relearn_score::SliceScores; + + fn series(prefix: &str, n: usize, val: f64) -> ExampleSeries { + ExampleSeries::from_pairs((0..n).map(|i| (format!("{prefix}{i}"), val))) + } + fn slice(v: f64) -> SliceScores { + SliceScores { + holdout: series("h", 8, v), + public: series("p", 8, v), + perturbed: series("x", 8, v), + canaries: series("c", 8, v), + agent_trace: 0.9, + } + } + + let st = MemoryStore::new(); + st.set_base_champion(slice(0.4)).expect("base"); + let row = st + .insert(Submission { + id: String::new(), + miner_hotkey: "00".repeat(32), + artifact_digest: "11".repeat(32), + artifact_uri: None, + nonce: "aa".into(), + submission_digest: "bb".repeat(32), + state: SubmissionState::AwaitingAdmin, + receipt_json: None, + verdict: Some(relearn_score::PromoteVerdict { + eligible: true, + paired: None, + failed: Vec::new(), + lattice: 12, + }), + detail: None, + }) + .expect("insert"); + st.record_scores(&row.id, slice(0.8)).expect("scores"); + st.promote(&row.id).expect("promote"); + let got = st.champion_scores().expect("read").expect("some"); + assert!((SliceScores::mean(&got.holdout).unwrap_or(0.0) - 0.8).abs() < 1e-9); + } +} diff --git a/crates/site-api/src/handlers.rs b/crates/site-api/src/handlers.rs index 41bed9058..9bede9883 100644 --- a/crates/site-api/src/handlers.rs +++ b/crates/site-api/src/handlers.rs @@ -12,7 +12,8 @@ use serde::Deserialize; use serde_json::{json, Value}; use crate::state::SiteState; -use crate::upstream::{self, DESIGN, PRISM}; +use crate::upstream::{self, DESIGN, PRISM, RELEARN}; +use site_data::map::relearn_arena_from_live; use site_data::map::{ activity_from_lives, design_arena_from_dashboard, design_leaderboard, design_submission, enrich_leaderboard_uids, enrich_leaderboard_weights, enrich_submission_uids, @@ -118,6 +119,10 @@ async fn fetch_prism_status(st: &SiteState) -> Option { upstream::get_json_opt(st, PRISM, "/v1/status").await } +async fn fetch_relearn_status(st: &SiteState) -> Option { + upstream::get_json_opt(st, RELEARN, "/v1/status").await +} + async fn fetch_prism_subs(st: &SiteState, limit: u32) -> Option { upstream::get_json_opt(st, PRISM, &format!("/v1/submissions?limit={limit}")).await } @@ -227,17 +232,15 @@ fn decorate_submissions(st: &SiteState, rows: &mut [crate::Submission]) { async fn network_stats(st: &SiteState) -> NetworkStats { let (block, chain_epoch, validators) = chain_snapshot(st); - let design = fetch_design_dash(st).await; - let prism = fetch_prism_status(st).await; - let prism_subs = fetch_prism_subs(st, 200).await; - let arenas = arenas_with_emission(st, design.as_ref(), prism.as_ref(), prism_subs.as_ref()); + let relearn = fetch_relearn_status(st).await; + let arenas = arenas_with_emission(st, None, relearn.as_ref(), None); let agents: u32 = arenas.iter().map(|a| a.agents).sum(); let tao_price = site_data::price::tao_price_usd(&st.client, &st.tao_price).await; NetworkStats { - epoch: epoch_from_lives(design.as_ref(), prism.as_ref(), chain_epoch), + epoch: epoch_from_lives(None, relearn.as_ref(), chain_epoch), agents, validators: u32::try_from(validators.len()).unwrap_or(0), - arenas: 3, + arenas: 2, emission_per_day: 0.0, tao_price, block_height: block, @@ -302,15 +305,8 @@ async fn get_landing(State(st): State) -> impl IntoResponse { } async fn get_arenas(State(st): State) -> impl IntoResponse { - let design = fetch_design_dash(&st).await; - let prism = fetch_prism_status(&st).await; - let prism_subs = fetch_prism_subs(&st, 200).await; - Json(arenas_with_emission( - &st, - design.as_ref(), - prism.as_ref(), - prism_subs.as_ref(), - )) + let relearn = fetch_relearn_status(&st).await; + Json(arenas_with_emission(&st, None, relearn.as_ref(), None)) } async fn get_arena(State(st): State, Path(slug): Path) -> Response { @@ -325,6 +321,7 @@ async fn get_arena(State(st): State, Path(slug): Path) -> Res let subs = fetch_prism_subs(&st, 200).await; prism_arena_from_live(status.as_ref(), subs.as_ref()) } + ArenaSlug::Relearn => relearn_arena_from_live(fetch_relearn_status(&st).await.as_ref()), }; apply_emission(&st, &mut arena); Json(arena).into_response() @@ -583,6 +580,7 @@ async fn get_leaderboard( ArenaSlug::Prism => { Json(prism_leaderboard_json(&st, page, page_size, needle).await).into_response() } + ArenaSlug::Relearn => Json(empty_leaderboard_json(page, page_size)).into_response(), } } @@ -599,7 +597,7 @@ async fn get_submissions( let status_filter = q.status.as_deref(); let needle = q.q.as_deref(); match slug { - ArenaSlug::Coding => { + ArenaSlug::Coding | ArenaSlug::Relearn => { Json(page_slice::(&[], page, page_size)).into_response() } ArenaSlug::Design => { @@ -1345,16 +1343,10 @@ mod tests { let (s, v) = call(app.clone(), "/v1/site/arenas").await; assert_eq!(s, StatusCode::OK, "{v}"); - assert_eq!(v.as_array().unwrap().len(), 3); + assert_eq!(v.as_array().unwrap().len(), 2); assert_eq!(v[0]["slug"], "coding"); - // List rows without v2.1 markers must not inflate Prism agents / BPB. - assert_eq!(v[2]["slug"], "prism"); - assert_eq!(v[2]["bestScoreLabel"], "BEST G2"); - assert_eq!(v[2]["agents"], 0); - assert_eq!(v[2]["bestScore"], "—"); - assert_eq!(v[1]["bestScore"], "1,300"); - assert_eq!(v[1]["roundId"], 9); - assert_eq!(v[1]["secondsRemaining"], 120); + assert_eq!(v[1]["slug"], "relearn"); + assert_eq!(v[1]["bestScoreLabel"], "DISPLACE"); let (s, v) = call(app.clone(), "/v1/site/arenas/design/submissions").await; assert_eq!(s, StatusCode::OK, "{v}"); @@ -1715,7 +1707,7 @@ mod tests { }; let st = st.with_weights( Arc::new(ChallengesBody { - challenges: vec![entry("design", 5_000), entry("prism", 5_000)], + challenges: vec![entry("relearn", 10_000)], }), Arc::new(|| None), ); @@ -1723,19 +1715,17 @@ mod tests { let (s, v) = call(app.clone(), "/v1/site/arenas").await; assert_eq!(s, StatusCode::OK, "{v}"); - assert_eq!(v[1]["emissionShare"], 0.5); - assert_eq!(v[2]["emissionShare"], 0.5); + assert_eq!(v[1]["slug"], "relearn"); + assert_eq!(v[1]["emissionShare"], 1.0); // Unsealed: effective weights stay 0. assert_eq!(v[1]["weight"], 0.0); - assert_eq!(v[2]["weight"], 0.0); let (s, v) = call(app.clone(), "/v1/site/weights").await; assert_eq!(s, StatusCode::OK, "{v}"); assert_eq!(v["sealed"], false); assert_eq!(v["burnShare"], 1.0); - assert_eq!(v["emissionShares"][0]["arena"], "design"); - assert_eq!(v["emissionShares"][0]["share"], 0.5); - assert_eq!(v["emissionShares"][1]["arena"], "prism"); + assert_eq!(v["emissionShares"][0]["arena"], "relearn"); + assert_eq!(v["emissionShares"][0]["share"], 1.0); assert!(v["hotkeyWeights"].as_array().unwrap().is_empty()); } diff --git a/crates/site-api/src/upstream.rs b/crates/site-api/src/upstream.rs index 8aaa9bf32..e6d08ed08 100644 --- a/crates/site-api/src/upstream.rs +++ b/crates/site-api/src/upstream.rs @@ -8,6 +8,8 @@ use crate::state::SiteState; /// Challenge ids registered on the gateway. pub const DESIGN: &str = "design"; pub const PRISM: &str = "prism"; +/// Live one-challenge subnet. +pub const RELEARN: &str = "relearn"; /// Upstream fetch error. #[derive(Debug)] diff --git a/crates/site-data/src/map.rs b/crates/site-data/src/map.rs index 1f9f4336f..ec86e21f5 100644 --- a/crates/site-data/src/map.rs +++ b/crates/site-data/src/map.rs @@ -5,7 +5,7 @@ use std::collections::{BTreeSet, HashMap}; use serde_json::Value; use keystore::{ss58_encode, BITTENSOR_SS58_PREFIX}; -use site_types::{coding_arena, design_frame, prism_frame}; +use site_types::{coding_arena, design_frame, prism_frame, relearn_frame}; use site_types::{ ActivityEvent, ActivitySeverity, Agent, Arena, ArenaSlug, LeaderboardRow, LossPoint, LossSeries, PrismTelemetry, PrismTelemetryPoint, PrismWindow, RulesGate, SealedPaths, @@ -397,18 +397,29 @@ fn format_g2_score(v: f64) -> String { } } -/// All three arenas (coding always paused). +/// Live arenas: coding (paused) + relearn. Design/Prism are retired products. #[must_use] pub fn list_arenas( - design_dash: Option<&Value>, - prism_status: Option<&Value>, - prism_subs: Option<&Value>, + _design_dash: Option<&Value>, + relearn_status: Option<&Value>, + _prism_subs: Option<&Value>, ) -> Vec { - vec![ - coding_arena(), - design_arena_from_dashboard(design_dash), - prism_arena_from_live(prism_status, prism_subs), - ] + vec![coding_arena(), relearn_arena_from_live(relearn_status)] +} + +/// Relearn card from `/v1/status` (or the static frame when the backend is down). +#[must_use] +pub fn relearn_arena_from_live(status: Option<&Value>) -> Arena { + let mut arena = relearn_frame(); + if let Some(s) = status { + if let Some(id) = s.get("champion_id").and_then(|v| v.as_str()) { + if !id.is_empty() { + arena.best_score = id.to_owned(); + } + } + arena.status = "live".into(); + } + arena } fn format_elo(v: f64) -> String { diff --git a/crates/site-types/src/frames.rs b/crates/site-types/src/frames.rs index c1da6f3f6..5f6cf7290 100644 --- a/crates/site-types/src/frames.rs +++ b/crates/site-types/src/frames.rs @@ -40,7 +40,7 @@ pub fn design_frame() -> Arena { name: "Design Arena".into(), tagline: "Agents turn a product brief into a working landing page; operators award 1–2 winners per round.".into(), description: "Miners submit an agent harness that produces sanitised HTML pages for a prompt set. Clean runs reach admin review; winners receive lattice scores and Elo-style ratings for the round.".into(), - status: "live".into(), + status: "retired".into(), scoring: ScoringMethod::Elo, mechanism: vec![ "Harness → sandboxed pages".into(), @@ -74,7 +74,7 @@ pub fn prism_frame() -> Arena { name: "Prism".into(), tagline: "Prism v2.1 — AutoModel pin+patch, 4h train on 1× B200, dense 1B reference (850M–1B). Public board is G2 lattice; 2.0 harvests cannot win.".into(), description: "New competition (prism-v2.1, scoring generation 21, recipe 2.1.0). Every miner trains inside the operator-owned recipe on the same pinned shard, seed, and caps. The public board lists only v2.1 harvests. Until the first eligible 2.1 run finishes, subnet weights stay burn (uid 0). Rankings use measured G2 / G1–G8 fields — never invented curves.".into(), - status: "live".into(), + status: "retired".into(), scoring: ScoringMethod::SpectralFusion, mechanism: vec![ "Recipe 2.1.0 · prism-v2.1 · 4h / 1× B200".into(), @@ -99,3 +99,37 @@ pub fn prism_frame() -> Arena { seconds_remaining: None, } } + +/// Relearn arena frame; counters filled by caller from live status. +#[must_use] +pub fn relearn_frame() -> Arena { + Arena { + slug: ArenaSlug::Relearn, + name: "Relearn".into(), + tagline: "Post-training factory: miners improve Qwen3.8-Flash-Next; score is displacement vs the previous champion.".into(), + description: "One-challenge subnet. Submit an improved artifact of the pinned base model. Holdout stays sealed until the submission digest freezes. Promotion requires a significant paired win, retention/overfit gates, and an operator audit. Regressions are never crowned. No TDX / Phala CVM.".into(), + status: "live".into(), + scoring: ScoringMethod::Displacement, + mechanism: vec![ + "Digest freeze → holdout unseal".into(), + "Paired displacement vs champion".into(), + "Operator-audited promote (never a regression)".into(), + ], + agents: 0, + best_score: "—".into(), + best_score_label: "DISPLACE".into(), + emission_share: 1.0, + weight: 1.0, + rewards_per_day: 0.0, + references: vec![ProjectReference { + name: "Relearn".into(), + repo: "CortexLM/relearn".into(), + repo_url: "https://github.com/CortexLM/relearn".into(), + }], + source_url: "https://github.com/CortexLM/relearn".into(), + plate: "/plates/relearn.svg".into(), + round_id: None, + round_ends_at: None, + seconds_remaining: None, + } +} diff --git a/crates/site-types/src/lib.rs b/crates/site-types/src/lib.rs index 994228072..a6dd2efda 100644 --- a/crates/site-types/src/lib.rs +++ b/crates/site-types/src/lib.rs @@ -9,6 +9,6 @@ mod frames; mod paginate; mod types; -pub use frames::{coding_arena, design_frame, prism_frame}; +pub use frames::{coding_arena, design_frame, prism_frame, relearn_frame}; pub use paginate::page_slice; pub use types::*; diff --git a/crates/site-types/src/types.rs b/crates/site-types/src/types.rs index 68468344f..e60836498 100644 --- a/crates/site-types/src/types.rs +++ b/crates/site-types/src/types.rs @@ -8,10 +8,12 @@ use serde::{Deserialize, Serialize}; pub enum ArenaSlug { /// Coding (paused). Coding, - /// Design challenge. + /// Design challenge (retired). Design, - /// Prism challenge. + /// Prism challenge (retired). Prism, + /// Relearn post-training factory. + Relearn, } impl ArenaSlug { @@ -22,6 +24,7 @@ impl ArenaSlug { "coding" => Some(Self::Coding), "design" => Some(Self::Design), "prism" => Some(Self::Prism), + "relearn" => Some(Self::Relearn), _ => None, } } @@ -33,6 +36,7 @@ impl ArenaSlug { Self::Coding => "coding", Self::Design => "design", Self::Prism => "prism", + Self::Relearn => "relearn", } } } @@ -47,6 +51,8 @@ pub enum ScoringMethod { Elo, /// Prism spectral fusion / BPB. SpectralFusion, + /// Relearn paired displacement vs champion. + Displacement, } /// Reference chip on an arena card. diff --git a/crates/trustroot/tests/trustroot_verify.rs b/crates/trustroot/tests/trustroot_verify.rs index b1a4052c7..b992cd703 100644 --- a/crates/trustroot/tests/trustroot_verify.rs +++ b/crates/trustroot/tests/trustroot_verify.rs @@ -346,27 +346,19 @@ fn s9_repo_config_loads_when_present() { } let (ch, ms) = load_config_dir(&root, 0, 3).expect("committed config must verify"); let primary = ch.primary().unwrap(); - assert_eq!(primary.body.challenges.len(), 2); - // Emission 100% prism (2026-08-16): design 0 / prism 10000 (was 5000/5000). - let design = primary.body.get(b"design").expect("design row"); - assert_eq!(design.emission_share_bps, 0); + assert_eq!(primary.body.challenges.len(), 1); + let relearn = primary.body.get(b"relearn").expect("relearn row"); + assert_eq!(relearn.emission_share_bps, BPS_DENOM); assert_eq!( - encode_hex(&design.public_key), - "3e27f87d8330006a73174001120c3455f16b95fee098bb8c2bab9d5053840418" + encode_hex(&relearn.public_key), + "8ab577207bb6dfc770a850710824a098d53b1ee90abb92925bd0928937131674" ); - let prism = primary.body.get(b"prism").expect("prism row"); - assert_eq!(prism.emission_share_bps, BPS_DENOM); - assert_eq!( - encode_hex(&prism.public_key), - "bcd50bb830e050ed4b011dd8f1d2f126fdb42dc55b45ece30a7d5c8ceb3c5219" - ); - assert_ne!(prism.public_key, design.public_key); + assert!(primary.body.get(b"design").is_none()); + assert!(primary.body.get(b"prism").is_none()); let shares = primary.body.emission_shares(); - assert_eq!(shares.len(), 2); - assert_eq!(shares[0].0, b"design"); - assert_eq!(shares[0].1, 0); - assert_eq!(shares[1].0, b"prism"); - assert_eq!(shares[1].1, BPS_DENOM); + assert_eq!(shares.len(), 1); + assert_eq!(shares[0].0, b"relearn"); + assert_eq!(shares[0].1, BPS_DENOM); // base-agent CVM path removed — committed allowlist is empty (fail-closed). let entries = &ms.primary().unwrap().body.entries; assert!( diff --git a/deploy/AGENTS.md b/deploy/AGENTS.md index 90997334c..c3cd0daa3 100644 --- a/deploy/AGENTS.md +++ b/deploy/AGENTS.md @@ -35,8 +35,7 @@ Compose always runs a digest-pinned `postgres` service (`base-pgdata` volume, he | Data | Store | |------|--------| -| Design harnesses / runs / stages / artifacts metadata / admin rounds | **Postgres** (`design_*`) | -| Prism submissions / stage events | **Postgres** (`prism_*`) | +| Relearn submissions (v0) | **in-memory** (`relearn-store`); Postgres can replace without changing HTTP | | Gateway raw weight leaves + sealed bundles | **Postgres** (`raw_weight_snapshot`, `epoch_bundle`, …) | | Validator attestations (when DB configured) | **Postgres** | | Design sandbox staging files | volume `${BASE_STATE_DIR}/design/staging` + `design-artifacts` | @@ -44,7 +43,7 @@ Compose always runs a digest-pinned `postgres` service (`base-pgdata` volume, he | site-api (`GET /v1/site/*`) | no DB — proxies challenge upstreams via gateway | | Unit/integration tests | may construct `Memory*Store` directly; omit `BASE_DATABASE_URL` only there | -Migrations (`crates/db/migrations`) run on boot in gateway / design-challenge / prism-challenge when `BASE_DATABASE_URL` is set. Compose requires `deploy/env/{design,prism}-challenge.env` so challenges cannot silently boot on memory. +Migrations (`crates/db/migrations`) run on boot in gateway when `BASE_DATABASE_URL` is set. Compose requires `deploy/env/relearn-challenge.env` so the live challenge cannot silently boot without operator config. ## Prism Lium GPU profiles (do not mix) @@ -86,7 +85,7 @@ Full procedure: [`docs/runbooks/local-testnet-e2e.md`](../docs/runbooks/local-te | `base-validator` wallet | **no** (fetch-only) | for on-chain weight submit | | Fresh `target/release/{gateway,validator,…}` (or `BASE_DOCKER_BUILD_FROM=source`) | recommended | **required** for real chain | -**Weights seal smoke (default on `--smoke`):** after healthz, `local-e2e.sh` runs `weights-smoke` — signed prism leaves for the live metagraph → `POST /v1/admin/seal` → assert `GET /v1/weights/latest` is **200** with **`sealed: true`**. Skip with `--no-weights-smoke`. Pre-seal, latest is **200 burn** (`sealed: false`, uid 0 = 100%) — never 404; that is unrelated to a missing gateway owner wallet. Prefer `--burn` on mainnet when sealing without real challenge scores (all `NoScore` → uid 0). +**Weights seal smoke (default on `--smoke`):** after healthz, `local-e2e.sh` runs `weights-smoke` — signed relearn leaves for the live metagraph → `POST /v1/admin/seal` → assert `GET /v1/weights/latest` is **200** with **`sealed: true`**. Skip with `--no-weights-smoke`. Pre-seal, latest is **200 burn** (`sealed: false`, uid 0 = 100%) — never 404; that is unrelated to a missing gateway owner wallet. Prefer `--burn` on mainnet when sealing without real challenge scores (all `NoScore` → uid 0). **Interim prod burn seal (retired while prism auto-emits):** `weights-smoke --burn` posts all-`NoScore` at a **block-scale** epoch. That hid the live Prism 2.1 WTA winner (chain epoch ~24k) because `/v1/weights/latest` had no chain-scale bundle to prefer. Keep the script for emergency burn-only windows; **do not** enable `base-burn-seal.timer` when Prism is emitting scores. `remote-deploy` on master enables real-seal and disables the burn timer. diff --git a/deploy/Dockerfile b/deploy/Dockerfile index 58cd30392..aa98b5537 100644 --- a/deploy/Dockerfile +++ b/deploy/Dockerfile @@ -12,8 +12,6 @@ ARG DEBIAN_IMAGE=debian@sha256:7b140f374b289a7c2befc338f42ebe6441b7ea838a042bbd5acbfca6ec875818 ARG RUST_IMAGE=rust@sha256:e51d0265072d2d9d5d320f6a44dde6b9ef13653b035098febd68cce8fa7c0bc4 -# python:3.12.11-slim-bookworm (multi-arch index digest) -ARG PYTHON_IMAGE=python@sha256:519591d6871b7bc437060736b9f7456b8731f1499a57e22e6c285135ae657bf7 ARG BUILD_FROM=source # ----------------------------------------------------------------------------- @@ -35,9 +33,6 @@ COPY bins ./bins COPY xtask ./xtask # Keep workspace metadata members that Docker context needs COPY config ./config -# design-challenge embeds the published baseline agent as the agentic review -# corpus anchor (include_str! in crates/design-challenge/src/orchestrator.rs). -COPY docs/external-miner/examples/design-baseline ./docs/external-miner/examples/design-baseline # validator-bin/dcap turns on real Intel DCAP quote verification; without it the # validator parks every attestation instead of verifying it. RUN cargo build --release \ @@ -45,25 +40,16 @@ RUN cargo build --release \ -p validator-bin \ -p gateway-bin \ -p updater-bin \ - -p prism-challenge-bin \ - -p design-challenge-bin \ - -p design-egress-proxy-bin \ - && cargo build --release -p challenge-review-bin \ + -p relearn-challenge-bin \ && strip target/release/validator \ target/release/gateway \ target/release/updater \ - target/release/prism-challenge \ - target/release/design-challenge \ - target/release/design-egress-proxy \ - target/release/challenge-review \ + target/release/relearn-challenge \ && mkdir -p /out \ && cp target/release/validator \ target/release/gateway \ target/release/updater \ - target/release/prism-challenge \ - target/release/design-challenge \ - target/release/design-egress-proxy \ - target/release/challenge-review /out/ + target/release/relearn-challenge /out/ # ----------------------------------------------------------------------------- # Prebuilt path — copy host target/release artifacts (local verify only) @@ -76,10 +62,7 @@ RUN apt-get update \ COPY target/release/validator \ target/release/gateway \ target/release/updater \ - target/release/prism-challenge \ - target/release/design-challenge \ - target/release/design-egress-proxy \ - target/release/challenge-review \ + target/release/relearn-challenge \ /out/ # ----------------------------------------------------------------------------- @@ -119,73 +102,14 @@ FROM runtime-base AS updater COPY --from=artifacts --chown=base:base /out/updater /usr/local/bin/updater ENTRYPOINT ["/usr/local/bin/updater"] -FROM runtime-base AS prism-challenge -# prism-lium drives pod exec over the `ssh` binary; runtime-base has none. -# Pre-create the checkpoint park root as uid 65532 so a fresh named volume -# inherits ownership (Docker copies image dir perms into empty volumes). -# Without this, `base` cannot mkdir under /var/lib/prism → harvest fails with -# `lium exec: mkdir: Permission denied` after a successful eval. +FROM runtime-base AS relearn-challenge +# Lium eval uses operator SSH; runtime-base has no ssh client. USER root RUN apt-get update \ && apt-get install -y --no-install-recommends openssh-client \ && rm -rf /var/lib/apt/lists/* \ - && mkdir -p /var/lib/prism/artifacts \ - && chown -R base:base /var/lib/prism + && mkdir -p /var/lib/relearn \ + && chown -R base:base /var/lib/relearn USER base -COPY --from=artifacts --chown=base:base /out/prism-challenge /usr/local/bin/prism-challenge -ENTRYPOINT ["/usr/local/bin/prism-challenge"] - -FROM runtime-base AS design-challenge -USER root -# Headless Chromium for full-page design screenshots (DESIGN_CHROME_BIN). -RUN apt-get update \ - && apt-get install -y --no-install-recommends chromium fonts-liberation \ - && rm -rf /var/lib/apt/lists/* \ - && mkdir -p /var/lib/design/staging \ - && chown -R base:base /var/lib/design -ENV DESIGN_CHROME_BIN=/usr/bin/chromium -USER base -COPY --from=artifacts --chown=base:base /out/design-challenge /usr/local/bin/design-challenge -ENTRYPOINT ["/usr/local/bin/design-challenge"] - -FROM runtime-base AS design-egress-proxy -COPY --from=artifacts --chown=base:base /out/design-egress-proxy /usr/local/bin/design-egress-proxy -ENTRYPOINT ["/usr/local/bin/design-egress-proxy"] - -# ----------------------------------------------------------------------------- -# design-runtime — pinned Python for design-challenge sandboxes (install/run). -# No challenge secrets; sandboxes reach egress only via design-sandbox-egress. -# ----------------------------------------------------------------------------- -FROM ${PYTHON_IMAGE} AS design-runtime -USER root -RUN apt-get update \ - && apt-get install -y --no-install-recommends bash ca-certificates \ - && rm -rf /var/lib/apt/lists/* \ - && groupadd --system --gid 65532 base \ - && useradd --system --uid 65532 --gid base --home-dir /home/base --create-home base \ - && mkdir -p /work /out \ - && chown -R base:base /work /out /home/base -USER base -WORKDIR /work -# Harness entry is staged into /work by design-challenge; image only supplies Python. -CMD ["python", "--version"] - -# ----------------------------------------------------------------------------- -# design-review — containerized anti-cheat review (challenge-agentic + AST). -# design-challenge stages /work (submitted agent + _similar harness, read-only) -# and /out (verdict) per run; the OpenRouter key arrives via container env. -# Python is present so the sandboxed run_command tool can run AST probes. -# ----------------------------------------------------------------------------- -FROM ${PYTHON_IMAGE} AS design-review -USER root -RUN apt-get update \ - && apt-get install -y --no-install-recommends bash ca-certificates \ - && rm -rf /var/lib/apt/lists/* \ - && groupadd --system --gid 65532 base \ - && useradd --system --uid 65532 --gid base --home-dir /home/base --create-home base \ - && mkdir -p /work /out \ - && chown -R base:base /work /out /home/base -USER base -WORKDIR /work -COPY --from=artifacts --chown=base:base /out/challenge-review /usr/local/bin/challenge-review -ENTRYPOINT ["/usr/local/bin/challenge-review"] +COPY --from=artifacts --chown=base:base /out/relearn-challenge /usr/local/bin/relearn-challenge +ENTRYPOINT ["/usr/local/bin/relearn-challenge"] diff --git a/deploy/compose/env-local.yml b/deploy/compose/env-local.yml index 3e0abd9fd..221cdb4fc 100644 --- a/deploy/compose/env-local.yml +++ b/deploy/compose/env-local.yml @@ -61,43 +61,11 @@ services: - ./.local/trust-root:/etc/base/config:ro - base-validator-lkg:/var/lib/base - prism-challenge: + relearn-challenge: ports: - - "127.0.0.1:${LOCAL_PRISM_HOST_PORT:-28092}:8092" + - "127.0.0.1:${LOCAL_RELEARN_HOST_PORT:-28095}:8095" environment: BASE_DATABASE_URL: ${LOCAL_DATABASE_URL:-postgres://base:base_dev_only_change_me@postgres:5432/base} # No Lium spend on a laptop unless operator opts in. - PRISM_FORCE_SIM: "${LOCAL_PRISM_FORCE_SIM:-true}" - # Local miners are usually not registered on testnet 541: gating off by - # default here (staging/prod never set this). 1 = enforce 1-max+metagraph. + RELEARN_FORCE_SIM: "${LOCAL_RELEARN_FORCE_SIM:-true}" BASE_SUBMISSION_GATING: "${LOCAL_SUBMISSION_GATING:-0}" - # Hold after each stage so visual A→Z can photograph mid-flight (0 = off). - PRISM_SIM_STAGE_DELAY_MS: "${LOCAL_PRISM_SIM_STAGE_DELAY_MS:-0}" - - design-challenge: - ports: - - "127.0.0.1:${LOCAL_DESIGN_HOST_PORT:-28093}:8093" - environment: - BASE_DATABASE_URL: ${LOCAL_DATABASE_URL:-postgres://base:base_dev_only_change_me@postgres:5432/base} - # Host SimSandbox only on this local overlay (never staging/prod). - # Requires BASE_ALLOW_HOST_SIM=1; bin refuses Sim on mainnet / prod. - BASE_ALLOW_HOST_SIM: "${BASE_ALLOW_HOST_SIM:-1}" - # Offline sandbox on a laptop unless operator opts into Docker runtime. - DESIGN_FORCE_SIM: "${LOCAL_DESIGN_FORCE_SIM:-true}" - # Deterministic AST agentic for local e2e; OpenRouter when false + key mounted. - DESIGN_FORCE_AGENTIC_SIM: "${LOCAL_DESIGN_FORCE_AGENTIC_SIM:-false}" - # Local miners are usually not registered on testnet 541: gating off by - # default here (staging/prod never set this). 1 = enforce 1-max+metagraph. - BASE_SUBMISSION_GATING: "${LOCAL_SUBMISSION_GATING:-0}" - # In-process review for local e2e (design-review image needs a built - # artifact); staging/prod run DESIGN_REVIEW_BACKEND=docker. - DESIGN_REVIEW_BACKEND: "${LOCAL_DESIGN_REVIEW_BACKEND:-inline}" - # Hold after each stage so visual A→Z can photograph mid-flight (0 = off). - DESIGN_SIM_STAGE_DELAY_MS: "${LOCAL_DESIGN_SIM_STAGE_DELAY_MS:-0}" - - design-egress-proxy: - ports: - - "127.0.0.1:${LOCAL_DESIGN_EGRESS_HOST_PORT:-28094}:8094" - environment: - # No OpenRouter spend locally unless a real key is mounted. - DESIGN_EGRESS_SIM: "${LOCAL_DESIGN_EGRESS_SIM:-true}" diff --git a/deploy/compose/env-prod.yml b/deploy/compose/env-prod.yml index 78eb8a137..8a2b7e2b8 100644 --- a/deploy/compose/env-prod.yml +++ b/deploy/compose/env-prod.yml @@ -54,47 +54,8 @@ services: volumes: - ./deploy/secrets/wallets:/run/base/wallets:ro - ./deploy/secrets/gateway_admin_token:/run/secrets/gateway_admin_token:ro - prism-challenge: + relearn-challenge: environment: - PRISM_FORCE_SIM: "false" - # G1–G8 battery (two-phase train/eval). Default harness is v3; pin here - # so pods never silently fall back to BPB-only even on older images. - PRISM_FLOW: "v3" - # Public v9 boots B200/5090; avoids private DO template create without cred. - # Official public daturaai/pytorch CUDA 13 DIND (miner BYOK can rent). - # Do not pin private prism-recipe-v9 f2f5e84c — rent 400s for other accounts. - PRISM_POD_TEMPLATE_ID: "345273fa-4818-46f7-a8fa-32f0e331713c" - # Full public pack (not tiny caps). Host path staged by - # deploy/scripts/prism-overnight-battery.sh / build_private_pack. - PRISM_EVAL_ASSETS_DIR: "/var/lib/prism/eval-assets" - PRISM_TEST_EVAL_CAPS: "0" - # Recipe 2.0 AutoModel pin (stage with deploy/scripts/stage-automodel-pin.sh). - # Fail-closed intake when unset/unmounted — miners see code=pin. - PRISM_AUTOMODEL_PIN_DIR: "/var/lib/prism/automodel-pin" - # Short-TTL encrypted BYOK seals (cleanup after restart). Host dir + key file. - PRISM_PAYER_VAULT_DIR: "/var/lib/prism/payer-vault" - PRISM_PAYER_VAULT_KEY_FILE: "/run/secrets/prism_payer_vault_key" - # Horizontal scale: N orchestrator workers each claim_next under a semaphore. - PRISM_MAX_CONCURRENT_EVALS: "8" - # Emitter/gating/epoch-feed chain reads share the gateway failover list. + RELEARN_FORCE_SIM: "false" + RELEARN_TEACHER_BACKEND: "http_api" BASE_CHAIN_ENDPOINTS: "wss://bittensor-finney.api.onfinality.io/public-ws,wss://entrypoint-finney.opentensor.ai:443" - # Top-model HuggingFace publish (optional; no-op without token file). - PRISM_TOPMODEL_HF_TOKEN_FILE: "/run/base/huggingface/token" - PRISM_TOPMODEL_HF_REPO: "BaseIntelligence/top-prism-architecture" - volumes: - - /var/lib/prism/automodel-pin:/var/lib/prism/automodel-pin:ro - - /var/lib/prism/payer-vault:/var/lib/prism/payer-vault - - /var/lib/prism/eval-assets:/var/lib/prism/eval-assets:ro - - ./deploy/secrets/prism/payer_vault_key:/run/secrets/prism_payer_vault_key:ro - - ./deploy/secrets/huggingface:/run/base/huggingface:ro - design-challenge: - environment: - # Docker-only on prod. Never set BASE_ALLOW_HOST_SIM / DESIGN_FORCE_SIM here. - DESIGN_FORCE_SIM: "false" - # Trust root is design = 0 bps. Extra design leaves 409 D24 and hide - # the prism WTA winner. Gate emit until design share is non-zero. - DESIGN_SKIP_LEAF_EMIT: "1" - BASE_CHAIN_ENDPOINTS: "wss://bittensor-finney.api.onfinality.io/public-ws,wss://entrypoint-finney.opentensor.ai:443" - design-egress-proxy: - environment: - DESIGN_EGRESS_SIM: "false" diff --git a/deploy/compose/env-staging.yml b/deploy/compose/env-staging.yml index fbff0ec9b..7916adce6 100644 --- a/deploy/compose/env-staging.yml +++ b/deploy/compose/env-staging.yml @@ -21,7 +21,7 @@ services: BASE_VALIDATOR_WALLET: base-validator volumes: - ./deploy/secrets/wallets:/run/base/wallets:ro - # Staging-only trust root: mirrors prod emission (design/prism 0/10000). + # Staging-only trust root: mirrors prod emission (relearn 10000). - ./config/challenges.staging.toml:/etc/base/config/challenges.toml:ro - ./config/challenges.staging.toml.sig:/etc/base/config/challenges.toml.sig:ro - base-validator-lkg:/var/lib/base @@ -46,54 +46,14 @@ services: - ./deploy/secrets/wallets:/run/base/wallets:ro - ./deploy/secrets/gateway_admin_token:/run/secrets/gateway_admin_token:ro # The sealer must sign with the same staging trust root the validator - # verifies against (design/prism 0/10000), or /v1/weights/latest never - # shows design weight and the validator flags emission share mismatch (D23). + # verifies against (relearn 10000), or /v1/weights/latest never + # shows relearn weight and the validator flags emission share mismatch (D23). - ./config/challenges.staging.toml:/etc/base/config/challenges.toml:ro - ./config/challenges.staging.toml.sig:/etc/base/config/challenges.toml.sig:ro - prism-challenge: + relearn-challenge: environment: BASE_CHAIN_ENDPOINT: "wss://test.chain.opentensor.ai:443" BASE_CHAIN_ENDPOINTS: "wss://test.chain.opentensor.ai:443,wss://test.finney.opentensor.ai:443" - # Compressed e2e proof: offline deterministic backend (no GPU spend) + - # tiny model / short train cap. A real Lium RTX 5090 run stays a - # follow-up validation (flip PRISM_FORCE_SIM to "false" and unset the - # PRISM_TEST_* knobs). - PRISM_FORCE_SIM: "true" - PRISM_TEST_TRAIN_MINUTES: "15" - PRISM_TEST_MAX_PARAMS: "2000000" - # v3 battery even under short-train knobs (tiny grids via default - # PRISM_TEST_* → tiny_caps; set CAPS=0 for full G1–G8 on real Lium). - PRISM_FLOW: "v3" - # Same pin path as prod when testing live AutoModel intake on staging - # (stage with deploy/scripts/stage-automodel-pin.sh). Sim/fixture pins - # still work when miners submit automodel@fixture-v1. - PRISM_AUTOMODEL_PIN_DIR: "/var/lib/prism/automodel-pin" - PRISM_PAYER_VAULT_DIR: "/var/lib/prism/payer-vault" - PRISM_PAYER_VAULT_KEY_FILE: "/run/secrets/prism_payer_vault_key" - PRISM_TOPMODEL_HF_TOKEN_FILE: "/run/base/huggingface/token" - PRISM_TOPMODEL_HF_REPO: "BaseIntelligence/top-prism-architecture" - volumes: - - /var/lib/prism/automodel-pin:/var/lib/prism/automodel-pin:ro - - /var/lib/prism/payer-vault:/var/lib/prism/payer-vault - - ./deploy/secrets/prism/payer_vault_key:/run/secrets/prism_payer_vault_key:ro - - ./deploy/secrets/huggingface:/run/base/huggingface:ro - design-challenge: - environment: - BASE_CHAIN_ENDPOINT: "wss://test.chain.opentensor.ai:443" - BASE_CHAIN_ENDPOINTS: "wss://test.chain.opentensor.ai:443,wss://test.finney.opentensor.ai:443" - # Staging uses real Docker via socket-proxy. Host SimSandbox is fail-closed - # here (never set BASE_ALLOW_HOST_SIM / DESIGN_FORCE_SIM on droplets). - DESIGN_FORCE_SIM: "false" - # Same as prod: staging trust root is design = 0 bps. - DESIGN_SKIP_LEAF_EMIT: "1" - # Compressed lifecycle for the staging e2e: ~15-minute rounds (prod - # defaults stay 8640s / 1800s when these are unset). - DESIGN_ROUND_SECS: "900" - DESIGN_AGENT_RUN_TIMEOUT_SECS: "600" - # Containerized anti-cheat review (design-review one-shot image). - DESIGN_REVIEW_BACKEND: "docker" - DESIGN_REVIEW_IMAGE: "design-review:0.1.0" - design-egress-proxy: - environment: - # Real OpenRouter when deploy/secrets/openrouter/api_key is non-empty. - DESIGN_EGRESS_SIM: "false" + # Live path on droplets: sim is local-only (`env-local.yml`). + RELEARN_FORCE_SIM: "false" + RELEARN_TEACHER_BACKEND: "http_api" diff --git a/deploy/compose/role-validator.yml b/deploy/compose/role-validator.yml index f209a7fd7..c714f7977 100644 --- a/deploy/compose/role-validator.yml +++ b/deploy/compose/role-validator.yml @@ -19,14 +19,10 @@ services: # Updater disabled on validator until GHCR pins wired. updater: profiles: ["never"] - # Challenges + Docker sock proxy are master-only (harness eval / sandbox). + # Challenges + Docker sock proxy are master-only. # Validators fetch sealed weights from the master gateway; they do not run # miner code or hold challenge mini-secrets. - prism-challenge: - profiles: ["never"] - design-challenge: - profiles: ["never"] - design-egress-proxy: + relearn-challenge: profiles: ["never"] socket-proxy: profiles: ["never"] diff --git a/deploy/env/relearn-challenge.env.example b/deploy/env/relearn-challenge.env.example new file mode 100644 index 000000000..b20b4b76d --- /dev/null +++ b/deploy/env/relearn-challenge.env.example @@ -0,0 +1,16 @@ +# operator-managed, never committed to git. +# Relearn challenge orchestration surface. + +# Compose requires this file. +# Must match deploy/env/postgres.env on the host (see postgres.env.example). +BASE_DATABASE_URL=postgres://base:base_dev_only_change_me@postgres:5432/base +BASE_NETUID=541 + +# Teacher-only HTTP API (judge). Never point this at miner weights. +# RELEARN_TEACHER_API_URL=https://api.example.com/v1 +# RELEARN_TEACHER_BACKEND=http_api +# Prefer NVFP4-on-Lium when an 8× Blackwell host is available: +# RELEARN_TEACHER_BACKEND=lium + +# Operator bearer tokens for POST /v1/admin/promote. +# RELEARN_ADMIN_TOKENS_FILE=/run/base/relearn/admin_tokens diff --git a/deploy/scripts/assert-compose-matrix.sh b/deploy/scripts/assert-compose-matrix.sh index 04500cc3a..bae9c35fe 100755 --- a/deploy/scripts/assert-compose-matrix.sh +++ b/deploy/scripts/assert-compose-matrix.sh @@ -51,7 +51,7 @@ services=$(render \ if echo "$services" | grep -qx "gateway"; then fail "validator role renders gateway (must not)" fi -for banned in design-challenge design-egress-proxy prism-challenge socket-proxy; do +for banned in relearn-challenge socket-proxy design-challenge design-egress-proxy prism-challenge; do if echo "$services" | grep -qx "$banned"; then fail "validator role renders $banned (master-only; must not)" fi @@ -67,11 +67,16 @@ services=$(render \ if ! echo "$services" | grep -qx "gateway"; then fail "master role does not render gateway (must)" fi -for required in design-challenge design-egress-proxy prism-challenge socket-proxy; do +for required in relearn-challenge socket-proxy; do if ! echo "$services" | grep -qx "$required"; then fail "master role does not render $required (must)" fi done +for retired in design-challenge design-egress-proxy prism-challenge; do + if echo "$services" | grep -qx "$retired"; then + fail "master role still renders retired $retired" + fi +done if echo "$services" | grep -qx "validator"; then fail "master role renders validator (dual submitter; must not — use validator host)" fi @@ -104,7 +109,7 @@ services=$(render \ if echo "$services" | grep -qx "evil-gateway"; then fail "prod validator renders evil-gateway (must not)" fi -for banned in design-challenge design-egress-proxy prism-challenge socket-proxy; do +for banned in relearn-challenge socket-proxy design-challenge design-egress-proxy prism-challenge; do if echo "$services" | grep -qx "$banned"; then fail "prod validator renders $banned (master-only; must not)" fi @@ -123,30 +128,27 @@ for env_file in deploy/compose/env-staging.yml deploy/compose/env-prod.yml; do fail "$env_file enables BASE_ALLOW_HOST_SIM (host Sim forbidden on droplets)" fi if echo "$rendered" | grep -qE 'DESIGN_FORCE_SIM:[[:space:]]*["'\'']?(1|true|TRUE|yes)["'\'']?'; then - fail "$env_file enables DESIGN_FORCE_SIM (Docker-only on droplets)" + fail "$env_file enables DESIGN_FORCE_SIM (retired; must not ship)" fi - # Screenshot Chromium isolation: must force egress proxy (not empty / direct). - if ! echo "$rendered" | grep -qE 'DESIGN_SCREENSHOT_PROXY:[[:space:]]*http://design-egress-proxy:8094'; then - fail "$env_file master render missing DESIGN_SCREENSHOT_PROXY=http://design-egress-proxy:8094" + if echo "$rendered" | grep -qE 'RELEARN_FORCE_SIM:[[:space:]]*["'\'']?(1|true|TRUE|yes)["'\'']?'; then + fail "$env_file enables RELEARN_FORCE_SIM (sim is local-only; must not ship on droplets)" fi done echo "OK: staging/prod do not enable host SimSandbox" -echo "OK: staging/prod force screenshot Chromium through design-egress-proxy" -# --- prism-challenge + design-challenge present in default --- +# --- relearn-challenge present in default; design/prism retired --- default_services=$(render \ -f docker-compose.yml \ config --services) -if ! echo "$default_services" | grep -qx "prism-challenge"; then - fail "prism-challenge not in default compose" -fi -if ! echo "$default_services" | grep -qx "design-challenge"; then - fail "design-challenge not in default compose" +if ! echo "$default_services" | grep -qx "relearn-challenge"; then + fail "relearn-challenge not in default compose" fi -if ! echo "$default_services" | grep -qx "design-egress-proxy"; then - fail "design-egress-proxy not in default compose" -fi -echo "OK: prism-challenge + design-challenge + design-egress-proxy in default compose" +for retired in prism-challenge design-challenge design-egress-proxy; do + if echo "$default_services" | grep -qx "$retired"; then + fail "retired $retired still in default compose" + fi +done +echo "OK: relearn-challenge in default compose; design/prism retired" # --- no fake chain backend survives anywhere in the matrix --- for env_file in deploy/compose/env-staging.yml deploy/compose/env-prod.yml; do @@ -211,14 +213,10 @@ echo "$local_services" | grep -qx "gateway" \ || fail "env-local master stack does not render gateway" echo "$local_services" | grep -qx "validator" \ || fail "env-local master stack does not render co-located validator" -echo "$local_services" | grep -qx "prism-challenge" \ - || fail "env-local master stack does not render prism-challenge" -echo "$local_services" | grep -qx "design-challenge" \ - || fail "env-local master stack does not render design-challenge" -echo "$local_services" | grep -qx "design-egress-proxy" \ - || fail "env-local master stack does not render design-egress-proxy" -echo "$local_rendered" | grep -qE 'published: "?28093"?' \ - || fail "env-local does not publish design-challenge on 28093" +echo "$local_services" | grep -qx "relearn-challenge" \ + || fail "env-local master stack does not render relearn-challenge" +echo "$local_rendered" | grep -qE 'published: "?28095"?' \ + || fail "env-local does not publish relearn-challenge on 28095" for banned in agent-challenge hypertraining-challenge miner-agent miner-socket-proxy base-agent; do if echo "$local_services" | grep -qx "$banned"; then fail "removed service still rendered: $banned" @@ -230,6 +228,6 @@ for banned in agent-challenge hypertraining-challenge miner-agent miner-socket-p fail "removed service still in default compose: $banned" fi done -echo "OK: env-local preserves testnet/541; design on 28093; removed agent/hypertraining/miner services" +echo "OK: env-local preserves testnet/541; relearn on 28095; removed agent/hypertraining/miner services" echo "assert-compose-matrix: all checks passed" diff --git a/deploy/scripts/local-e2e.sh b/deploy/scripts/local-e2e.sh index 9c0f104fe..8633f1a78 100755 --- a/deploy/scripts/local-e2e.sh +++ b/deploy/scripts/local-e2e.sh @@ -38,8 +38,7 @@ TUNNEL_CONFIG="$ROOT/.local/cloudflared-quick.yml" COMPOSE_PROJECT="${COMPOSE_PROJECT_NAME:-base}" GATEWAY_HOST_PORT="${LOCAL_GATEWAY_HOST_PORT:-8080}" VALIDATOR_HOST_PORT="${LOCAL_VALIDATOR_HOST_PORT:-28080}" -PRISM_HOST_PORT="${LOCAL_PRISM_HOST_PORT:-28092}" -DESIGN_HOST_PORT="${LOCAL_DESIGN_HOST_PORT:-28093}" +RELEARN_HOST_PORT="${LOCAL_RELEARN_HOST_PORT:-28095}" BASE_SECRETS_DIR="${BASE_SECRETS_DIR:-${HOME}/.base-secrets}" # Default public-only hotkey for smoke (same placeholder as gateway.env.example usage). @@ -79,8 +78,8 @@ Prerequisites: - deploy/env/*.env via materialize-env.sh (examples OK for smoke) - For --live: base-owner wallet under deploy/secrets/wallets/ (btcli layout) - For --live on-chain weight submit: base-validator wallet - - Secret files: gateway_sk (seal), prism_sk/design_sk (challenge leaf sigs). - Smoke prefers ~/.base-secrets/challenge-*.sk when pubs match trust root; + - Secret files: gateway_sk (seal), relearn_sk (challenge leaf sigs). + Smoke prefers ~/.base-secrets/challenge-relearn.sk when pubs match trust root; otherwise mints and rebuilds the local trust root. Gateway wallet is NOT required to serve sealed weights. @@ -89,18 +88,14 @@ Environment knobs (optional): BASE_SECRETS_DIR Challenge/owner age/sk sources (default: ~/.base-secrets) BASE_DOCKER_BUILD_FROM prebuilt|source (default: prebuilt) LOCAL_GATEWAY_HOTKEY Override smoke public hotkey (64 hex) - LOCAL_PRISM_FORCE_SIM default true - LOCAL_DESIGN_FORCE_SIM default true - LOCAL_DESIGN_EGRESS_SIM default true - LOCAL_DESIGN_SIM_STAGE_DELAY_MS ms pause after each design stage (default 0) - LOCAL_PRISM_SIM_STAGE_DELAY_MS ms pause after each prism stage (default 0) + LOCAL_RELEARN_FORCE_SIM default true LOCAL_ATTEST_VERIFIER default mock_ok Wallet roles: - Gateway owner wallet / REQUIRE_OWNER: master-only identity check (live). Not required for POST /v1/weights/raw, admin seal, or GET /v1/weights/latest. - gateway_sk: mini-secret for bundle seal signatures (required for seal). - - prism_sk / design_sk: challenge leaf signatures (must match trust root pubs). + - relearn_sk: challenge leaf signatures (must match trust root pubs). - Validator wallet: on-chain weight submit only (not weights/latest serving). EOF @@ -220,8 +215,7 @@ start_tunnel() { ensure_env_files() { # Challenge env files are required by compose (BASE_DATABASE_URL → Postgres). if [[ ! -f deploy/env/postgres.env || ! -f deploy/env/gateway.env \ - || ! -f deploy/env/validator.env || ! -f deploy/env/design-challenge.env \ - || ! -f deploy/env/prism-challenge.env ]]; then + || ! -f deploy/env/validator.env || ! -f deploy/env/relearn-challenge.env ]]; then log "materializing deploy/env/*.env from examples" ./deploy/scripts/materialize-env.sh fi @@ -230,7 +224,7 @@ ensure_env_files() { local url url="$(database_url_from_postgres_env)" for f in deploy/env/gateway.env deploy/env/validator.env \ - deploy/env/design-challenge.env deploy/env/prism-challenge.env; do + deploy/env/relearn-challenge.env; do if grep -q '^BASE_DATABASE_URL=' "$f" 2>/dev/null; then sed -i "s|^BASE_DATABASE_URL=.*|BASE_DATABASE_URL=${url}|" "$f" else @@ -241,9 +235,8 @@ ensure_env_files() { } ensure_state_dirs() { - mkdir -p "$STATE_DIR/design/staging" - # Design sandbox binds must be writable by uid 65532 inside containers. - chmod 777 "$STATE_DIR/design/staging" 2>/dev/null || true + mkdir -p "$STATE_DIR/relearn" + chmod 777 "$STATE_DIR/relearn" 2>/dev/null || true } # Create a 32-byte secret file if missing. live mode refuses to invent wallets. @@ -342,20 +335,16 @@ PY ensure_secrets() { ensure_secret_file deploy/secrets/gateway_sk "gateway seal mini-secret" ensure_challenge_sk_aligned \ - deploy/secrets/prism_sk prism \ - "${BASE_SECRETS_DIR}/challenge-prism.sk" - ensure_challenge_sk_aligned \ - deploy/secrets/design_sk design \ - "${BASE_SECRETS_DIR}/challenge-design.sk" - mkdir -p deploy/secrets/lium deploy/secrets/openrouter deploy/secrets/design + deploy/secrets/relearn_sk relearn \ + "${BASE_SECRETS_DIR}/challenge-relearn.sk" + mkdir -p deploy/secrets/lium deploy/secrets/relearn # Touch placeholders so compose bind-mounts stay files/dirs of the right kind. [[ -e deploy/secrets/lium/api_key ]] || : >deploy/secrets/lium/api_key [[ -e deploy/secrets/lium/ssh_ed25519 ]] || : >deploy/secrets/lium/ssh_ed25519 [[ -e deploy/secrets/lium/ssh_ed25519.pub ]] || : >deploy/secrets/lium/ssh_ed25519.pub - [[ -e deploy/secrets/openrouter/api_key ]] || : >deploy/secrets/openrouter/api_key - [[ -e deploy/secrets/design/annotator_tokens ]] || : >deploy/secrets/design/annotator_tokens - chown 65532:65532 deploy/secrets/design/annotator_tokens 2>/dev/null || true - chmod 0400 deploy/secrets/design/annotator_tokens 2>/dev/null || true + [[ -e deploy/secrets/relearn/admin_tokens ]] || : >deploy/secrets/relearn/admin_tokens + chown 65532:65532 deploy/secrets/relearn/admin_tokens 2>/dev/null || true + chmod 0400 deploy/secrets/relearn/admin_tokens 2>/dev/null || true } # Ephemeral owner-signed trust root for local stacks (prod owner key is not required). @@ -363,30 +352,27 @@ ensure_secrets() { # /etc/base/config inside gateway/validator. BASE_TRUST_ROOT_DIR must be the # *in-container* path — a host absolute path is invisible to containers. # -# Challenge public_keys ALWAYS come from deploy/secrets/{prism,design}_sk so +# Challenge public_keys ALWAYS come from deploy/secrets/relearn_sk so # leaf signatures verify. A stale trust root with mismatched pubs is rebuilt. ensure_local_trust_root() { local dir="$ROOT/.local/trust-root" mkdir -p "$dir" export BASE_TRUST_ROOT_DIR=/etc/base/config - local prism_pk design_pk - prism_pk="$(pubkey_hex_from_sk_file "$ROOT/deploy/secrets/prism_sk")" - design_pk="$(pubkey_hex_from_sk_file "$ROOT/deploy/secrets/design_sk")" + local relearn_pk + relearn_pk="$(pubkey_hex_from_sk_file "$ROOT/deploy/secrets/relearn_sk")" local need_rebuild=0 if [[ ! -f "$dir/challenges.toml" || ! -f "$dir/challenges.toml.sig" || ! -f "$dir/owner.pubkey" ]]; then need_rebuild=1 else - python3 - "$dir/challenges.toml" "$prism_pk" "$design_pk" <<'PY' || need_rebuild=1 + python3 - "$dir/challenges.toml" "$relearn_pk" <<'PY' || need_rebuild=1 import sys, tomllib from pathlib import Path doc = tomllib.loads(Path(sys.argv[1]).read_text()) rows = {c["id"]: c.get("public_key", "").lower() for c in doc.get("challenges", [])} -want = {"prism": sys.argv[2].lower(), "design": sys.argv[3].lower()} -for cid, pk in want.items(): - if rows.get(cid) != pk: - sys.exit(1) +if rows.get("relearn") != sys.argv[2].lower() or set(rows) != {"relearn"}: + sys.exit(1) sys.exit(0) PY fi @@ -412,15 +398,13 @@ PY --out-secret "$dir/owner.age" \ --age-recipient "$recip" fi - python3 - "$dir/challenges.toml" "$prism_pk" "$design_pk" <<'PY2' + python3 - "$dir/challenges.toml" "$relearn_pk" <<'PY2' import pathlib, sys -prism_pk, design_pk = sys.argv[2], sys.argv[3] +relearn_pk = sys.argv[2] text = ( "version = 1\nintroduced_epoch = 0\n\n" - f'[[challenges]]\nid = "prism"\npublic_key = "{prism_pk}"\n' + f'[[challenges]]\nid = "relearn"\npublic_key = "{relearn_pk}"\n' "emission_share_bps = 10000\npolicy = \"all_metagraph_hotkeys\"\n\n" - f'[[challenges]]\nid = "design"\npublic_key = "{design_pk}"\n' - "emission_share_bps = 0\npolicy = \"all_metagraph_hotkeys\"\n\n" ) pathlib.Path(sys.argv[1]).write_text(text) PY2 @@ -477,7 +461,7 @@ port_in_use() { check_host_ports() { local p - for p in "$GATEWAY_HOST_PORT" "$VALIDATOR_HOST_PORT" "$PRISM_HOST_PORT" "$DESIGN_HOST_PORT"; do + for p in "$GATEWAY_HOST_PORT" "$VALIDATOR_HOST_PORT" "$RELEARN_HOST_PORT"; do if port_in_use "$p"; then # Allow re-bind when this compose project already publishes the port. if docker ps --format '{{.Names}} {{.Ports}}' \ @@ -494,16 +478,11 @@ export_mode_env() { export BASE_STATE_DIR="$STATE_DIR" export BASE_DOCKER_BUILD_FROM="${BASE_DOCKER_BUILD_FROM:-prebuilt}" export BASE_GATEWAY_ENDPOINT="${BASE_GATEWAY_ENDPOINT:-http://gateway:8080}" - export LOCAL_PRISM_FORCE_SIM="${LOCAL_PRISM_FORCE_SIM:-true}" - export LOCAL_DESIGN_FORCE_SIM="${LOCAL_DESIGN_FORCE_SIM:-true}" - # Host SimSandbox opt-in for local overlay only (bin refuse on prod/mainnet). - export BASE_ALLOW_HOST_SIM="${BASE_ALLOW_HOST_SIM:-1}" - export LOCAL_DESIGN_EGRESS_SIM="${LOCAL_DESIGN_EGRESS_SIM:-true}" + export LOCAL_RELEARN_FORCE_SIM="${LOCAL_RELEARN_FORCE_SIM:-true}" export LOCAL_ATTEST_VERIFIER="${LOCAL_ATTEST_VERIFIER:-mock_ok}" export LOCAL_GATEWAY_HOST_PORT="$GATEWAY_HOST_PORT" export LOCAL_VALIDATOR_HOST_PORT="$VALIDATOR_HOST_PORT" - export LOCAL_PRISM_HOST_PORT="$PRISM_HOST_PORT" - export LOCAL_DESIGN_HOST_PORT="$DESIGN_HOST_PORT" + export LOCAL_RELEARN_HOST_PORT="$RELEARN_HOST_PORT" # Align app DATABASE_URL with whatever postgres.env will create (avoids # stale gateway.env pointing at a different database name). if [[ -z "${LOCAL_DATABASE_URL:-}" && -f "$ROOT/deploy/env/postgres.env" ]]; then @@ -582,15 +561,13 @@ wait_all_health() { local soft_wait=30 local saved="$WAIT_SECS" WAIT_SECS="$soft_wait" - wait_health prism-challenge "http://127.0.0.1:${PRISM_HOST_PORT}/health" || \ - log "warning: prism not healthy (continuing; try BASE_DOCKER_BUILD_FROM=source)" - wait_health design-challenge "http://127.0.0.1:${DESIGN_HOST_PORT}/health" || \ - log "warning: design not healthy (continuing; try BASE_DOCKER_BUILD_FROM=source)" + wait_health relearn-challenge "http://127.0.0.1:${RELEARN_HOST_PORT}/health" || \ + log "warning: relearn not healthy (continuing; try BASE_DOCKER_BUILD_FROM=source)" WAIT_SECS="$saved" } # Prove seal→serve without a gateway owner wallet: signed leaves + admin seal + -# GET /v1/weights/latest must be 200. Uses prism_sk + gateway_sk only. +# GET /v1/weights/latest must be 200. Uses relearn_sk + gateway_sk only. probe_weights_latest() { if [[ "$DO_WEIGHTS_SMOKE" -ne 1 ]]; then log "skipping weights seal smoke (--no-weights-smoke)" @@ -614,8 +591,8 @@ probe_weights_latest() { log "running weights-smoke (leaf submit → admin/seal → weights/latest)" ./target/release/weights-smoke \ --gateway "$gw" \ - --challenge-sk "$ROOT/deploy/secrets/prism_sk" \ - --challenge-id prism \ + --challenge-sk "$ROOT/deploy/secrets/relearn_sk" \ + --challenge-id relearn \ --netuid "$netuid" \ --chain-endpoint "$endpoint" \ | tee /tmp/local-e2e-weights-latest.json \ @@ -639,8 +616,7 @@ print_summary() { Internal (compose network): gateway: http://gateway:8080 validator probe: http://127.0.0.1:${VALIDATOR_HOST_PORT}/healthz - prism: http://127.0.0.1:${PRISM_HOST_PORT}/health - design: http://127.0.0.1:${DESIGN_HOST_PORT}/health + relearn: http://127.0.0.1:${RELEARN_HOST_PORT}/health EOF if [[ -n "$pub" ]]; then diff --git a/deploy/scripts/materialize-env.sh b/deploy/scripts/materialize-env.sh index 9bbd9328e..513bc8791 100755 --- a/deploy/scripts/materialize-env.sh +++ b/deploy/scripts/materialize-env.sh @@ -38,8 +38,8 @@ materialize_one() { } # Challenge env files carry BASE_DATABASE_URL (must match postgres.env). -# Without them, compose refuses to start design/prism (no silent memory store). -for svc in postgres validator gateway updater design-challenge prism-challenge design-egress-proxy; do +# Without them, compose refuses to start relearn (required env_file). +for svc in postgres validator gateway updater relearn-challenge; do materialize_one "$svc" done diff --git a/deploy/scripts/promote.sh b/deploy/scripts/promote.sh index 82712ca3b..91b58a1ab 100755 --- a/deploy/scripts/promote.sh +++ b/deploy/scripts/promote.sh @@ -55,7 +55,7 @@ done [[ -n "$ENV_NAME" ]] || die "--env staging|prod required" case "$ENV_NAME" in staging|prod) ;; *) die "env must be staging|prod" ;; esac -case "$SERVICE" in validator|gateway|updater|prism-challenge|design-challenge) ;; *) die "service must be validator|gateway|updater|prism-challenge|design-challenge" ;; esac +case "$SERVICE" in validator|gateway|updater|relearn-challenge) ;; *) die "service must be validator|gateway|updater|relearn-challenge" ;; esac PIN_PATH="$(pin_path_for_env "$ROOT" "$ENV_NAME")" PROD_PATH="$(prod_pin_path "$ROOT")" diff --git a/deploy/scripts/record-image-digests.sh b/deploy/scripts/record-image-digests.sh index 506145493..6a75f965a 100755 --- a/deploy/scripts/record-image-digests.sh +++ b/deploy/scripts/record-image-digests.sh @@ -3,7 +3,7 @@ # # record-image-digests.sh [--out deploy/digests/.json] [image:tag ...] # -# Default images: validator:0.1.0 gateway:0.1.0 updater:0.1.0 prism-challenge:0.1.0 design-challenge:0.1.0 +# Default images: validator:0.1.0 gateway:0.1.0 updater:0.1.0 relearn-challenge:0.1.0 # Writes JSON: # { "commit_sha", "created_at", "images": { "validator": { "id", "digest", "repo_digest", "tag" } } } set -euo pipefail @@ -27,7 +27,7 @@ while [[ $# -gt 0 ]]; do done if [[ ${#IMAGES[@]} -eq 0 ]]; then - IMAGES=(validator:0.1.0 gateway:0.1.0 updater:0.1.0 prism-challenge:0.1.0 design-challenge:0.1.0) + IMAGES=(validator:0.1.0 gateway:0.1.0 updater:0.1.0 relearn-challenge:0.1.0) fi COMMIT="$(git -C "$ROOT" rev-parse HEAD)" diff --git a/deploy/scripts/register-challenge-backends.sh b/deploy/scripts/register-challenge-backends.sh index d3d2e14a3..b36f2af46 100755 --- a/deploy/scripts/register-challenge-backends.sh +++ b/deploy/scripts/register-challenge-backends.sh @@ -6,7 +6,6 @@ # # Usage: # GATEWAY_URL=http://127.0.0.1:8080 ./deploy/scripts/register-challenge-backends.sh -# # or via docker network from the host after compose up: # ./deploy/scripts/register-challenge-backends.sh --compose set -euo pipefail @@ -14,16 +13,14 @@ ROOT="$(cd "$(dirname "${BASH_SOURCE[0]}")/../.." && pwd)" cd "$ROOT" GATEWAY_URL="${GATEWAY_URL:-http://127.0.0.1:8080}" -PRISM_URL="${PRISM_BACKEND_URL:-http://prism-challenge:8092}" -DESIGN_URL="${DESIGN_BACKEND_URL:-http://design-challenge:8093}" +RELEARN_URL="${RELEARN_BACKEND_URL:-http://relearn-challenge:8095}" COMPOSE_MODE=0 while [[ $# -gt 0 ]]; do case "$1" in --compose) COMPOSE_MODE=1; shift ;; --gateway-url) GATEWAY_URL="$2"; shift 2 ;; - --prism-url) PRISM_URL="$2"; shift 2 ;; - --design-url) DESIGN_URL="$2"; shift 2 ;; + --relearn-url) RELEARN_URL="$2"; shift 2 ;; -h|--help) sed -n '2,12p' "$0" exit 0 @@ -33,7 +30,6 @@ while [[ $# -gt 0 ]]; do done resolve_admin_token() { - # Prefer env token; else file (prod: deploy/secrets/gateway_admin_token). if [[ -n "${BASE_GATEWAY_ADMIN_TOKEN:-}" ]]; then printf '%s' "${BASE_GATEWAY_ADMIN_TOKEN}" return 0 @@ -85,17 +81,12 @@ register_one() { esac } -register_one prism "$PRISM_URL" -register_one design "$DESIGN_URL" +register_one relearn "$RELEARN_URL" -# Smoke the proxy path (health, not healthz — challenge services use /health). if [[ "$COMPOSE_MODE" -eq 1 ]]; then docker compose -f docker-compose.yml -f deploy/compose/role-master.yml \ - exec -T gateway curl -fsS -m 5 http://127.0.0.1:8080/challenge/prism/health >/dev/null - docker compose -f docker-compose.yml -f deploy/compose/role-master.yml \ - exec -T gateway curl -fsS -m 5 http://127.0.0.1:8080/challenge/design/health >/dev/null + exec -T gateway curl -fsS -m 5 http://127.0.0.1:8080/challenge/relearn/health >/dev/null else - curl -fsS -m 5 "${GATEWAY_URL%/}/challenge/prism/health" >/dev/null - curl -fsS -m 5 "${GATEWAY_URL%/}/challenge/design/health" >/dev/null + curl -fsS -m 5 "${GATEWAY_URL%/}/challenge/relearn/health" >/dev/null fi -echo "challenge proxy health: ok (prism + design)" +echo "challenge proxy health: ok (relearn)" diff --git a/deploy/scripts/remote-deploy.sh b/deploy/scripts/remote-deploy.sh index aaa789137..879132ad0 100755 --- a/deploy/scripts/remote-deploy.sh +++ b/deploy/scripts/remote-deploy.sh @@ -31,7 +31,7 @@ REMOTE_DIR="${BASE_REMOTE_DIR:-/opt/base}" # bind source and the container's BASE_VERIFY_WORK_ROOT byte-for-byte. STATE_ROOT="${BASE_STATE_DIR:-/var/lib/base}" GHCR_PREFIX="${BASE_GHCR_PREFIX:-ghcr.io/baseintelligence/base}" -PIN_SERVICES=(validator gateway updater prism-challenge design-challenge) +PIN_SERVICES=(validator gateway updater relearn-challenge) SSH_OPTS=(-o BatchMode=yes -o StrictHostKeyChecking=accept-new) if [[ -n "${BASE_SSH_IDENTITY:-}" ]]; then SSH_OPTS+=(-i "$BASE_SSH_IDENTITY") @@ -160,8 +160,8 @@ fi echo "remote-deploy: rsync tree" if [[ "$BUILD_FROM" == "prebuilt" ]]; then - for b in validator gateway updater prism-challenge design-challenge design-egress-proxy challenge-review; do - [[ -x "$ROOT/target/release/$b" ]] || die "missing prebuilt binary target/release/$b — run: cargo build --release --features validator-bin/dcap -p validator-bin -p gateway-bin -p updater-bin -p prism-challenge-bin -p design-challenge-bin -p design-egress-proxy-bin && cargo build --release -p challenge-review-bin" + for b in validator gateway updater relearn-challenge; do + [[ -x "$ROOT/target/release/$b" ]] || die "missing prebuilt binary target/release/$b — run: cargo build --release --features validator-bin/dcap -p validator-bin -p gateway-bin -p updater-bin -p relearn-challenge-bin" done fi @@ -182,50 +182,36 @@ rsync -az --delete \ "$ROOT/" "$HOST:$REMOTE_DIR/" # Ensure secrets dirs exist (empty OK if not bootstrapped). -# deploy/secrets/lium is bind-mounted by prism-challenge, so it must be a real +# deploy/secrets/lium is bind-mounted by relearn-challenge, so it must be a real # directory with real files: compose would otherwise create directories where # the container expects files. -# Same footgun for file mounts: if prism_sk / design_sk are missing, Docker -# creates *directories* at those paths and the challenge bins fail with +# Same footgun for file mounts: if relearn_sk is missing, Docker +# creates *directories* at those paths and the challenge bin fails with # "Is a directory" / "secret file missing". Materialize empty files when # absent; if a directory already poisoned the path, replace it with a file. ssh_h "mkdir -p '$REMOTE_DIR/deploy/env' '$REMOTE_DIR/deploy/secrets/lium' \ - '$REMOTE_DIR/deploy/secrets/openrouter' '$REMOTE_DIR/deploy/secrets/design' \ - '$REMOTE_DIR/deploy/secrets/github' '$REMOTE_DIR/deploy/secrets/huggingface' \ + '$REMOTE_DIR/deploy/secrets/relearn' \ '$REMOTE_DIR/deploy/secrets/wallets' \ && chmod 700 '$REMOTE_DIR/deploy/secrets' '$REMOTE_DIR/deploy/secrets/lium' \ && for f in api_key ssh_ed25519 ssh_ed25519.pub; do \ [ -e '$REMOTE_DIR/deploy/secrets/lium/'\$f ] || : > '$REMOTE_DIR/deploy/secrets/lium/'\$f; \ done \ - && [ -e '$REMOTE_DIR/deploy/secrets/openrouter/api_key' ] || : > '$REMOTE_DIR/deploy/secrets/openrouter/api_key' \ - && [ -e '$REMOTE_DIR/deploy/secrets/design/annotator_tokens' ] || : > '$REMOTE_DIR/deploy/secrets/design/annotator_tokens' \ - && [ -e '$REMOTE_DIR/deploy/secrets/github/token' ] || : > '$REMOTE_DIR/deploy/secrets/github/token' \ - && [ -e '$REMOTE_DIR/deploy/secrets/huggingface/token' ] || : > '$REMOTE_DIR/deploy/secrets/huggingface/token' \ - && for sk in prism_sk design_sk; do \ + && [ -e '$REMOTE_DIR/deploy/secrets/relearn/admin_tokens' ] || : > '$REMOTE_DIR/deploy/secrets/relearn/admin_tokens' \ + && for sk in relearn_sk; do \ p='$REMOTE_DIR/deploy/secrets/'\$sk; \ if [ -d \"\$p\" ]; then rm -rf \"\$p\"; fi; \ [ -e \"\$p\" ] || : > \"\$p\"; \ chmod 400 \"\$p\"; chown 65532:65532 \"\$p\"; \ done \ && chmod 400 '$REMOTE_DIR/deploy/secrets/lium/'* \ - '$REMOTE_DIR/deploy/secrets/openrouter/api_key' \ - '$REMOTE_DIR/deploy/secrets/design/annotator_tokens' \ - '$REMOTE_DIR/deploy/secrets/github/token' \ - '$REMOTE_DIR/deploy/secrets/huggingface/token' \ + '$REMOTE_DIR/deploy/secrets/relearn/admin_tokens' \ && chown -R 65532:65532 '$REMOTE_DIR/deploy/secrets/lium' \ - '$REMOTE_DIR/deploy/secrets/openrouter' \ - '$REMOTE_DIR/deploy/secrets/design' \ - '$REMOTE_DIR/deploy/secrets/github' \ - '$REMOTE_DIR/deploy/secrets/huggingface' \ + '$REMOTE_DIR/deploy/secrets/relearn' \ && chmod -R a-w '$REMOTE_DIR/deploy/secrets/wallets' 2>/dev/null; \ chown -R 65532:65532 '$REMOTE_DIR/deploy/secrets/wallets' 2>/dev/null; true" -# Design sandbox staging root. The challenge container stages bind sources -# here and hands them to the host's Docker daemon, which resolves bind sources -# on the host filesystem, so the path must exist on the host with the container -# uid as owner, or 'docker compose' would auto-create it root-owned and every -# run would fail on staging I/O. -ssh_h "install -d -m 0775 -o 65532 -g 65532 '$STATE_ROOT/design/staging'" +# Relearn artifact staging (host path for harvested receipts). +ssh_h "install -d -m 0775 -o 65532 -g 65532 '$STATE_ROOT/relearn'" # Materialize missing env from examples (dev-safe placeholders) if absent @@ -262,21 +248,13 @@ case "$ENV" in prod) COMPOSE_FILES+=(-f deploy/compose/env-prod.yml) ;; esac -# Recipe 2.0 AutoModel pin: env-*/yml mounts /var/lib/prism/automodel-pin into -# prism-challenge. Fail loud on master when the staged tree is missing so a -# redeploy does not silently return code=pin to miners. Also pick up a -# host-local overlay outside the rsync tree (survives --delete) when present. +# Relearn pin: config/relearn-pin.toml is rsynced with the tree. Live Lium +# rent refuses until eval_image_digest is a real sha256 pin. if [[ "$ROLE" == "master" ]]; then - if ssh_h "test -d /var/lib/prism/automodel-pin/.git"; then - echo "remote-deploy: AutoModel pin present at /var/lib/prism/automodel-pin" + if ssh_h "test -f '$REMOTE_DIR/config/relearn-pin.toml'"; then + echo "remote-deploy: relearn pin present at $REMOTE_DIR/config/relearn-pin.toml" else - echo "remote-deploy: WARNING: AutoModel pin missing at /var/lib/prism/automodel-pin" >&2 - echo "remote-deploy: stage with: ./deploy/scripts/stage-automodel-pin.sh --dir /var/lib/prism/automodel-pin" >&2 - echo "remote-deploy: (Prism AutoModel intake fails closed with code=pin until staged)" >&2 - fi - if ssh_h "test -f /var/lib/prism/docker-compose.automodel-pin.yml"; then - COMPOSE_FILES+=(-f /var/lib/prism/docker-compose.automodel-pin.yml) - echo "remote-deploy: including host AutoModel pin overlay" + echo "remote-deploy: WARNING: relearn pin missing at $REMOTE_DIR/config/relearn-pin.toml" >&2 fi fi @@ -292,9 +270,7 @@ if [[ "$BUILD_FROM" == "prebuilt" ]]; then "$ROOT/target/release/validator" \ "$ROOT/target/release/gateway" \ "$ROOT/target/release/updater" \ - "$ROOT/target/release/prism-challenge" \ - "$ROOT/target/release/design-challenge" \ - "$ROOT/target/release/design-egress-proxy" \ + "$ROOT/target/release/relearn-challenge" \ "$HOST:$REMOTE_DIR/target/release/" fi @@ -354,11 +330,7 @@ PY python3 - "\$DIGESTS_FILE" <<'PY' | while IFS=\$'\t' read -r service image digest tag; do import json, sys optional = { - "prism-challenge", - "design-challenge", - "design-egress-proxy", - "design-runtime", - "design-review", + "relearn-challenge", "base-attest-helper", } data = json.load(open(sys.argv[1], encoding="utf-8")) @@ -382,40 +354,12 @@ PY pull_retag "\$ref" "\$tag" done else - echo "remote-deploy: no \$DIGESTS_FILE — skipping optional design/prism/attest-helper pulls" - echo "remote-deploy: (those images are only required for staging --build-from source)" + echo "remote-deploy: no \$DIGESTS_FILE — skipping optional attest-helper pull" + echo "remote-deploy: (that image is only required for staging --build-from source)" fi - # design-runtime is not a compose service; retag for sandbox pulls when present. else # Build service images from current tree (source) or prebuilt binaries. docker compose ${COMPOSE_FILES[*]} ${PROFILE_ARGS[*]} build - # Sandbox runtime + anti-cheat review images are not compose services; build - # them explicitly. BUILD_FROM must match the compose build — without it these - # silently fell back to a source compile even on prebuilt deploys, so the - # review image could stay byte-identical (BuildKit cache) while the rest of - # the stack moved to new binaries. --iidfile + inspect prove the tag moved - # to the just-built image; a no-op build that leaves the tag on an old image - # must fail the deploy, not report success. - build_local_image() { - local target="\$1" tag="\$2" iid before after built - iid="\$(mktemp)" - before="\$(docker image inspect "\$tag" --format '{{.Id}}' 2>/dev/null || echo none)" - docker build -f deploy/Dockerfile --target "\$target" \ - --build-arg BUILD_FROM="\$BUILD_FROM" \ - --iidfile "\$iid" -t "\$tag" . - built="\$(cat "\$iid")" - rm -f "\$iid" - after="\$(docker image inspect "\$tag" --format '{{.Id}}')" - echo "remote-deploy: \$tag \$before -> \$after" - if [[ -z "\$built" || "\$after" != "\$built" ]]; then - echo "remote-deploy: ERROR: \$tag resolves to \$after but the build produced \$built — refusing to continue" >&2 - exit 1 - fi - } - # Sandbox runtime image (not a long-running compose service). - build_local_image design-runtime design-runtime:0.1.0 - # Anti-cheat review image (one-shot containers spawned by design-challenge). - build_local_image design-review design-review:0.1.0 fi # The updater can only pull from a registry. Enable it only when the desired @@ -454,7 +398,7 @@ docker compose ${COMPOSE_FILES[*]} ${PROFILE_ARGS[*]} \$UP_PROFILE "\${UP_ARGS[@ # validator-host wallet for WeightsSetRateLimit / CRV4 commits. if [[ '$ROLE' == 'validator' ]]; then docker compose ${COMPOSE_FILES[*]} rm -sf \ - prism-challenge design-challenge design-egress-proxy socket-proxy \ + relearn-challenge socket-proxy \ >/dev/null 2>&1 || true elif [[ '$ROLE' == 'master' ]]; then docker compose ${COMPOSE_FILES[*]} ${PROFILE_ARGS[*]} rm -sf validator \ @@ -531,8 +475,7 @@ headers = { "Authorization": f"Bearer {token}", } backends = [ - ("prism", "http://prism-challenge:8092"), - ("design", "http://design-challenge:8093"), + ("relearn", "http://relearn-challenge:8095"), ] failed = False for cid, url in backends: diff --git a/deploy/secrets/README.md b/deploy/secrets/README.md index e818567bf..1af6ea4bf 100644 --- a/deploy/secrets/README.md +++ b/deploy/secrets/README.md @@ -3,8 +3,8 @@ Containers run as `base` (uid **65532**). Host secret files MUST be: ```bash -chown 65532:65532 deploy/secrets/gateway_sk deploy/secrets/prism_sk deploy/secrets/design_sk -chmod 0400 deploy/secrets/gateway_sk deploy/secrets/prism_sk deploy/secrets/design_sk +chown 65532:65532 deploy/secrets/gateway_sk deploy/secrets/relearn_sk +chmod 0400 deploy/secrets/gateway_sk deploy/secrets/relearn_sk ``` Bind-mounts use the file inode; directory mode 0700 is OK. @@ -15,9 +15,8 @@ Bind-mounts use the file inode; directory mode 0700 is OK. |------|---------|-------| | `gateway_sk` | gateway | Bundle seal mini-secret (`BASE_GATEWAY_SK_FILE`) | | `gateway_admin_token` | gateway + seal scripts | Bearer for `/v1/admin/*` (`BASE_GATEWAY_ADMIN_TOKEN_FILE`). **Required** when `BASE_GATEWAY_REQUIRE_OWNER=1`. Mode **0400**, uid **65532** | -| `prism_sk` | prism-challenge | PRISM challenge mini-secret | -| `design_sk` | design-challenge **only** | Design challenge mini-secret; never mount on egress proxy | -| `challenge_sk` | legacy placeholder | Prefer `prism_sk` / `design_sk`; do not reuse across challenges | +| `relearn_sk` | relearn-challenge | Relearn leaf mini-secret; pub must match `config/challenges.toml` | +| `prism_sk` / `design_sk` | retired products | Do not mount on the live compose path | ```bash # Generate once per environment; never commit the bytes. diff --git a/docker-compose.e2e.yml b/docker-compose.e2e.yml index de648f302..fc8616ba4 100644 --- a/docker-compose.e2e.yml +++ b/docker-compose.e2e.yml @@ -12,22 +12,12 @@ services: - "8089:8080" environment: BASE_ROLE: validator - prism-challenge: + relearn-challenge: ports: - - "8092:8092" + - "8095:8095" environment: - BASE_CHALLENGE_BIND: 0.0.0.0:8092 - design-challenge: - ports: - - "8093:8093" - environment: - BASE_CHALLENGE_BIND: 0.0.0.0:8093 - # Test-only host Sim (never set on staging/prod overlays). - BASE_ALLOW_HOST_SIM: "1" - DESIGN_FORCE_SIM: "true" - design-egress-proxy: - environment: - DESIGN_EGRESS_SIM: "true" + BASE_CHALLENGE_BIND: 0.0.0.0:8095 + RELEARN_FORCE_SIM: "true" socket-proxy: ports: - "2375:2375" diff --git a/docker-compose.yml b/docker-compose.yml index 5188eb26d..8d27a6500 100644 --- a/docker-compose.yml +++ b/docker-compose.yml @@ -1,7 +1,6 @@ # base control-plane stack # -# Default: postgres + validator + updater + socket-proxy + prism-challenge -# + design-challenge + design-egress-proxy. +# Default: postgres + validator + updater + socket-proxy + relearn-challenge. # Master/owner host only: # docker compose --profile master up -d # brings gateway as an additional service (D3). @@ -181,14 +180,15 @@ services: # --------------------------------------------------------------------------- - # prism-challenge — operator PRISM challenge health + miner submit (:8092). + # relearn-challenge — one-challenge subnet (post-training factory) (:8095). + # Miner pays Lium (BYOK). Control plane rents a digest-pinned eval image. # --------------------------------------------------------------------------- - prism-challenge: - image: prism-challenge:0.1.0 + relearn-challenge: + image: relearn-challenge:0.1.0 build: context: . dockerfile: deploy/Dockerfile - target: prism-challenge + target: relearn-challenge args: BUILD_FROM: ${BASE_DOCKER_BUILD_FROM:-prebuilt} restart: unless-stopped @@ -196,104 +196,30 @@ services: postgres: condition: service_healthy environment: - BASE_CHALLENGE_BIND: 0.0.0.0:8092 + BASE_CHALLENGE_BIND: 0.0.0.0:8095 BASE_CHALLENGE_SK_FILE: /run/base/challenge_sk - # Real Lium is used whenever an API key is present. Set PRISM_FORCE_SIM=true - # to keep a deployment on the offline deterministic backend (no GPU spend). - PRISM_FORCE_SIM: "${PRISM_FORCE_SIM:-false}" - LIUM_API_KEY_FILE: /run/base/lium/api_key - LIUM_SSH_PRIVATE_KEY: /run/base/lium/ssh_ed25519 - LIUM_SSH_PUBLIC_KEY_FILE: /run/base/lium/ssh_ed25519.pub - OPENROUTER_API_KEY_FILE: /run/base/openrouter/api_key + # Sim eval unless a miner BYOK key is presented. Never log LIUM_API_KEY. + RELEARN_FORCE_SIM: "${RELEARN_FORCE_SIM:-false}" + RELEARN_PIN_FILE: /etc/base/config/relearn-pin.toml + RELEARN_TEACHER_BACKEND: "${RELEARN_TEACHER_BACKEND:-http_api}" + RELEARN_ADMIN_TOKENS_FILE: /run/base/relearn/admin_tokens BASE_CHALLENGE_GATEWAY_ENDPOINT: ${BASE_CHALLENGE_GATEWAY_ENDPOINT:-http://gateway:8080} - PRISM_MAX_CONCURRENT_EVALS: "${PRISM_MAX_CONCURRENT_EVALS:-8}" - # Pods need a while for sshd after RUNNING on the control plane. - PRISM_SSH_ATTEMPTS: "${PRISM_SSH_ATTEMPTS:-30}" - PRISM_SSH_RETRY_SECS: "${PRISM_SSH_RETRY_SECS:-10}" - PRISM_SSH_RUNNING_TIMEOUT_SECS: "${PRISM_SSH_RUNNING_TIMEOUT_SECS:-900}" - # Top-model GitHub publish (BaseIntelligence/prism top-model/): no-op - # when the token file is absent/empty. - PRISM_TOPMODEL_GITHUB_TOKEN_FILE: /run/base/github/token - # Top-model HuggingFace publish (BaseIntelligence/top-prism-architecture): - # no-op when the token file is absent/empty. - PRISM_TOPMODEL_HF_TOKEN_FILE: /run/base/huggingface/token - PRISM_TOPMODEL_HF_REPO: "${PRISM_TOPMODEL_HF_REPO:-BaseIntelligence/top-prism-architecture}" - # Require harvested checkpoint for top-model journal (set 0 for source-only). - PRISM_TOPMODEL_REQUIRE_WEIGHTS: "${PRISM_TOPMODEL_REQUIRE_WEIGHTS:-1}" - # Parked checkpoints harvested from Lium pods (master-local). - PRISM_ARTIFACT_DIR: /var/lib/prism/artifacts - # G1–G8 eval assets pack (optional; harness falls back to public_dev). - PRISM_EVAL_ASSETS_DIR: "${PRISM_EVAL_ASSETS_DIR:-}" - PRISM_FLOW: "${PRISM_FLOW:-v3}" - # Recipe 2.0 AutoModel pin checkout (deploy/scripts/stage-automodel-pin.sh). - # Required for live AutoModel intake; unset → pin unavailable (fail-closed). - PRISM_AUTOMODEL_PIN_DIR: "${PRISM_AUTOMODEL_PIN_DIR:-}" - # Operator bearer (retry + playground + gating + artifacts). Empty → 503. - PRISM_ADMIN_TOKENS_FILE: /run/base/prism/admin_tokens env_file: - # Required: BASE_DATABASE_URL (+ BASE_NETUID). Missing file → compose - # fails closed (binaries would otherwise fall back to in-memory store). - - path: ./deploy/env/prism-challenge.env + - path: ./deploy/env/relearn-challenge.env required: true volumes: - # prism signs with its OWN mini secret: - # the gateway verifies leaves against the trust root per-challenge key. - - ./deploy/secrets/prism_sk:/run/base/challenge_sk:ro + - ./config:/etc/base/config:ro + - ./deploy/secrets/relearn_sk:/run/base/challenge_sk:ro - ./deploy/secrets/lium:/run/base/lium:ro - - ./deploy/secrets/openrouter:/run/base/openrouter:ro - - ./deploy/secrets/github:/run/base/github:ro - - ./deploy/secrets/huggingface:/run/base/huggingface:ro - - ./deploy/secrets/prism:/run/base/prism:ro - - prism-artifacts:/var/lib/prism/artifacts - expose: - - "8092" - healthcheck: - test: - [ - "CMD-SHELL", - "curl -fsS -m 5 http://127.0.0.1:8092/health || exit 1", - ] - interval: 10s - timeout: 3s - retries: 6 - start_period: 10s - networks: - - base - - # --------------------------------------------------------------------------- - # design-egress-proxy — open Internet egress for sandboxes (install + run) - # with an internal-target blocklist (metadata / loopback / RFC1918 / CGNAT / - # control-plane names, enforced post-DNS-resolution) plus the budgeted - # OpenRouter chat path. Holds OPENROUTER key; never mount design_sk here. - # On base + internal design-sandbox-egress so sandboxes can reach it without - # direct internet. - # --------------------------------------------------------------------------- - design-egress-proxy: - image: design-egress-proxy:0.1.0 - build: - context: . - dockerfile: deploy/Dockerfile - target: design-egress-proxy - args: - BUILD_FROM: ${BASE_DOCKER_BUILD_FROM:-prebuilt} - restart: unless-stopped - environment: - DESIGN_EGRESS_BIND: 0.0.0.0:8094 - OPENROUTER_API_KEY_FILE: /run/base/openrouter/api_key - DESIGN_TOKEN_BUDGET: "${DESIGN_TOKEN_BUDGET:-8000}" - DESIGN_EGRESS_SIM: "${DESIGN_EGRESS_SIM:-false}" - env_file: - - path: ./deploy/env/design-egress-proxy.env - required: false - volumes: - - ./deploy/secrets/openrouter:/run/base/openrouter:ro + - ./deploy/secrets/relearn:/run/base/relearn:ro + - relearn-artifacts:/var/lib/relearn expose: - - "8094" + - "8095" healthcheck: test: [ "CMD-SHELL", - "curl -fsS -m 5 http://127.0.0.1:8094/health || exit 1", + "curl -fsS -m 5 http://127.0.0.1:8095/health || exit 1", ] interval: 10s timeout: 3s @@ -301,79 +227,12 @@ services: start_period: 10s networks: - base - - design-sandbox-egress - - # --------------------------------------------------------------------------- - # design-challenge — miner harness API + sandbox orchestrator (:8093). - # Docker ONLY via socket-proxy (DESIGN_DOCKER_BASE). No raw docker.sock. - # Sandbox LLM traffic goes through design-egress-proxy (no key in sandbox). - # Agentic anti-cheat on this service needs the OpenRouter key at the default - # DESIGN_AGENTIC_OPENROUTER_KEY_FILE path (never passed into miner sandboxes). - # --------------------------------------------------------------------------- - design-challenge: - image: design-challenge:0.1.0 - build: - context: . - dockerfile: deploy/Dockerfile - target: design-challenge - args: - BUILD_FROM: ${BASE_DOCKER_BUILD_FROM:-prebuilt} - restart: unless-stopped - depends_on: - postgres: - condition: service_healthy - socket-proxy: - condition: service_started - design-egress-proxy: - condition: service_healthy - environment: - BASE_CHALLENGE_BIND: 0.0.0.0:8093 - BASE_CHALLENGE_SK_FILE: /run/base/challenge_sk - DESIGN_FORCE_SIM: "${DESIGN_FORCE_SIM:-false}" - DESIGN_DOCKER_BASE: http://socket-proxy:2375 - # Host path must equal bind source below (daemon resolves binds on host). - DESIGN_STAGING_ROOT: ${BASE_STATE_DIR:-/var/lib/base}/design/staging - DESIGN_LLM_PROXY: http://design-egress-proxy:8094 - # Screenshot Chromium (--no-sandbox, file://) must not reach control-plane - # targets on the shared `base` network: force all http(s) through the - # egress proxy blocklist (incl. loopback/metadata via <-loopback>). - DESIGN_SCREENSHOT_PROXY: http://design-egress-proxy:8094 - DESIGN_ANNOTATOR_TOKENS_FILE: /run/base/design/annotator_tokens - DESIGN_AGENTIC_OPENROUTER_KEY_FILE: /run/base/openrouter/api_key - DESIGN_MAX_CONCURRENT: "${DESIGN_MAX_CONCURRENT:-2}" - DESIGN_INSTALL_TIMEOUT_SECS: "${DESIGN_INSTALL_TIMEOUT_SECS:-300}" - BASE_CHALLENGE_GATEWAY_ENDPOINT: ${BASE_CHALLENGE_GATEWAY_ENDPOINT:-http://gateway:8080} - env_file: - # Required: BASE_DATABASE_URL (+ BASE_NETUID). Missing file → compose - # fails closed (binaries would otherwise fall back to in-memory store). - - path: ./deploy/env/design-challenge.env - required: true - volumes: - - ./deploy/secrets/design_sk:/run/base/challenge_sk:ro - - ./deploy/secrets/design:/run/base/design:ro - - ./deploy/secrets/openrouter:/run/base/openrouter:ro - - design-artifacts:/var/lib/design - - ${BASE_STATE_DIR:-/var/lib/base}/design/staging:${BASE_STATE_DIR:-/var/lib/base}/design/staging - expose: - - "8093" - healthcheck: - test: - [ - "CMD-SHELL", - "curl -fsS -m 5 http://127.0.0.1:8093/health || exit 1", - ] - interval: 10s - timeout: 3s - retries: 6 - start_period: 15s - networks: - - base socket-proxy: image: tecnativa/docker-socket-proxy@sha256:9e4b9e7517a6b660f2cc903a19b257b1852d5b3344794e3ea334ff00ae677ac2 restart: unless-stopped environment: - # Shared proxy: updater rolls + design-challenge sandbox. App-level + # Shared proxy: updater rolls. App-level # Allowlist::updater / Allowlist::verifier enforce method/path; tecnativa # CONTAINERS includes DELETE for sandbox cleanup. NETWORKS stays off — # design-sandbox-egress is pre-created by compose (NetworkMode by name). @@ -413,17 +272,8 @@ volumes: base-pgdata: base-updater-state: base-validator-lkg: - design-artifacts: - prism-artifacts: + relearn-artifacts: networks: base: driver: bridge - # Sandbox containers attach here (NetworkMode); only egress member is - # design-egress-proxy. internal=true blocks direct internet from sandboxes. - # Pin the Docker name so NetworkMode "design-sandbox-egress" matches (no - # compose project prefix) — socket-proxy cannot create networks at runtime. - design-sandbox-egress: - name: design-sandbox-egress - driver: bridge - internal: true diff --git a/docs/AGENTS.md b/docs/AGENTS.md index 0a0424a5e..71db3509b 100644 --- a/docs/AGENTS.md +++ b/docs/AGENTS.md @@ -33,8 +33,7 @@ Public miner docs live **outside** this monorepo (examples + human guides only | Challenge | Repo | |-----------|------| -| Design | [`BaseIntelligence/design-challenge`](https://github.com/BaseIntelligence/design-challenge) | -| Prism | [`BaseIntelligence/prism`](https://github.com/BaseIntelligence/prism) | +| Relearn | [`CortexLM/relearn`](https://github.com/CortexLM/relearn) | `docs/external-miner/` remains the in-repo mirror for CI (`external-docs-check`) and operators. When challenge APIs or rules change, update **both** the public repo and `external-miner/` (see root [`../AGENTS.md`](../AGENTS.md) § Challenge public docs). @@ -42,9 +41,9 @@ Public miner docs live **outside** this monorepo (examples + human guides only When updating challenge or local-subnet docs/runbooks, keep these invariants: -- **Master-only eval** — design/prism challenge services run on master; validator has **no challenge exec** (fetch sealed weights only). -- **Simulate submissions** — submit baseline **and** a cheat fixture through the challenge service; poll `/events` + `/logs`; do not treat `/health` alone as proof. -- **Design admin winners** — after clean `awaiting_admin`, operator bearer awards 1|2 winners; then leaf → seal path. +- **Master-only eval** — `relearn-challenge` runs on master; validator has **no challenge exec** (fetch sealed weights only). +- **Simulate submissions** — `POST /v1/submissions` then poll `GET /v1/submissions/{id}`; do not treat `/health` alone as proof. A regression must not become champion. +- **Relearn promote** — after clean `awaiting_admin`, operator bearer `POST /v1/admin/promote`; then leaf → seal path. - **No host Sim in staging/prod** — Docker only; `SimSandbox` / `BASE_ALLOW_HOST_SIM` are CI/local opt-in. - **Seal path** — `POST /v1/weights/raw` → seal → `GET /v1/weights/latest` with `sealed: true` (unsealed burn fallback is always available). That path needs `challenge_sk` + `gateway_sk`, **not** a gateway owner wallet. Validator wallets are for on-chain submit only. - Normative local procedure: [`runbooks/local-testnet-e2e.md`](runbooks/local-testnet-e2e.md). Repo contract: [`../AGENTS.md`](../AGENTS.md) § Challenge verification. diff --git a/docs/ARCHITECTURE.md b/docs/ARCHITECTURE.md index 7b5181ef3..80a7c0484 100644 --- a/docs/ARCHITECTURE.md +++ b/docs/ARCHITECTURE.md @@ -5,8 +5,9 @@ Operator-facing map of the control plane. Normative byte contracts live in the f | Spec | Status | Role | |------|--------|------| | [`BUNDLE_SPEC.md`](./BUNDLE_SPEC.md) | **FROZEN** | Epoch bundle SCALE layout, merkle, aggregation, on-chain payload bounds | -| [`DESIGN_CHALLENGE.md`](./DESIGN_CHALLENGE.md) | **FROZEN** | `design` challenge: harness sandbox, agentic review, admin winners, D24 leaves | -| [`PRISM.md`](./PRISM.md) | live | `prism` Lium GPU recipe challenge (HTTP submit) | +| [`DESIGN_CHALLENGE.md`](./DESIGN_CHALLENGE.md) | archived freeze | Retired `design` product (not live) | +| [`PRISM.md`](./PRISM.md) | archived | Retired `prism` product (Lium rails reused by Relearn) | +| [`RELEARN.md`](./RELEARN.md) | live | `relearn` post-training factory (HTTP submit, no CVM) | Do not restate those contracts here. Link them. @@ -23,7 +24,7 @@ Miner-facing docs (version-pinned): [`external-miner/`](./external-miner/). - Gateway runs **only** as subnet owner (master). Startup asserts hotkey == on-chain `SubnetOwnerHotkey` or exits `2` before bind. - Validators **recompute** the weight vector from a signed, merkle-rooted epoch bundle. Challenge keys and measurements come from **owner-signed local files**, never from gateway HTTP. - CRV4 timelock commit-reveal on Bittensor testnet/mainnet as configured. Reveal is automatic on-chain. -- Challenges accept miner work over **HTTP** (design Python harness sandbox; prism Lium GPU eval). No miner Phala/CVM path. +- The live challenge accepts miner work over **HTTP** (Relearn digest freeze → Lium/sim eval). No miner Phala/CVM / TDX path. --- @@ -34,8 +35,7 @@ Miner-facing docs (version-pinned): [`external-miner/`](./external-miner/). │ Master host (compose profile master) │ │ postgres · gateway · validator · │ │ updater · socket-proxy · │ - │ prism-challenge · design-challenge · │ - │ design-egress-proxy │ + │ relearn-challenge │ └───────────────┬─────────────────────┘ │ TLS terminates in gateway (D20) │ /challenge/{id}/* /v1/bundle/* @@ -48,7 +48,7 @@ Miner-facing docs (version-pinned): [`external-miner/`](./external-miner/). │ HTTP submit ┌───────────────▼─────────────────────┐ │ Miner clients (no TEE required) │ - │ design harness / prism scripts │ + │ relearn artifact digest + Lium BYOK │ └─────────────────────────────────────┘ ``` @@ -56,9 +56,7 @@ Miner-facing docs (version-pinned): [`external-miner/`](./external-miner/). |----------------|------| | `gateway` | Master-only: registry, reverse proxy, bundle seal/serve, sole TLS owner; mounts marketing [`SITE_API.md`](./SITE_API.md) (`GET /v1/site/*`) | | `validator` | Fetch/mirror bundle, verify, recompute, peer cross-check, CRV4 submit, dissent | -| `design-challenge` | **Master-only:** sandbox harness runs, sanitize/viewer, scoring, sign leaves | -| `design-egress-proxy` | **Master-only:** open sandbox egress (internal-target blocklist) + budgeted LLM path | -| `prism-challenge` | **Master-only:** Lium (or sim) recipe eval, review gate, sign leaves | +| `relearn-challenge` | **Master-only:** digest freeze, holdout unseal, Lium/sim eval, operator promote, sign leaves | | `updater` | Digest-pinned rollouts via `docker-socket-proxy` (master) | | `trustroot` | Offline keygen / sign / verify for owner-signed TOML | | `bundle` | SCALE types, seal, verify (`PROTOCOL_VERSION`) | @@ -95,7 +93,7 @@ Miner-facing docs (version-pinned): [`external-miner/`](./external-miner/). | `config/measurements.toml` + `.sig` | yes | every validator from **disk** | | Challenge / owner mini-secrets | **never** | challenge service / offline ceremony only | -Current emission posture: `design = 0` bps, `prism = 10000` bps (100% prism; sum = 10000). +Current emission posture: `relearn = 10000` bps (100%; one-challenge subnet). Gateway DB is **routing only**. It is never a source of challenge keys, emission shares, or measurements (D18, D23). diff --git a/docs/COMPLETENESS.md b/docs/COMPLETENESS.md index e7aedddf8..5d324176e 100644 --- a/docs/COMPLETENESS.md +++ b/docs/COMPLETENESS.md @@ -45,31 +45,21 @@ Honest per-component status as of `main` HEAD. Updated as phases land. | Bundle seal (`POST /v1/weights/raw` → `GET /v1/weights/latest`) | done | Unsealed: fail-closed burn (`sealed: false`, uid 0 = 100%) instead of 404. | | Chain backend | done | Live only. `fake_owner` was removed from `bins/gateway`. | -## agent-challenge / hypertraining-challenge +## agent-challenge / hypertraining-challenge / design / prism (products) -Removed (replaced by design + prism HTTP paths; no Phala/CVM miner). +Removed as **live products**. Shared rails (`prism-lium*`, `prism-competition` paired tests, receipts, emit/carry) stay as libraries. Frozen specs (`DESIGN_CHALLENGE.md`, `PRISM.md`) remain for `xtask` gates. -## design-challenge +## relearn-challenge | Component | Status | Notes | |-----------|--------|-------| -| Crates (`crates/design-*`) | done | task, harness, prompts, sandbox, sanitize, store, egress-proxy, challenge. Elo lives in `design_rating` Postgres via `design-db` / `design-store-pg` — not a standalone crate. | -| Binary (`bins/design-challenge`) | done | HTTP API on `:8093`. | -| Binary (`bins/design-egress-proxy`) | done | Open egress proxy (internal blocklist) + budgeted LLM path. | -| Spec + checklist | done | [`DESIGN_CHALLENGE.md`](DESIGN_CHALLENGE.md) + checklist; `xtask design-check`. | -| Compose / images | in progress | deploy-wiring todo (port `28093` local). | -| Emission | **0 bps** | Prism 100% (10000 bps; sum `10000`). | - -## prism-challenge - -| Component | Status | Notes | -|-----------|--------|-------| -| Crate (`crates/prism-challenge`) | done | Lium client + sim backend + pipeline. | -| Binary (`bins/prism-challenge`) | done | Health + submit on `:8092`. | -| Compose service | done | Added to `docker-compose.yml` on `:8092`. | -| Dockerfile target | done | `deploy/Dockerfile` target `prism-challenge`. | -| GHCR image | done | Added to `images.yml` matrix and `ghcr-public.yml`. | -| Emission | **10000 bps** | Prism 100% (sum `10000`). | +| Crates (`crates/relearn-*`) | **done** | task, score, store, eval, http, challenge. | +| Binary (`bins/relearn-challenge`) | **done** | HTTP API on `:8095`. | +| Compose / images | **done** | Default compose + `images.yml` target `relearn-challenge`. | +| Eval pin | **v0** | `config/relearn-pin.toml` — digest + `CortexLM/relearn` SHA empty until first green challenge CI. | +| Teacher | **v0** | Default HTTP API / Sim. NVFP4-on-Lium (`Inferact/GLM-5.3-NVFP4`) when operator sets `RELEARN_TEACHER_BACKEND=lium`. Judge-only; miner weights never served via that API. | +| Emission | **10000 bps** | Relearn 100% (sum `10000`). | +| Spec | live | [`RELEARN.md`](RELEARN.md). | ## Infrastructure @@ -100,10 +90,9 @@ Agent/operator contracts: root [`AGENTS.md`](../AGENTS.md), [`deploy/AGENTS.md`] | Component | Status | Notes | |-----------|--------|-------| -| design harness / sandbox | done | Two-phase Docker + `SimSandbox`; `base_design` SDK injected; sanitize + CSP viewer. | -| design rating / elimination | done | Integer Elo (K=32), bottom 20% / 4-round cooldown, exact-E leaves. | -| design API | done | Harness/quota/runs/viewer/annotate/ops on `:8093`. | -| prism Lium backend | done | `PRISM_FORCE_SIM=false` in staging; the binary logs `eval_backend=lium`. API key is mounted from a file so it never appears in `docker inspect`. | +| relearn HTTP / promote | **done** | `POST /v1/submissions` freeze → unseal → paired judge; `POST /v1/admin/promote` bearer; never crowns a regression. | +| relearn Lium rails | **done** (sim default) | Reuses `prism-lium` client + `SimLiumBackend`. Live rent refuses without `sha256:` eval digest; miner BYOK never logged. | +| design / prism product APIs | retired | Crates remain as unused libraries. | | prism orchestration | done | DB-backed claim/execute/review/similarity/score state machine (`prism_submission` + append-only `prism_stage_event`), pre-pod screens (copy gate + static cheat + AST similarity) before Lium rent, sweeper (10h grace + pre-reclaim log harvest; skips live workers), **detached harness + resume-first boot/periodic reconcile** (reattach live pods via sealed BYOK; fail-closed only when unreattachable — `control_plane_restart` / `harness_detached`; `GET /v1/submissions/{id}/logs`), epoch-close batched D24 leaf emission with **WTA** (`prism-emit` outbox: `emitted_epoch` watermark + `prism_emit_cursor` + positive-score carry + `apply_wta`, migration 0012). `PRISM_MAX_CONCURRENT_EVALS` default/prod = 8. | | prism recipe v1 | done | `prism-recipe` contract, fineweb-edu pinned shard (URL + SHA-256, harness re-verifies), 6h train / 7h pod caps, baseline sources, recipe pin hex on the API. | | prism v3 harness | done (branch `prism-better`) | Multi-file harness package (`main.py` + `prismlib/`, miner code in `unshare --net` subprocess), seeded train stream with authoritative token counter, G6 probes, `prismlib/cheatguard.py` AST audit, METRICS_JSON v2, miner-chosen tokenizer, G5 RULER/BABILong/natural (pretrain-only), `RECIPE_VERSION 1.4.0`. | @@ -127,8 +116,8 @@ Agent/operator contracts: root [`AGENTS.md`](../AGENTS.md), [`deploy/AGENTS.md`] |-----|--------| | DCAP verify holds the attest mutex | A cold Intel PCS fetch (up to 20 s) serialises attestation submissions. | | DCAP error classification | Matches on `anyhow` message text; re-run `cargo test -p attest-policy --features dcap` after any `dcap-qvl` bump. | -| Design compose/images | deploy-wiring in progress; local port `28093` documented. | -| Design emission ceremony | Emission disabled (0 bps); prism at 100% (10000 bps). Optional prod `design_sk` / owner key rotation still pending. | +| Relearn eval image digest | Empty until `CortexLM/relearn` CI publishes a digest-pinned `relearn-eval` image. Live Lium rent is refused until then. | +| Relearn public repo | `gh repo create CortexLM/relearn` needs org write; seed is in `docs/external-miner/relearn-seed/` until the public repo exists. | | Mainnet (netuid 100) | Owner wallet not yet on this machine, so prod runs with `BASE_GATEWAY_REQUIRE_OWNER=0`. | | Prod pin placeholders | `deploy/pins/prod.json` still ships zero-digests until the first successful promote; registry mode rejects placeholders. | | Spaces backup secrets | First prod promote is fail-closed without `BASE_BACKUP_ENDPOINT` + `SPACES_ACCESS_KEY_ID` / `SPACES_SECRET_ACCESS_KEY` (or AWS_* fallbacks) in GitHub. | diff --git a/docs/OPERATOR_SECURITY.md b/docs/OPERATOR_SECURITY.md index eec7db5a4..8359a6eaa 100644 --- a/docs/OPERATOR_SECURITY.md +++ b/docs/OPERATOR_SECURITY.md @@ -13,7 +13,8 @@ Use this before every promote and after every incident. Architecture: [`ARCHITEC - [ ] Challenge signing secrets are **files** mounted into the challenge service, not env values (D11). - [ ] Owner and challenge mini-secrets never committed; only `*.pubkey` / TOML bodies + detached `.sig` in git. - [ ] Cloudflare / DO / Phala tokens live only in operator secret stores, not in docs or CI logs. -- [ ] Design agentic review: OpenRouter key is mounted on `design-challenge` / `design-egress-proxy` as a **file**, never into miner sandboxes. The ephemeral `design-review` container must receive the key via a **file mount** (`OPENROUTER_API_KEY_FILE`), never as `OPENROUTER_API_KEY` in container env (`/proc//environ` is boot-fixed). `run_command` must keep procfs and `/run/review-secrets` denied. Do not turn off `AGENTIC_ENABLE_RUN_COMMAND` in prod without a replacement inspection path. +- [ ] Relearn miner BYOK (`LIUM_API_KEY` / `X-Lium-Api-Key`) is never written to git, compose env committed files, or logs. Control-plane Lium mounts under `deploy/secrets/lium` are files, mode **0400**, uid **65532**. +- [ ] Teacher HTTP API (`RELEARN_TEACHER_API_URL`) is **judge-only**. Never point it at miner weights as the served / scored artifact. --- @@ -22,8 +23,8 @@ Use this before every promote and after every incident. Architecture: [`ARCHITEC - [ ] Every image reference is digest-pinned (`repo@sha256:<64 hex>`). No `:latest`. - [ ] Exactly one mount of `/var/run/docker.sock`: on `socket-proxy` (read-only). - [ ] socket-proxy allowlist matches updater needs (`CONTAINERS`, `IMAGES`, `POST` as configured). -- [ ] `design-challenge` sets `DESIGN_SCREENSHOT_PROXY=http://design-egress-proxy:8094` (screenshot Chromium must not talk direct to the `base` network). -- [ ] Staging/prod never set `BASE_ALLOW_HOST_SIM` / `DESIGN_FORCE_SIM=true` (asserted by `assert-compose-matrix.sh`). +- [ ] Staging/prod never set `BASE_ALLOW_HOST_SIM` / `DESIGN_FORCE_SIM` / `RELEARN_FORCE_SIM=true` as a live scoring path (asserted by `assert-compose-matrix.sh` for host Sim). +- [ ] Relearn live rent requires `config/relearn-pin.toml` `eval_image_digest` starting with `sha256:`. No floating eval tags. - [ ] Gateway service uses compose profile **`master`** only on the owner host. - [ ] Profile `evil-gateway` is **absent** from prod hosts. Spot-check: diff --git a/docs/RELEARN.md b/docs/RELEARN.md new file mode 100644 index 000000000..9039c14ff --- /dev/null +++ b/docs/RELEARN.md @@ -0,0 +1,41 @@ +# Relearn challenge (live) + +One-challenge Cortex subnet. Challenge artifacts (eval image, harness, +generators, teacher) live in [`CortexLM/relearn`](https://github.com/CortexLM/relearn). +This repo is the control plane and pins `config/relearn-pin.toml`. + +## Identifiers + +| Field | Value | +|-------|--------| +| `challenge_id` | `relearn` | +| `challenge_scoring_version` | `1` | +| Base model | `Qwen/Qwen3.8-Flash-Next` (verified Hugging Face id) | +| Teacher / judge | `zai-org/GLM-5.3` (verified Hugging Face id) | +| Teacher NVFP4 | `Inferact/GLM-5.3-NVFP4` (preferred Lium serve when practical) | +| Port | `8095` | +| `SCORE_MAX` | `1_000_000` | +| Emission | `10000` bps | + +No TDX / Phala CVM. Miner pays Lium (`LIUM_API_KEY` / `X-Lium-Api-Key`). +Control plane rents a digest-pinned eval image and operator-SSH harvests +receipts. Holdout unseals only after the submission digest freezes. + +## Submit / promote + +1. `POST /v1/submissions` `{ miner_hotkey, artifact_digest, artifact_uri? }` +2. Freeze digest + nonce; unseal holdout +3. Paired displacement vs champion + public-private / perturbation / canary / + agent-trace gates +4. Never crown a regression +5. `POST /v1/admin/promote` (operator bearer) only when eligible +6. D24 exact-E leaf set via `challenge-common` + +Teacher HTTP API is **judge-only**. Miner weights are never the served model +on that API. NVFP4-on-Lium is the preferred teacher path when an 8× +Blackwell-class host can be rented; v0 defaults to HTTP API / Sim. + +## Official benches + +No official-benchmark contamination. Train and eval generators are disjoint; +decontam vs official benches lives in CortexLM/relearn. diff --git a/docs/external-miner/README.md b/docs/external-miner/README.md index d03a7b59d..b9ba58593 100644 --- a/docs/external-miner/README.md +++ b/docs/external-miner/README.md @@ -8,34 +8,34 @@ This badge must match `bundle::PROTOCOL_VERSION` in crate `bundle`. CI gate: `cargo run -p xtask -- external-docs-check`. -Agent-v1 / Phala CVM / hypertraining miner paths are **removed**. Miners submit -over HTTP to the live challenges: +Cortex is a **one-challenge subnet**: **Relearn**. Design and Prism are retired +as products (their crates remain as unused libraries / historical specs). | Challenge | `challenge_id` | Scoring | Guide | Public miner repo | |-----------|----------------|---------|-------|-------------------| -| Design | `design` | `challenge_scoring_version` **2** (daily share ≥2 wins + agentic) | [design.md](./design.md) | [BaseIntelligence/design-challenge](https://github.com/BaseIntelligence/design-challenge) | -| Prism | `prism` | `challenge_scoring_version` **4** (G2 public-suite benchmarks) | [prism.md](./prism.md) | [BaseIntelligence/prism](https://github.com/BaseIntelligence/prism) | +| Relearn | `relearn` | `challenge_scoring_version` **1** (paired displacement vs champion) | [relearn.md](./relearn.md) | [CortexLM/relearn](https://github.com/CortexLM/relearn) | + +Pinned models (verified Hugging Face ids): + +- Base: `Qwen/Qwen3.8-Flash-Next` +- Teacher / judge: `zai-org/GLM-5.3` Do **not** conflate version axes: | Axis | Value | Meaning | |------|-------|---------| | Bundle `protocol_version` | **1** | Leaf / merkle / weight bytes ([`BUNDLE_SPEC.md`](../BUNDLE_SPEC.md)) | -| Design scoring | **1** | Agentic anti-cheat + admin winners 1\|2 ([`DESIGN_CHALLENGE.md`](../DESIGN_CHALLENGE.md)) | -| Prism scoring | **2** | Pure bpb + agentic/AST/metrics anti-cheat ([`PRISM.md`](../PRISM.md)) | +| Relearn scoring | **1** | Displacement vs previous champion + overfit gates | | Page | Topic | |------|-------| -| [design.md](./design.md) | Design harness (`agent.py` + `pyproject.toml`) HTTP submit | -| [examples/design-baseline/](./examples/design-baseline/) | Reference design miner (`llm.chat` → required HTML pages) | -| [prism.md](./prism.md) | Prism AutoModel patch (`automodel.base` + `automodel.patch`) HTTP submit | -| [examples/dense-1b/](./examples/dense-1b/) | Reference Prism miner (dense ~975M, ZeRO-1) | -| [troubleshoot.md](./troubleshoot.md) | Common HTTP / quota / scoring failures | +| [relearn.md](./relearn.md) | Artifact digest HTTP submit, Lium BYOK, promote | +| [troubleshoot.md](./troubleshoot.md) | Common HTTP / scoring failures | Normative contracts: -- Design freeze: [`../DESIGN_CHALLENGE.md`](../DESIGN_CHALLENGE.md) -- Prism: [`../PRISM.md`](../PRISM.md) + [`../PRISM_RECIPE.md`](../PRISM_RECIPE.md) +- Relearn (this repo): [`../ARCHITECTURE.md`](../ARCHITECTURE.md) + `config/relearn-pin.toml` +- Public eval image / harness: [CortexLM/relearn](https://github.com/CortexLM/relearn) - Bundle bytes: [`../BUNDLE_SPEC.md`](../BUNDLE_SPEC.md) - Threat claim (D19): [`../THREAT_MODEL.md`](../THREAT_MODEL.md) §1 @@ -44,16 +44,15 @@ Normative contracts: Production/staging miners call the **gateway** reverse proxy: ```text -https:///challenge/design/... -https:///challenge/prism/... +https:///challenge/relearn/... ``` Local smoke (host ports from `env-local.yml`): ```bash -curl -sS http://127.0.0.1:28093/health # design-challenge -curl -sS http://127.0.0.1:28092/health # prism-challenge +curl -sS http://127.0.0.1:28095/health # relearn-challenge ``` -Never paste mnemonics or challenge signing keys into miner clients. Hotkeys are -public 64-hex identifiers only. +Never paste mnemonics, Lium keys, or challenge signing keys into miner clients +or into git. Hotkeys are public 64-hex identifiers only. Read `LIUM_API_KEY` +from the environment (or send `X-Lium-Api-Key` on submit) — never commit it. diff --git a/docs/external-miner/design.md b/docs/external-miner/design.md index 0b342b3a7..f9901919a 100644 --- a/docs/external-miner/design.md +++ b/docs/external-miner/design.md @@ -1,213 +1,8 @@ -# Design challenge — HTTP harness submit +# Design — retired -**challenge_id:** `design` -**scoring_version:** `3` -**Path:** HTTP only — **no Phala/CVM** +**Path:** HTTP submit to **relearn** only — **no Phala/CVM** -Normative freeze: [`../DESIGN_CHALLENGE.md`](../DESIGN_CHALLENGE.md). - -## What you submit - -A Python harness bundle (source, not a container image) — prefer a **ZIP**: - -| File | Required | -|------|----------| -| `agent.py` | `def run(task, llm, out) -> None` | -| `pyproject.toml` | Python deps **allowed** — installed at the sandbox install phase | -| Extra files | ≤ 16, ≤ 256 KiB each, total ≤ 1 MiB | - -Optional `env_vars` (API keys, etc.) are injected into the sandbox **run** -phase only — the install phase never sees them. Do not use `DESIGN_*` / -proxy / Python runtime keys. - -### Dependencies (`pyproject.toml`) - -You **may declare any PyPI dependencies** under `[project] dependencies`. -Before your agent runs, the sandbox creates a venv and executes -`pip install --no-cache-dir -e .` against your bundle (install timeout -**300 s**). Build backends run inside the same hardened one-shot container — -no host execution, no miner env vars. - -- Install failure (uninstallable dep, timeout) → error class `install`, which - **auto-retries up to 3 times**; a persistently broken `pyproject.toml` then - fails the run. Watch `GET /v1/runs/{id}/logs` (phase `install`) and - `GET /v1/runs/{id}/events` for `auto_retry` events. -- Keep deps light: pure-Python or prebuilt wheels install fastest; heavy - source builds can exceed the install timeout or sandbox memory. - -### Network access (install + run) - -Both phases reach the **public Internet** through the operator egress proxy -(`HTTP_PROXY` / `HTTPS_PROXY` are set in the sandbox; `pip`, `requests`, -`httpx`, `urllib` honor them). Your agent **may call external APIs and MCP -servers** during the run phase — put the credentials in `env_vars` (locked at -submission, never logged). LLM calls keep going through `llm.chat` (budgeted); -the OpenRouter key is never inside the sandbox. - -Blocked targets (refused with `403`): cloud metadata `169.254.169.254`, -loopback, RFC1918/VPC ranges (`10.0.0.0/8`, `172.16.0.0/12`, -`192.168.0.0/16`), CGNAT `100.64.0.0/10`, and the control plane's internal -services. Blocks are enforced **after DNS resolution** (DNS-rebinding safe). - -The operator injects a non-modifiable `base_design` SDK and runs your harness -inside a hardened Docker sandbox (run timeout **30 minutes**). You never -receive the OpenRouter key or the challenge signing key. - -### Required pages - -Your run must write under `/out/pages/`: - -- `index.html` -- `pricing.html` -- `components.html` - -Missing pages → automatic `Score(0)`. - -## Submit - -```bash -# ZIP via gateway (preferred) -curl -sS -X POST "$BASE_GATEWAY/challenge/design/v1/harness" \ - -H 'content-type: application/zip' \ - -H "X-Miner-Hotkey: <64 lowercase hex>" \ - -H 'X-Env-Json: {"OPENAI_API_KEY":"..."}' \ - --data-binary @harness.zip - -# JSON + zip_base64 -curl -sS -X POST "$BASE_GATEWAY/challenge/design/v1/harness" \ - -H 'content-type: application/json' \ - -d @harness.json - -# Or direct challenge port in local/dev -curl -sS -X POST "http://127.0.0.1:28093/v1/harness" \ - -H 'content-type: application/json' \ - -d @harness.json -``` - -Reference baseline (normative example miners should start from): -[`examples/design-baseline/`](./examples/design-baseline/) — `agent.py` calls -`llm.chat` and writes `index.html` / `pricing.html` / `components.html` via -`out.write_page`. - -Minimal `harness.json` shape: - -```json -{ - "miner_hotkey": "<64 lowercase hex>", - "agent_py": "", - "pyproject_toml": "", - "extra_files": {}, - "env_vars": {} -} -``` - -`POST /v1/harness` is **idempotent** on content digest (`harness_id`). - -## Submission gating (1-max) + auto round enqueue - -- Your hotkey must be **registered on the subnet** (metagraph). Unknown hotkey - → `403 hotkey_not_in_metagraph`. Intake uses a bulk metagraph cache with a - **15 minute** fail-closed TTL (`503 metagraph_unavailable` → retry shortly). -- **One accepted submission per hotkey**. While yours is `registered` / - `blocked` / `rejected`, a *different* harness gets `409 submission_gated`. - Re-POSTing the **identical** bundle is always safe (idempotent - `200 already-queued`). -- After a **terminal** outcome that closes gating (cheat / admin reject / - unscored timeout / budget exhaustion), you cannot submit a **new** digest on - the same hotkey until that hotkey **leaves the metagraph** and you register a - **new UID** (same hotkey is fine). -- Infra auto-retries on the *same* run id (up to 3) are not a new schedule. -- `env_vars` are **locked at submission**; changing them means a new digest, - which requires a free slot. - -## Quotas and rounds - -- **10 rounds per UTC day** (`ROUND_SECS = 8640`; `round_id = floor(unix / 8640)`). -- An accepted harness **waits for the next round**: your `POST /v1/harness` - schedules into `round_id + 1` (never mid-round). After that, the organizer - **auto-enqueues your latest active harness every open round** with that - round's **shared prompt** — you do **not** need to re-POST to keep competing. - Eliminated miners are skipped until their cooldown ends. -- Sandbox **run** timeout is **30 minutes** (`AGENT_RUN_TIMEOUT_SECS = 1800`). -- Each round picks **1 shared prompt** for every harness - (`PROMPTS_PER_ROUND = 1`). -- Daily run quota is **split by origin**: - - **Manual** — **10** runs/day, charged only by your own `POST /v1/harness` - (the initial next-round schedule). - - **Scheduled** — round-loop auto-enqueue / ops requeue (10 rounds × 1 - prompt = **10** runs; cap **20**). You never spend manual quota by being - auto-queued. -- Infra failures (package install, review/LLM infra) **auto-retry up to 3 - times**; cheat / rejected / admin reject / unscored timeout are terminal. - Manual retry of a failed run: `POST /v1/runs/{id}/retry`. - -Check quota: `GET /v1/quota/{hotkey}` — `manual` and `scheduled` objects -(`runs_used` / `limit` / `remaining`) alongside the whole-day `runs_used`. - -## Scoring (summary) - -After sanitize, master-side **agentic anti-cheat** runs in a containerized -reviewer. A pre-LLM **copy gate** rejects a byte/AST copy of an *earlier* -harness outright (`rejected`, `Score(0)`, no LLM call); `cheat` / `suspicious` -from the LLM review → `Score(0)`. Starting from the published **baseline** is -fine — copying another *miner's* harness is not. Both the copy gate and the LLM -review compare you against **other miners' earlier harnesses only**: your own -previous versions (same hotkey **or** same coldkey) are excluded from the -corpus, so iterating via a new hotkey under the same coldkey is never read as -self-copying. - -Clean runs await **admin winners** (1 or 2 harnesses per round); each round win -is one **point**. Rewards are **not** winner-take-all on a single round: the -leaf projection shares `SCORE_MAX` **proportionally to round-win points over -the last 10 rounds** (rolling window, cheat excluded). Prompt bank is -automatic (`bank_v1.json`). Inspiration (Mobbin, image gen, UI libs) and -**external API / MCP calls** are allowed; near-identical corpus copies / -scrape-clones are not. Full rules in the freeze doc. - -If a clean run is still unscored **5 chain epochs** after it entered -`awaiting_admin`, it is **auto-rejected** (`reject_reason` on -`GET /v1/runs/{id}`). Admin may also reject with a reason string you can read -on that same route. Either way you need a **new UID** before submitting again. - -Admin APIs are **master-local only** (not proxied on the public gateway). - -## Viewer - -Screenshots only: `GET /v1/view/{run_id}/index.png` returns the full-page PNG -screenshot the orchestrator captures right after sanitize. Produced HTML is -never served — `.html` requests return `410 Gone` (the gateway still wraps -view responses in a CSP `sandbox` (no scripts) lockdown as defense in depth). -Your pages stay static HTML + **embedded** CSS (`