[FEAT] Valkey adapters for the state ports, the counters, and the Durable boot arm

[justin13888]

Description

Lane W-VALKEY of the unattended backlog run (issue #403), stacked on #435 (feat/server-binary-config-operator-commands-401).

Summary

• capsule-server/src/store/valkey.rs: redis-rs adapters for all six state ports (AuthStateStore, UploadSessionStore, ChallengeStore, EnrollmentStore, ChannelStore, CohortStore) over one multiplexed ConnectionManager, plus the ValkeyStores bundle. Every multi-key mutation or decide-and-write is one Lua script (EVALSHA, SCRIPT LOAD on NOSCRIPT); every derived index heals on read; expiry is decided by the injected Clock and written into each record, with PEXPIRE as the collector.
• capsule-server/src/counter/valkey.rs: ValkeyCounters, one hash and one script per key — open, charge or refuse in one server-side step.
• capsule-server/src/counter/conformance.rs: the counter port's suite, extracted from counter/tests.rs and generic over the store, with a racing case on a multi-threaded runtime; the in-memory adapter now runs it case by case and in one pass.
• capsule-server/tests/valkey.rs: the store and counter suites plus a contested finalize claim and a contested counter hit, against a valkey/valkey container (CAPSULE_TEST_VALKEY=1) or a running server (CAPSULE_TEST_VALKEY_URL); skipped with one line otherwise. .config/nextest.toml puts it in the one-thread containers group.
• capsule-server/src/boot.rs: the Durable arm connects to and PINGs VALKEY_URL first (BootError::Valkey on failure, never quoting the URL), then refuses naming DATABASE_URL and server: Postgres adapters and a conformance suite for every durable port #402 until the Postgres half lands. .env.example, local-development.md, dependencies.md (Volatile state row) and SLICES.md (S-C29) say so.

Validation

See the lane report; filled in as gates complete.

Risks and rollout

• The in-memory adapters and Backends::Memory are untouched; the Valkey adapters are reached only through Backends::Durable, which still refuses at the end of the boot path until server: Postgres adapters and a conformance suite for every durable port #402. Nothing persists a schema; every key is under capsule: and carries a TTL except the cohort hash.
• CohortStore on Valkey is interim (decision 3); the key has no TTL so the data migrates with one HGETALL per account.
• The container-gated suite needs a container runtime the gate host did not have; see Validation.

Related Issues

Closes #403. Refs #402 (the Postgres half of the Durable arm, DATABASE_URL).

Decisions taken

Issue 403 - server: Valkey adapters for the auth-state and upload-session ports
Plan:     A-1 (planned against f433d918; executed on the head of lane #401's branch)
Branch:   feat/valkey-adapters-403
Base:     feat/server-binary-config-operator-commands-401 (head of the W-SERVE PR), stacked; the PR targets that branch
Worktree: /var/mnt/scratch/golem/dev/Capsulsaurus/Capsule.worktrees/Capsule-feat-valkey-adapters-403
Cause:    -
Touches:  capsule-server/Cargo.toml (redis; dev: testcontainers, testcontainers-modules), capsule-server/src/store/{mod.rs (pub mod valkey), valkey.rs (new)}, capsule-server/src/counter/{mod.rs, conformance.rs (new), valkey.rs (new), tests.rs}, capsule-server/src/boot.rs (Durable arm's Valkey half: ConnectionManager + PING), capsule-server/tests/valkey.rs (new, env-gated CAPSULE_TEST_VALKEY=1), .config/nextest.toml (containers group filterset), capsule-docs/src/content/docs/design/dependencies.md (redis-rs row: scope + primitives), Cargo.lock, SLICES.md (S-C29 owed note ONLY — coordinate: #402 edits S-C2/S-C37 notes), capsule-server/.env.example (Valkey half of the Backends comment), capsule-docs/src/content/docs/development/local-development.md (the sentence that says VALKEY_URL refuses, if present)
Will not: write a Postgres adapter; enable the multi-replica share limiter or revocation cache; support Redis Cluster; introduce a generic TTL/CAS type; use bb8
Lane:     serialised behind #401; parallel with #402 (shared: boot.rs Durable arm — one hunk each; .config/nextest.toml — adjacent overrides; SLICES.md S-C29 row — both lanes append notes to the same row: this lane owns the row, #402 records its S-C29 remainder in its PR body instead)
Settled:  Separate typed ports, no generic TTL/CAS (AGENTS.md). Barrels (#399). Config/boot seam (#401). Base = head of PR #418 → stacks on #399 → #401.

Decisions taken.

1. Deliverable boundary - all six store ports plus the counters in one lane.
   Taken:    AuthStateStore, UploadSessionStore, ChallengeStore, EnrollmentStore, ChannelStore, CohortStore (interim Valkey hash, no expiry) + ValkeyCounters, the container-gated suite, and the Durable boot arm. store::conformance.rs:61-76 makes Harness demand all six, so a partial set cannot run run_all.
   Rejected: the two named ports only - Backends::Durable would mix Valkey and in-memory doubles in one deployment profile, which filesystem/server.md:21 forbids.
   Reverses: delete the ceremony/cohort impls from ValkeyStores; the Harness impl stops compiling, which is the signal.
   Filed:    -

2. Connection strategy and CAS
   Taken:    One redis::aio::ConnectionManager plus one Lua script per multi-key mutation (EVALSHA with SCRIPT LOAD fallback); no bb8 (the bb8/bb8-redis workspace pins stay unused).
   Rejected: bb8 + WATCH/MULTI - WATCH needs an exclusive connection held across the optimistic loop, reintroducing the read-then-write window claim_finalize and counter hit exist to close.
   Reverses: swap ConnectionManager for bb8::Pool and each Script::invoke for a WATCH/MULTI closure; call sites unchanged.

3. Where the device-cohort map lives in the Durable arm
   Taken:    A Valkey hash capsule:cohorts:{user_id} with no expiry, documented as interim until #402's remainder lands a Postgres CohortStore (note: #402 ships PostgresCohorts in this run — the lane wires ValkeyStores::cohorts to satisfy Harness and boot.rs prefers the Postgres adapter when DATABASE_URL is set; record which was wired).
   Rejected: InMemoryCohorts in Durable (authentication.md:211: durable); deferring (Harness has no optional member).
   Reverses: point ValkeyStores::cohorts at the Postgres adapter and delete ValkeyCohorts; data migrates with one HGETALL.

4. How the Valkey harness moves time
   Taken:    Real tokio::time::sleep in Harness::advance against stores built with a 250 ms uniform TTL; the containers group is max-threads = 1.
   Rejected: a fresh container per expiry case with production TTLs.
   Reverses: change with_uniform_ttl's argument and the sleep.

Decisions taken inside the manifest.

5. How expiry is decided - the injected Clock, with PEXPIRE as the collector (reverses 4)
   Taken:    Every record hash carries an adapter-internal expires_at (microseconds) written from the injected Clock when the record is opened; every script that reads a record checks it against a `now` argument and deletes an expired one; PEXPIRE is set on the same key with the same lifetime and only collects keys nothing reads again. The Valkey harness in tests/valkey.rs therefore advances a ManualClock exactly as InMemoryStores does, with a one-minute uniform TTL and no sleeps. The counter window is likewise measured from the port's own `at` argument (opened_at in the hash), so counter::conformance passes absolute instants and drives both adapters unchanged.
   Rejected: Sleeping (decision 4). store::conformance's touching_a_session_records_activity_without_extending_its_life advances to ttl - 1 ns and expects the session live, then 1 ns more and expects it gone; no real-clock harness can land a round trip inside the last nanosecond of a window, so a sleeping harness needs a heuristic that under-sleeps sub-TTL advances and repays them on the next - a claim about the suite rather than the adapter, and one the counter suite's own sub-window steps break. Also rejected: INCR plus first-hit EXPIRE for counters, which measures the window on the server's clock and cannot answer the suite's `at`.
   Reverses: delete the expires_at/opened_at fields and the `live`/`expired` Lua helpers; the harness must then sleep, and decision 4's caveat returns.
   Filed:    -

6. What the Durable arm does once Valkey answers
   Taken:    boot::valkey connects, PINGs and builds ValkeyStores first; boot::durable then returns BootError::AdapterUnavailable { key: "DATABASE_URL", issue: "#402 (Postgres)" }. A new BootError::Valkey names an unreachable server without the URL. #402's merge fills durable's body; the hunk is one function each.
   Rejected: assembling the App over Valkey stores plus in-memory doubles for the index, accounts, albums and the rest - the mixed profile store/mod.rs forbids. Also rejected: leaving the old VALKEY_URL refusal in place - the Valkey half would be unreachable from any boot path.
   Reverses: replace durable's body with the assembly; delete nothing.
   Filed:    -

7. The eviction view excludes a claimed session
   Taken:    capsule:upload:progress holds only Pending and Uploading sessions; claim_finalize and set_status(WaitingForProcessing) ZREM the id, per the port's claim_finalize doc ("the winner leaves the progress view rather than being evicted out from under itself"). The in-memory double keeps a WaitingForProcessing session as a candidate (is_active includes it); the conformance suite does not distinguish the two, and memory.rs is outside this lane's manifest.
   Rejected: mirroring the double.
   Reverses: make is_evictable equal to is_active.
   Filed:    - (reported to the orchestrator as a finding against memory.rs)

8. Test-side environment surface
   Taken:    CAPSULE_TEST_VALKEY=1 (container via testcontainers), CAPSULE_TEST_VALKEY_URL (an already-running server, takes precedence), CAPSULE_TEST_VALKEY_TAG (image tag; default 9.0.4, the compose.yaml image). The plan named CAPSULE_TEST_VALKEY_IMAGE; testcontainers-modules' Valkey image fixes the repository and varies only the tag, so the variable is named for what it sets.
   Rejected: no URL override - the gate host's rootless podman could not run an image from its default storage root, and a URL override is also the developer path against `mise run serve-deps`.
   Reverses: delete the branch in tests/valkey.rs::server.
   Filed:    -

9. redis 1.2.2, not the 1.6.0 the caret pin resolves to
   Taken:    Cargo.lock pins redis at 1.2.2, the version the workspace manifest declares, via `cargo update -p redis --precise 1.2.2`; features tokio-rustls-comp and script are added in capsule-server/Cargo.toml (the root manifest is outside the manifest).
   Rejected: accepting 1.6.0 - the adapter was written against the 1.2.2 API and the declared pin should be the locked one.
   Reverses: `cargo update -p redis`.
   Filed:    -

Unresolved review notes

[cloudflare-workers-and-pages]

Deploying capsule with    Cloudflare Pages

Latest commit:	a959cb3
Status:	✅  Deploy successful!
Preview URL:	https://aee3ba89.capsule-22k.pages.dev
Branch Preview URL:	https://feat-valkey-adapters-403.capsule-22k.pages.dev

View logs