Skip to content
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 2 additions & 0 deletions .github/workflows/verify.yml
Original file line number Diff line number Diff line change
Expand Up @@ -32,6 +32,7 @@ jobs:
python3 tools/test_wire_format_reference.py
python3 tools/interop-reference.py verify
python3 tools/interop-v2-reference.py verify
python3 tools/test_encryption_verify.py
python3 tools/encryption-verify.py
python3 tools/wire-format-reference.py verify

Expand All @@ -40,6 +41,7 @@ jobs:
pip install cryptography==49.0.0 msgpack==1.2.1 lz4==4.4.5
python3 tools/interop-reference.py verify
python3 tools/interop-v2-reference.py verify
python3 tools/test_encryption_verify.py
python3 tools/encryption-verify.py --require-seal
python3 tools/wire-format-reference.py verify --require-extras

Expand Down
24 changes: 24 additions & 0 deletions CHANGELOG.md
Original file line number Diff line number Diff line change
Expand Up @@ -4,6 +4,30 @@ All notable changes to the CacheKit Protocol Specification.

## [Unreleased]

### Encryption — keyring conformance vectors + status reconciliation (LAB-687)

- [`test-vectors/encryption.json`](test-vectors/encryption.json) gains a `keyring`
block: two master keys (`k1`, `k2`) for tenant `keyring-conformance`, one entry
sealed under each, and per-vector `key_fingerprint_hex` — the fingerprint of the
HKDF-derived per-tenant encryption key, as cachekit-py stores it. Frozen names
`encrypted_with_k1` / `encrypted_with_k2`; append-only like the main set.
- [`tools/encryption-verify.py`](tools/encryption-verify.py) enforces
[`spec/encryption.md` § Key Rotation (Keyring)](spec/encryption.md#key-rotation-keyring):
`[k2, k1]` decrypts both at the declared entry, `[k2]` alone rejects the k1
entry, and the stored fingerprint selects the derived key — a master-key
fingerprint cannot select. Entry derivation, metadata, AAD and selection run
in the stdlib lane; only the decrypt attempts need `cryptography`.
- New [`tools/test_encryption_verify.py`](tools/test_encryption_verify.py) mutation
suite runs ahead of the verifier in both CI lanes (same doctrine as the
wire-format guard): every keyring guard is proven to go red by poisoning a copy
of the fixture. Added after the LAB-687 panel found three vacuous passes in the
first revision.
- Status banners reconciled with shipped code: `spec/encryption.md` and
`decisions/key-rotation.md` no longer say "not yet implemented"; the
[feature matrix](sdk-feature-matrix.md#encryption) Key rotation row is ✅ for
Python, Rust and TypeScript (cachekit-py#261, cachekit-rs#63, cachekit-ts#103).


### Wire format — compressed-byte reproducibility scoped per-vector (LAB-1751)

- LZ4 compressed bytes are **not canonical** across conforming block encoders.
Expand Down
4 changes: 2 additions & 2 deletions decisions/key-rotation.md
Original file line number Diff line number Diff line change
Expand Up @@ -4,11 +4,11 @@

| | |
| :--- | :--- |
| **Status** | Proposed (accepted on merge) |
| **Status** | Accepted (protocol [#34](https://github.com/cachekit-io/protocol/pull/34), 2026-07-23); implemented in all three SDKs by 2026-08-15 |
| **Date** | 2026-07-23 |
| **Ticket** | LAB-516 (filed by the LAB-275 cross-SDK feature-gap audit) |
| **Normative spec** | [`spec/encryption.md` → Key Rotation (Keyring)](../spec/encryption.md#key-rotation-keyring) — the spec section owns the rules; this record owns the rationale and runbooks. |
| **Implementation** | Not yet shipped in any SDK — tracked as LAB-516 sub-issues. `ZeroKnowledgeEncryptor::rotate_key()` returns `NotImplemented` (`cachekit-core/src/encryption/core.rs:492`). The [feature matrix](../sdk-feature-matrix.md#encryption) rotation row reads ❌ fleet-wide as of the LAB-1400 consolidation and flips per SDK only as each implementation ships. |
| **Implementation** | Shipped. Shared decrypt helper `Keyring` in cachekit-core 0.5.0 ([cachekit-core#67](https://github.com/cachekit-io/cachekit-core/pull/67), LAB-683 — also deleted the `rotate_key()` stub, `KeyRotationState` and `RotationAwareHeader`); SDK surfaces in [cachekit-py#261](https://github.com/cachekit-io/cachekit-py/pull/261) (LAB-684), [cachekit-rs#63](https://github.com/cachekit-io/cachekit-rs/pull/63) (LAB-686) and [cachekit-ts#103](https://github.com/cachekit-io/cachekit-ts/pull/103) (LAB-685). The [feature matrix](../sdk-feature-matrix.md#encryption) rotation row reads ✅ for all three; conformance is enforced by [`tools/encryption-verify.py`](../tools/encryption-verify.py) against the keyring vectors in [`test-vectors/encryption.json`](../test-vectors/encryption.json) (LAB-687). |

---

Expand Down
14 changes: 6 additions & 8 deletions sdk-feature-matrix.md
Original file line number Diff line number Diff line change
Expand Up @@ -6,9 +6,9 @@

**Feature parity and compliance status across all CacheKit SDK implementations.**

*Last updated: 2026-08-04 — LAB-1400 consolidation of ten open matrix PRs into one end-state. Every version-keyed claim is verified against the **published artifact** (registry metadata, and the `.crate`/`.tgz` contents where an embedded dependency version decides the answer), not against a repo branch — see [decisions/matrix-version-verification.md](decisions/matrix-version-verification.md) for why and how. Per-PR fold verdicts are in [CHANGELOG.md](CHANGELOG.md); per-row history is `git log sdk-feature-matrix.md`.*
*Last updated: 2026-09-02 — LAB-687 keyring conformance and documentation reconciliation (following LAB-1400's matrix baseline correction). Every version-keyed claim is verified against the **published artifact** (registry metadata, and the `.crate`/`.tgz` contents where an embedded dependency version decides the answer), not against a repo branch — see [decisions/matrix-version-verification.md](decisions/matrix-version-verification.md) for why and how. Per-PR fold verdicts are in [CHANGELOG.md](CHANGELOG.md); per-row history is `git log sdk-feature-matrix.md`.*

*__Cells that reversed — check these if you built on them:__ Key rotation (py/rs ✅ → ❌ fleet-wide), Rust `::secure` preset and Rust sync support (both ✅ → do not exist), Builder API (py/ts ✅ → ❌), Hardware acceleration (rs ✅ → not re-exported, ts N/A → ❌), TypeScript Arrow (🔜 → ❌), Python's encrypted read path (documented fail-closed → **fail-open by default**), and `cache.secure.wrap()` in TypeScript (implied encryption → **no guarantee**). The TypeScript protocol-1.1 `bin` rollout also reversed twice in two days: it is **not** shipped on either ts path (per-artifact evidence in the [cachekit-core architecture note](#architecture-notes)).*
*__Cells that reversed — check these if you built on them:__ Rust `::secure` preset and Rust sync support (both ✅ → do not exist), Builder API (py/ts ✅ → ❌), Hardware acceleration (rs ✅ → not re-exported, ts N/A → ❌), TypeScript Arrow (🔜 → ❌), Python's encrypted read path (documented fail-closed → **fail-open by default**), and `cache.secure.wrap()` in TypeScript (implied encryption → **no guarantee**). The TypeScript protocol-1.1 `bin` rollout also reversed twice in two days: it is **not** shipped on either ts path (per-artifact evidence in the [cachekit-core architecture note](#architecture-notes)).*

</div>

Expand Down Expand Up @@ -66,19 +66,17 @@
| HKDF-SHA256 key derivation | ✅ via Rust FFI | ✅ | ✅ via NAPI (Rust) | 🔜 Planned |
| Per-tenant key isolation | ✅ | ✅ | ✅ via TenantKeys NAPI | 🔜 Planned |
| AAD v0x03 (cache_key binding) | ✅ | ✅ | ✅ | ❌ |
| Key rotation | ❌ mismatch detection only⁵ | ❌⁵ | ❌ — nonce-exhaustion detection exists but is **unobservable** to the caller⁵ ⁸ | ❌ |
| Key rotation | ✅ keyring; derived-key fingerprint selection⁵ | ✅ keyring; sequential attempts⁵ | ✅ keyring; sequential attempts⁵ | ❌ |
Comment thread
coderabbitai[bot] marked this conversation as resolved.
| **Tamper / wrong-key failure mode** | ⚠️ **fail-OPEN by default** — warn + recompute; switchable with `CACHEKIT_ENCRYPTION_FAIL_CLOSED=true`¹⁸ | ✅ **Fails closed** — `decrypt(…)?` propagates (`client.rs:830`, `:847`), and `#[cachekit(secure)]` emits no fail-open arm (`cachekit-macros/src/lib.rs:439-451`) | ⚠️ **fail-OPEN on reads, silently drops writes, not switchable**⁸ | — |
| **Does the `secure` API enforce encryption?** | ✅ Raises without a key | ✅ `secure()` returns `Err` | ❌ **`cache.secure.wrap()` is an unconditional alias for `wrap()`** — silently caches plaintext on any instance not built by `createCache.secure()` (LAB-513, CWE-311); see [Intent-preset semantics](#intent-preset-semantics-parity-not-presence) | — |
| Hardware acceleration detection | ✅ surfaced (`hardware_acceleration_enabled()`) | ⚠️ core-internal, not re-exported⁶ | ❌ not exposed⁶ | N/A |
| Counter-based nonces | ✅ via Rust | ✅ | ✅ via NAPI (Rust) | ❌ use random |

> [!IMPORTANT]
> AAD v0x03 is required for protocol compliance. SDKs without it cannot safely interoperate with encrypted payloads from compliant SDKs — the auth tag will fail verification. Python, Rust, and TypeScript construction was code-verified byte-identical on 2026-07-21 (LAB-272); the normative byte layout and the frozen `True`/`False` compressed tokens (protocol#12) are defined in [spec/encryption.md](spec/encryption.md#additional-authenticated-data-aad). Python's auto serializers append the optional `original_type` fifth component; Rust, TypeScript, and interop mode always emit exactly four. When encryption is enabled but a stored entry claims plaintext, Python never returns it — the entry is converted to a miss and evicted rather than raising (LAB-241, [cachekit-py#215](https://github.com/cachekit-io/cachekit-py/pull/215)); see note ⁵ for what that means if you are monitoring for hard errors.
> AAD v0x03 is required for protocol compliance. SDKs without it cannot safely interoperate with encrypted payloads from compliant SDKs — the auth tag will fail verification. Python, Rust, and TypeScript construction was code-verified byte-identical on 2026-07-21 (LAB-272); the normative byte layout and the frozen `True`/`False` compressed tokens (protocol#12) are defined in [spec/encryption.md](spec/encryption.md#additional-authenticated-data-aad). Python's auto serializers append the optional `original_type` fifth component; Rust, TypeScript, and interop mode always emit exactly four. When encryption is enabled but a stored entry claims plaintext, Python never returns it — the entry is converted to a miss and evicted rather than raising (LAB-241, [cachekit-py#215](https://github.com/cachekit-io/cachekit-py/pull/215)).
<!-- -->
> [!WARNING]
> ⁵ **Key rotation ships in no SDK. Rotating a master key today invalidates every encrypted entry.** Corrected 2026-08-04 (LAB-1400) from a previous Python ✅ / Rust ✅ that no code supported. `cachekit-core`'s `ZeroKnowledgeEncryptor::rotate_key()` returns `EncryptionError::NotImplemented` (`src/encryption/core.rs:492`); cachekit-rs neither re-exports nor calls any rotation API; cachekit-ts classifies the core's nonce-exhaustion message into a `NonceExhaustedError` that **no caller can observe**, because it is raised inside the `set` callback and absorbed there (note ⁸). Note that nonce exhaustion does *not* make rotation mandatory — the core fails closed and a fresh encryptor instance clears it; reaching for a master-key rotation instead would destroy every existing entry, which is exactly the trap this note exists to prevent.
>
> **Do not build on cachekit-py's importable `KeyRotationState`.** cachekit-py ships a live PyO3 binding — `#[pyclass(name = "KeyRotationState")]` with `start_rotation()` / `complete_rotation()` / `is_rotating()` (`rust/src/python_bindings.rs:258`, `:283`, registered on `_rust_serializer` at `:394`) — so `from cachekit._rust_serializer import KeyRotationState` works today, `start_rotation(new_key)` **returns success**, and `is_rotating()` then reports `True`. Nothing in `src/cachekit/` consults any of it: no dual-key read path exists, so an operator who takes that success as a grace window and swaps `CACHEKIT_MASTER_KEY` renders every encrypted entry undecryptable. Removal is tracked under LAB-516. `cachekit-core` likewise exports `RotationAwareHeader` (`src/encryption/mod.rs:20`, aliased as the "canonical" `EncryptionHeader`) which no SDK ever writes to the wire.
> ⁵ **Keyring rotation ships in Python, Rust, and TypeScript.** Each SDK accepts one current encryption key plus up to three decrypt-only previous keys, rejects a configuration that repeats the current key in that list, writes only with the current key, and keeps reading old-key entries until the operator removes that key — retirement is manual, not TTL-driven. Python selects a keyring entry by the stored fingerprint of its HKDF-derived per-tenant encryption key; Rust and TypeScript attempt the current key followed by decrypt-only keys with identical AAD. The protocol reference verifier ([`tools/encryption-verify.py`](tools/encryption-verify.py)) checks the [keyring vectors](test-vectors/encryption.json): both entries decrypt under `[k2, k1]` at the declared entry, the k1 entry is rejected under `[k2]`, and each stored fingerprint selects the HKDF-derived (not master) key; no SDK consumes these vectors yet. Use the [key rotation runbook](https://docs.cachekit.io/concepts/key-rotation/) (normative source: [decisions/key-rotation.md § Runbooks](decisions/key-rotation.md#runbooks-normative-for-docs)) for the required three-phase rollout and compromise response.
>
> ¹⁸ **Python's encrypted read path is fail-OPEN by default.** `EncryptionWrapper(fail_closed=False)` (`serializers/encryption_wrapper.py:113`) from `encryption_fail_closed: bool = Field(default=False)` (`config/settings.py:225`): on a key-fingerprint mismatch the default is a warning plus a recompute, not a raise — that flag gates the fingerprint pre-check — and the same setting drives the AES-GCM authentication-failure policy through `handle_decrypt_failure` (`cache_handler.py:1307`, `:1317`; resolved at `:550-552`, L1 path `decorators/wrapper.py:1210`). Both default to recompute. Set `CACHEKIT_ENCRYPTION_FAIL_CLOSED=true` if you are relying on hard errors as your wrong-key or tamper alarm — otherwise the signal is a log line and a billable miss. (Separately, a stored entry that claims plaintext while encryption is enabled is never returned to the caller: it is converted to a miss and evicted, deliberately, for plaintext→encrypted migration — LAB-241, [cachekit-py#215](https://github.com/cachekit-io/cachekit-py/pull/215).)
>
Expand All @@ -88,7 +86,7 @@
>
> - **Read** — a tampered payload or wrong key makes `cache.get()` return `null`, an ordinary miss. Through `wrap()` the function is re-executed and the result re-stored.
> - **Write** — an encrypt, NAPI or key failure makes `set()` **resolve like success while storing nothing**; every later `wrap()` re-executes the origin, forever, silently.
> - **Nonce exhaustion** — `NonceExhaustedError` is raised *inside* `encrypt()` (`encryption/manager-core.ts`), i.e. inside the `set` callback, so it is absorbed too. **Nonce reuse is not the risk:** `cachekit-core` fails closed at 2³² operations per encryptor instance — `generate_nonce()` returns `NonceCounterExhausted` once `counter >= u32::MAX`, and the counter is an `AtomicU64` specifically so it *stays* exhausted rather than wrapping (`encryption/core.rs:168-185`, `:297-311`). The unobservable consequence is instead that **every** encrypted `set()` on that encryptor silently stores nothing for the remaining life of the process. An operator seeing only a rising miss rate has no way to reach the real remedy — a fresh encryptor instance — and per note ⁵ the instinct to rotate the master key would destroy every existing entry.
> - **Nonce exhaustion** — `NonceExhaustedError` is raised *inside* `encrypt()` (`encryption/manager-core.ts`), i.e. inside the `set` callback, so it is absorbed too. **Nonce reuse is not the risk:** `cachekit-core` fails closed at 2³² operations per encryptor instance — `generate_nonce()` returns `NonceCounterExhausted` once `counter >= u32::MAX`, and the counter is an `AtomicU64` specifically so it *stays* exhausted rather than wrapping (`encryption/core.rs:168-185`, `:297-311`). The unobservable consequence is instead that **every** encrypted `set()` on that encryptor silently stores nothing for the remaining life of the process. An operator seeing only a rising miss rate has no way to reach the real remedy — a fresh encryptor instance — follow the [key rotation runbook](https://docs.cachekit.io/concepts/key-rotation/) for the safe scheduled procedure.
>
> Degradation is on unless explicitly disabled (`degradationEnabled = config.degradation !== false`, `reliability/executor.ts:39`) and `createCache.secure()` / `.production()` / `.io()` all set it `true` (`intents-core.ts:186`); only `minimal` sets `false`, and `minimal` carries no encryption. **There is no `failClosed` option anywhere in cachekit-ts** — Python's `CACHEKIT_ENCRYPTION_FAIL_CLOSED` has no counterpart, so the only lever is `reliability: { degradation: false }`, which also gives up backend-outage degradation. If you rely on a thrown error as your tamper, wrong-key or nonce alarm, TypeScript raises none; the sole signal is the `errors_total` counter.
>
Expand Down
18 changes: 11 additions & 7 deletions spec/encryption.md
Original file line number Diff line number Diff line change
Expand Up @@ -342,13 +342,17 @@ by `tools/encryption-verify.py`.

## Key Rotation (Keyring)

> [!WARNING]
> **Status: specified, not yet implemented** (decision record:
> [decisions/key-rotation.md](../decisions/key-rotation.md), 2026-07-23;
> implementation tracked under LAB-516). Until an SDK ships this section, rotating
> its master key invalidates every encrypted entry — fail-open readers take misses,
> fail-closed readers take errors. The [feature matrix](../sdk-feature-matrix.md)
> reflects per-SDK implementation status.
> [!NOTE]
> **Status: shipped in Python, Rust, and TypeScript** (decision record:
> [decisions/key-rotation.md](../decisions/key-rotation.md), 2026-07-23; LAB-516).
> The shared decrypt helper is `cachekit_core::encryption::keyring::Keyring`
> ([cachekit-core#67](https://github.com/cachekit-io/cachekit-core/pull/67)); the SDK
> surfaces are [cachekit-py#261](https://github.com/cachekit-io/cachekit-py/pull/261),
> [cachekit-rs#63](https://github.com/cachekit-io/cachekit-rs/pull/63) and
> [cachekit-ts#103](https://github.com/cachekit-io/cachekit-ts/pull/103). An SDK that
> does not implement this section invalidates every encrypted entry when its master
> key rotates — fail-open readers take misses, fail-closed readers take errors. The
> [feature matrix](../sdk-feature-matrix.md) reflects per-SDK implementation status.
>
> Earlier revisions of this spec described a 32-byte `RotationAwareHeader` prepended
> to ciphertext. **That header was never written by any SDK and is not part of the
Expand Down
Loading
Loading