[FEAT] capsule-core::media on rawshift-image: still decode, the LQIP producer, and typed unsupported formats - #436
Open
justin13888 wants to merge 17 commits into
Conversation
`capsule-core::media` becomes the Capsule-side owner of still detection,
decode, orientation, metadata normalisation and derivative generation,
over `rawshift-image` 0.1.1 from crates.io (a registry dependency, not
the pinned submodule) behind a new `media` feature that `native` implies
and the wasm32 sealing build excludes.
Rawshift owns the codecs; this module owns every decision Capsule has to
make around them:
- the closed sets — `StillFormat` (what counts as a still) and
`DerivativeFormat` (what a signed `DerivativeManifest.format` may
say, with the `original` sentinel);
- detection, because the crate's own `detect_standard_format` gates its
HEIC arm on the HEIC codec, so delegating would make the typed refusal
for a format depend on whether it can be decoded;
- a pre-decode pixel budget and an unwind boundary, because a pre-1.0
decoder is fed untrusted bytes on the import path;
- tier sizing and a deterministic integer area-average downscale, since
the crate has no resize and a derivative's bytes are signed;
- the metadata strip: every encode passes `MetadataEmbedOptions::none()`
because the crate's default embeds EXIF, GPS included.
Decode covers JPEG, PNG, JXL, TIFF, GIF and WebP; encode covers WebP,
which produces the 256 px q=50 thumbnail tier. HEIC, AVIF, RAW and a
lossy JXL encoder each need a system library or an assembler the cross
and cargo-ndk builds do not carry, so each is a typed
`MediaError::UnsupportedFormat` or a recorded per-format deferral rather
than a silent gap.
`DerivativeCore.format` keeps its `String` type: the same field carries
the `embedding/{model_id}` grammar, and a typed field would turn an
unrecognised value into a parse failure before any signature is
examined. The closed set is enforced at production and at verification
instead.
`lifecycle/import.rs` hard-coded `(exif dimensions, None, DeferredNoCodec)` for every still, so `capsule_core::lqip` — fully tested since `S-B14` — had no production caller and `DerivativeStatus` had one reachable value. `Workspace::prepare_still` replaces the constant triple with one decode pass that yields the header-derived `content_type`, pixel `dimensions`, the chromahash `lqip`, and the signed thumbnail derivatives; `persist_derivatives` writes them under `derivatives/` at the layout the upload-bundle reader already looks for. Both run inside the existing signed write path, so nothing about the sealing order moves. Pixel dimensions win over EXIF because they are post-orientation: a quarter-turned JPEG's `PixelXDimension` is its *stored* width, which is transposed relative to what a viewer shows. Derivatives are persisted **after** the asset's own files are durable, and a write failure is logged rather than returned: a derivative is regenerable and must never fail an import whose signed original is already committed. Nothing here can fail an import over unreadable pixels — every path degrades to "signed, encrypted, verifiable original, without a placeholder" and records which reason applied. `ImportOutcome::Imported` gains `deferred_formats`, summarised by `ImportExecutionSummary::deferred_format_count()`. It counts *format variants* missing from assets that do have a thumbnail, where `deferred_derivative_count()` counts *assets* with none — a decoded JPEG reports two (the JXL master, the AVIF delivery variant), which is the number that falls to zero as the encoders land. The `S-B13` distinction the executor test lost to `S-C59` is observable again, and now rests on the bytes rather than the extension: a HEIC is `DeferredNoCodec` (recognised, no codec here, backfillable) while a `.jpg` that is not a JPEG is `DecodeFailed` (a format we do decode, failing on these bytes). Both still land as signed, self-verifying backups.
Deploying capsule with
|
| Latest commit: |
42d21ee
|
| Status: | ✅ Deploy successful! |
| Preview URL: | https://cd5f92e1.capsule-22k.pages.dev |
| Branch Preview URL: | https://feat-media-rawshift-still-de.capsule-22k.pages.dev |
`capsule_core::lqip` compiled identically on all three surfaces and was reachable from one: the import pipeline now encodes a placeholder, so the readers need an entry point or the module's whole reason for living at the crate root goes unexercised. - `capsule-wasm`: `decodeLqip` returns `WasmLqipImage` — packed RGBA the share viewer hands to `putImageData`, band-limited to the box being painted rather than decoded at a fixed size. The whole of the logic lives in a pure helper and the boundary is a `map`/`ok_or_else`, because `JsError` cannot be constructed off-wasm: a host test reaching the error arm through the exported function aborts the test binary instead of failing an assertion. - `capsule-core-ffi`: `render_lqip` → `LqipPlaceholder`. A free function rather than a `Catalog` method, deliberately: the `assets` table's `chromahash`/`dominant_color` columns are NULL and must stay so until `library::rebuild` projects them identically, or a rebuilt index would disagree with a freshly written one. So it takes the record the caller already holds from the decrypted sidecar rather than pretending the index has it. Both are infallible over a malformed record — an unknown version or a payload the parser rejects paints the `dominant_color` fill — because a reader must never misrender a placeholder and a gallery must never fail to draw a cell over one. The wasm boundary throws only on a `dominant_color` that is not three bytes, where there is no colour to fall back to; the FFI paints black, the conventional empty cell. Both are asserted byte-identical to `Lqip::decode_capped`, which is the `S-B14` cross-surface criterion at the two boundaries where a second implementation could have crept in.
`SLICES.md` had S-B1, S-B5 and S-B13 as `RETIRED`/`ready` and S-B14 owing a wasm entry point. Three of the four moved: - **S-B1** — re-landed on `rawshift-image`; the injected `StillEncoder` seam is gone, because it existed only to work around core linking no codec. `done*`, owing the JXL master, the AVIF delivery variant, the preview tier and HEIC/RAW decode to #437, each blocked on a system library or an assembler rather than on a design question. - **S-B5** — `ACTIVE` and still unimplemented: `rawshift-video` is unpublished and the transcode toolchain shares nothing with the still path. Owed to #438, with the licensing gate named up front. - **S-B13** — `done`. There are no stubs to make uninhabited any more: the coverage table is a gate checked before any decoder runs, and the two-reason distinction is observable again — and now rests on the bytes rather than the extension. - **S-B14** — the owed wasm entry point exists, and so does the FFI one. `thumbnails.md` gains an implementation-status note under the tier table. The table stays the contract; the note says what is generated today, names the toolchain blocking each missing cell, and records that the distance between the two is a number the import run reports rather than something a reader has to infer. The "Where LQIP Lives" rationale is restated on the ground that outlived the teardown: `media` is `native`-only wherever it exists, so a placeholder every client needs cannot live inside it and still reach the browser.
Two products in `downscale_rgba8` were computed at widths that a reachable input overflows, both found by re-reading the diff rather than by a failing test: - the destination-to-source boundary `(y + 1) * src_h` reaches `dst_edge * src_edge`. A 1 x 300000 frame reduced to a 256 px long edge makes that 7.7e10, past a 32-bit `usize` — and `armv7-linux-androideabi` and `i686-linux-android` are both CI-gated targets; - the per-channel accumulator was `u32` and reaches `count * 255`, where `count` is the whole frame when the function is called with a cap of 1. `downscale_rgba8` is a `pub` entry point, so that cap is reachable even though the tier table only ever passes 256. A debug build panics on either; a release build wraps into wrong pixels or an out-of-bounds index — inside a derivative whose bytes are signed. Both are now `u64`, with a test at each shape. Also merges the identical `match` arms clippy's `match_same_arms` flagged (`standard_format`'s container mapping, `gamut_of`'s sRGB default) and drops two other lint-level nits. The merged arms lose nothing: the RAW families map to the container `rawshift-image` actually sees, which is the same TIFF for all of them, and one wildcard is honester than an explicit list beside a catch-all with the same body.
Two repairs found after the first push, in the same files.
**The thumbnail tier moves from WebP to JXL, on CI evidence.** WebP was
chosen because `image/webp` is in the tier table and `libwebp` exposes
exactly the q=50 knob the table specifies. It does not compile:
`rawshift-image-0.1.1/src/codecs/webp.rs:164,177,190` pass
`b"EXIF".as_ptr() as *const i8` to `WebPMuxSetChunk`, whose `libwebp-sys`
0.14.4 signature (`ffi.rs:881`) takes `*const core::ffi::c_char` — and
`c_char` is `u8` on aarch64, so it is an E0308 on every 64-bit ARM
target, which is every mobile target Capsule ships. `codecs/mod.rs:13`
compiles that module under `any(webp-decode, webp-encode)`, so
decode-only does not escape it either.
The `webp` feature is therefore dropped and the tier encodes JXL through
the pure-Rust `zune-jpegxl` backend — `image/jxl` is the table's
committed *master* format, so the format that ships first is the one the
table already puts first. The cost is that `JxlSimpleEncoder` is
lossless, so the declared q=50 is advisory and a thumbnail costs more
bytes than intended; a test asserts the losslessness rather than letting
it be discovered. A `cfg(target_arch)` gate was rejected: thumbnails on
desktop and none on any phone is worse than one lossless format
everywhere. `StillFormat::WebP` becomes recognised-but-undecodable, which
is a real user-visible gap for a common export format, so it is filed
rather than absorbed.
**The hardening**, from an adversarial read of the diff:
- the `original` sentinel copied the whole original into
`derivatives/{uuid}.thumbnail.{ext}`, putting the source's EXIF and GPS
into a derivative blob and duplicating a file two directories up. The
contract's word is *references*: a sentinel now carries no bytes and
its manifest content-addresses the original;
- a derivative-generation failure propagated and failed the whole import,
trading a missing thumbnail for a missing backup. It is warned and
reported as `DecodeFailed` instead;
- the unwind boundary covered only `Decoder::decode` while the module
claimed no codec could abort an import; `media::guarded` now wraps the
chromahash placeholder and the encode too;
- `capped_dimensions` divided by zero on a zero dimension, reachable
through a `pub` entry point;
- `MediaMetadata::gamut` claimed to carry the source colour space.
`probe_standard_image` hard-codes `Srgb` for every format, so it never
does — documented as the fidelity limitation it is, with `gamut_of`
kept as the seam;
- `MAX_DECODE_PIXELS`' note counted one buffer at a time and understated
the peak 3-4x. The real peak is ~2.5 GB, and `native` implies `media`,
so it lands on a phone: the budget drops to 128 Mpx, still ~25% above a
102 Mpx medium-format frame;
- the HEIC-detection rationale overstated the crate's blind spot, and
`encode`'s unreachable arm returned an error naming a `StillFormat`
that was not at fault.
Three intra-doc links from public items to private ones are also dropped,
so the rustdoc gate passes under `--document-private-items`.
The dependency row, the tier-table status note, `S-B1` and the `AGENTS.md` sentence all named WebP as the format that ships. They now name JXL, and each says why WebP is absent — it is a compile failure on every aarch64 target, not a preference, so the reason belongs beside the choice rather than only in the issue tracker (#444). The status note gains the honest asterisk on the tier table: the pure-Rust JXL backend is lossless, so the declared q=50 is advisory and a thumbnail costs more bytes than the table intends. That is the one place this build knowingly departs from the contract, and the note says so rather than leaving a reader to infer it from a byte count. Decode coverage narrows with the feature: WebP is recognised and refused alongside HEIC, AVIF and the RAW families, because the crate compiles the broken module for decode as well as encode.
…cause The barrel's own module doc explained the fully-qualified `crate::media::…` links by asserting that a module's documentation is resolved before its `pub use` items are in scope. That is a guess at rustdoc's resolution rules, not something this lane verified, and it read as fact. What was actually observed is the asymmetry: the bare names fail under the gate (`cargo doc --no-deps`) and resolve under `--document-private-items`, which is why the failure surfaced only in CI. The comment now says that, and says the qualified path is used because it holds either way.
Derivative blobs were pushed in the clear. `capsule-sdk::push` shipped `DerivativeBlob::bytes` verbatim while the original went as ciphertext, so a field named `ciphertext_hash` addressed plaintext and a thumbnail — a recognisable low-resolution copy of a private photo — reached the server readable. Encryption's opening clause admits no exception: "every asset — original bytes, derivative bytes, metadata blob — is encrypted client-side", and the upload protocol adds "each encrypted independently". `DerivativeCore` gains a **required** `nonce_prefix`, the same type `ManifestCore` carries. Required rather than `Option` because a receiver that cannot recover it cannot open the blob at all — an absent prefix would be an unopenable derivative, not a tolerable gap — and it is safe to require because no real `derivative-manifest/v1` has ever been written: derivatives were unconditionally `DeferredNoCodec` until the decoder landed, and `crate::ml` constructs none. Nothing to stay compatible with, so the schema string does not move. Generation encrypts each derivative with the same construction the original uses — `encrypt_asset_rekey` under the source asset's `file_id` and the album's AMK, a fresh CSPRNG prefix per derivative — and signs the **ciphertext's** address. The ciphertext is discarded: the client keeps the plaintext derivative locally, because that is what the local gallery paints, and `derivative_blobs` re-derives the ciphertext at push time from the recorded prefix, exactly as `upload_bundle` already does for the original. That ordering forced one change in `import_asset_with`: the original is encrypted *before* derivatives are generated, because the `original` sentinel is a signed reference to that blob and commits to its address and prefix, neither of which existed yet. `media` gains a narrow `DerivativeSealer` seam rather than the AMK: the codec module still names no key material, and `lifecycle` still names no codec. Two further skips at `derivative_blobs`, both previously missing: `verify_still_format` now runs there, so a still-role manifest naming a format outside the closed set is the structural rejection the tier table specifies; and the byte-free `original` sentinel is recognised as an expected reference and skipped at `debug!`, since an expected absence logged as a warning is how people learn to ignore warnings. The warning stays for a non-sentinel manifest whose bytes have gone. **Also repairs two claims the previous commit made and did not deliver.** Its message said the unwind boundary had been widened to the placeholder and the encode, and that a generation failure was reported rather than propagated. Neither edit actually applied — a silent find-and-replace miss — and no test covered either path, so both went unnoticed. They are applied here, and `guarded` is no longer an unused import.
`prepare_still` reached nine parameters when the AMK and the original's committed pair joined it, and clippy's `too_many_arguments` is right about what that means here: the signature had grown two *kinds* of input — the file being imported, and the crypto identity it commits under — without saying so. The four file facts (`plaintext`, `ext`, `src`, `exif`) become `StillSource`. They are one thing, always passed together, and naming them makes the remaining parameters read as the identity half. Silencing the lint would have kept the signature and hidden the reason it grew.
…en the guards Review round 1 findings F1-F16. The two that mattered: **F1 (critical).** `prepare_still` propagated any derivative-generation failure as `LifecycleError::Io`, and it did so *before* the asset's files were written — so an encoder refusing a frame lost the original from the backup entirely. That contradicted this module's own header, `S-B13`, and the decision recorded for it. `MediaError` gains a `Sign` variant so a workspace fault (a hardware signer refusing, a missing epoch key) is distinguishable at the type level from a codec refusing pixels. Only the former propagates; every codec, resize and encode failure degrades to `DerivativeStatus::DecodeFailed` with the real dimensions and placeholder kept, and the import commits. **F2 (high).** The unwind boundary guarded only the decode, while the chromahash placeholder and the JXL encode — both pre-1.0, both running on the same untrusted pixels — ran bare, so one panicking frame could abort a twenty-thousand-photo import part way through. Every stage that runs foreign code over pixels is now guarded, with the stage named so a caught unwind is attributable. `guarded` is `pub(crate)`: `lifecycle` is its only caller and no client of this crate has pixels of its own. The rest: - **F9** `DerivativeFormat` and `verify_still_format` move to an unconditional crate-root module. They were behind the `media` feature, which `native` implies — so `capsule-server` and `capsule-wasm`, the two crates that *receive* a manifest they did not author, could not link the check at all. A closed set only its producer can evaluate is not a closed set. `media` re-exports both names. - **F5** derivative bytes are addressed by `(role, format)`, not by a role prefix that took whichever filename sorted first — which would have silently skipped both variants the moment AVIF lands beside JXL. - **F14** a role's chain continues across generation runs instead of restarting per invocation, so a backfill extends the record rather than forking it. - **F6** the 32-bit overflow test now genuinely crosses `u32::MAX` (its arithmetic was off by 1000x), and the boundary-product claim is restated as the defensive measure it actually is. - **F13** the HEIC executor fixture carries a real `ftyp` header, so the test exercises the byte sniffing its own docs claim rather than the extension fallback. - **F11** `decodeLqip` no longer throws on a malformed record: it paints the same fallback fill the native FFI paints. One record answered two ways by two clients is the divergence `capsule-core::lqip` exists to prevent. - **F12** JPEG/PNG **encoders** move to `[dev-dependencies]`; only the fixtures used them, and shipping them put `jpeg-encoder`'s conjunctive IJG arm into every release binary. `cargo tree -e normal -i jpeg-encoder` is now empty while `cargo deny --all-features` still sees it, so the exception stays matched. - **F7, F8, F10** stale docs: the budget is 128 Mpx in `SLICES.md`, and the `libwebp`/"vendored C" claims left over from before the JXL swap are corrected.
… media feature The module moved out of `media` so the receivers can link it, and its doc comments moved with it — still pointing at `StillFormat`, `MediaError` and `GeneratedDerivative`, none of which exist in a `--no-default-features` build. Rustdoc caught it as four unresolved intra-doc links. They become prose naming the `media::` path instead of links to it. A module that exists precisely so a feature-gated stack is not a prerequisite must not re-acquire that prerequisite through its documentation.
`FALLBACK_FILL` is a private constant, and `decode_lqip`'s doc linked it — which resolves only under `--document-private-items` and fails the rustdoc gate as written. The sentence names the colour instead, which is what a reader of the public API actually needs to know.
…ature `pub(crate) use self::decode::guarded` was unconditional, but `lifecycle` is the only caller and `lifecycle` is `native`-gated. A `--features media` build without `native` — which the aarch64 cross-check uses, to isolate the codecs from SQLite's C build — carried it as an unused import. Found by that cross-check rather than by `check-rust`, whose clippy pass runs the default feature set where `native` is on. A feature combination no gate compiles is a feature combination that rots.
`de756e90` rewrote the `read_derivative_bytes` doc comment by replacing the tail of the file from that comment onward, and the replacement did not carry the last two lines with it. `#[cfg(test)] mod tests;` was deleted, so `lifecycle/upload/tests.rs` stayed tracked, stayed green in review, and stopped being compiled at all. Thirteen tests went dark. Nine were the ones that prove this PR's central claims — the decision-18 KAT that a derivative ships as ciphertext and decrypts back to the bytes on disk, the tampered-derivative skip, the sentinel contributing no blob, both arms of the closed-format check, the missing-bytes skip, the pushed-thumbnail-differs assertion, the survives-a-reopen case (F3) and the two-formats-by-format case (F5). The last two were added in the same commit that deleted the declaration, so they had never been compiled even once. Four more were pre-existing S-D18 coverage that had passed at `4f8b8bda`. All thirteen pass unmodified against the current `derivative_blobs(&self, asset, album, epoch)` signature, so the tests were right and only their declaration was missing. This is the second time in this branch that a whole-region replacement silently dropped code — the same failure class recorded for `fe1e3c97`. The difference is that a lost `mod` declaration cannot be caught by reading the diff of the file it belongs to: it presents as a passing suite. `cargo nextest list` is the check that sees it, and its census for this module now goes in the pull request rather than a summary line.
Review round 2, findings M2-M4 and L5-L10.
**M4 — the reuse refusal now exists.** The encryption doc is normative:
"the writer additionally refuses to emit a `nonce_prefix` it has already
used for that `file_id` … the same rule governs derivative
re-encryption". The sealer passed `replaces: None`, so nothing was ever
refused and the sentence was false for every derivative. It now carries
the set of prefixes already spent on this `file_id` — the original's, plus
every prefix in the existing bundle, which the bundle reader already had
to open — redraws a collision, and adds each sealed prefix before the
next seal. An exhausted draw is `MediaError::Sign`: a 1-in-2^56 collision
eight times running is a broken CSPRNG, which is a workspace fault, and
decision 22 propagates those rather than writing one derivative fewer. A
prefix is folded into the file-key salt, so reusing one reuses the *key*
— two blobs under one keystream, which is what the construction exists to
prevent.
**M2 — an unheld epoch no longer panics the bundle.** `file_key` indexes
`album.amks[&epoch]` with an epoch read off an unverified `.cbor`, inside
a function contracted never to fail. It needs no tampering to reach: an
album recovered from a backup holds only the epochs it escrowed. It is
now the fifth skip reason, and the rustdoc enumerates five.
**M3 — embedding-role manifests are named out of scope.** `F5` keyed the
reader by `(role, format)`, and `embedding/{model_id}` parses to no still
format, so every embedding manifest fell through to "no bytes on disk" —
a misleading warning for an artefact with no writer, since `crate::ml`
produces none. They are skipped at `debug!` and the doc says why.
**L9/L10 — the two failure paths are tested through the real import.**
The previous F2 test called `guarded` itself, so deleting every
production call site left it green. Both now drive a fault through
`Workspace::import_asset_with` via a `#[cfg(test)]` hook inside the
sealer — absent from a release build, not merely disabled — and assert
what actually matters: `DecodeFailed` reported, and the original
committed, signed and self-verifying, with real dimensions and a real
placeholder. Both were mutation-checked: reverting the match arm to `?`
fails the first; removing the guard aborts the second.
L5-L8 are doc corrections: the closed set is named at
`crate::derivative_format` and described as linkable without `media`; the
two `# Errors` blocks route sealing to `Sign`; the `{uuid}.{role}.`
prefix-scan description is replaced by the exact-path composition that
superseded it; and two WebP leftovers in the tests.
Both `# Errors` blocks in `media::derivative` still routed signing and sealing failures to `MediaError::Encode`. That has been untrue since the `Sign` variant was introduced: `sign_derivative` returns `Sign` for a signer refusal and for a manifest that will not serialise, and the only `DerivativeSealer` implementation returns `Sign` both when the encryption refuses and when it cannot draw an unused nonce prefix inside its retry budget. The distinction is the contract, not bookkeeping, which is why a stale doc here is worth a commit of its own: the import path **degrades** an `Encode` or `ZeroDimension` to "this asset has no thumbnail" and commits the original anyway, and **propagates** `Sign`, because a workspace that cannot author a signed record is broken in a way a missing derivative is not. A reader following the old text would have concluded the two were interchangeable. The trait block also drops its reference to "a drawn prefix that collides with the one being replaced": `replaces` is always `None` for a derivative, which supersedes nothing. Non-reuse is enforced against the set of prefixes already spent on that `file_id` instead. Documentation only; no behaviour change. Recorded because it is the third instance on this branch: these two edits were claimed in `5a486852`'s message and never landed. A batch script computed several replacements against one file and wrote once at the end, an `assert` on a later pattern aborted it, and every earlier in-memory edit to that file was discarded while an earlier *file*'s write had already succeeded — so the per-edit progress output looked like success. Each edit here was written and read back separately.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Description
capsule-core::mediabecomes the Capsule-side owner of still detection, decode, orientation, metadata normalisation and derivative generation, overrawshift-image0.1.1 from crates.io behind a newmediafeature thatnativeimplies and thewasm32-unknown-unknownsealing build excludes. The fully testedcapsule-core::lqipmodule gains its first production caller, andDerivativeStatusgets more than one reachable value for the first time sinceS-C59.Summary
The module (
capsule-core/src/media/, private submodules behind one barrel). Rawshift owns codecs; this owns every decision Capsule has to make around them:detect—StillFormat, the closed set of stills Capsule models (JPEG, PNG, WebP, JXL, TIFF, GIF, Netpbm, AVIF, HEIC and the six RAW families), with a Capsule-owned magic-byte table, an extension fallback used in exactly two places a header cannot settle, andis_decodable()as the single codec-coverage predicate.decode— theDecoderseam: sniff, gate, budget, decode, apply the EXIF orientation, normalise to packed opaque RGBA8.guardedputs an unwind boundary around every step that runs foreign code over pixels.resize— a deterministic integer area-average downscale, because the crate has no resize and a derivative's bytes are signed.derivative—DerivativeFormat(with theoriginalsentinel),DerivativeTier,generate_still_derivatives,verify_still_format, and the two-signature manifest chain.error—MediaError, includingUnsupportedFormat { format, op }.Import wiring.
Workspace::prepare_stilldecodes once per still and yields the header-derivedcontent_type, pixeldimensions(post-orientation, so a quarter-turned JPEG's pair is transposed relative to its EXIF tags), the chromahashlqip, and the signed thumbnail derivatives.persist_derivativeswrites them at the layout the upload-bundle reader already looks for, after the asset's own files are durable, logging rather than returning on failure — a derivative is regenerable and must never fail an import whose signed original is already committed.Per-format deferral.
ImportOutcome::Importedgainsdeferred_formats, summarised byImportExecutionSummary::deferred_format_count(). It counts format variants missing from assets that do have a thumbnail, wheredeferred_derivative_count()counts assets with none. A decoded JPEG reports two — the JXL master and the AVIF delivery variant — which is the number that falls to zero as #437 lands.The
S-B13distinction is observable again, and now rests on the bytes rather than the extension: a HEIC isDeferredNoCodec(recognised, no codec here, backfillable) while a.jpgthat is not a JPEG isDecodeFailed.Exports.
capsule-wasm'sdecodeLqipandcapsule-core-ffi'srender_lqip, both asserted byte-identical toLqip::decode_capped— theS-B14cross-surface criterion at the two boundaries where a second implementation could have crept in.Why JXL is the format that ships first — and why it is not WebP
WebP was the first choice, for good reasons:
image/webpis an admitted value in the tier table,LibwebpEncodeConfigexposes exactly the q=50 knob the table specifies, andlibwebp-sysis MIT with pre-generated bindings and a vendored static build.CI refuted it.
rawshift-image-0.1.1/src/codecs/webp.rs:164,177,190passb"EXIF".as_ptr() as *const i8toWebPMuxSetChunk, whoselibwebp-sys0.14.4 signature (ffi.rs:881) takes*const core::ffi::c_char.c_charisu8on aarch64, so the cast is an E0308 on every 64-bit ARM target — which is every mobile target Capsule ships — and the Apple leg additionally fails at link time.codecs/mod.rs:13compiles that module underany(webp-decode, webp-encode), so decode-only does not escape it. The run may not patch, vendor, or fork a dependency.So the tier encodes JXL through the pure-Rust
zune-jpegxlbackend.image/jxlis the table's committed master format, so the format that ships first is the one the table already puts first, and every enabled codec is now pure Rust — no C links at all, which is what keeps the mobile cross-builds clean. The cost is thatJxlSimpleEncoderis lossless, so the declared q=50 is advisory and a thumbnail costs more bytes than the table intends;the_jxl_thumbnail_is_lossless_todayasserts that rather than leaving it to be discovered.Rejected: gating WebP behind
cfg(target_arch)— thumbnails on desktop and none on any phone is worse than one lossless format everywhere. Rejected: JPEG, which is not in the format table.The consequence beyond the encoder is that
StillFormat::WebPis recognised and not decodable, because the broken module is shared by both directions. WebP is a common export format, so that is a real user-visible gap and is filed (#444) rather than absorbed. A lossy JXL still needs C libjxl and AVIF still needsnasm(#437).Privacy: three separate holes, all closed
rawshift-core'sMetadataEmbedOptions::default()isall(), so a default-configured encode copies the source's EXIF — GPS included — into the thumbnail. Every Capsule encode passesMetadataEmbedOptions::none()and hands the encoderImageMetadata::default(), so the source's metadata is never read at all. A test demonstrates the leak using the crate's own default and then asserts Capsule's derivative carries noEXIF/XMP/ICCPchunk and none of the source's GPS rationals.capsule-sdk::pushshippedDerivativeBlob::bytesverbatim while the original went as ciphertext, so a field namedciphertext_hashaddressed plaintext. Derivatives are now encrypted with the same construction the original uses — a fresh per-derivative nonce prefix, signed into the manifest — and re-derived at push from the plaintext the library keeps locally. The read half is filed (media: decrypt fetched derivative blobs — capsule_sdk::fetch::open_representation returns ciphertext with no key path #448).originalsentinel (found by an adversarial read of this diff, after the first push). The retired implementation copied the whole original intoderivatives/{uuid}.thumbnail.{ext}whenever the source was inside the 256 px cap — putting the original's EXIF and GPS into a derivative blob, which is the one place a re-encoded thumbnail is metadata-free by construction. The contract's word is references, so a sentinel now carries no bytes: its manifest content-addresses the original the holder already has, and nothing is written beside it. Three tests pin the absence.Validation
Run inside the worktree at
/var/mnt/scratch/golem/dev/Capsulsaurus/Capsule.worktrees/Capsule-feat-media-rawshift-still-decode-410.cargo test -p capsule-wasmcargo test -p capsule-core-fficargo test -p capsule-cli --test import_round_tripmise run test-rustTEST_RUST=0at5a486852—1764 tests run: 1764 passed, 0 skipped, then785 passed, then160 passed: 2709 tests, 0 failures across the workspace,capsule-core --features ffiandcapsule-sdk --features ffi. The count rose by 44 from the previous round, which is the thirteen tests C1 restored plus this round's new coverage.mise run check-rustCHECK_RUST=0at5a486852. All fourteen sub-tasks ran, ending atverify-examples: format-check-rust, lint-check-rust, doc-check-rust, i18n-check, i18n-guard, openapi-check-kynos, architecture-check, license-check, translate-readme-check, build-rust, build-check-wasm, build-ffi, lint-check-ffi, gen-bindings, verify-examples.mise run license-checklicenses ok, re-run afterlibwebp-sysleft the graph.zune-jpegxl0.5.2 isMIT OR Apache-2.0 OR Zlib, already allow-listed, sodeny.tomlandNOTICEneed no change: the MPL list still namesrawshift-core/rawshift-image, andjpeg-encoder's IJG exception is still matched.mise run build-check-wasmcargo tree -p capsule-core --target wasm32-unknown-unknown --no-default-features -e normal -i rawshift-imageWASM=0, empty —warning: nothing to print.-e normalis now the honest form of this check: decision 27 put the JPEG/PNG encoders in[dev-dependencies], so the default edge set shows a dev-only path torawshift-imagethat no wasm artefact links. The shipping graph is clean.mise run build-ffimise run gen-bindingssurface check passedmise run check-docs-truth5a486852—cross-links: 473 link(s) checked, all resolve./endpoint-census: 84 citation(s) checked, all resolve./module-paths: 119 path(s) checked, all resolve.mise run check-md5a486852—Linting: 168 files/Summary: 0 issues in 0 filesmise run check-docs5a486852—59 page(s) built,Complete!(afterbun installincapsule-docs/; see below)cargo check -p capsule-core --no-default-features --features mediacargo check -p capsule-core --no-default-features --features media --target aarch64-linux-androidARM=0at5a486852— the local proof for the codec swap: the JXL-only codec set compiles for a real aarch64 target, the samec_char = u8platform WebP fails on.nativeis off so no C build script is in the graph, which isolates the codecs fromrusqlite/bundled. (The target was added withrustup target add;mise run targets-addis the repo's own sanctioned way to do that.)mise run doc-check-rustDOC_CHECK_RUST=0at42d21ee5. This is the gate that madeRust (fmt + clippy + build)red at 72e5921; see the diagnosis below.cargo nextest run -p capsule-core --features mediaNEXTEST=0at42d21ee5—780 tests run: 780 passed, 0 skipped. Re-run for42d21ee5(docs(core): say which failures are Sign and which are Encode), which corrects the two# Errorsblocks inmedia::derivativethat still routed signing and sealing failures toMediaError::Encode. Documentation only, no behaviour change.capsule importsmoke on a temp dirimported: 1, errors: 0,generated: 1, deferred: 2, and on diskderivatives/{uuid}.thumbnail.jxl(35812 bytes, magicff 0a— a real bare JXL codestream) beside a signed{uuid}.derivatives.cbor, with the sidecar carryinglqip/chromahash. This is the plan's acceptance criterion, end to end.Failures classified
Rust (fmt + clippy + build)red at 72e5921 —caused, and fixed. The cause isdoc-check-rust, not theinstall-actionnoise higher in the log:cargo doc --no-depsfailed with three unresolved intra-doc links incapsule-core/src/media/mod.rs(StillFormat,DerivativeFormat,MediaError::UnsupportedFormat) plus one private-item link inlifecycle/import.rs:315. The observable fact — stated without guessing which of rustdoc's resolution rules produces it — is an asymmetry: a bare[`StillFormat`]inmedia/mod.rsdoes not resolve undercargo doc --no-deps, and does resolve when the same command is given--document-private-items. That is exactly why it never reproduced locally: my pre-emptive runs used the private-items form, anticipating the base branch's gate change. All four links now use their fullcrate::media::…path, which holds under both, and the module records the asymmetry rather than a theory about it.mise run doc-check-rustnow exits 0.Rust cross (android)/(apple)/(linux-arm64)red at 72e5921 —caused, and fixed by the codec swap above. Verified in-tree against the unpacked crate sources rather than taken from the log:rawshift-image-0.1.1/src/codecs/webp.rs:164,177,190vslibwebp-sys-0.14.4/src/ffi.rs:881, andcodecs/mod.rs:13gating the module onany(webp-decode, webp-encode).Build & test Capsule.appred at 72e5921 —caused, same dependency, failing at link time (symbol(s) not found for architecture arm64) rather than compile time.mise run check-docsfirst failed atbuild-docswithastro: command not found— unavailable, not caused:node_moduleshad never been installed in this worktree.bun install --frozen-lockfileincapsule-docs/provisioned it (gitignored, tree still clean) and the gate then passed.cargo check -p capsule-core --target aarch64-apple-ios— unavailable on this Linux host:error occurred in cc-rs: failed to find tool "xcrun", fromrusqlite/bundled's C build, not from any media crate. Theaarch64-linux-androidcheck above is the substitute, and it isolates the question better anyway.doc-check-ruston the merged tree: pass. An earlier draft of this section claimed a set of rustdoc findings were "pre-existing at 718bc82" — that cited the commit before the base repaired those links, so the claim was not evidence of anything. It is replaced by the gate's own result on this head:mise run doc-check-rustexits 0, and it runs inside thecheck-rustresult recorded above.mise run build-ffifailure was caused and self-inflicted: it ran against a tree mid-edit.lifecycle::upload::test census. The confirming round found thatde756e90had deleted this module's#[cfg(test)] mod tests;, so thirteen tests were tracked but never compiled — nine of them the ones proving this PR's central claims, two of which had never been compiled even once. Restored infc7a6d17; all thirteen then passed unmodified. Verbatim fromcargo nextest list -p capsule-core --features media, filtered to this module — 15 tests:Of these,
derivative_blobs_ship_ciphertext_that_decrypts_to_the_bytes_on_diskis the decision-18 KAT: plaintext → encrypted at import → manifest →derivative_blobs→ the blob's bytes are the re-derived ciphertext, its hash is the signedciphertext_hash, and decrypting with the recordednonce_prefixreturns the on-disk plaintext.Risks and rollout
jxl-oxideandzune-jpegxlfor JXL, plustiffandgif— so this adds noccinvocation and no system library anywhere. That is load-bearing rather than incidental: it is why the aarch64 cross-builds work at all, and the reason WebP is absent (see above).MAX_DECODE_PIXELSis 128 Mpx — ~25% headroom over a 102 Mpx medium-format frame. The honest peak at that ceiling is ~2.5 GB, not the ~1 GB a single buffer suggests:zune-png'su16samples, the realloc that drops alpha, Capsule's RGBA8 copy, and the widening back to RGBu16for the encode. Sincemediais implied bynative, that peak happens on a phone too, where it is an OOM kill rather than an error — which is why the ceiling is not set as high as an allocation bomb would require.dimensionsfrom decoded pixels,lqippopulated,content_typeheader-derived. Nothing rewrites an existing sidecar, andlqipwas alreadyOptionand signature-covered per asset, so a revert stops producing placeholders and leaves every existing sidecar valid.derivatives/. A revert orphans them; they are regenerable by design, and the read side already skips a derivative whose bytes no longer content-address its manifest.content_typefor a non-still changed. The extension table lost its still rows (header-derived now) and gained the common video suffixes, so a.movisvideo/quicktimerather thanapplication/octet-stream.asset_type_forclassifies both as before.lifecycle/upload.rs::derivative_blobslogsderivative manifest has no bytes on disk; skippingand omits the blob, which is correct behaviour for a reference — but it is a per-asset warning that reader was not written to expect. Noted below;upload.rsis outside this lane's manifest.Related Issues
Closes #410
Refs #437 (the S-B1 remainder: a lossy JXL master, AVIF encode, the preview tier, HEIC/RAW decode)
Refs #438 (S-B5: video derivatives)
Refs #444 (WebP is blocked upstream:
rawshift-image's codec does not compile on aarch64)Refs #448 (the read half of derivative encryption:
capsule_sdk::fetch::open_representation)Refs #449 (the closed-format check on receipt:
capsule_server::upload::envelope::check_envelope)Decisions taken
Decisions taken inside the manifest by this lane
Decisions taken in the repair round (orchestrator authority)
Decisions taken in review round 1 (orchestrator authority)
Decisions taken in review round 2 (orchestrator authority)
Unresolved review notes
An adversarial read of this diff produced five notes, and review round 1 produced sixteen findings and six design questions. All of the findings and all but two of the notes are closed by decisions 18-29; what remains is below.
Derivative blobs are uploaded in the clear.Closed by decision 18 — derivatives are encrypted client-side under a fresh per-derivative nonce prefix, andDerivativeBlob::bytesis ciphertext. The read half is filed as media: decrypt fetched derivative blobs — capsule_sdk::fetch::open_representation returns ciphertext with no key path #448, namingcapsule_sdk::fetch::open_representationas the function that will own it.verify_still_formathas no production caller.derivative_blobs.The sentinel's byte-free manifest makesClosed by decision 20 — recognised as an expected reference and skipped atderivative_blobswarn per small asset.debug!.derivative_blobsskips them atdebug!and names them out of scope. Nothing produces one —crate::mlwrites no derivative manifest — so there is no issue to file; when it does, this reader needs an embedding arm and theembedding/{model_id}grammar needs a home.capsule_server::upload::envelope::check_envelopeand the two shape questions it has to settle.capsule-serveris outside this lane's manifest..webpimport lands as a signed original with no placeholder and no thumbnail. WebP is a common export format; media: WebP delivery encode is blocked — rawshift-image 0.1.1's webp codec does not compile on aarch64 (c_char mismatch vs libwebp-sys 0.14.4) #444 is the fix, and it is one upstream cast.SLICES.md's Lane B intro paragraph is stale: it still sayscapsule_core::media"does not" survive and that "there is no decoder in the workspace at all". This lane's edits toSLICES.mdare restricted to the S-B1/S-B5/S-B13/S-B14 rows and detail blocks, and that paragraph is neither. It belongs to whoever owns the prose.Two things this PR got wrong and then fixed, worth flagging rather than burying. Both are the same failure class — a whole-region find-and-replace that silently dropped code — and the second is the more instructive.
(1) Commit
fe1e3c97's message claims the unwind boundary was widened to cover the chromahash placeholder and the encode, and that a derivative-generation failure is reported rather than propagated. Neither edit actually applied — a silent find-and-replace miss — and no test covered either path, so the suite stayed green over a commit message that was not true of the code. Both are applied inde756e90, which says so. The lesson is in the test gap, not the typo: a claim about a failure path needs a test that enters that path.(2)
de756e90's rewrite of a doc comment at the end oflifecycle/upload.rsoverran the file and deleted#[cfg(test)] mod tests;. Thirteen tests stopped being compiled — nine of them the ones proving this PR's central claims, two of which had never been compiled even once — and the suite stayed green, because a lostmoddeclaration presents as a passing run. Restored infc7a6d17; all thirteen passed unmodified, so the tests were right and only their declaration was gone.cargo nextest listis the check that sees this, which is why its census is in the Validation section above rather than a summary line.Contributor Checklist
## Validationfor the exact commands and classifications.