Add low-memory JPEG decode path - #1
Draft
mariusandra wants to merge 15 commits into
Draft
Conversation
- pixie/decodebudget: runtime per-decode memory budget (replaces the compile-time frameosEmbedded consts); decoders raise catchable PixieErrors instead of exhausting memory - jpeg: decode plan checked against the budget before any image-sized allocation; progressive JPEGs now use target-sized channel masks; sampling resolution clamps itself to the budget, trading sharpness for a successful decode - jpeg: streaming decode (decodeJpegStreamScaled/Into) pulls the compressed input through a 32K sliding window so files never need to be fully buffered; bit-identical output across the test suite - jpeg: decodeJpegInfo probe + jpegDecodeIntermediateBytes for pre-decode budget planning - jpeg/png: ScaledDecodeFit (stretch/cover/contain) on all scaled decode paths, enabling aspect-correct decode-into-canvas; cover inflates mask sampling density for the cropped region within budget - png: budget check at IHDR; inflated scanlines released before the pixel seq allocation Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
seekEntropyMarker could set state.pos behind the sliding window start on a long 0xFF run in damaged entropy data; clamp to windowStart so recovery matches the buffered decoder instead of failing the whole decode. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Unfiltering allocated a second scanline-sized buffer, putting the decode plan for a canvas-sized RGBA PNG at pixels + 2x scanlines (4.5MB for 480x800). Compacting the [filter byte][row] stride in place drops the plan to pixels + scanlines (3.0MB), which fits the decode budget on ESP32-class devices. Interlaced images keep the per-pass copy and the old plan formula. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Non-interlaced PNGs now decode row by row through zippy's streaming inflate (pinned to the FrameOS fork): scanlines are unfiltered against a single previous-row buffer and written straight into the pixel data, so the whole-image inflate buffer disappears. Full decodes plan pixels + a fixed ~64KB; decodePngScaled/Into sample rows on the fly and never allocate the full-size pixel buffer at all, so a huge PNG can scale into display bounds like a streamed JPEG. Interlaced and 16-bit-scaled decodes keep the buffered path. Verified pixel-identical to the previous decoder across the whole pngsuite corpus, plus differential streamed-vs-buffered scaled decode tests and tightened budget assertions (480x800 RGBA: full decode within 2MB, scaled-into within 256KB). Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
A transitive URL requirement breaks nimble's CI resolution (two zippy sources for one package name); frameos.nimble pins the FrameOS zippy fork at the root instead, the same proven pattern used for pixie. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
CI's nimble cannot reliably resolve a second forked package (transitive or root URL requirements both produced incomplete nimble.paths), so the streaming inflate now lives inside pixie as a self-contained module (huffman machinery vendored from zippy 0.10.16, MIT) raising PixieError directly. The dependency graph returns to the CI-proven shape: one forked package (pixie), stock guzba zippy. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
The streaming inflate only took one contiguous buffer, so PNGs that split their compressed data across many IDAT chunks (libpng emits 8K chunks; a 2.6MB gallery PNG has 326) were concatenated into a second multi-MB allocation before decoding — the allocation that OOMed streamed decodes on fragmented ESP32 PSRAM. The bit reader now consumes a list of InflateSegments, crossing chunk boundaries byte-wise and keeping the fast word-load path within a segment. Verified: a real 2.6MB multi-IDAT PNG stream-decodes into a 1200x1600 target under a 1MB decode budget, pixel-identical to the buffered path; IDATs re-chunked at 1..8192 bytes decode identically; PNG fuzzer clean over 10k iterations. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
decodePngScaledInto now accepts a list of InflateSegments, so a PNG whose bytes arrived in fixed-size download chunks decodes without ever being assembled into one contiguous buffer: the segmented parser validates every chunk CRC across boundaries (incremental crc32) and hands IDAT spans to the segmented inflater as sub-slices. Non-interlaced <=8-bit images stream scanlines into the target; interlaced/16-bit fall back to coalescing plus the buffered decode. streamIdatRows now takes inflater segments directly, shared by the contiguous and segmented paths. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
decodePngStreamScaledInto(source, totalLen, target, fit) decodes a PNG read sequentially from a callback (e.g. a download spilled to disk on a device without the memory to buffer it). The chunk walker validates CRCs as bytes stream by and feeds IDAT payloads to the inflater through a 16KB read buffer, so peak memory is that buffer plus the fixed streaming overhead — the compressed file is never resident. The streaming inflater gains InflatePull, a pull-based input the reader falls back to when its segment list is exhausted; input consumption is strictly sequential so each pulled buffer may be reused. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
The orientation SHORT occupies the first two bytes of the 4-byte IFD data field. After the full-word maybeSwap it sits in the LOW word for little-endian files, but the value was always taken from the high word (`shr 16`) — so orientation from II-endian cameras (Sony, Canon) was silently read as 0 and photos rendered sideways. Big-endian (MM) files, which all the existing f1..f8 fixtures use, were unaffected. Adds II-endian variants of the f1..f8 orientation fixtures and a test asserting they decode pixel-identically to the MM originals. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
PNG and JPEG each declared their own pull-source callback with an identical signature, and frameos already passed one file closure to both. Name it once as ImageSourceProc and keep PngSourceProc/JpegSourceProc as aliases so callers compile unchanged. scaledFitRects moves to common.nim alongside it: BMP and PPM need the same fit arithmetic, and a second copy would be a second place for fitContain's untouched-border rule to drift. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
A spilled BMP had no file-backed decoder, so a download too big for RAM could only fail — the whole point of spilling to storage is that the decode never needs the full body back. BMP is the easiest format to stream: no entropy coder, fixed stride, strictly sequential rows. decodeBmpStreamScaledInto pulls rows through the same engine PNG uses — one source row in RAM, a monotonic target cursor, nearest-neighbour sampling — with the row cursor running upward for bottom-up files so the trailing flipVertical (and its whole-image buffer) disappears. Rows no target row samples are consumed without conversion, and the walk stops once the last sampled row is read, so a fitCover crop never touches the file's tail. decodeDib is rebuilt on the same header parse and row converter, and now checks the decode budget before newImage: a 20000x20000 header used to walk straight into a 1.6 GB allocation. PPM P6 gets the same treatment; ASCII P3 raises rather than buffering back. Also fixes a pre-existing crash found while fuzzing: decodeBmp checked for 14 bytes and then indexed byte 14, so an exactly-14-byte file raised IndexDefect instead of a catchable PixieError. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Needed this to be able to load JPEGs on an ESP32...