fix(stream): stop PriorityStreamer::analyze() from duplicating object-nested arrays on reconstruction - #397
Merged
Conversation
…-nested arrays on reconstruction
extract_patches's Object branch pushed a Set patch carrying the full
cloned value of a field, then unconditionally recursed into it. For a
non-empty array field this produced a redundant Append patch for the
same elements, so JsonReconstructor applied both and doubled every
object-nested array on reconstruction (e.g. {"items":[1,2,3]} became
{"items":[1,2,3,1,2,3]}).
Set patches now carry an array-emptied skeleton of the value
(skeletonize_arrays) instead of a full clone, with the array's own
Append patch supplying the data, and are priority-hoisted to
max(own priority, highest Append priority in their subtree) so a Set
can never be applied after an Append it must precede -- closing three
data-loss regressions an adversarial review round found in an earlier
skeletonize-only attempt (same-path and parent-object priority
inversion wiping already-appended data, and permanent loss of arrays
under JsonPath-unencodable keys). Array priority for chunked arrays is
now computed once for the whole array before chunking, keeping chunk
order stable regardless of per-chunk priority differences.
Wire format is unchanged; this is a fix-scoped change.
Closes #394
bug-ops
enabled auto-merge (squash)
August 17, 2026 21:53
github-actions
Bot
added
documentation
WASM Bundle Size Report
|
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
PriorityStreamer::analyze()'sextract_patchesemitted aSetpatch carrying an object field's full cloned value, then unconditionally recursed into that same value. For a non-empty array field this produced a redundantAppendpatch for the same elements, soJsonReconstructorapplied both and duplicated every object-nested array on reconstruction (e.g.{"items":[1,2,3]}round-tripped to{"items":[1,2,3,1,2,3]}).Setpatches now carry an array-emptied skeleton of the value (skeletonize_arrays) instead of a full clone, with the array's ownAppendpatch supplying the data.Setpriority is hoisted tomax(own priority, highest Append priority anywhere in its subtree)so aSetcan never be applied after anAppendit must precede — this closes three data-loss regressions an adversarial review round found in an earlier skeletonize-only attempt: same-path priority inversion, parent-object priority inversion (including a chunked-array tail variant), and permanent loss of arrays nested under JsonPath-unencodable keys.arr.chunks(max_patch_size)) is now computed once for the whole array before chunking, keeping chunk order stable regardless of per-chunk priority differences.PatchOperation/JsonPatch/PriorityStreamFrameshapes untouched) —fix-scoped change, not breaking.Closes #394
Test plan
cargo +nightly fmt --checkcargo clippy --workspace --all-targets --all-features -- -D warningscargo nextest run --workspace --all-features --lib --bins(1019/1019 pass)crates/pjs-core/src/stream/priority.rs(analyze()->JsonReconstructor-> assert structural equality), covering: exact issue repro, multi-entity payload, chunked arrays (including a divergent-chunk-priority case), nesting at depth, array-of-arrays, arrays inside array elements, empty arrays, top-level bare arrays, mixed payloads, and the three priority-inversion/unencodable-key regressions found during reviewCHANGELOG.mdupdated under[Unreleased]/### Fixed