feat(compression): add compressMetadata and decompressMetadata - #839
Open
woahwhattheheck wants to merge 5 commits into
Open
feat(compression): add compressMetadata and decompressMetadata#839woahwhattheheck wants to merge 5 commits into
woahwhattheheck wants to merge 5 commits into
Conversation
Adds a JSON + base64url round trip for invoice metadata small enough to sit in a Stellar transaction memo or an IPFS payload, with a size ceiling so an oversized object fails at encode time rather than at submission. Both directions reject bad input with SdkError CONTRACT_REJECTED: a non-serialisable object, an encoded string outside the base64url alphabet, one that does not contain JSON, and one that decodes to something other than an object.
6 tasks
compression.ts is isomorphic - it feature-detects CompressionStream and falls back to node:zlib - but compressMetadata/decompressMetadata reached for the Node Buffer global, which is undefined in a browser bundle without a polyfill. Both would have thrown 'Buffer is not defined' there, and the Node test process could never surface it. Uses TextEncoder/TextDecoder with btoa/atob instead, which exist in browsers and Node >= 16. Output is byte-identical to the previous encoding.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Closes #619.
Adds a JSON + base64url round trip for invoice metadata small enough to sit in
a Stellar transaction memo or an IPFS payload, with a size ceiling so an
oversized object fails at encode time rather than at submission.
Changes
src/compression.tscompressMetadata(metadata, maxBytes = 512)— JSON-serialises thenbase64url-encodes, without padding.
decompressMetadata(encoded)— decodes and parses back to the object.DEFAULT_METADATA_MAX_BYTES— exported,512.src/index.ts— all three re-exported from the package root.Both directions reject bad input with
SdkErrorcarryingSdkErrorCode.CONTRACT_REJECTED, matching the codes this SDK already uses.What counts as bad input
Encoding rejects anything that is not a serialisable plain object, and reports
the actual size against the limit when the ceiling is exceeded:
null, arrays, strings and numbers — the signature saysRecord<string, unknown>, and silently accepting an array would produce avalue
decompressMetadatathen refuses.BigIntvalues. Both makeJSON.stringifythrow a raw
TypeError; caught and re-raised asSdkErrorso a callerswitching on
err.codesees it like every other SDK failure.BigIntisworth calling out — amounts elsewhere in this SDK are
bigint, so puttingone in metadata is an easy mistake to make.
Decoding rejects a string outside the base64url alphabet, one that does not
contain JSON, and one that decodes to something other than an object — so
decompressMetadataeither returns aRecordor throws, never a surprisearray.
Padding is tolerated on decode but never emitted
Output has no
=padding, per the issue. Decoding accepts optional trailingpadding anyway, so a value that was padded elsewhere in a caller's pipeline
still round-trips rather than failing on a cosmetic difference.
Validation
npx vitest run test/compression.metadata.test.ts test/compression.test.ts— 33/33 passing. The pre-existing
compression.test.tsis included as aregression check, since this change touches the same module.
+or/),the default and a custom
maxBytes, size reported in the error details,every rejected input type, circular and
BigIntpayloads, invalidmaxBytesvalues, non-base64url and non-JSON input, encoded values thatdecode to an array/number/string/null, padded input, and round trips over
empty, flat, nested, null/boolean, unicode and quote-containing keys —
including a second round trip producing a byte-identical string.
npx tsc --noEmitreports 210 errors on this branch and 210 on unmodifiedmain— identical, so this adds none.src/compression.tsis clean; theerrors reported in
src/index.tsare the pre-existing missing-export onesat lines 411–1417, all of which predate this change.
Note on overlap
PR #815 also implements this issue, but bundles it with
dedup.ts,horizonPaginator.ts,invoiceReminderScheduler.ts,search.tsandwebhooks/verify.ts, and currently shows as conflicting. This PR is scoped to#619 alone —
compression.ts, the root export, and a separate test file thatleaves the existing
compression.test.tsuntouched — so it can be taken ordropped without affecting the other issues.
Update — removed a Node-only dependency
The first version of this used
Buffer.from(...)for the base64url step. Thatwas wrong for this module:
compression.tsis isomorphic — it feature-detectsCompressionStream/Bloband falls back tonode:zlib— so reaching for theNode
Bufferglobal would have thrownBuffer is not definedin a browserbundle without a polyfill. The unit tests could not have caught it, since the
test process is Node.
Now uses
TextEncoder/TextDecoderwithbtoa/atob, which exist inbrowsers and in Node >= 16. The output is byte-identical to the previous
encoding — verified across empty, flat, nested, unicode, quote-escaped,
+//-alphabet and 5 KB payloads.Two details in that helper worth noting:
String.fromCharCode(...bytes), because spreading a large array overflows thecall stack — and the size limit is only checked after encoding, so a large
input does reach this line.
bytesfor themaxBytescomparison is nowencoded.length. base64url isASCII-only, so the character count is the byte count.
Added a test that deletes
globalThis.Bufferand round-trips through bothfunctions, so the browser path is covered rather than assumed.