Skip to content

feat(compression): add compressMetadata and decompressMetadata - #839

Open
woahwhattheheck wants to merge 5 commits into
Stellar-split:mainfrom
woahwhattheheck:feat/619-metadata-encoding
Open

feat(compression): add compressMetadata and decompressMetadata#839
woahwhattheheck wants to merge 5 commits into
Stellar-split:mainfrom
woahwhattheheck:feat/619-metadata-encoding

Conversation

@woahwhattheheck

@woahwhattheheck woahwhattheheck commented Sep 6, 2026

Copy link
Copy Markdown

Closes #619.

Adds a JSON + base64url round trip for invoice metadata small enough to sit in
a Stellar transaction memo or an IPFS payload, with a size ceiling so an
oversized object fails at encode time rather than at submission.

Changes

  • src/compression.ts
    • compressMetadata(metadata, maxBytes = 512) — JSON-serialises then
      base64url-encodes, without padding.
    • decompressMetadata(encoded) — decodes and parses back to the object.
    • DEFAULT_METADATA_MAX_BYTES — exported, 512.
  • src/index.ts — all three re-exported from the package root.

Both directions reject bad input with SdkError carrying
SdkErrorCode.CONTRACT_REJECTED, matching the codes this SDK already uses.

What counts as bad input

Encoding rejects anything that is not a serialisable plain object, and reports
the actual size against the limit when the ceiling is exceeded:

  • null, arrays, strings and numbers — the signature says
    Record<string, unknown>, and silently accepting an array would produce a
    value decompressMetadata then refuses.
  • Circular structures and BigInt values. Both make JSON.stringify
    throw a raw TypeError; caught and re-raised as SdkError so a caller
    switching on err.code sees it like every other SDK failure. BigInt is
    worth calling out — amounts elsewhere in this SDK are bigint, so putting
    one in metadata is an easy mistake to make.

Decoding rejects a string outside the base64url alphabet, one that does not
contain JSON, and one that decodes to something other than an object — so
decompressMetadata either returns a Record or throws, never a surprise
array.

Padding is tolerated on decode but never emitted

Output has no = padding, per the issue. Decoding accepts optional trailing
padding anyway, so a value that was padded elsewhere in a caller's pipeline
still round-trips rather than failing on a cosmetic difference.

Validation

  • npx vitest run test/compression.metadata.test.ts test/compression.test.ts
    — 33/33 passing. The pre-existing compression.test.ts is included as a
    regression check, since this change touches the same module.
  • New coverage: no padding emitted, base64url alphabet (never + or /),
    the default and a custom maxBytes, size reported in the error details,
    every rejected input type, circular and BigInt payloads, invalid
    maxBytes values, non-base64url and non-JSON input, encoded values that
    decode to an array/number/string/null, padded input, and round trips over
    empty, flat, nested, null/boolean, unicode and quote-containing keys —
    including a second round trip producing a byte-identical string.
  • npx tsc --noEmit reports 210 errors on this branch and 210 on unmodified
    main
    — identical, so this adds none. src/compression.ts is clean; the
    errors reported in src/index.ts are the pre-existing missing-export ones
    at lines 411–1417, all of which predate this change.

Note on overlap

PR #815 also implements this issue, but bundles it with dedup.ts,
horizonPaginator.ts, invoiceReminderScheduler.ts, search.ts and
webhooks/verify.ts, and currently shows as conflicting. This PR is scoped to
#619 alone — compression.ts, the root export, and a separate test file that
leaves the existing compression.test.ts untouched — so it can be taken or
dropped without affecting the other issues.

Update — removed a Node-only dependency

The first version of this used Buffer.from(...) for the base64url step. That
was wrong for this module: compression.ts is isomorphic — it feature-detects
CompressionStream/Blob and falls back to node:zlib — so reaching for the
Node Buffer global would have thrown Buffer is not defined in a browser
bundle without a polyfill. The unit tests could not have caught it, since the
test process is Node.

Now uses TextEncoder/TextDecoder with btoa/atob, which exist in
browsers and in Node >= 16. The output is byte-identical to the previous
encoding — verified across empty, flat, nested, unicode, quote-escaped,
+//-alphabet and 5 KB payloads.

Two details in that helper worth noting:

  • The binary string is built in 0x8000 chunks rather than
    String.fromCharCode(...bytes), because spreading a large array overflows the
    call stack — and the size limit is only checked after encoding, so a large
    input does reach this line.
  • bytes for the maxBytes comparison is now encoded.length. base64url is
    ASCII-only, so the character count is the byte count.

Added a test that deletes globalThis.Buffer and round-trips through both
functions, so the browser path is covered rather than assumed.

Adds a JSON + base64url round trip for invoice metadata small enough to sit in
a Stellar transaction memo or an IPFS payload, with a size ceiling so an
oversized object fails at encode time rather than at submission.

Both directions reject bad input with SdkError CONTRACT_REJECTED: a
non-serialisable object, an encoded string outside the base64url alphabet,
one that does not contain JSON, and one that decodes to something other than
an object.
compression.ts is isomorphic - it feature-detects CompressionStream and falls
back to node:zlib - but compressMetadata/decompressMetadata reached for the
Node Buffer global, which is undefined in a browser bundle without a polyfill.
Both would have thrown 'Buffer is not defined' there, and the Node test process
could never surface it.

Uses TextEncoder/TextDecoder with btoa/atob instead, which exist in browsers
and Node >= 16. Output is byte-identical to the previous encoding.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Add compressMetadata and decompressMetadata to compression.ts

1 participant