Skip to content

fix(cpp): handle TS2DIFF float prefixes in batch decode - #901

Open
kkzi wants to merge 1 commit into
apache:developfrom
kkzi:fix/cpp-ts2diff-float-double-batch-prefix
Open

fix(cpp): handle TS2DIFF float prefixes in batch decode#901
kkzi wants to merge 1 commit into
apache:developfrom
kkzi:fix/cpp-ts2diff-float-double-batch-prefix

Conversation

@kkzi

@kkzi kkzi commented Aug 7, 2026

Copy link
Copy Markdown

Summary

  • make C++ FLOAT/DOUBLE TS_2DIFF batch decoding consume the Java-compatible scale/overflow prefix
  • route batch reads through the existing segment-aware scalar decoders
  • add regression coverage for multiple FLOAT segments and a DOUBLE overflow bitmap prefix

Root cause

The FLOAT/DOUBLE batch overrides delegated directly to the INT32/INT64 TS_2DIFF batch decoders and then bit-cast the results. Those integer decoders expect a delta-block header at the current stream position, but FLOAT/DOUBLE segments place a maxPointNumber or overflow-bitmap prefix before that header. The prefix was therefore decoded as block metadata, leaving the stream and decoder state inconsistent. In tree-reader batch paths this could result in unbounded decoding at end-of-input.

The scalar read_float and read_double implementations already consume the prefix and apply scale/overflow handling. Reusing those implementations restores correctness for normal, overflow, and legacy raw segments. This intentionally gives up the integer SIMD fast path for FLOAT/DOUBLE until a prefix-aware optimized implementation is available.

Tests

  • FloatDoubleTS2DIFFCodecTest.*
  • TS2DIFFCodecTest.*
  • FloatTS2DIFFEncoderResetTest.*
  • EncodingCoverage.TS2DIFF*
  • TreeQueryByRowTest.QueryByRow_TabletMultiType_PartialPaths
  • TsFileWriterTest.WriteDiffrentTypeCombination
  • clang-format --dry-run --Werror cpp/src/encoding/ts2diff_decoder.h cpp/test/encoding/ts2diff_codec_test.cc

Fixes #900

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

[Bug] C++ FLOAT/DOUBLE TS_2DIFF batch decoding skips segment prefix

1 participant