Add decimal support to pyo3 and minarrow-py Python bridges - #188
Merged
Conversation
Polars only supports i128-backed decimals so Decimal32/64 round-trips are not valid through that path without widening first.
pyo3: - Add `decimal` feature flag forwarding minarrow/decimal - Map Decimal32/64/128 ArrowType variants to pa.decimal128() in arrow_type_to_pyarrow (PyArrow only supports 128-bit decimal) - Zero-copy export/import through existing Arrow C Data Interface FFI - Document decimal type mappings in the FFI module - Add Decimal128 roundtrip tests (array, high precision, RecordBatch) minarrow-py: - Add `decimal` feature flag - Add Decimal32/64/128 variants to PyArrowType enum with From impls - Add DType::Decimal variant with Numeric group membership - Add precision/scale properties on PyArray (None for non-decimal) - Handle decimal dtype strings: decimal128(P,S), decimal64(P,S), decimal32(P,S), decimal(P,S) alias for Decimal128 - Construct DecimalArray from Python int sequences with dtype param - Return Python decimal.Decimal values from element access - Scale-aware formatting in repr via format_decimal_string - 23 pytest tests covering construction, inspection, element access, display, slicing, ArrowType, and PyArrow round-trip
pbower
changed the title
pbower
force-pushed
the
MA-EPC86-DECIMAL-TSK383-PYTHON
branch
3 times, most recently
from
August 31, 2026 07:43
7c2e955 to
fed0ca2
Compare
Precision and scale are properties of numeric data, not decimal-specific. Integer types report their digit capacity with scale 0, float types report their significant-digit capacity, and decimal types report their configured precision and scale.
pbower
force-pushed
the
MA-EPC86-DECIMAL-TSK383-PYTHON
branch
from
August 31, 2026 07:46
fed0ca2 to
3b82851
Compare
pbower
added a commit
that referenced
this pull request
Aug 31, 2026
* Add DecimalArray foundation and i128 trait impls (#181) * Add Decimal32, Decimal64, Decimal128 variants to ArrowType (#180) * Wire DecimalArray into NumericArray, Array, Scalar with From impls (#182) * Wire DecimalArray into NumericArray, Array, and Scalar enums with From impls and tests (#182) * Implement decimal conversion arms in NumericArray try_ accessors * Add decimal FFI import/export with arrow-rs and Polars round-trips (#186) * Add decimal FFI import/export with arrow-rs and Polars round-trips * Integrate DecimalArray into Value, ArrayV, NumericArrayV, SuperArray, SuperArrayV, and Consolidate (#187) * Add decimal broadcast dispatch and scale-aware print formatting (#184) * Add decimal arithmetic kernels with checked overflow and auto-promotion + fix null mask drop on broadcasting (#185) * Add decimal arithmetic kernels with checked overflow and auto-promotion * Propagate null mask through broadcast for all array types * Add DecimalArray type conversions and width conversion methods (#183) * Add decimal support to pyo3 and minarrow-py Python bridges (#188) * Add Decimal32 and Decimal64 round-trip tests for arrow-rs * Add decimal support to pyo3 and minarrow-py Python bridges * Make precision and scale report values for all numeric types
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
pa.decimal128()for export/import via the existing Arrow C Data Interface FFI.list[int]viadtype="decimal128(P,S)", element access returnsdecimal.Decimal, scale-awarerepr,.precisionand.scalepropertiesTest plan
cargo test --features "decimal,cast_arrow"passes (all three widths round-trip)cargo test --features decimalpasses