build(deps): support genoray 4.x - #347
Merged
Merged
Conversation
Widen the genoray constraint from >=3.4.0,<4 to >=3.4.0,<5 in pyproject.toml,
pixi.toml, and docs/requirements.txt, and move the lock to genoray 4.0.0.
genoray 4.0.0's two breaking changes are on APIs GVL never calls:
- SparseVar2.from_vcf(tune=) removal — GVL never passed tune=.
- SparseVar2.from_vcf_list(max_mem=) becoming a whole-process budget rather
than a per-chunk cap — GVL does not use from_vcf_list at all.
Every genoray private symbol GVL imports (_types.{POS,V_IDX,DOSAGE}_TYPE,
_contigs.ContigNormalizer, _utils.{format,parse}_memory, _svar2_batch's
MAX_END_SHIFT and _find_ranges_chunked, _svar2_fields.{StoredField,FormatField,
InfoField,_META_DTYPE}, _svar.dense2sparse, _svar._convert._dense2sparse_with_length,
exprs.ILEN) is unchanged between 3.4.0 and 4.0.0; _svar2_batch.py is byte-identical
across the two tags.
The 3.x floor stays: 4.0.0 requires seqpro>=0.21.1,<0.23, which the pinned
seqpro==0.22.0 satisfies, so both majors resolve.
The Rust genoray_core/svar2-codec git pin is deliberately left at rev 66ba734
(genoray 3.0.0) — it already lagged the Python package, and the svar2 tests
exercise the Python-writer/Rust-reader format boundary end to end.
Verified against genoray 4.0.0: 1130 passed / 58 skipped / 4 xfailed (full
tree, after regenerating the stale 1kg fixtures), 6 passed in the slow tier,
134 + 4 cargo tests, ruff check + format clean, pyrefly 0 errors.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
The svar2-codec / genoray_core git pin sat at rev 66ba734 (genoray 3.0.0)
while the Python package moved to 3.4.0 and now 4.0.0. Move both crates to
rev 1bac5f9 (tag 4.0.0) so the Rust reader and the Python writer are built
from the same genoray revision, removing a skew that could let an svar2
on-disk format change slip past the Python-writer/Rust-reader tests.
No GVL source change was needed: the genoray_core APIs GVL uses
(query::{ContigReader,FieldView,BatchResultSplit,KeyRef,dense_abs_row,
unpack_vk_src,pack_vk_src}, field::StorageDtype, layout::{ContigPaths,FieldSub},
bits_get_bit) and the svar2_codec encode/decode helpers are unchanged.
Cost: genoray 4.0.0 declares `tracing` and `tracing-subscriber` as
unconditional dependencies rather than feature-gated ones, so this adds 11
transitive crates (tracing*, matchers, sharded-slab, thread_local, nu-ansi-term,
lazy_static, valuable) that GVL never uses. Gating them upstream would let this
pin carry no extra build cost.
Verified: 134 + 4 cargo tests, 1130 passed / 58 skipped / 4 xfailed pytest.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Contributor
|
Rejoice! 👼 🎉 |
This was referenced Aug 21, 2026
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Widens the
genorayconstraint from>=3.4.0,<4to>=3.4.0,<5and moves the lock to genoray 4.0.0. 3.x compatibility is retained — the floor is unchanged, so both majors resolve.Why it's safe
genoray 4.0.0's two breaking changes are on APIs GVL never calls:
SparseVar2.from_vcf(tune=)removed — GVL never passedtune=.SparseVar2.from_vcf_list(max_mem=)became a whole-process byte budget rather than a per-chunk cap — GVL does not usefrom_vcf_listat all.GVL leans on a lot of genoray private API, so those were checked specifically. Every private symbol GVL imports is unchanged between the
3.4.0and4.0.0tags:_types.{POS_TYPE,V_IDX_TYPE,DOSAGE_TYPE}·_contigs.ContigNormalizer·_utils.{format_memory,parse_memory}·_svar2_batch.{MAX_END_SHIFT,_find_ranges_chunked}·_svar2_fields.{StoredField,FormatField,InfoField,_META_DTYPE}·_svar.dense2sparse·_svar._convert._dense2sparse_with_length·exprs.ILEN_svar2_batch.pyis byte-identical across the two tags.genoray 4.0.0 requires
seqpro>=0.21.1,<0.23, which the pinnedseqpro==0.22.0satisfies.Verification (against genoray 4.0.0)
pytest tests -q(full tree)pytest tests -q -m slowcargo-testruff check/ruff format --checktypecheck(pyrefly)The first run errored on 3
test_ds_haps_1kgtests, but that was a stale local fixture (untracked, written by gvl 0.35.0 at dataset format v1 vs the current v2) — pre-existing and unrelated to genoray. Regenerated viagen-1kgso the 1kg parity tests actually ran.Commits
build(deps): support genoray 4.x— the constraint widening inpyproject.toml,pixi.toml,docs/requirements.txt(whosegenoray>=2.3.1floor was long stale), plus the lock move.build(deps): align the Rust genoray pin with genoray 4.0.0— optional, see below.The second commit is separable
The Rust
svar2-codec/genoray_coregit pin sat at rev66ba734(genoray 3.0.0) — it already lagged the Python package before this PR. Commit 2 moves both crates to rev1bac5f9(tag4.0.0) so the Rust reader and Python writer are built from the same genoray revision, closing a skew that could let an svar2 on-disk format change slip past the Python-writer/Rust-reader tests. No GVL source change was needed.Cost: genoray 4.0.0 declares
tracingandtracing-subscriberas unconditional dependencies rather than feature-gating them, sodefault-features = falsedoesn't avoid them. This adds 11 transitive crates GVL never uses (tracing*,matchers,sharded-slab,thread_local,nu-ansi-term,lazy_static,valuable). Gating them upstream in genoray would make this pin free.Drop commit 2 if that build cost isn't worth it — commit 1 passes the full suite on its own.
Notes
pixi installrewrote some cosmetic lock formatting (droppedvirtual-packagesblocks,./→.) from a newer pixi version. It's in commit 1's diff and unrelated to genoray; no package other than genoray changed in the lock.pyproject.toml:11says>= 3.14 blocked by pyarrow/genoray, but genoray 4.0.0 now allows<3.15and pyarrow is uncapped>=3.10. The<3.14cap is likely still correct (conda-pinnednumba==0.59.1), but confirming that needs a py314 env — separate change.skills/genvarloader/SKILL.mdorapi.mdupdate is required; no prose doc asserts a genoray version.🤖 Generated with Claude Code