Skip to content

Align the jax stack, name the CUDA extra for CUDA, and drop Python 3.11 - #10

Merged
mahdi-shafiei merged 3 commits into
mainfrom
chore/align-jax-ecosystem-and-drop-py311
Aug 29, 2026
Merged

Align the jax stack, name the CUDA extra for CUDA, and drop Python 3.11#10
mahdi-shafiei merged 3 commits into
mainfrom
chore/align-jax-ecosystem-and-drop-py311

Conversation

@mahdi-shafiei

Copy link
Copy Markdown
Collaborator

Brings DiffBio onto the stack the sibling repositories now share: jax 0.11.1, jaxlib 0.11.1, flax 0.12.9, optax 0.2.8, grain 0.2.18, on Python 3.12.

gpucuda12

JAX names its own extras for the CUDA major version — cuda12, cuda12-local, cuda13 — and publishes no gpu extra. This project also ships metal, which is a GPU, so gpu could not distinguish the two. setup.sh already spoke of a cuda12 backend while syncing --extra gpu.

The extra also drops its jaxlib line: every jax[cuda12*] extra already requires jaxlib at its own version, so restating it only creates a second place for the two to disagree.

Floors that state requirements

A lock refresh does not move a dependency that still satisfies its floor, so each of these is declared rather than left to the resolver:

declaration reason
flax>=0.12.1 first release with nnx.Variable.set_value, which the pipeline calls
optax>=0.2.8 below it, optax sets jax_pmap_shmap_merge, removed in jax 0.10 — importing flax raises AttributeError and collection dies for the entire suite
dm-haiku>=0.0.17 (constraint) arrives via jax-md; up to 0.0.16 it binds jax.core.DropVar at import, removed in jax 0.11.0

Python 3.11

Follows jax 0.11.0 dropping it. This also collapses the resolution to a single unforked lockfile, which is most of the lock diff. Raising ruff's target to py312 enables the PEP 695 rules; the four aliases converted here have no runtime uses anywhere, and the module-level TypeVar its generic replaced is removed with it.

Two defects found while verifying, each in its own commit

top_k_mask returned values outside its documented [0, 1] — one float32 ULP over, from the isotonic solver's pooled averaging. The same input gives the same value on the previous jax release, so this predates the version move; the property test only reaches it on some seeds. The forward value is saturated onto the bound while the backward pass stays on the projection's own gradient — a plain clip would zero the gradient of exactly the entries the mask selects. Over 540 combinations the gradients are unchanged and only the four out-of-bound masks move.

A benchmark bypassed the optimizer ownership boundary, failing a contract test on main. It went unnoticed because no CI job runs tests/benchmarks/ — the core-and-misc shard ignores it, no other shard names it, and the e2e and performance jobs select by marker and collect nothing there. Routing it through create_benchmark_optimizer required exposing weight_decay, because OptimizerConfig defaults it to 0.0 while optax.adamw defaults to 1e-4; passing 1e-4 keeps the optimizer bit-identical, verified against the previous call with a deliberately wrong value as a control.

Verification

All five unit shards on Python 3.12 with the extras CI installs: 3126 passed, 48 skipped, 0 failed. ruff check, ruff format --check (606 files) and lint-imports (6 contracts) all clean.

tests/benchmarks/ remains outside every CI job's selection. That gap is left as-is here rather than widened into a CI change alongside a dependency move.

top_k_mask documents a mask "bounded in [0, 1]" and returns the Euclidean
projection onto the capped simplex, where that bound holds in exact arithmetic.
The isotonic solver averages pooled blocks in float32 and can finish one ULP
outside it: for n=4 with scores drawn from seed 202 the largest entry comes back
as 1.0000001, which is what the property test asserts against.

The same input produces the same value on the previous jax release, so this is a
long-standing gap rather than something the version move introduced; the property
test only reaches it on some seeds, which is why it surfaced now.

Saturate the forward value onto the bound and leave the backward pass on the
projection's own gradient. A plain clip would zero the gradient of exactly the
saturated entries, which are the ones the mask selects. Over 540 score/seed
combinations the gradients are unchanged and only the four out-of-bound masks
move, each by at most 2.4e-07.
tests/benchmarks/test_ownership_boundaries.py requires benchmarks to build
optimizers through benchmarks/_optimizers.py rather than calling optax directly.
benchmarks/singlecell/compute_cost.py constructed optax.adamw inline, so the
contract has been failing on main.

It went unnoticed because no CI job runs tests/benchmarks: the core-and-misc
shard ignores that directory, no other shard names it, and the end-to-end and
performance jobs select by marker and collect nothing there. Confirmed by
collecting each job's selection and checking the file directly as a control.

create_benchmark_optimizer gains a weight_decay argument because OptimizerConfig
defaults it to 0.0 while optax.adamw defaults it to 1e-4; passing 1e-4 keeps the
optimizer identical, verified by comparing updates over five steps against the
previous call, with a deliberately wrong value as the control.
Resolves to jax 0.11.1, jaxlib 0.11.1, flax 0.12.9, optax 0.2.8 and grain
0.2.18, matching the sibling repositories.

Rename the gpu extra to cuda12. JAX names its own extras for the CUDA major
version (cuda12, cuda12-local, cuda13) and publishes no gpu extra, and this
project also ships metal, which is a GPU. setup.sh already spoke of a cuda12
backend while syncing --extra gpu. The extra also drops its jaxlib line: every
jax cuda extra already requires jaxlib at its own version, so restating it only
lets the two disagree.

Raise floors to state requirements rather than leave them to the resolver:

- flax 0.12.1 is where nnx Variable gained set_value.
- optax below 0.2.8 sets jax_pmap_shmap_merge, which jax removed in 0.10, so
  importing flax raises AttributeError and collection dies for the whole suite.
- dm-haiku arrives through jax-md and up to 0.0.16 binds jax.core.DropVar at
  import time, which jax removed in 0.11.0; 0.0.17 guards it. Constrained rather
  than declared, since nothing here depends on it directly.

Requiring Python 3.12 follows jax 0.11.0 dropping 3.11, and collapses the
resolution to a single unforked lockfile, which is most of the lock diff.

Raising ruff's target to py312 enables the PEP 695 rules. The four type-alias
conversions stand, but the generic function is reverted and UP046, UP047 and
UP049 are disabled: the docstring gate is pydocstyle 6.3.0, whose parser predates
PEP 695 and reports missing docstrings for generic functions on definitions whose
docstring is present. 6.3.0 is its final release. No workflow runs pydocstyle
today, but the hook is configured, so the failure reaches contributors.

Renaming the extra also moved its key, and the sort_pyproject hook enforces the
ordering; the file parses to the same document before and after.

All five unit shards pass on Python 3.12: 3126 passed, 48 skipped.
@mahdi-shafiei
mahdi-shafiei force-pushed the chore/align-jax-ecosystem-and-drop-py311 branch from d977c7d to cfa17af Compare August 28, 2026 22:33
@mahdi-shafiei
mahdi-shafiei merged commit 536c794 into main Aug 29, 2026
14 checks passed
@mahdi-shafiei
mahdi-shafiei deleted the chore/align-jax-ecosystem-and-drop-py311 branch August 29, 2026 01:14
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant