ZJIT: lobsters perf burndown

[k0kubun]

I'm focusing on lobsters again, so I filed a separate issue from #1030. Take 3 of the effort after #833 and #713.

I have a branch where ZJIT is faster than YJIT on lobsters. The goal of this issue is to upstream the most reasonable ideas out of it in cleaner patches, not necessarily porting everything.

yjit: ruby 4.1.0dev (2026-08-26T21:52:46Z master 0968fd4b3b) +YJIT +PRISM [x86_64-linux]
zjit-before: ruby 4.1.0dev (2026-08-26T21:52:46Z master 0968fd4b3b) +ZJIT +PRISM [x86_64-linux]
zjit-after: ruby 4.1.0dev (2026-08-26T23:15:25Z zjit/min-straight d2330a712c) +ZJIT +PRISM [x86_64-linux]

--------  ------------  ----------------  ---------------  -------------------  ------------------  ----------------  ---------------
bench        yjit (ms)  zjit-before (ms)  zjit-after (ms)  zjit-before 1st itr  zjit-after 1st itr  yjit/zjit-before  yjit/zjit-after
lobsters  421.7 ± 6.3%      461.9 ± 4.8%     399.5 ± 5.0%                0.533               0.459             0.913            1.056
--------  ------------  ----------------  ---------------  -------------------  ------------------  ----------------  ---------------

Branch

• Branch

This is a branch that originally had more patches, which got reduced to fewer patches and then rebased.

List of patches

I have previously benchmarked each revision so we can know the impact of each change. Note: The latency was measured on a different CPU from the above.

sha	speedup	ms/itr	commit
b59ea8efc7	1.000	450.9	ZJIT: Reserve a label reference's bytes by writing them (ruby#18519) (base)
fbccb8ac4f	0.999	451.5	ZJIT: Expose bucket counts on DistributionSummary
93970417c8	1.003	449.4	ZJIT: Guard the profiled receiver types of megamorphic call sites
72b5ea6116	1.010	446.4	ZJIT: Guard receiver types for sends with a literal block
e3e36fd91b	1.002	449.8	ZJIT: Dispatch megamorphic-by-class call sites on one ancestor guard
41ea490868	1.002	450.2	ZJIT: Look megamorphic send targets up in a class table
f182afce8c	0.999	451.4	ZJIT: Fix invokeblock block-handler profiling and gate its ISEQ chain
8706bad1f8	1.004	448.9	ZJIT: Dispatch to blocks that only return with a direct JIT-to-JIT call
99006aca80	1.002	450.1	ZJIT: Inline a block whose non-local return escapes to the frame we compile
607937d7ab	1.000	450.8	ZJIT: Destructure a lone yielded Array for the direct block dispatch
088b42a163	1.002	450.2	ZJIT: Destructure a lone yielded Array at polymorphic yield sites too
cae94dab8f	1.006	448.3	ZJIT: Specialize yield whose arity does not match the block
dbe5aa7e8f	1.002	449.9	ZJIT: Let a yield-unlocking iterator past the caller's inline budget
ab56cf176f	1.007	448.0	ZJIT: Lay the inlined block frame out on the reshaped yield arguments
6ac1117571	1.002	450.2	ZJIT: Drop the now-unreachable invokeblock arity-mismatch fallback reason
8d9005fe9d	1.005	448.6	ZJIT: Read the throwing frame's ISEQ through CFP_ISEQ() in vm_throw_start
5e8fa8b101	1.005	448.7	ZJIT: Dispatch a yield to a block that can break directly
2a286ff97e	1.003	449.8	ZJIT: Call megamorphic ISEQ targets without leaving JIT code
7d77398b9b	0.997	452.4	ZJIT: Size megamorphic send tables to the shapes that need them
52c435946c	1.001	450.5	ZJIT: Compile direct sends to def foo(...) callees
921e5ce21c	1.005	448.8	ZJIT: Compile direct sends to callees with a **rest parameter
73f2c20acf	1.001	450.3	ZJIT: Compile direct sends to callees with an anonymous **
7fe4ce55a4	1.005	448.6	ZJIT: Narrow test reg, imm to the immediate's width
57a28d54ab	1.015	444.2	ZJIT: Allocate JITFrames below INT32_MAX
2de98f9bf0	1.009	446.7	ZJIT: Compile send/send with a profiled method name
f32800b920	1.012	445.7	ZJIT: Compile splat calls instead of dispatching them dynamically
8bc85404bc	1.012	445.5	ZJIT: Bump VM_INSTRUCTION_SIZE for the new profiled splatarray
3d0a8c2fb5	1.011	445.9	ZJIT: Read nil for an out-of-bounds Array#[] instead of side-exiting
c0fed27163	1.013	445.2	ZJIT: Forward a splat straight to the callee's rest parameter
09ebf123c2	1.021	441.5	ZJIT: Pass a &blk argument through to a direct send
b0ae5b4177	1.020	442.1	ZJIT: Only branch on the block param proxy for an ISEQ callee
2c8127083b	1.019	442.6	ZJIT: Pass a &blk argument through to a direct C call
37cabd0c65	1.018	442.9	ZJIT: Pin the forwardable callee's fallback on a &blk call site
720e77f911	1.022	441.1	ZJIT: Inline callees that a &blk call site passes a block handler to
da07755ead	1.022	441.3	ZJIT: Stop side-exiting forever at unstable super sites
f200ed58b3	1.020	442.1	ZJIT: Dispatch polymorphic super on the frame's method entry
013722e299	1.027	439.2	ZJIT: Compile protected calls behind a check on the caller's self
0b1ed96a03	1.025	440.0	ZJIT: Compile a yield to a Symbol block as the send it stands for
e913ce9375	1.029	438.3	ZJIT: Drop the now-unreachable invokeblock may-throw fallback reason
f7a0a83372	1.013	444.9	ZJIT: Let a frozen ivar dispatch earn back the shape it is missing
ffd1019e36	1.025	440.0	ZJIT: Fall back to a generic ivar access when a shape chain misses
2c3db83855	1.024	440.2	ZJIT: Back ivar shape-chain misses with a per-name shape table
d34ae52b40	1.022	441.3	ZJIT: Weigh an ivar respecialization by the traffic it would take off the fallback
e372630f06	1.027	439.1	ZJIT: Test the ivar reprofile budget inline
55d4065554	1.024	440.4	ZJIT: Let a getivar site's profile forget boot-time shapes
00ef8c1683	1.023	440.6	ZJIT: Cache the ivar location of class and module shapes
be07f449b2	1.027	438.9	ZJIT: Test profile eviction and cached class ivars
f6ff443528	1.022	441.3	ZJIT: Size ivar shape tables to the names that need them
e5e533c148	1.026	439.3	ZJIT: Hand the SP register back when returning past inlined frames
b2af99f1da	1.027	439.0	ZJIT: Dispatch yield to a run-time block ISEQ without leaving JIT code
a97775546b	1.023	441.0	ZJIT: Dispatch a Symbol block's send through the megamorphic class table
5aae4cfcf5	1.021	441.8	ZJIT: Insert the missing zjit_splatarray in the checked-in bindings
887ae5a22e	1.024	440.5	ZJIT: Inline def foo(...) callees and compile the bar(...) inside them
6c9e6a68ee	1.041	433.3	ZJIT: Specialize bar(...) in standalone forwardable ISEQs
cd8025014a	1.043	432.3	ZJIT: Fuse Test and fixnum compares into CondBranch
3b2d0a7e8b	1.052	428.8	ZJIT: Fuse type tests into the branch that consumes them
bd9f496b4f	1.061	425.0	ZJIT: Split cold code into an outlined code region
f5bafc5aea	1.065	423.3	ZJIT: Assert with_write_ptr patches the half it is on
3e2c13a5ab	1.058	426.2	ZJIT: Stop each half of the code region at its own end
4b910e0aa9	1.062	424.8	ZJIT: Allocate callee-saved registers to call-crossing VRegs
5dd4a434ed	1.067	422.7	ZJIT: Remove CheckInterrupts in empty inline frames
f8e5349cdb	1.061	424.9	ZJIT: Remove the trivial inliner
c749360d5f	1.064	423.9	ZJIT: Remove NoTracePoint patch points in empty inline frames
61974e367f	1.061	425.0	ZJIT: Elide inline frames around leaf builtin calls
b1ab8d9cd1	1.063	424.2	ZJIT: Retry inlining annotated builtins after type inference
504d291dfd	1.061	424.9	ZJIT: Update opt_tests snapshots for the empty-inline-frame elimination
2f22d768e9	1.063	424.0	ZJIT: cut the cost of the GC mark hooks
63d34ffe94	1.067	422.7	ZJIT: Coalesce block params with the arguments feeding them
41e90501eb	1.064	423.7	ZJIT: Rematerialize block params that can only hold one value
d064b6c3ac	1.071	421.1	ZJIT: Make the block-param coalescing analysis linear
a4d8070e04	1.068	422.2	ZJIT: Shrink side-exit stubs with a shared metadata table
d1ce2014fc	1.069	421.7	ZJIT: Skip the bounds check on the side-exit metadata lookup
ff371cafae	1.063	424.1	ZJIT: Profile ISEQs before function stub compilation
1297d69808	1.062	424.5	ZJIT: Compile the once instruction
e848a42abb	1.059	425.9	ZJIT: Grow the array instead of side-exiting on out-of-range Array#[]=
37a00b6c38	1.063	424.3	ZJIT: Compute the coderange instead of side-exiting when it is unknown
e3ccdd7fad	1.067	422.7	ZJIT: Profile expandarray and compile one shape per site
c124fa60d5	1.068	422.3	ZJIT: Renumber YJIT's instruction bindings for zjit_expandarray
4dc9d55c82	1.065	423.5	ZJIT: Inline a fast path for Array#push and Array#<<
2174f109b8	1.068	422.0	ZJIT: Skip redundant cfp->sp stores
2dc25a0444	1.067	422.8	ZJIT: Skip redundant local spill stores
b9596c23b1	1.067	422.6	ZJIT: Test that spill elision keeps frame locals observable
3303693b7e	1.068	422.1	ZJIT: Don't break spilled parallel-copy cycles through SCRATCH_REG
0f2743ee57	1.069	421.8	ZJIT: Break down zjit_alloc_bytes by subsystem
bcadbeadfb	1.067	422.8	ZJIT: Release an ISEQ's profile and GC offsets when the ISEQ is freed
ab396b7bd0	1.070	421.3	ZJIT: Size operand type profiles exactly
f0638f31ae	1.066	423.1	ZJIT: Box the polymorphic tail of a type distribution
c9e6722195	1.067	422.4	ZJIT: Box the rarely-used profile side tables
5ba271156d	1.066	423.0	ZJIT: Stop over-reserving in long-lived metadata vectors
6129f47787	1.064	423.9	ZJIT: Pin the profiling metadata layout the memdiet merge depends on
4a162a7816	1.071	421.1	ZJIT: Free the payload of a GC-freed ISEQ and account dead-version bytes
cfea460e2f	1.077	418.7	ZJIT: Let invalidated ISEQs recompile instead of exiting forever
4455e638ef	1.087	415.0	ZJIT: Compile past polymorphic splatkw instead of side-exiting
b6072e749d	1.103	409.0	ZJIT: Recognize an unpassed block at unspecialized block-param sites
4f0c06814f	1.102	409.1	ZJIT: Initialize the block param local when a function stub exits
4c1e741e67	1.104	408.4	ZJIT: Compile exception handler entries (jit_exception)
480d6a3ba3	1.102	409.1	ZJIT: Materialize frames before pushing a caught break value
60e78ec08b	1.110	406.2	ZJIT: Re-enter JIT code for frames unwound back into after a throw
6821f4f286	1.102	409.0	ZJIT: Dispatch exception handler entries over several continuations
302657336c	1.100	410.1	ZJIT: Keep the profiled shape reachable through guards and dispatch arms
e001eb49c6	1.099	410.2	ZJIT: Branch on the ivar shape for attr_accessor sends in polymorphic arms
b59ea8efc7	1.083	416.4	YJIT row — base commit run with --yjit (vs base ZJIT)

Details

• Biggest single wins:
  • 6c9e6a68ee +1.7% — specializes the bar(...) inside a def foo(...) forwarder that was not itself inlined, which is the shape Rails delegation generates everywhere
  • b6072e749d +1.5% — a site whose block param is never passed a block no longer pays the generic block-param dispatch
  • ffd1019e36 +1.1% — the last arm of an ivar shape chain was a guard whose only miss target was a side exit, and the recompile it asked for was gated on a reprofile that on rubocop never happened
  • 57a28d54ab +1.0% — lets every JIT-to-JIT call site store its JITFrame pointer with the narrow form instead of the wide one
  • 4455e638ef +0.9% — a generic Insn::ToHash for splatkw, plus an InvalidationCause split so an invalidation grant is not spent on a respecialization request
  • bd9f496b4f +0.9% — moves side-exit stubs and other cold code out of the hot half, so a hot function's instructions sit closer together
  • 09ebf123c2 +0.9% — foo(&blk) no longer forces the dynamic send path
  • 3b2d0a7e8b +0.8% — a HasType feeding a CondBranch built an 89-byte diamond that materialized 0/1 into a block parameter and then re-tested it
• Biggest apparent local regressions:
  • f7a0a83372 −1.5% — lets an ivar dispatch compiled under no_side_exits earn back a shape its pre-invalidation profile never saw; sets up the +1.1% two commits later
  • e3e36fd91b −0.8% — one ancestor guard in place of a chain; the compile-time no-override walk it needs costs a pass over every class in the process
  • 3e2c13a5ab −0.7% — bounds the hot half at outlined_start — a correctness fix for the cold split, and the check is on the emit path
  • 6821f4f286 −0.7% — splits handler entry into continuations; more entry points to compile
  • 7d77398b9b −0.6% — shrinks the per-call-shape send tables, trading hit rate for memory
  • 2de98f9bf0 −0.6% — guards the profiled method name at send/__send__ sites
• Bug fixes in the branch, as opposed to optimizations:
  • f182afce8c — invokeblock profiled block handlers by object identity, so every IFUNC/Proc yield site looked megamorphic; also gated the ISEQ dispatch chain on profile coverage
  • 8d9005fe9d — vm_throw_start read a ZJIT-pushed frame's ISEQ via raw cfp->_iseq
  • 57a28d54ab — JITFrame pointers on the Rust heap need the wide store form; allocate them below INT32_MAX (also the largest single win here)
  • 8bc85404bc — checked-in ZJIT bindings still said VM_INSTRUCTION_SIZE = 259 after the profiled splatarray made it 260
  • 5aae4cfcf5 — YARVINSN_zjit_splatarray itself was never added, leaving every YARVINSN_zjit_* from 229 up one lower than the built VM
  • c124fa60d5 — the same staleness in YJIT's copy of the instruction table — asking YJIT for stats panicked
  • 37cabd0c65 — a &blk call site stored a callinfo describing an already-popped argument, so a later sendforward replayed the call wrong
  • e5e533c148 — gen_return() left the SP register stale when returning past inlined frames
  • f5bafc5aea — a half switch inside with_write_ptr would hand back an empty written range to remove_gc_offsets
  • 3e2c13a5ab — bounds were checked per region rather than per half, so a body that filled the inlined half overwrote exit stubs in the outlined half
  • 63d34ffe94 — three soundness defects in the ported coalesce_block_params() — this commit is the port with its follow-up fix folded in (see below)
  • 3303693b7e — resolve_ssa() broke spilled parallel-copy cycles through SCRATCH_REG, which scratch_split also stages memory-to-memory moves in
  • bcadbeadfb — an IseqPayload was never freed — 86 MB of dead profiles on an RDoc-over-stdlib run
  • 4a162a7816 — rb_zjit_iseq_free used the creating getter, allocating and leaking a fresh payload for every GC-freed ISEQ
  • 4f0c06814f — prepare_for_exit skipped the block-param local, leaving it uninitialized when a function stub exits to the interpreter
  • 480d6a3ba3 — vm_exec_handle_exception pushed the caught break value before materializing the frame, so the stack map overwrote it