perf(fastsac): fast async learner pipeline — batched drain, CUDA-IPC ring, fused updates - #74
Merged
Merged
Conversation
wlgys8
force-pushed
the
perf/fastsac-batched-async-drain
branch
from
September 23, 2026 03:37
f171ebb to
8dad42b
Compare
There was a problem hiding this comment.
Copilot review overview
🟡 Changes recommended
There are correctness issues in the new per-device GPU name extraction and in IPC-ring device matching semantics that can violate the “same GPU” assumption under common configurations.
Get a fresh assessment by requesting another Copilot review.
Review effort: Lite
Findings: 3
Open (3)
What changed in this PR
This PR accelerates FastSAC’s async trainer by optimizing learner-side ingestion and update execution (batched drains, CUDA-IPC transition ring, fused target updates) while expanding training-panel observability with richer per-device system metrics.
Changes:
- Added a transport layer supporting both host shared-memory and CUDA-IPC transition rings with contiguous-run (
read_span/commit_reads) ingestion and replay-bufferextend_batch. - Fused target-network soft updates into the compiled critic step and tightened CUDA-graph-safe metric ownership, with regression tests.
- Expanded system metrics and console panel UI to support per-core CPU spectrum and per-GPU device rows, while keeping overview aggregate-only.
| File | Description |
|---|---|
| wiki/design/fastsac-async-heterogeneous-trainer.md | Updated design doc to describe IPC ring transport and batched drain protocol. |
| scripts/bench_fastsac_collector_inference.py | Removed standalone collector inference benchmark script. |
| motrix_rl/tests/test_system_metrics.py | Added coverage for CPU model name/per-core stats and per-device GPU stats aggregation. |
| motrix_rl/tests/test_rl_sim_backend.py | Updated ring API usage from single-slot reads to read_span. |
| motrix_rl/tests/test_fastsac_learner.py | Added CUDA-graph regression ensuring returned metrics remain readable across later replays. |
| motrix_rl/tests/test_fastsac_ipc_ring.py | New correctness tests for CUDA-IPC transition ring protocol and learner drain path. |
| motrix_rl/tests/test_fastsac_collector.py | Updated imports and timing expectations (first-level env_step sub-stages only). |
| motrix_rl/tests/test_fastsac_buffer.py | Added tests for extend_batch wrap behavior and read_span FIFO/no-wrap contract. |
| motrix_rl/tests/test_console.py | Updated panel rendering API and added system-view tests (per-core/per-GPU details). |
| motrix_rl/src/motrix_rl/system_metrics.py | Added per-core CPU metrics, CPU model name, per-device GPU usage and device-name support. |
| motrix_rl/src/motrix_rl/fastsac/sync/train.py | Wired per-device GPU stats into training panel stats for sync trainer. |
| motrix_rl/src/motrix_rl/fastsac/config.py | Documented and added transition_ipc async option. |
| motrix_rl/src/motrix_rl/fastsac/buffer.py | Implemented extend_batch to ingest contiguous slot-major runs efficiently. |
| motrix_rl/src/motrix_rl/fastsac/async_impl/worker.py | Added IPC ring gating, IPC ring handshake slot shipping, and panel per-device GPU stats. |
| motrix_rl/src/motrix_rl/fastsac/async_impl/transport/weight_channel.py | Updated shared helpers import path to new transport package. |
| motrix_rl/src/motrix_rl/fastsac/async_impl/transport/ring.py | New shared-memory ring with cursor protocol + read_span/commit_reads. |
| motrix_rl/src/motrix_rl/fastsac/async_impl/transport/ipc_ring.py | New CUDA-IPC device-resident transition ring with event-ordered cursor publishing. |
| motrix_rl/src/motrix_rl/fastsac/async_impl/transport/common.py | Updated module references to transport package. |
| motrix_rl/src/motrix_rl/fastsac/async_impl/transport/__init__.py | Exposed new transport surface (IPC ring + cursors) and updated docs. |
| motrix_rl/src/motrix_rl/fastsac/async_impl/train.py | Selected host vs IPC ring in parent and updated transport wiring. |
| motrix_rl/src/motrix_rl/fastsac/async_impl/shm/ring.py | Removed old shm ring implementation (superseded by transport/ring.py). |
| motrix_rl/src/motrix_rl/fastsac/async_impl/learner.py | Implemented batched drain with pinned staging/copy stream and IPC-ring D2D ingest; added ingest synchronization for checkpoints. |
| motrix_rl/src/motrix_rl/fastsac/async_impl/collector.py | Updated to accept IPC ring transport and pruned timing stats to first-level sub-stages. |
| motrix_rl/src/motrix_rl/fastsac/agent.py | Fused target soft update into compiled critic step and adjusted metrics ownership for CUDA graph safety. |
| motrix_rl/src/motrix_rl/console.py | Added System view tab with per-core spectrum and per-GPU table; kept overview aggregate-only. |
| configs/algo_base/motrix.fastsac.yaml | Documented/added transition_ipc config option. |
💡 Add a code-review agent skill or configure MCP servers for context-aware, tailored reviews. Learn more in the docs.
wlgys8
force-pushed
the
perf/fastsac-batched-async-drain
branch
4 times, most recently
from
September 23, 2026 04:20
9707f69 to
1f22539
Compare
wlgys8
force-pushed
the
perf/fastsac-batched-async-drain
branch
from
September 23, 2026 04:28
1f22539 to
2b98e78
Compare
There was a problem hiding this comment.
Copilot review overview
🟡 Changes recommended
The collector/learner handshake currently assumes a new slot_queue payload shape and breaks existing in-repo callers unless backward-compatible decoding (or equivalent fixes) is applied.
Get a fresh assessment by requesting another Copilot review.
Review effort: Lite
Findings: 1
Open (2)
Resolved since last review (1)
…ring, fused updates Learner-side ingest and update-path performance work, plus training panel observability: * Batched async drain: contiguous ring runs (read_span/commit_reads) move through pinned staging as one non-blocking H2D per field on a dedicated copy stream that overlaps gradient updates; replay buffer gains extend_batch for slot-major multi-transition writes. * CUDA-IPC transition ring (transition_ipc auto/on/off): when collector inference and the learner share one GPU, slots live in one fused device tensor — the collector publishes each env-step batch with a single fused H2D, the learner ingests pure D2D strided copies. GPU writes are ordered behind per-slot CUDA events before cursor bumps; endpoint-local issued counters index slots (the published cursor lags in-flight copies). Host ring remains the fallback transport. * Fused learner updates: the target-net soft update moved into the compiled critic step (replays in the same CUDA graph); _own output clones are restricted to what outlives a graph generation, with the gated actor pair always owned (metrics must survive future replays — regression-tested). Multi-step compile blocks and module-only compile variants were benchmarked and rejected (torch.compile splits regions at backward/optimizer.step; per-step boundary is the optimum). * Timing stats report only first-level env_step sub-stages (panel and TensorBoard), removing duplicate child paths. * Training panel: per-GPU utilization/VRAM samplers (sample_per_device + sample_gpu_devices), a dedicated System tab (keyboard 1/2/3) with separate CPU and GPU blocks — aggregate CPU summary in the card title, per-core utilization spectrum (height-block glyphs with a connected ceiling cap; MOTRIX_PANEL_CPU_SPECTRUM selects the style for fonts that flatten block glyphs), per-GPU device table — while overview/timing and the plain-text fallback stay aggregate-only. Measured (microduck-walk-flat scale): learner drain 0.46-0.48 -> 0.14-0.16 ms/batch at 256 envs (3x), 1.86 -> 0.34 ms per 8-slot run at 2048 envs (5.5x); collector-side fused push costs +0.03 ms/batch.
wlgys8
force-pushed
the
perf/fastsac-batched-async-drain
branch
from
September 23, 2026 04:40
2b98e78 to
65587c2
Compare
There was a problem hiding this comment.
Copilot review overview
🔵 Needs a closer look
It introduces substantial new CUDA-IPC and cross-process ordering logic that warrants final validation by a human reviewer on representative multi-GPU hardware.
Review effort: Lite
Findings: None
Resolved since last review (2)
wlgys8
added a commit
that referenced
this pull request
Sep 23, 2026
…MA binding Multi-collector async FastSAC training on the post-#74 transport stack: - One SPSC ring + weight channel per collector (single-writer invariants unchanged); the learner merges shard slots into full num_envs batches by generation before GPU ingest, while the single-collector topology keeps the batched read_span / pinned-staging / CUDA-IPC fast paths. - DDP multi-learner option (num_learners, learner_devices): update counts derive from the global progress basis so ranks stay lockstep; rank 0 owns weight publishing, stats aggregation, logging and checkpointing. - NUMA placement is automatic: collectors spread round-robin over the host's nodes (multi-collector only), learners bind to their GPU's PCIe-local node via pynvml + sysfs (no CUDA context in the parent); no manual numa_nodes / learner_numa_nodes config. - cpus_per_collector chunks each collector's CPU slice from its node.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Learner-side ingest and update-path performance work, plus training
panel observability:
move through pinned staging as one non-blocking H2D per field on a
dedicated copy stream that overlaps gradient updates; replay buffer
gains extend_batch for slot-major multi-transition writes.
inference and the learner share one GPU, slots live in one fused
device tensor — the collector publishes each env-step batch with a
single fused H2D, the learner ingests pure D2D strided copies. GPU
writes are ordered behind per-slot CUDA events before cursor bumps;
endpoint-local issued counters index slots (the published cursor lags
in-flight copies). Host ring remains the fallback transport.
compiled critic step (replays in the same CUDA graph); _own output
clones are restricted to what outlives a graph generation, with the
gated actor pair always owned (metrics must survive future replays —
regression-tested). Multi-step compile blocks and module-only compile
variants were benchmarked and rejected (torch.compile splits regions
at backward/optimizer.step; per-step boundary is the optimum).
TensorBoard), removing duplicate child paths.
separate CPU and GPU blocks — aggregate CPU summary in the card
title, per-core utilization spectrum (height-block glyphs with a
connected ceiling cap; MOTRIX_PANEL_CPU_SPECTRUM selects the style
for fonts that flatten block glyphs), per-GPU device table — while
overview/timing and the plain-text fallback stay aggregate-only.
Measured (microduck-walk-flat scale): learner drain 0.46-0.48 ->
0.14-0.16 ms/batch at 256 envs (3x), 1.86 -> 0.34 ms per 8-slot run at
2048 envs (5.5x); collector-side fused push costs +0.03 ms/batch.