Publish-ready main: measured results, contribution setup, and honest scope

[Shaurya-M002]

What this changes

Brings dev into main as the first publish-ready state: measured benchmark results for the two kernels that had none, a tidied tree, a contribution setup, and two scope claims narrowed to what is actually measured.

Numbers

Fused linear cross-entropy, the kernel whose harness had never been executed, on an idle A100-80GB, bf16, h=4096, v=128256, full fwd+bwd:

BT	latency before	after	vs eager	peak before	after	eager
1024	262.1 ms	15.2 ms	0.96x	4024 MB	1269 MB	1260 MB
8192	321.7 ms	113.1 ms	1.02x	4135 MB	2085 MB	6012 MB

Two causes, both in the chunk loop: a per-chunk (V, H) fp32 temporary that made chunking buy no memory at all, and a BT/(V/H) chunk rule that re-read the weight 32 times per pass. Result files are committed under kernels/cross_entropy/benchmarks/results/.

forge.patch at inference, real Qwen2.5-0.5B weights, one kernel at a time — correct but slower, so the scope is now stated plainly in the README:

patched	prefill	vs eager
nothing	31.81 ms	1.00x
rmsnorm	34.80 ms	0.91x
swiglu	35.20 ms	0.90x
everything	37.44 ms	0.85x

• GPU was idle when measured (nvidia-smi), device pinned with CUDA_VISIBLE_DEVICES
• Result CSV/JSON committed

Correctness

• uv run ruff check . — green, with a narrow rule set configured rather than reformatting ~700 inherited findings
• 660 tests pass, 4 skip, 1 xfail, 0 fail, each suite in its own process

Both documented pytest commands were broken and now work. pytest tests/ could not be collected at all (9 errors) because tests/layernorm/ is a package, so pytest put tests/ on sys.path instead of the repo root. Fixing that surfaced 4 real failures in the LayerNorm dW comparison, which turned out to be tolerance rather than gradient: against a float64 reference the kernel is closer to the truth than eager (2.16e-05 vs 2.92e-05), so a flat atol=1e-5 was failing the more accurate result.

Anything a reviewer should know

• Nothing was renamed or rewritten. The pre-cleanup tree is preserved on hackathon-archive.
• Corrected two wrong entries in the correctness table: LoRA QKV was listed at 18 tests when it has 90, and GeGLU as having no committed run when its 89 tests pass.
• The forge package is not installable today — the built wheel contains zero kernels and import forge fails outside a checkout. Filed as a blocker; not addressed here.

Made with Cursor

[Shaurya-M002]

Added 7e3fa10, which corrects a claim made earlier in this PR.

The README section titled "This is a training library" said forge.patch helps training and costs 15% at inference. That was measured at batch 1 and generalised too far. Bisecting across shapes shows the deciding variable is activation elements per call (rows × width), not the training/inference distinction:

Mode	batch × seq	elements/call	vs eager
infer	1 × 128	0.11 M	0.85×
infer	1 × 1024	0.92 M	0.87×
infer	4 × 1024	3.67 M	1.12×
train	4 × 1024	3.67 M	1.20×

Inference at batch 4 is 1.12× faster. Below ~1024 rows a 0.5B model is launch-bound rather than compute-bound — eager needs the same 32 ms for 128 tokens as for 1024 — so 73 extra kernel launches can only cost. Training also gains 36% peak memory (21.9 GB → 14.1 GB), nearly all of it from fused linear cross-entropy: Qwen2.5-0.5B pairs an 896-wide hidden state with a 151936-token vocabulary, so not materialising the logits saves 6.2 GB on its own.

forge.patch now dispatches per call against MIN_FUSED_ELEMENTS, calibrated on measurements that hold across a 4.5× change in width. Worst case moves 0.85× → 0.98×; the wins are unchanged.

Also in the commit:

• mode="infer" skips the loss and LoRA kernels, recording why in model._forge_skipped.
• RoPE declines head_dim values it cannot handle instead of raising a bare AssertionError inside attention (ForgeRoPEv3 requires a power-of-two head_dim, excluding models with head_dim 80 or 96 #18).
• forge/tests/test_lora_mlp.py went 0/8 → 8/8: stale positional args put a bool in a tensor slot, so the kernel had not actually been checked against its reference (LoRAMLPv6.apply has 17 positional parameters and two call sites have already silently broken #19).
• forge/tests/test_shape_guard.py, 7 tests, including that a declined call is bitwise identical to unpatched.
• forge/benchmarks/bench_patch_bisect.py, the tool the threshold came from.

Verification after the change, all on an idle A100: test_shape_guard 7/7, test_lora_mlp 8/8, test_lora_qkv 12/12, test_embedding_padding_idx PASS, verify_lora_qwen_patch and verify_qwen_patch_backward pass with unpatch restoring, and verify_fsdp2_lora_qwen on 2 GPUs passes 7/7 with 5-step loss matching the single-GPU reference to 9.2e-05. Closes #9.