One dense reasoning core. Six capabilities. Zero architectural exotics.
A 25.89B dense multimodal reasoning foundation model engineered around deep reasoning · coding · tool calling · long-horizon agent · image & video understanding — maximizing intelligence per compute, not parameter count.
This is an engineering specification, not marketing. Every claim is tagged, every number is computed, and where evidence is insufficient the docs say "not enough data to conclude."
Most frontier models buy capabilities with architectural exotics and opaque training recipes. NEXUS-R1D takes the opposite bet: one modern dense backbone, trained and controlled with reasoning-first discipline.
| Capability | How it's actually achieved |
|---|---|
| 🧠 Deep reasoning | Channel-think-then-act with adaptive compute budgets (0.5K–32K tokens, hard-capped) + trained verification + wrong→detect→correct→recover data |
| 💻 Coding as execution | Nothing is "done" until it compiles, tests pass, behavior matches — RL rewards are execution-grounded |
| 🔧 Native tool calling | JSON-schema-validated, dependency-aware parallel/sequential use with error-recovery budgets — a trained skill, not JSON prompting |
| 🤖 Long-horizon agent | 9-state loop with explicit goal/plan/state memory + expected-vs-actual completion evidence (no false completion, ever) |
| 🖼️ Vision | Dynamic-resolution ViT with tiered token budgets (256 → 4096 visual tokens) |
| 🎬 Video as time | Timestamp-grounded temporal reasoning — 2 fps, 144 tok/frame, temporal tools. Never "all frames into attention" |
One reasoning core. Every modality projects into the same backbone — there is no separate "vision model" or "reasoning model."
flowchart LR
subgraph inputs[" "]
T["📝 Text<br/>131K vocab"]
I["🖼️ Image<br/>any res / 28px"]
V["🎬 Video<br/>2fps + timestamps"]
A["🔊 Audio<br/>16kHz log-Mel"]
end
I --> VE
V --> VE
subgraph towers["Multimodal Towers · 870M"]
VE["VisionEncoder<br/>403M · 24L · window 4:1 global<br/>2×2 patch-merge"]
AE["AudioEncoder<br/>315M · 12L · Whisper-style"]
end
T --> EMB
VE -->|"<b>MultimodalProjector</b><br/>2-layer gated MLP"| EMB
AE -->|"<b>MultimodalProjector</b>"| EMB
EMB["Embeddings<br/>6144 · untied LM head"]
EMB --> BB
subgraph core["The ONLY reasoner · 25.02B dense backbone"]
BB["60 × pre-norm blocks<br/>GQA 48:8 · QK-Norm · RoPE θ=500K + YaRN×8<br/>SwiGLU 6144→16384 · RMSNorm<br/>BF16 train / FP8 GEMM · FlashAttention"]
end
BB --> OUT["Channel-split output<br/>think → act → verify → answer"]
subgraph ctrl["Behavior controllers <i>(enforce, not decide)</i>"]
RC["ReasoningController<br/>budgets · rounds · repetition"]
TI["ToolInterface<br/>schema-validate · timeout · recovery"]
AG["Agent 9-state loop<br/>no-false-completion"]
MC["MemoryController<br/>Keep/Compress/Discard/Retrieve"]
end
OUT -.-> RC
RC -.-> TI
TI -.-> AG
AG -.-> MC
Parameter Count 25.89B total (25.02B backbone + 0.87B multimodal)
Layers 60 · hidden 6144 · SwiGLU 16384
Attention GQA 48:8 (6:1, head dim 128) + QK-Norm
Positional RoPE θ=500K + YaRN ×8 (32K native → 256K served)
Vocabulary 131,072 · untied embedding + LM head
Vision Encoder SigLIP2-NAFlex-style ViT, 403M, 4:1 window:global
Video per-frame ViT @2fps → 144 tokens/frame + temporal tools
Audio Encoder Whisper-style, 315M, 12L, 16kHz log-Mel
Temporal Compressor 67M — OFF in 1.0 (identity; 1.1-gated on ablation A-9)
Reasoning channel-think-then-act · adaptive budgets · in-loop verify
Precision BF16 + FP8 GEMM (train) · BF16/FP8/INT8/INT4 (serve)
All computed by python3 nexus_r1d/config.py — no invented figures:
| Quantity | Value |
|---|---|
| Total / backbone / multimodal params | 25.886B / 25.016B / 870.0M |
| — vision tower · audio tower · projectors | 403.3M · 314.7M · 84.9M |
| Pretraining corpus / compute | 15T tokens · 2.33e24 FLOPs |
| KV cache | 240 KiB/token (BF16) → 60 GiB @ 256K ctx |
| Decode cost | ~46.8 GFLOP/token (GEMM path) |
| Training cluster | 512×H200 · TP8×PP8×DP8 · ~92 days |
| Serving tiers | 8×H200 BF16 · 1×H200 INT4 |
| Video 120s clip budget | 170 frames · 24,480 visual tokens · chunked |
Why 26B @ 15T: Chinchilla-optimal (~500B) is far from best for reasoning post-training [EVIDENCE: R1/Qwen3-class over-training]. At fixed ≈2.33e24 FLOPs, 26B @ 15T (~600 tok/param) sits in the empirically strongest region for RL-scaling while remaining single-node-servable in INT4. It is the best-supported point, not a claimed optimum — iso-compute ablations (A-13) are mandated.
├── README.md ← you are here
├── configs/
│ └── nexus_r1d_1.0.json ← canonical machine-readable config
├── nexus_r1d/ ← PyTorch skeleton — importable & smoke-tested
│ ├── config.py ← single source of truth · prints all param math
│ ├── layers.py ← RMSNorm · QK-Norm · YaRN-RoPE · GQA · SwiGLU
│ ├── llm.py ← 60-layer dense backbone (KV-cache correct)
│ ├── vision.py ← ViT tower (windowed/global, patch-merge)
│ ├── audio.py ← Whisper-style encoder tower
│ ├── video.py ← sampler · temporal compressor (off) · budgets
│ ├── model.py ← NexusR1D: towers + projectors + one backbone
│ ├── reasoning.py ← ReasoningController (budgets/rounds) + Verifier
│ ├── tools.py ← registry · schema validation · dispatch · policy
│ ├── memory.py ← TaskRecord + Keep/Compress/Discard/Retrieve
│ ├── agent.py ← 9-state agent loop · no-false-completion
│ └── api.py ← chat/generate/reason/tool/vision/agent/embeddings
├── docs/ ← 7-part engineering specification
├── run_tiny.py ← live probe: Qwen3-0.6B through the NEXUS loop
├── run_iot.py ← INT8 edge bench: adaptive CoT + tools
├── run_vision.py ← vision probe: SmolVLM-256M OCR/chart reading
├── test_smoke.py ← 15 deterministic tests · every tensor path
└── demos/ ← task files + ground-truth images
# 1. The full parameter accounting (works everywhere, ~1s)
python3 nexus_r1d/config.pyNEXUS-R1D 1.0 | total 25.886B (backbone 25.016B + multimodal 870.0M) | L=60 d=6144
heads=48/8 hd=128 ffn=16384 vocab=131072 ctx=262144
pretrain FLOPs (15T tok): 2.330e+24
KV cache/token BF16: 240.0 KiB
decode FLOPs/token: 46.8 GFLOP
# 2. Video token budget policy (controllers are behavior-complete)
python3 -c "
import sys; sys.path.insert(0, '.')
from nexus_r1d.config import NexusR1DConfig
from nexus_r1d.video import VideoController
print(VideoController(NexusR1DConfig()).budget(120.0))"{'frames': 170, 'visual_tokens': 24480, 'chunked': True}
# 3. Full test suite — every tensor path verified
python3 test_smoke.py[config] PASS t_config
[layers/rope] PASS t_rope_shapes · t_rope_highdim_matches_halfdim
[backbone] PASS t_backbone_forward · t_backbone_incremental_decode · t_backbone_padding_mask
[vision] PASS t_vision
[audio] PASS t_audio · t_audio_pos_broadcast
[model fusion] PASS t_model_fusion
[video] PASS t_video_controller
[reasoning] PASS t_reasoning
[tools] PASS t_tools
[memory/agent] PASS t_memory_agent
[runners] PASS t_run_tiny_toolflow
============================================================
PASS 15 / FAIL 0
The 26B backbone is substituted by an open tiny model so the real NEXUS loop — reasoning → tool → verify → answer — actually runs on a laptop:
pip install torch transformers pillow
# Reasoning + tool calling + error recovery (Qwen3-0.6B)
python3 run_tiny.py --task demos/tasks.json[task math-tool] What is (17 + 23) * 7? Use the calculator tool.
answer: 280
tool: calculator({"expression": "(17 + 23) * 7"}) -> success
[task error-recovery] read_file 'does_not_exist.txt'
answer: The file 'does_not_exist.txt' does not exist...
tool: read_file({...}) -> error ← §25 recovery, no blind retry
# INT8 edge deployment: difficulty-routed compute budgets
python3 run_iot.py --bf16-baseline
# Vision: OCR + chart understanding against ground truth
python3 run_vision.pydemos/vision_ocr.png PASS 'NEXUS-R1D 1.0 / 25.89B parameters'
demos/vision_chart.png PASS 'Benchmark Scores: Math, Code, Agent, Vision'
test_smoke.py asserts the properties that actually matter — not just "runs without error":
| Test | Property proven |
|---|---|
t_backbone_incremental_decode |
Cached decode == teacher-forced forward, bit-comparable — the RoPE offset + causal mask are position-exact |
t_rope_highdim_matches_halfdim |
RoPE emits exactly D/2 frequencies θ^(-i/dim) matching the rotate-half convention (the old code silently emitted D/4) |
t_model_fusion |
Projected modality tokens really enter the sequence at explicit positions |
t_reasoning |
§82 no-false-completion: an unverified "done" claim is fail, never pass |
t_tools |
Timeout, exception, unknown-tool, and schema-validation paths all return structured ToolResults |
t_video_controller |
120s → 170 frames / 24,480 tokens / chunked; compressor carries zero dead parameters while disabled |
- No architectural exotics without multi-lab evidence — the rejected-components table (docs/01 §1) names what did not ship and what evidence would unblock it.
- Reasoning quality over reasoning length — correctness-vs-tokens Pareto is a release benchmark; overthinking is a regression.
- Tool calling is a skill, not serialization — validated schemas, dependency-aware parallelism, recovery budgets, hallucinated tools hard-rejected pre-execution.
- Video is temporal, not frames — timestamps are first-class; evidence-grounded answers cite them.
- Code is done when it executes — training data, RL rewards, and the agent state machine all enforce it.
- No capability claims before measurement — every target carries uncertainty; every benchmark table is a plan.
- "Not enough data to conclude" is a valid answer — used where it is the truth (visual compression, temporal compressor, learned memory).
Every statement in the spec is tagged — and the tags mean something:
- [FACT] — established, mechanically verifiable
- [EVIDENCE] — supported by published benchmark evidence (cited)
- [HYPOTHESIS] — unproven; must be ablated before shipping
- [TRADE-OFF] — explicit capability/cost judgment with both sides stated
No benchmark numbers are invented anywhere. Targets are labeled targets; hypotheses are labeled hypotheses.
| Version | Scope | Gate |
|---|---|---|
| 1.0 | baseline production architecture | — |
| 1.1 | training/reasoning upgrades — temporal compressor, visual compression, distilled verifier, learned memory | each gated on its ablation (A-9, A-8, A-5, A-10) |
| 1.2 | tool/agent upgrades — adaptive video control, SSD-paged 1M-context exploration | 1.1 measured |
| 1.3 | vision/video + hardware stacks | 1.2 measured |
| 2.0 | architecture change | only if 1.x is beaten by ≥5pt aggregate at ≤1.15× FLOPs |
Newer is not better until measured.
| Document | Contents |
|---|---|
| docs/01-architecture.md | scaling decision · math spec · parameter accounting · attention/FFN/RoPE |
| docs/02-reasoning-coding-tools-agent.md | reasoning channels · coding-as-execution · tool grammar · agent loop |
| docs/03-multimodal.md | vision · video · audio · documents · fusion strategy |
| docs/04-training.md | data · pretraining · SFT · RL · curriculum |
| docs/05-inference-hardware.md | inference · quantization · distributed · FLOPs/VRAM/latency |
| docs/06-evaluation.md | benchmarks · ablation program · failure analysis |
| docs/07-api-deployment-roadmap.md | API · deployment · limitations · roadmap |
- Skeleton + tests: Python 3.10+, PyTorch 2.x — nothing else
- Live probes:
transformers(v5+),pillow; models auto-download on first run (~0.6–2GB) - Everything runs on CPU — no GPU required for any path in this repo
NEXUS-R1D 1.0 — engineering specification, not marketing.
Every number computed. Every claim tagged. Every capability gated on evidence.