A system substrate for building embodied intelligence across heterogeneous robots.
robonix.ai · Documentation · Package catalog · Quick start
Robonix is a general-purpose agentic operating system designed for embodied artificial intelligence. It employs large language models (LLMs) and vision-language models (VLMs) to convert natural-language tasks into Robot Task Description Language (RTDL) programs, which are then executed to complete the specified tasks. The primary advantages of Robonix include the following:
- (1) Hardware–software decoupling. Robonix utilizes layered capability abstractions (Task, Skill, Service, Primitive) to separate software functionality from robot-specific hardware. These abstractions establish standard interface definition language (IDL) interfaces, enabling the reuse of compatible primitives, services, and skills across different robotic platforms.
- (2) Adaptability to diverse user tasks, robot capabilities, and environments. Robonix leverages LLMs and VLMs to generate RTDL programs tailored to each user's task, the robot's capabilities, registered skills, environmental conditions, and body state. This approach enables Robonix to accommodate a wide range of natural language tasks, robotic hardware, and operational environments.
- (3) Observable and verifiable execution. Robonix explicitly represents the model's program in RTDL, allowing for inspection of its structure. The system performs program validation prior to execution and can verify specified outcomes upon completion.
Figure 1. The architecture of Robonix.
Figure 1 illustrates the overall architecture of Robonix. The user interacts with Liaison, which verifies the user with Keystone. Liaison delegates the task to Pilot. The model orchestrates skills and services for the task by writing an RTDL program; Pilot checks it and forwards it to Executor. Sentinel monitors Executor, while Scene keeps the semantic map (objects and their relations) and the spatial map of the outer world. Custom services sit beside Scene: navigation, SLAM mapping, object detection, grasp-pose estimation, motion planning, memory, and more from the package catalog. Atlas, Nexus, Chronos, Scribe, and Vitals provide the capability catalog, messaging, time, logging, and health that the rest build on. At the bottom, Soma is the body abstraction. A URDF gives the body's geometry and coordinate frames; a YAML body description gives its component tree (base, arm, gripper, camera) and the primitive capabilities each component provides. On the right, the skill library holds skills such as picking, handing over an item, pouring, and folding, often vision-language-action (VLA) models; skills can be learned in a simulator such as Webots, MuJoCo, or Isaac Sim and then deployed to real robots through Robonix. Where it meets Soma sit the preset skills the vendor ships with the body, such as a dance or a backflip.
| Abstraction | Definition |
|---|---|
| primitive | A single hardware function — sensing or actuation — behind a software interface. |
| service | Reusable software that implements a Robonix interface, such as planning, execution, interaction and scene management. |
| skill | A learned model or an algorithmic procedure, packaged as a reusable executable unit. |
| task | A goal, its constraints, and the conditions that count as completion. |
Each primitive, service, and skill specifies its offerings in a versioned contract and registers with Atlas at startup. Callers bind to the contract rather than the underlying implementation, enabling task transfer to another body and allowing implementations to be replaced without modifying callers.
The System Architecture section lists every component.
Install these from their own documentation first — Robonix does not provide or install them for you.
| Tool | Why it is needed | Install |
|---|---|---|
| Rust (stable) | Most components are Rust; make install builds them with cargo |
rustup.rs |
| uv | Resolves and runs the Python workspace — services and primitives | docs.astral.sh/uv |
| Docker | Runs the Webots simulator stack, and any capability provider you choose to containerise | docs.docker.com |
| NVIDIA Container Toolkit | Only on a machine with an NVIDIA GPU: Scene then runs its detector in a container with --gpus all, which fails without it |
docs.nvidia.com |
| System packages | A C toolchain, pkg-config and OpenSSL headers for the Rust build, and ALSA's arecord/aplay for the audio driver |
sudo apt install build-essential pkg-config libssl-dev git curl alsa-utils |
Rust and uv install into your home directory, so put them on PATH before continuing:
export PATH="$HOME/.local/bin:$HOME/.cargo/bin:$PATH"git clone --recursive https://github.com/syswonder/robonix.git
cd robonix
make installThis builds the Rust components and the rbnx command-line interface (CLI) into ~/.cargo/bin. See Host Platforms for what is regularly tested.
Start the Webots simulator in one terminal:
export DISPLAY=:0
bash examples/webots/sim/start.shBoot Robonix in a second terminal with any OpenAI-compatible VLM endpoint:
export RMW_IMPLEMENTATION=rmw_zenoh_cpp
export VLM_BASE_URL=https://api.openai.com/v1
export VLM_API_KEY=sk-...
export VLM_MODEL=your-model-name
cd examples/webots
rbnx build
rbnx bootThen run rbnx chat in a third terminal. Try go to room 101, what can you see?, or explore the office. See the Getting Started guide for the complete walkthrough.
The Robonix package template contains a mock primitive, a service, and a skill that boot without robot hardware:
git clone https://github.com/syswonder/template-rbnx.git
cd template-rbnx
cp .env.example .env
# Fill in the three VLM values in .env.
set -a; source .env; set +a
rbnx build
rbnx bootRun rbnx caps to inspect the live providers, then try rbnx chat and ask the robot to say hello. Each example package keeps its manifest, config.spec, build/start scripts, implementation, and optional capability definitions in one directory. Start there, then follow the package integration guide to publish a reusable package.
Robot bodies published to the catalog, with more on the way: wheeled, tracked, and quadruped bases, fixed and dual arms, standalone dexterous hands, a humanoid, and four simulated bodies. They span several vendors' chassis software development kits (SDKs), both versions of the Robot Operating System (ROS 1 and ROS 2), and both grippers and five-finger hands, while running the same components, capability contracts, and skills.
Per-robot hardware and links
| Robot | Hardware | Maintainer | Links |
|---|---|---|---|
| AgileX Ranger Mini v3 | Ranger Mini v3 chassis; Livox MID-360 lidar and IMU; RealSense D435i RGB-D; optional Piper arm; audio | syswonder | repo · catalog |
| DEEP Robotics Lite3 | Lite3 quadruped; Livox MID-360 lidar and IMU; Orbbec Gemini 330 RGB-D | Bunnycxk | repo · catalog |
| DEEP Robotics Lynx S10 | Lynx S10 wheeled-quadruped over UDP; Orbbec Gemini 336L RGB-D; InternVLA navigation | 1mujue | repo · catalog |
| Unitree Go2 | Go2 quadruped; onboard lidar, camera, IMU; audio bridge | Origamii520 | repo · catalog |
| Yobotics Y20W | Y20W chassis and posture; Livox MID-360 lidar; RealSense D435i RGB-D; speech | chenx1118 | repo · catalog |
| MirrorMe BPX | BPX quadruped; odometry; guarded stand and sit posture control | nonkr | repo · catalog |
| WHEELTEC R550 | R550 tracked chassis and IMU; LSLIDAR N10P; Orbbec Astra S RGB-D | sherry-part | repo · catalog |
| Yahboom ROSMASTER X3 | Mecanum chassis on Jetson TX2 NX; RPLidar; guarded ROS 1 navigation | luoyg0831-a11y | repo · catalog |
| Hantewin Benben | Benben chassis; Livox MID-360 and LSLIDAR LakiBeam1; RealSense camera; audio | Futaba19-c | repo · catalog |
| BeingBeyond D1 | Fixed-base 6-DOF arm; 2-DOF head; five-finger hand; head RGB-D; pick, place, stack, sort | Ciliphen | repo · catalog |
| AgileX Dual Piper | Two Piper arms and CAN grippers on separate buses; per-arm joint telemetry | syswonder | repo · catalog |
| WowRobo Roboarm | Five-axis LeRobot Koch arm; Orbbec Gemini 215 RGB-D; audio | gaoyz1235 | repo · catalog |
| AUBO i5H | i5H six-axis collaborative arm over the official AUBO SDK; wave skill | gaoyz1235 | repo · catalog |
| LinkerHand O6 | Standalone six-axis five-finger hand over CAN; gesture and finger-motion skills | Ciliphen | repo · catalog |
| Webots TIAGo Lite (sim) | Simulated differential-drive base; head RGB-D; Hokuyo lidar; audio | syswonder | repo · catalog |
| Minecraft Bot (sim) | Minecraft player body; camera, chassis, world state, inventory, navigation | ZZJJWarth | repo · catalog |
| Ranger + Piper (sim) | MuJoCo Ranger Mini v3 and Piper; web or native viewer; mapping, navigation, manipulation | syswonder | repo · catalog |
| Unitree Go2 (sim) | MuJoCo Go2 quadruped; web or native viewer; exploration and semantic navigation | syswonder | repo · catalog |
Each deployment links the complete robot manifest and its primitive, service, and skill dependencies. Published deployment metadata does not replace the hardware-specific safety, commissioning, and acceptance gates documented by each repository. See the robot catalog for published integrations.
Every robot above is assembled from packages, not written as one program. A package declares the capabilities it provides against the shared contracts, so what it offers does not depend on which body it was written for, and a deployment can swap one implementation for another without the layers above noticing.
Browse them in the package catalog, or publish your own with the package integration guide.
The Rust components and the Python packages are written to stay portable across architectures; x86-64, arm64 (NVIDIA Jetson), and LoongArch64 are tested.
| Arch | OS / Distribution | Status |
|---|---|---|
| x86_64 | Ubuntu 22.04 | ✅ Tested |
| x86_64 | Debian 13 | ✅ Tested |
| arm64 | NVIDIA Jetson — JetPack 6.2 (L4T 36.4.3, Ubuntu 22.04) | ✅ Tested |
| LoongArch64 | Loongson 3A6000 + AMD Radeon 7900 XTX — Loong ArchLinux 2026.08.07 | ✅ Tested |
| x86_64 / arm64 | Ubuntu 24.04 and newer | 🚧 Planned |
Note (LoongArch64): Robonix itself runs on the Loongson 3A6000 host, while the simulation platform (Webots) runs on a separate x86_64 machine with Ubuntu 22.04. The two machines are connected over Ethernet on the same local area network (LAN).
"Tested" means the full Robonix pipeline runs end-to-end on that platform — in simulation or on a real robot: voice & interaction, task execution, body movement, scene & mapping (semantic map + spatial map), navigation, and skill execution. Other Linux distributions will likely work but are not regularly verified.
Capability providers that use ROS 2 are built and tested against ROS 2 Humble.
Robonix ships the components below. Deployments add custom services from the package catalog, such as navigation, SLAM mapping, object detection, grasp-pose estimation, motion planning with MoveIt, and memory.
| Component | Responsibility |
|---|---|
| liaison | User input — text and voice |
| keystone | Identity, configuration, and access policy |
| pilot | Turns a task into an RTDL program with the model, and validates it |
| executor | Runs the program, dispatching each call to its provider |
| sentinel | Decides whether a capability call is allowed |
| scene | The environment: a semantic map of objects and their relations, and a spatial map |
| atlas | Capability catalog — every running provider and its contract |
| nexus | Communication over gRPC, the Model Context Protocol (MCP), and ROS 2 |
| chronos | One time source across sensors, actuators, and components |
| scribe | Structured logging |
| vitals | Onboard health: temperatures, voltage, joint motors |
| soma | Body abstraction — URDF for geometry and frames, a YAML component tree with each component's primitive capabilities, and the body's runtime state |
Preset skills, such as a dance or a backflip, ship with a robot body. The skill library holds the rest, such as picking, pouring, and folding. Soma launches deployed skills, Pilot discovers them through Atlas, and Executor invokes them.
Every contract lives in capabilities/, whether a service or a package implements it. Primitives and skills live in their own repositories. Currently, all 12 components are under the system directory for historical reasons.
Full documentation lives at book.robonix.ai.
Getting started
- Quickstart — the full version of the Webots walkthrough in this README
- Host Platforms — what is tested, and what is not
Understanding the system
- Architecture overview — the control plane, and one full request end to end
- Namespaces & contracts — how
robonix/primitive/*,robonix/service/*,robonix/skill/*, androbonix/system/*relate - Interface catalog — every primitive and service contract, generated from
capabilities/
Building on it
- Package integration guide — write a package and publish it to the catalog
- Package catalog — every published package, browsable by kind
- Robot catalog — every published deployment, with its full dependency tree
Release history lives in CHANGELOG.md, formatted per Keep a Changelog. Contributors add entries under ## [Unreleased]; they are moved into a versioned section at release time.
See CONTRIBUTING.md for the repository's license headers, code style, validation commands, commit format, human-authorship policy, and AI assistance disclosure rules.
Mulan Permissive Software License, Version 2 (MulanPSL-2.0). See LICENSE.