Skip to content
Draft
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
3 changes: 3 additions & 0 deletions .gitignore
Original file line number Diff line number Diff line change
Expand Up @@ -54,3 +54,6 @@ ge_check_op.json
# Local tutorial environments and results
.reference/
results/

# XR-1 local runtime environment
.venv-xr1*/
3 changes: 3 additions & 0 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -880,6 +880,8 @@ and the expected results. Build times, FA2 slim-build flags and ccache are in

## Supported Models

- **Xiaomi-Robotics-1** (`config="xr1"`) — gfx1151 BF16 reference and native HIP paths for the published 5B and RoboCasa checkpoints; [installation, inference, and validation](docs/xr1_gfx1151.md).

- **Pi0.5** (`config="pi05"`) — [quickstart](#quick-start), [API reference](USAGE.md#api-reference), [NVFP4 notes](USAGE.md#nvfp4-pi05-only), [Thor example](examples/thor/README.md), [RTX 5090 example](examples/blackwell/README.md)
- **Pi0** (`config="pi0"`) — [API snippets](#api-snippets), [usage guide](USAGE.md#api-reference)
- **GROOT N1.6** (`config="groot"`) — [API snippets](#api-snippets), [GROOT embodiment slots](#groot-n16-embodiment-slots)
Expand Down Expand Up @@ -916,6 +918,7 @@ have been built and tested, and the dispatch map that selects them.
|---|---:|---|---|
| Jetson AGX Thor | SM110 | Production target | Pi0, Pi0.5, GROOT N1.6 and N1.7, Pi0-FAST, LingBot-VLA, Hy-Embodied-0.5-VLA, Alpamayo 1.5 (full request and FlashDrive streaming), Cosmos3-Edge AV / Reasoner, Qwen3.6, and Qwen3-VL BF16 with opt-in W8/W4 decode; reproduction guides for [π0.5](docs/thor/pi05.md) and [GR00T N1.7](docs/thor/groot-n17.md). |
| RTX 5090 | SM120 | Production target | Pi0/Pi0.5/GROOT/Pi0-FAST RTX paths, Qwen3.6, Qwen3-8B, Qwen3-VL, Higgs Audio v3 FP8, Motus, Wan2.2, Cosmos3-Nano, and HF Kernel Hub package validation; see [RTX 5090 latency](examples/blackwell/README.md#vla-latency-rtx-5090). |
| AMD Radeon / Ryzen AI Max+ | gfx1151 (RDNA 3.5) | Supported | Pi0.5 BF16 and XR-1 5B/RoboCasa BF16 inference; [XR-1 deployment](docs/xr1_gfx1151.md). |
| AMD Instinct MI350X | gfx950 (CDNA4) | Supported | Pi0.5 at **16.4 ms** FP8 and GROOT N1.7 at **16.0 ms** full-frame, on a standalone ROCm/HIP backend: hand-written HIP kernels, hipBLASLt FP8 GEMM, MFMA attention, HIP graph capture. Uses the gfx950 source set, checked at build and runtime. See [AMD deployment](docs/deployment_amd.md) and [Pi0.5 on MI350X](docs/deployment_amd_pi05.md). |
| AMD Instinct MI300X | gfx942 (CDNA3) | Supported | Architecture-specific BF16/FP8 FNUZ kernels and HIP graphs. See [AMD deployment](docs/deployment_amd.md), [π0.5](docs/deployment_amd_pi05.md), and [GR00T N1.7](docs/deployment_amd_groot_n17.md). |
| Ascend 910B4 | Ascend NPU | Supported | Pi0.5 and GR00T N1.7 backend with native AscendC kernels and NPU graphs. See [π0.5 deployment](docs/deployment_npu.md) and [GR00T N1.7 deployment](docs/deployment_npu_groot_n17.md). |
Expand Down
22 changes: 21 additions & 1 deletion docs/stable_api.md
Original file line number Diff line number Diff line change
Expand Up @@ -64,6 +64,8 @@ def load_model(
use_fp16: bool = False,
# Robot output schema (RDNA Pi0.5 / Jetson Pi):
action_dim: int | None = None,
# XR-1 torch/gfx1151 opt-in, keyword-only:
use_graph: bool = False,
) -> VLAModel
```

Expand All @@ -77,7 +79,25 @@ Returns a `VLAModel` wrapping the appropriate frontend for the detected
keep their output conventions.
- `decode_cuda_graph`, `decode_graph_steps`, `max_decode_steps` apply to
Pi0-FAST.
- `embodiment_tag` and `action_horizon` apply to GROOT.
- `embodiment_tag` and `action_horizon` apply to GROOT. XR-1 also uses
`embodiment_tag` for the robot normalization entry; its action horizon is
checkpoint-defined.
- `config="xr1"` supports torch on `hardware="amd_rdna35"` (gfx1151).
Pass three views and robot state. `attention="sdpa"` (default) or `"eager"`
selects VLM attention; `fuse=True` enables the native HIP adapters and
`fuse=False` is the reference default. `num_steps` defaults to five.
`action_dim` defaults to 7 for RoboCasa and 60 for the published 5B policy.
`use_graph=True` captures the complete GPU vision/VLM/DiT forward on first
prediction; it supports both reference and native modes. The default remains
uncaptured. Each frontend keeps one profile and rebuilds it when prompt tokens,
image layout, tensor shapes, embodiment, prefix length or `num_steps` change.
Image/state/prefix values and `seed` are refreshed on every replay, with no
reuse of image features or K/V across observations. Setup includes an exact
comparison with the uncaptured forward and raises on failure. First use and
profile changes therefore have additional setup latency. Instances support
sequential calls. A true flag on other model/framework/hardware routes is
rejected; other frontends' existing graph controls are unaffected.
See [XR-1 deployment](xr1_gfx1151.md) for checkpoints and input schemas.
- `use_fp4`, `fp4_layers`, `use_awq`, `awq_alpha`, and
`use_p1_split_gu` apply to the Pi0.5 torch and JAX NVFP4 encoder path on
Thor. The JAX path loads Orbax checkpoints; the torch path loads
Expand Down
Loading