[Goo-Leo]

Add an opt-in BF16 Metal pipeline that records patch embedding, vision, projector, prefix encoding and all denoising steps into one reusable ICB. Plan construction reuses the existing Pi0.5 model traversal; replay uses persistent buffers with fixed shapes and bindings.

The replay backend uses Metal indirect command buffers (ICBs) because this workload is already lowered to custom Metal kernels with fixed buffer bindings, launch geometry and dependencies. ICBs let us encode those commands once and execute them repeatedly. The current runtime submits the recorded ICB range through a new command buffer for each inference, avoiding per-kernel host encoding while preserving explicit kernel and memory ownership. Apple's documentation describes both ICB command reuse and the CPU encoding/replay model.

MPSGraph represents tensor operations as a symbolic graph and compiles that graph into an executable. Using it here would require a graph-backend lowering and validation of its layouts, kernels and numerical behavior. ICB supplies command replay for the existing custom-kernel path; it does not perform graph-level fusion or automatically apply MPSGraph/MLX compilation optimizations. Kernel dependencies and execution stalls can still remain inside an ICB. Graph compilation and command replay address different parts of the optimization problem.

• Organize Metal kernels under csrc/apple/, including TensorOps GEMM, D72 vision attention, split decode attention and merged encoder gate/up GEMM.
• Write prompt and noise inputs directly into compute storage. Production replay contains 2,665 dispatches and zero buffer-copy operations.
• Remove the copy shader and preserve diagnostic snapshots through native Metal blits. Reset mutable inputs before each profiler replay.
• Add SDK/CMake builds, export checks, numerical tests, checkpoint comparisons and trace-analysis tools.

Validated on Apple M5 Pro, 24 GiB, macOS 27.0.1, macOS SDK 27.0, Apple Clang 21, Python 3.12.14 and PyTorch 2.14.1:

• Both build paths, 132 Python tests and 7 CTest checks passed.
• Two checkpoint samples passed all 29 diagnostic probes; direct/ICB, frontend and repeated execution outputs matched exactly.
• Copy cleanup preserved saved outputs bit-for-bit. Natural-prompt CPU comparison reached raw robot-action cosine 0.99999549.

Short paired measurements used BF16, two camera views, prefix M=522, decode M=10, 10 denoising steps, three warmups and five timed iterations:

Path	p50	p95
Torch MPS	276.11 ms	277.79 ms
Full native ICB	140.79 ms	141.14 ms

These measure prepared-input execution: native includes CPU input upload and CPU action output; Torch uses MPS tensor inputs/outputs. Image preprocessing, tokenization and action unnormalization are excluded. These results describe the complete native path, not an isolated speedup from copy removal.

Reproduction commands, with local checkpoint/tokenizer paths supplied:

bash scripts/apple/build_native.sh

cmake -S . -B .cache/apple-cmake \
  -DFLASHRT_BUILD_APPLE_NATIVE=ON \
  -DFLASHRT_TEST_APPLE_TENSOROPS=ON \
  -DCMAKE_BUILD_TYPE=Release
cmake --build .cache/apple-cmake --parallel 2
ctest --test-dir .cache/apple-cmake --output-on-failure --parallel 1

MTL_DEBUG_LAYER=1 .venv/bin/python -m pytest -q \
  tests/test_apple_native_contracts.py \
  tests/test_apple_native_plan.py tests/test_apple_profiling.py

.venv/bin/python -m pytest -q \
  tests/test_apple_backend.py tests/test_pi05_torch_pipeline.py

PYTHONPATH=. .venv/bin/python examples/apple/benchmark_native_pipeline.py \
  --checkpoint "$CHECKPOINT" --tokenizer "$TOKENIZER" \
  --output .cache/apple-qualification \
  --prompt-length 10 --samples 2 --warmup 3 --iters 5 --profile-kernels

PYTHONPATH=. .venv/bin/python examples/apple/pi05.py \
  --checkpoint "$CHECKPOINT" --tokenizer "$TOKENIZER" \
  --output .cache/apple-cpu-reference \
  --native-pipeline --cpu-reference --samples 2 --warmup 1 --iters 3

Checkpoint: Fisher-Wang/pi05-libero-pytorch, revision ab107fbe6661e8abfec0f13dc27bfd3fafe24c69. M=522 uses a truncated prompt for shape qualification. Accuracy checks compare shared model providers; independent OpenPI/LIBERO task-success evaluation and CUDA/RDNA device validation are outside this validation.