eagle3

Train speculative decoding models effortlessly and port them smoothly to SGLang serving.

A PyTorch native library for training speculative decoding models

AMD ROCm (gfx1030) inference fork with RotorQuant/TurboQuant KV compression, PHANTOM-X zero-copy draft speculation, EAGLE3 speculative decoding, 12 RDNA2 crash fixes, and PrismML Bonsai Q1_0_G128 1-bit GGUF support.

This is a research project to efficiently accelerate speculative decoding with token-level cross-paradigm routing, accepted at EMNLP 2026.

JAX/TPU-native training framework for EAGLE3 speculative decoding draft models

Deployment guide: Qwen3-VL-8B on Jetson AGX Thor with NVIDIA TensorRT-Edge-LLM — NVFP4 quantization and ONNX export on x86, native TensorRT engine build on Thor, EAGLE-3 speculative decoding (67–92 tok/s)

Train workload-specific speculative decoding drafts for LLM inference — DSpark, DFlash2, EAGLE3 and more.

Some benchmarks on running speculative decoding with Qwen-3.6-35B-A3B with EAGLE3, DFlash and MTP.

Mistral Small 4 119B NVFP4 + EAGLE-3 on NVIDIA Jetson AGX Thor with vLLM — TRITON_MLA, chat/reasoning_effort patches for the Jetson image, 33.7 tok/s (1.93x)

Воспроизведение EAGLE-3 на Qwen3: где спекулятивное декодирование ускоряет, а где мешает. Два ноутбука, полный прогон на Kaggle T4.

Controlled EAGLE-3 draft adaptation for frozen Qwen3-8B: Rust latency, replication, scaling controls, and serving trade-offs.