You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
{{ message }}
Repository navigation
Connect validated four-model RNGD producers to live PD serving #234
The four-model matrix has validated saved RNGD KV exports, but live HTTP PD still lacks integration of the validated Qwen3-4B, KONI and Gemma producers. Gemma production imports are enabled by #231; its HTTP evidence currently replays recorded exports.
Evidence
The existing #199 Qwen3 worker already implements SubprocessPrefillWorker's shared-memory/request protocol, but the service fixes Qwen3-0.6B, context512 and width<=512. #227 supplies validated model-specific producers and true-length export semantics at longer widths. Production #232 handles fresh Gemma hybrid KV, with numerical limits retained in #233.
Scope
Reuse the existing shared-memory worker contract and production scheduler/HTTP owner. Refactor model producers into callable adapters used by both measurement and live requests; support Qwen3-0.6B, Qwen3-4B, KONI-8B and Gemma3-1B with explicit compiled capacity and checkpoint geometry. Validate input boundaries, EOS, cancellation/reuse, device ownership and same-fixture native comparisons. Start with fresh full-prefix transfers; retain cache/overlap gaps explicitly under #201 instead of claiming completion.
Done when
Each model handles real live RNGD prefill -> ATOM HTTP decode in an owned allocation with recorded correctness, lifecycle and timing evidence, and runs are distinguished from saved-KV replay. Numerical mismatches and unsupported cache paths stay explicit; communication-library changes only if the existing protocol is demonstrated insufficient.
Activity
test: record four-model long live RNGD to ATOM HTTP results (#234)