You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
{{ message }}
Repository navigation
Validate four-model live GPU PD HTTP over long inputs and lifecycle events #240
The four-model coverage has standalone GPU HTTP and offline RNGD-to-GPU continuation, but lacks live GPU PD HTTP evidence over long inputs and concurrent clients.
Evidence
The production serve_batched factory already supports a separate GPU prefill device. The HTTP matrix validator does not expose that option or verify remote prefill counters.
Hypothesis and scope
Exercise actual GPU0-prefill/GPU1-decode for Qwen3-0.6B, Qwen3-4B, KONI-8B and Gemma3-1B using the existing HTTP matrix (128/2048/8192 inputs, queued clients, cancellation/disconnect, invalid requests, cache conditions where supported). Add only CLI wiring and evidence checks to the validator. Preserve numerical differences and failures. Timings from this lifecycle harness are observational, not a controlled native-vs-PD speedup claim.
Done when
Four-model outcomes and exact commands/versions are recorded, actual remote work is verified by counters, discovered defects are fixed or linked to explicit investigations, and limitations including cross-host RNGD-to-GPU live transport remain visible.
Owner
Codex research implementation; maintainer review/merge.
Activity
examples: HTTP validation and benchmark harness for PD serving (#240, #…
gspark-etri commentedon Oct 30, 2026
Fixed by #286, squash-merged into optimus-rm.