You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
{{ message }}
Repository navigation
fix: map dense subprocess KV into hybrid GPU cache groups #258
Live RNGD-to-GPU Gemma3-1B fails during first KV admission: subprocess prefill returns a dense model-layer tensor, but PagedKV expects one tensor per attention group. The owner exits with a shape mismatch instead of serving the request.
Evidence
Runtime3e16c33, Gemma3-1B BF16, four GPU slots/context9216/page64/cache off; RNGD width8192. Actual first128-token request fails in PDBatcher._admit_prefilled -> PagedKV.write_span: value [26,1,128,256] cannot broadcast to destination [22,1,128,256]. Gemma has22 sliding and4 global layers. Native GPU and grouped Torch producer paths pass existing matrices. Raw failure retained as local201-gemma-matrix-a.json and server log; no completed performance claim.
Scope
Map validated dense model-layer KV spans to receiver groups without changing wire protocol. Preserve existing grouped spans. Reject invalid shape/group input before any partial write. Add focused regression and real tiny Gemma PD coverage, then rerun hardware.
Done when
Dense and grouped imports yield identical per-layer cache contents and decode outputs; invalid input is rejected before writes; existing non-hybrid and grouped paths pass; live Gemma RNGD/GPU reaches and completes its matrix. Numerical parity is a separate measured gate, not presumed fixed.
Owner
Codex; maintainer review. Related #201, #231, #248.
Activity
exp: validate live Gemma dense KV fix on RNGD and GPU (#258)
pd: import dense subprocess KV into hybrid GPU cache groups (#258) (#283
gspark-etri commentedon Oct 30, 2026
Fixed by #283, squash-merged into optimus-rm.