Skip to content

fix: map dense subprocess KV into hybrid GPU cache groups #258

Closed

Description

@gspark-etri

Problem

Live RNGD-to-GPU Gemma3-1B fails during first KV admission: subprocess prefill returns a dense model-layer tensor, but PagedKV expects one tensor per attention group. The owner exits with a shape mismatch instead of serving the request.

Evidence

Runtime3e16c33, Gemma3-1B BF16, four GPU slots/context9216/page64/cache off; RNGD width8192. Actual first128-token request fails in PDBatcher._admit_prefilled -> PagedKV.write_span: value [26,1,128,256] cannot broadcast to destination [22,1,128,256]. Gemma has22 sliding and4 global layers. Native GPU and grouped Torch producer paths pass existing matrices. Raw failure retained as local201-gemma-matrix-a.json and server log; no completed performance claim.

Scope

Map validated dense model-layer KV spans to receiver groups without changing wire protocol. Preserve existing grouped spans. Reject invalid shape/group input before any partial write. Add focused regression and real tiny Gemma PD coverage, then rerun hardware.

Done when

Dense and grouped imports yield identical per-layer cache contents and decode outputs; invalid input is rejected before writes; existing non-hybrid and grouped paths pass; live Gemma RNGD/GPU reaches and completes its matrix. Numerical parity is a separate measured gate, not presumed fixed.

Owner

Codex; maintainer review. Related #201, #231, #248.

Activity

  1. gspark-etri commented on Oct 30, 2026

    @gspark-etri
    CollaboratorAuthor

    Fixed by #283, squash-merged into optimus-rm.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions