Skip to content

Connect validated four-model RNGD producers to live PD serving #234

Open

Description

@gspark-etri

Problem

The four-model matrix has validated saved RNGD KV exports, but live HTTP PD still lacks integration of the validated Qwen3-4B, KONI and Gemma producers. Gemma production imports are enabled by #231; its HTTP evidence currently replays recorded exports.

Evidence

The existing #199 Qwen3 worker already implements SubprocessPrefillWorker's shared-memory/request protocol, but the service fixes Qwen3-0.6B, context512 and width<=512. #227 supplies validated model-specific producers and true-length export semantics at longer widths. Production #232 handles fresh Gemma hybrid KV, with numerical limits retained in #233.

Scope

Reuse the existing shared-memory worker contract and production scheduler/HTTP owner. Refactor model producers into callable adapters used by both measurement and live requests; support Qwen3-0.6B, Qwen3-4B, KONI-8B and Gemma3-1B with explicit compiled capacity and checkpoint geometry. Validate input boundaries, EOS, cancellation/reuse, device ownership and same-fixture native comparisons. Start with fresh full-prefix transfers; retain cache/overlap gaps explicitly under #201 instead of claiming completion.

Done when

Each model handles real live RNGD prefill -> ATOM HTTP decode in an owned allocation with recorded correctness, lifecycle and timing evidence, and runs are distinguished from saved-KV replay. Numerical mismatches and unsupported cache paths stay explicit; communication-library changes only if the existing protocol is demonstrated insufficient.

Owner

Codex; maintainer review. Related #199/#201/#227/#231/#233.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions