Skip to content

experiment: connect and profile Qwen3 RNGD ATOM serving #199

Open

Description

@gspark-etri

Problem

The validated Qwen3 RNGD prefill graph is only exercised by finite tensor fixtures, not the existing batched HTTP serving path. Its warm prefill latency remains roughly 719 ms without phase attribution.

Evidence

PR #198 validates Qwen3-0.6B host-mediated KV handoff and 3/3 output matches to ATOM. KV export currently follows all layers; there is no source overlap evidence.

Scope

Profile input, layer compute, host KV export and output head; connect the experimental Qwen3 worker to the existing subprocess/shared-memory and ATOM batched service interfaces. Validate bounded prompts, error handling, repeated HTTP requests and output parity on allocated hardware. Preserve model-family boundaries; no automatic production backend registration or DMA claim.

Done when

Reproducible measurements distinguish compute from transfer; actual HTTP requests exercise RNGD prefill and ATOM decode; supported capacity and remaining limitations are recorded.

Owner

Codex; maintainer review. Depends on #197 / #198 and the existing optimus-rm serving package.

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions