Skip to content

experiment: validate Qwen3 RNGD prefill and ATOM handoff #197

Open

Description

@gspark-etri

Problem

The current research RNGD prefill accepts only Qwen2. Qwen3-0.6B ATOM standalone matches CPU on the three tested prompts, but no heterogeneous handoff is validated.

Evidence

Qwen3-0.6B has hidden_size1024,16query heads,8KV heads,head_dim128,28layers, Q/K normalization and no QKV biases. The current Qwen2 graph assumes query width equals hidden size and biased projections.

Scope

Add a separate, strictly scoped research Qwen3 prefill runner preserving Q/K normalization and independent query width; verify KV/logits against CPU, then host-mediated ATOM admission using a dedicated compatible paged-KV compiled artifact. No transport ABI changes or automatic model-family enablement.

Done when

Recorded actual-device numerical and handoff checks, with remaining disagreements explicit. Unsupported quantized/sliding/MoE configs rejected before device work.

Owner

Codex (experimental implementation; maintainer review).
Related: #194, #195.

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions