Skip to content

Add experimental Qwen3 RNGD prefill and ATOM handoff validation - #198

Draft
gspark-etri wants to merge 1 commit into
mainfrom
exp/197-qwen3-prefill
Draft

gspark-etri wants to merge 1 commit into
mainfrom
exp/197-qwen3-prefill

Conversation

@gspark-etri

Copy link
Copy Markdown
Collaborator

Background

The existing research RNGD prefill implements Qwen2 semantics. Qwen3-0.6B needs Q/K normalization and a query projection width2048 independent of hidden_size1024, so enabling its model name in that graph would be incorrect.

Changes

Add a separate experimental denseQwen3 graph, strict SDK-independent model checks, finite prefill/ATOM runners, and measured validation records. Existing Qwen2/model registration and communication ABI are unchanged. Host-mediated KV import reuses the original and portable importers from PR195.

Verification

  • CPU contract tests:2passed; unsupported model/quantization/RoPE/sliding/bias/head configurations rejected.
  • Ruff format/check, py_compile, git diff --check.
  • RNGD2026.4,fusion4/Mesh2,width128; ATOM rebel0.11.0,optimum-rbln0.11.0.post1,torch2.11.0+cpu,B1/context512/3KV blocks.
  • Qwen3-0.6B,three natural prompts,thinking disabled,fixed16output tokens:3/3first tokens matchCPU/ATOM;3/3PD sequences match nativeATOM;3/3full decode-logit sequences exact between importers;336/336KV slot readbacks exact.
  • Relative RMS versus CPU FP32:K0.470–0.491%,V1.757–1.911%,last logits1.164–1.604%. First RNGD prefill1.321s,subsequent0.723/0.714s including hostKV/logits: no native-performance parity claim.
  • Initial compile rejectedBF16 epsilon/exponent inputs; corrected toFP32 as required by rms_dsl. Failure is retained in the research record.

Compatibility and limits

Experimental finite runner, not production serving registration. Requires optimus-rm/rngd_pd and PR195's portable importer/validated container launcher onPYTHONPATH. Only0.6B,width128 is hardware-validated. NoDMA/source-overlap claim; export follows complete prefill. Longer inputs,concurrency,cancellation,recovery and other sizes remain unvalidated. The shared incorrectIncheon answer demonstrates that token parity is not factual-quality validation. Fixed-length checks continue pastEOS.

README contains reproduction commands; validation.json contains exact token and error measurements. Existing shared installations/artifacts were not modified; dedicatedPod removed.

Related Issue

Related #197, #194, #195.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant