Repository navigation
Provide a supported Qwen3-4B ATOM B4 lane-consistent export and cache path #273
Description
Problem
Qwen3-4B ATOM B4 greedy output depends on the occupied lane with the current compiled artifact. Request concurrency must not silently change the advertised deterministic serving behavior. The all-attention-output diagnostic graph restores fixture consistency but has a measured throughput cost; it is not a supported production solution.
Evidence
Reproduction and archived raw hashes: PR #272, evidence commit ad70924, experiments/evidence270/QWEN4-LANE-REPRO.md and NUMERICS-DIAGNOSTICS.md. Runtime PR266 33d6cb08ec806f693db4ecfd7d630331a3bd1bb2f; TP1/B4/k5/context9216, rebel-compiler0.11.0, optimum-rbln0.11.0.post1, torch2.11.0+cpu, Transformers5.8.1.
- Direct SDK forced-history lane reproduction remains after same-card prefill-placement controls. First-layer stored KV passes bitwise equality and mutation/restore controls. Instrumentation changes the graph and numerical outcome: no original faulty operator or compiler transformation has been identified.
- Same-card original/all-core/all-core/original HTTP control: each original arm fails12 full-token and12 timed-text comparisons; both candidate arms pass66responses. All four arms have the same device binding and occupancy histogram. Candidate appends36 core-attention outputs while retaining37 hidden-state outputs; all292 forced-history recorded outputs and full-vocabulary logits are lane-equal.
- Mean of two timed batches per variant, input512/output128/c16:75.976 original versus62.894 candidate tok/s (-17.22%). Input8192/output128:18.253 versus16.489 (-9.66%). Different cross-variant reference tokens preclude equivalent-output performance claims.
- Candidate native HTTP264responses and RNGD-to-ATOM whole/stream528responses pass within-route parity. Whole/stream references match8/8, native/PD only4/8. Frozen native sequential/concurrent and PD whole/stream quality each pass36 strict answers/formats. PD quality occupancy reaches only2/3 lanes; the fixed-length PD performance test observes4 lanes. No general-quality claim.
Independent auditors audit_atom270_allcore_abba.py and audit_atom270_allcore_pd.py were rerun successfully from preserved raw results on2026-10-25. Original arm failures remain failures; cleanup is verified. This is distinct from RNGD/GPU prefill numerical drift in #256.
Requested scope
Engine owner to provide a supported Qwen3-4B compiled-batch correctness path, preferably through a validated compiler/runtime fix or narrowly scoped supported export configuration. Preserve original artifacts and provenance. Do not adopt the all-output diagnostic graph by default or silently treat its token changes as an accuracy oracle. If a supported B4 fix is unavailable, expose the restriction explicitly and evaluate a documented fallback with its capacity/performance cost. File vendor compiler reproduction with the existing portable fixture as appropriate.
Done when
- The direct SDK lane/history fixture and native B4 HTTP regression pass on the chosen supported artifact/runtime, with repeated same-device controls.
- Fresh export and cached reload identify the actual compiler/export variant; incompatible old artifacts cannot silently reuse a passing identity.
- Native and RNGD-to-ATOM whole/stream cover128/512/2048/8192 input and32/128 output, with occupancy, EOS, numerical differences and cancellation/reuse behavior recorded separately.
- Frozen quality baselines do not regress; claims about full-batch semantic quality require measured full-batch occupancy.
- Same-device repeated latency/throughput measurements expose the cost of the selected correction; a numerical consistency fix is not called a speedup.
- Qwen3-0.6B, KONI and Gemma retain their supported behavior. A reviewable engine-owned PR documents SDK requirements, cache migration and limits. Maintainer reviews and merges.
Owner
Optimus Prime engine maintainer owns production implementation and merge under the organization engine-boundary rule. Codex supplies isolated reproduction, raw evidence and hardware follow-up. Related #201, #270, #272; no existing failure is closed by this request.
AI-assisted report, prepared and independently checked with Codex.
Activity