Skip to content

Provide a supported Qwen3-4B ATOM B4 lane-consistent export and cache path #273

Open

Description

@gspark-etri

Problem

Qwen3-4B ATOM B4 greedy output depends on the occupied lane with the current compiled artifact. Request concurrency must not silently change the advertised deterministic serving behavior. The all-attention-output diagnostic graph restores fixture consistency but has a measured throughput cost; it is not a supported production solution.

Evidence

Reproduction and archived raw hashes: PR #272, evidence commit ad70924, experiments/evidence270/QWEN4-LANE-REPRO.md and NUMERICS-DIAGNOSTICS.md. Runtime PR266 33d6cb08ec806f693db4ecfd7d630331a3bd1bb2f; TP1/B4/k5/context9216, rebel-compiler0.11.0, optimum-rbln0.11.0.post1, torch2.11.0+cpu, Transformers5.8.1.

  • Direct SDK forced-history lane reproduction remains after same-card prefill-placement controls. First-layer stored KV passes bitwise equality and mutation/restore controls. Instrumentation changes the graph and numerical outcome: no original faulty operator or compiler transformation has been identified.
  • Same-card original/all-core/all-core/original HTTP control: each original arm fails12 full-token and12 timed-text comparisons; both candidate arms pass66responses. All four arms have the same device binding and occupancy histogram. Candidate appends36 core-attention outputs while retaining37 hidden-state outputs; all292 forced-history recorded outputs and full-vocabulary logits are lane-equal.
  • Mean of two timed batches per variant, input512/output128/c16:75.976 original versus62.894 candidate tok/s (-17.22%). Input8192/output128:18.253 versus16.489 (-9.66%). Different cross-variant reference tokens preclude equivalent-output performance claims.
  • Candidate native HTTP264responses and RNGD-to-ATOM whole/stream528responses pass within-route parity. Whole/stream references match8/8, native/PD only4/8. Frozen native sequential/concurrent and PD whole/stream quality each pass36 strict answers/formats. PD quality occupancy reaches only2/3 lanes; the fixed-length PD performance test observes4 lanes. No general-quality claim.

Independent auditors audit_atom270_allcore_abba.py and audit_atom270_allcore_pd.py were rerun successfully from preserved raw results on2026-10-25. Original arm failures remain failures; cleanup is verified. This is distinct from RNGD/GPU prefill numerical drift in #256.

Requested scope

Engine owner to provide a supported Qwen3-4B compiled-batch correctness path, preferably through a validated compiler/runtime fix or narrowly scoped supported export configuration. Preserve original artifacts and provenance. Do not adopt the all-output diagnostic graph by default or silently treat its token changes as an accuracy oracle. If a supported B4 fix is unavailable, expose the restriction explicitly and evaluate a documented fallback with its capacity/performance cost. File vendor compiler reproduction with the existing portable fixture as appropriate.

Done when

  • The direct SDK lane/history fixture and native B4 HTTP regression pass on the chosen supported artifact/runtime, with repeated same-device controls.
  • Fresh export and cached reload identify the actual compiler/export variant; incompatible old artifacts cannot silently reuse a passing identity.
  • Native and RNGD-to-ATOM whole/stream cover128/512/2048/8192 input and32/128 output, with occupancy, EOS, numerical differences and cancellation/reuse behavior recorded separately.
  • Frozen quality baselines do not regress; claims about full-batch semantic quality require measured full-batch occupancy.
  • Same-device repeated latency/throughput measurements expose the cost of the selected correction; a numerical consistency fix is not called a speedup.
  • Qwen3-0.6B, KONI and Gemma retain their supported behavior. A reviewable engine-owned PR documents SDK requirements, cache migration and limits. Maintainer reviews and merges.

Owner

Optimus Prime engine maintainer owns production implementation and merge under the organization engine-boundary rule. Codex supplies isolated reproduction, raw evidence and hardware follow-up. Related #201, #270, #272; no existing failure is closed by this request.

AI-assisted report, prepared and independently checked with Codex.

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions