Skip to content

experiment: validate KONI long-prefix RNGD handoff to standard decoders #219

Open

Description

@gspark-etri

Problem

KONI8B has long standalone GPU/ATOM coverage but RNGD-to-ATOM evidence only uses an older short external-cache graph. The selected standard B1/context9216/k1 artifact has not consumed long RNGD KV.

Hypothesis

The existing Llama3.1 FullPrefillEngine can produce KONI2K KV with its existing RoPE scaling and geometry. Export group tensors on the host into the same portable format as the Qwen matrix, then compare GPU and standard ATOM continuation.

Scope

Validate the exact KONI checkpoint/geometry; reuse FullPrefillEngine kernels. Add a small host-export adapter and a repeatable long-input runner, exercise2K first. Permit the ATOM checker to select its already-validated k1 artifact. Record numerical disagreement and compilation failures. No default engine or shared SDK changes, and no overlap/performance claim from these correctness runs.

Done when

Actual-device repeatability and decoder handoff have evidence, or a reproducible failure identifies the missing work. GPU/ATOM output and KV readback comparisons are reported separately. Full four-model goal remains open.

Owner

Codex, maintainer review. Related #201, #208, #217 and #194.

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions