Repository navigation
docs: record native RNGD coverage and KONI parity limits - #275
Open
gspark-etri wants to merge 8 commits into
Open
gspark-etri wants to merge 8 commits into
gspark-etri wants to merge 8 commits into
Conversation
gspark-etri
marked this pull request as ready for review
October 24, 2026 21:11
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Background
The four-model serving task needs native RNGD evidence separate from research-engine PD results. On the tested SDK2026.4 registry configurations, exact Qwen3-0.6B, Qwen3-4B and Gemma3-1B are rejected; KONI builds and serves but changes greedy output under concurrency. This PR records those limits, reproduction clients and controls. It does not implement a production engine fix.
Changes
Publish one experiment record, portable HTTP clients and fixture checks, a SHA256 source manifest and small evidence summaries. Retain failed measurements and distinguish execution, exact-token parity, strict answer quality, cache reuse and lifecycle behavior.
Verification
Evidence head: 052f84f; base main fd89265. One RNGD, TP8, context9216, furiosa-llm2026.4.0 b1baffa75, compiler2026.4.0 ae27d44cd8. Model/FXB hashes, exact commands, settings, timings, raw artifact hashes and cleanup records are documented in
experiments/2026-10-25-native-rngd-coverage.mdandexperiments/evidence274/.Final review checks rerun from the published source copies:
All eight checks pass. These are measurement-client fixtures, not substitutes for the hardware outcomes above. All27 source-manifest hashes and
git diff --checkverified. Changed-file sensitive scan clean; the full-repository scan retains seven findings in six unchanged baseline files and is not clean. Owned hardware experiments are stopped and Pod deletion verified.Compatibility and limits
Evidence-only change; no engine behavior or default changes. Exact registry rejections are version/configuration scoped. Single timed batches do not establish stable or equivalent-output speedups. Frozen36-case synthetic quality is not a broad quality benchmark. Reported top-k probabilities do not identify a compiler/kernel root cause.
Lifecycle inputs repeat valid token1000. Streaming victims close at first nonempty content, observed at44/80generated token IDs, so this is mid-decode cancellation rather than first-token or prefill cancellation. Abort counters and idle gauges do not prove physical KV allocation reclamation. Earlier native batch/shape drift remains unresolved.
Related Issue
Related to #201, #274, #276, #277, #278, #261 and #263. Does not close the overall four-model task. Production export/cache fixes and their acceptance remain separately tracked in #271 and #273; transport implementation is in PR266.
AI-assisted work: prepared and verified with Codex; maintainer review and merge required.