Repository navigation
exp: characterize KONI numerical drift and output-head precision limits - #262
Draft
gspark-etri wants to merge 1 commit into
Draft
gspark-etri wants to merge 1 commit into
gspark-etri wants to merge 1 commit into
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Background
KONI-8B live RNGD→GPU completes requests but differs from nativeGPU beforeEOS on128-token outputs. Lifecycle success does not explain numerical drift. This PR retains failed comparisons and controlled probes without changing production sampling.
Changes
Add original live matrix/audit, matched-prefill and padded-prefill controls, shortKV repeat/replay metadata, identical-prefix BF16/FP32/head-onlyFP32 probes, and unforced output-head precision controls. Include portable forced-prefix entry point and exact local research scripts. No production code changes.
Verification
Runtime32ce5d5, checkpoint0172bf7e73977ae213613529f3d2d5400a33f678; A10080GB+RNGD, SDK2026.4, GPU Torch2.13. Original inputs128/8192,outputs32/128,c1/4,3repeats,4paged lanes/context9216/cacheoff, RNGDwidth8192.
Hardware command/results are in experiments/2026-10-24-koni-numerics261.md. All raw complete flags/counts verified; portable CLI syntax/help and scoped sensitive/diff scans passed. CLI adaptation itself was not hardware rerun; equivalent original script was.
Compatibility and limits
Diagnostic evidence, not a universal numerical/quality fix. Firstdifference measures equality, not semantic accuracy. FP32native changes prefill/decode arithmetic; importedKV remainsBF16.17MBshort and1.1GBlong KV dumps remain local; capture script/dependencies documented. Local research scripts need their environment paths adapted; matching Qwen control inputs are in #257. No equivalent-output speedup claim.
Related Issue
Refs #261,#201,#256. Does not close numerical investigation. Stacked on #259; draft for review and broader validation. Maintainer merges.