Repository navigation
exp: validate live RNGD GPU cancellation and producer replacement - #260
Draft
gspark-etri wants to merge 3 commits into
Draft
gspark-etri wants to merge 3 commits into
gspark-etri wants to merge 3 commits into
Conversation
This was referenced Oct 23, 2026
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Background
Live heterogeneous PD needs evidence that cancellation and producer death do not leave a reusable decoder with stale work or linked shared memory. Performance parity alone does not test this.
Changes
Add the hardware lifecycle harness actually run against a real RNGD subprocess and GPU decoder, a native-call entry observer, and completed Qwen3-0.6B/Qwen3-4B/KONI-8B/Gemma3-1B evidence. No production runtime changes. Allfour baseline lifecycle runs are complete; this remains a draft for review.
Verification
Runtime32ce5d5 (#259); A10080GB GPU+one RNGD, Furiosa Torch/kernels2026.4, GPU Torch2.13.0+cu130. BF16,4paged lanes/page64/context9216/cacheoff, RNGDwidth8192. Cancellation/death victims use8192input32output; before/after controls use128input32output and exact nativeGPU tokens.
Allfour models pass: actual prefill entry observed, cancellation499 and exact reuse; owned producer termination yields pending500, subsequent admission503/readiness503; old slab unlinked; replacement readiness200 and exact tokens. This is actual producer execution, not a gate or KV replay.
Full commands/dependencies and raw JSON/logs: opt_prime/examples/pd/http_matrix/experiments/2026-10-24-live-gpu-fault201.md and evidence201/. Harness CLI help, artifact assertions, diff check and scoped sensitive scan passed. GPU performance tests were finished before faults began.
Compatibility and limits
Requires installed vendor SDK, research producer factories from #235/#245 and matching model weights. Independent lifecycle coverage does not relabel Qwen4/KONI long-output numerical failures. Additional concurrent-decode isolation now passes allfour models: producer terminated when currentGPUrequest had emitted2of512tokens; current request completesHTTP200 with512tokens exactly equal its independent same-route control. Pending victim500/subsequent503/readiness503/slab-unlink/replacement gates remain passing. Baseline and isolation raw results are retained separately.512-token comparison is same-route stability, not nativeGPU parity or answer quality. No performance or answer-quality claim.
Related Issue
Refs #201; stacked on #259. Does not close the overall four-model task. Maintainer reviews and merges.