Repository navigation
feat: support fresh hybrid KV imports in ATOM PD scheduler - #232
Draft
gspark-etri wants to merge 3 commits into
Draft
gspark-etri wants to merge 3 commits into
gspark-etri wants to merge 3 commits into
Conversation
This was referenced Oct 23, 2026
gspark-etri
added a commit
that referenced
this pull request
Oct 30, 2026
Lane-local sliding cache tables and global block tables for hybrid (Gemma3) external-cache serving; fresh full-prefix hybrid PD; batch-1 single-block PD with prefix/host caching off. Hybrid prefix and host caching stay rejected. From feat/236-atom-single-pd (#210, #232, #237): atom/batching.py, tests/test_atom_hybrid.py, tests/test_atom_batching.py. ATOM.md section rewritten to the supported scope. Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Background
Gemma hybrid caches were rejected whenever a remote prefill worker was configured. Fresh remote prefixes need different placement for sliding and global attention, and source dtype metadata can disagree with physical KV storage.
Changes
Verification
PYTHONPATH=opt_prime python -m pytest -q opt_prime/tests/test_atom_hybrid.py opt_prime/tests/test_atom_batching.py opt_prime/tests/test_pd_recovery.py opt_prime/tests/test_serve_batched.py: 37 passed, one existing warning. Initial regression failed before the fix; partial-result and malformed KV geometry recovery verified red/green. Hardware/runtime errors still poison the scheduler; only validation errors before device writes are request-scoped. The exception follow-up also passed the long-input hardware rerun described below.opt_prime/examples/pd/http_matrix/.Compatibility and limits
Stacked on #230. Fresh full-prefix PD only; hybrid prefix/host cache restoration remains rejected. Hardware producer is recorded RNGD KV replay, not live prefill. No overlap, hardware batch >1 or speedup claim. Vendor SIGPIPE backtraces on socket disconnect persist despite successful reuse; separate #214/#215 addresses that issue. Corrected Gemma artifact depends on upstream optimum-rbln #827.
Related Issue
Closes #231. Parent validation #201/#227 remains active.