Skip to content

[PD] Separate KV transfer backends and validate RNGD-to-ATOM continuation #194

Open

Description

@gspark-etri

Problem

The experimental native PD path combines a build-pinned RNGD DMA hook with Qwen3-specific KV geometry and a CUDA-only receiver. Reuse across decoder devices needs explicit backend boundaries.

Evidence

PR #193 improves native KV handoff. Stage attribution finds 34–51 ms mean handoff, ~10 ms prepared wait, ~1 ms final GPU import. Existing ATOM paths live on optimus-rm and have separate cache-format/runtime contracts.

Scope

Separate physical-run planning from RNGD ABI hooks; separate KV geometry and upload completion/ownership from CUDA operations; retain the validated native wire contract. Integrate and validate RNGD-to-ATOM continuation using available ATOM runtime, with explicit compatibility checks. Do not claim arbitrary model/vendor support or modify network protocols without evidence.

Done when

Existing GPU correctness passes; generic geometry and backend ownership tests pass; RNGD-to-ATOM hardware continuation is measured or a concrete runtime/model compatibility blocker is recorded. Documentation states backend-specific limits and remaining scheduler work.

Owner

Codex implements and verifies; maintainer reviews and merges. Related #192, #136, #132.

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions