Skip to content

Validate four-model live GPU PD HTTP over long inputs and lifecycle events #240

Closed

Description

@gspark-etri

Problem

The four-model coverage has standalone GPU HTTP and offline RNGD-to-GPU continuation, but lacks live GPU PD HTTP evidence over long inputs and concurrent clients.

Evidence

The production serve_batched factory already supports a separate GPU prefill device. The HTTP matrix validator does not expose that option or verify remote prefill counters.

Hypothesis and scope

Exercise actual GPU0-prefill/GPU1-decode for Qwen3-0.6B, Qwen3-4B, KONI-8B and Gemma3-1B using the existing HTTP matrix (128/2048/8192 inputs, queued clients, cancellation/disconnect, invalid requests, cache conditions where supported). Add only CLI wiring and evidence checks to the validator. Preserve numerical differences and failures. Timings from this lifecycle harness are observational, not a controlled native-vs-PD speedup claim.

Done when

Four-model outcomes and exact commands/versions are recorded, actual remote work is verified by counters, discovered defects are fixed or linked to explicit investigations, and limitations including cross-host RNGD-to-GPU live transport remain visible.

Owner

Codex research implementation; maintainer review/merge.

Activity

  1. gspark-etri commented on Oct 30, 2026

    @gspark-etri
    CollaboratorAuthor

    Fixed by #286, squash-merged into optimus-rm.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions