Skip to content

Expose accumulated batched PD producer and transfer durations #249

Closed

Description

@gspark-etri

Problem

Batched Torch PD retains producer prefill_s and transfer_s internally but does not export them. The existing single-stage transfer histogram stays zero on the batched route, so benchmarking cannot distinguish unobserved time from free transfer.

Evidence

#248 Qwen3-0.6B native/PD/DP2 measurements: PD metrics record42remote prefills while airm_kv_transfer_seconds_count remains0. PDBatcher._admit_prefilled accumulates the two durations; BatchedService._pd_metrics exports only counts/tokens.

Scope

Expose cumulative completed-handoff producer compute and transfer durations as distinct batched PD counters, with explicit semantics and missing-stat compatibility. Do not fabricate per-request histograms or infer overlap from summed durations. Add real tiny-model serving tests including repeated scrapes and missing-stat cases.

Done when

Public metrics show nonzero producer compute after actual prefill, transfer totals match the reported handoff durations without increasing on scrape, legacy workers remain supported, and focused/full CPU tests pass. Subsequent performance runs can capture these counters.

Owner

Codex implementation; maintainer review/merge. Related #248 and #47.

Activity

  1. gspark-etri commented on Oct 30, 2026

    @gspark-etri
    CollaboratorAuthor

    Fixed by #282, squash-merged into optimus-rm.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions