You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
{{ message }}
Repository navigation
Expose accumulated batched PD producer and transfer durations #249
Batched Torch PD retains producer prefill_s and transfer_s internally but does not export them. The existing single-stage transfer histogram stays zero on the batched route, so benchmarking cannot distinguish unobserved time from free transfer.
Evidence
#248 Qwen3-0.6B native/PD/DP2 measurements: PD metrics record42remote prefills while airm_kv_transfer_seconds_count remains0. PDBatcher._admit_prefilled accumulates the two durations; BatchedService._pd_metrics exports only counts/tokens.
Scope
Expose cumulative completed-handoff producer compute and transfer durations as distinct batched PD counters, with explicit semantics and missing-stat compatibility. Do not fabricate per-request histograms or infer overlap from summed durations. Add real tiny-model serving tests including repeated scrapes and missing-stat cases.
Done when
Public metrics show nonzero producer compute after actual prefill, transfer totals match the reported handoff durations without increasing on scrape, legacy workers remain supported, and focused/full CPU tests pass. Subsequent performance runs can capture these counters.
Owner
Codex implementation; maintainer review/merge. Related #248 and #47.
Activity
pd: producer failure handling, readiness gate and stage metrics (#238, …
gspark-etri commentedon Oct 30, 2026
Fixed by #282, squash-merged into optimus-rm.