Measure matched four-model native, data-parallel and PD HTTP performance #248

Question

With model/precision/input/output/cache/chunk policies matched, when does real HTTP PD improve latency or throughput relative to native serving and a same-two-GPU data-parallel baseline?

Evidence

#240/#241 showed that unmatched producer cache/chunk defaults can change BF16 outputs. Existing lifecycle timings include warmup and are not controlled speedup evidence. Native RNGD-to-ATOM output/lifecycle paths now pass but performance and actual overlap remain open.

Scope

Add a reproducible HTTP benchmark recording per-request TTFT, token timings, wall throughput and exact output checks. Start with available two-A100 hardware: one native backend, two native replicas with explicit round-robin routing, and GPU0-prefill/GPU1-decode. Pin producer cache-off/chunk256, decoder policy, input IDs and fixed output lengths. Warm each serving process before measurement; balance mode order across blocks and retain raw data and failures. Extend matched measurement to the established RNGD+ATOM path separately within the hypothesis, without claiming chunked compute/copy overlap from throughput alone.

Done when

Four-model matched baseline/PD data and correctness checks are recorded over short/long inputs and varied concurrency/output lengths, with resource differences and sample limits explicit; actionable bottlenecks/improvements are identified from measured stage/serving evidence.

Owner

Codex benchmark implementation and analysis; maintainer review/merge.