[Research] Factory compiled-width cost and shared-weight buckets #268

Problem

The four-model research factory path accepts one compiled width. Current width8192 serving pads short128/512-token prompts to that width, leaving Qwen4 short-prompt PD substantially slower than native GPU even after layer streaming.

Evidence

PR#266 evidence501718a: Qwen3-4B c16,input128/output128 median TTFT native5.22s,whole23.29s,stream18.40s. At8192/128:21.94s/26.95s/21.51s. One batch per condition, different resource counts; these observations do not establish padding as the cause. Factory loader parses int(width). KONI PartitionedPrefill already inherits shared projection ownership and smallest-fitting bucket selection; Qwen/Gemma constructors separate width-dependent tensors conceptually but re-upload weights per instance.

Hypothesis / Scope

For identical short prompts, reducing compiled width8192 to512 reduces source latency without changing acceptable outputs. First isolate source computation/export using separate processes, retaining initialization, memory observations, all measured repeats, KV/logit artifacts and numerical differences. Then validate decoder output and serving latency. Only if justified, generalize factory width selection with shared weight ownership; avoid complete model duplication. Four models: Qwen3-0.6B,Qwen3-4B,Gemma3-1B,KONI-8B. Existing#132 covers legacy engine buckets; this hypothesis addresses the factory adapters.

Done when

• Matched128/512-token inputs measured at widths512/8192 for all four models, with cold/warm distinction and retained failures.
• KV/logit numerical differences and full decoder outputs independently checked; faster but divergent runs are not silently passed.
• If implementation follows, resource ownership, alternating width requests and capacity errors are tested, then native/whole/stream serving remeasured.
• Report initialization and memory costs; do not infer device DMA overlap or equal-resource advantage.

Owner

@gspark-etri

Research tracking: npu_offload research-hub; analysis/serving-matrix-201/factory-width-review.md. Production integration base remains a separate pending clarification; initial probes are isolated experiments.