docs: qualify four-model prefix-cache page geometry controls

.research/events.jsonl

@@ -0,0 +1 @@
+{"id": "733a5fdd21ec4b2b91130ef4c477dbdd", "ts": "2026-10-30T13:12:09+09:00", "task": "Four-model serving matrix (aicomp #201)", "result": "Native GPU page256/chunk256 grid: eight arms, 192 conditions, 2752 responses independently verified; zero token/text mismatches. All processes exit0 and fresh GPU idle verification recorded.", "next": "Maintainer review of PR279; heterogeneous cache coverage continues under #201.", "state": "active", "done": false, "blocker": "", "metrics": {}, "params": {}, "artifacts": [{"path": "experiments/cache-history-212/full-grid-summary.json", "size": 96053, "sha256": "de53c0fbf1d9cab65f4674a9194baeba1253f9e9b70d3b724c83108fe3e2975c"}], "experiments": ["experiments/cache-history-212/full-grid-controls.md"], "labels": ["benchmark"], "review": true, "todos": [], "note": "Public scoped copy of research-hub event; measured runtime is 8f9f2a05178bcbb859c97dce2e5ee1bbc23eca3e. Weight hashes are historical; metadata hashes/file sizes rechecked.", "agent": "codex", "host": "GPU host", "branch": "docs/212-cache-history", "commit": "2178e6b9876c6b51fd9861a081043d7190ae9747"}

experiments/cache-history-212/README.md

@@ -0,0 +1,77 @@
+# Prefix-cache page geometry and request history (#212)
+
+## Background
+
+On the four selected checkpoints, prefix-cache reuse can change greedy output relative to prior requests. A preceding512/2048 homogeneous request matrix did not reproduce the problem: Qwen3-0.6B page64/page256 C0+ABBA controls completed220responses with100/100 token and100/100 SSE comparisons equal to cache-off. That pass did not cover the original128-token partially warmed prefix.
+
+## Setup and criterion
+
+Runtime a3497e57ac9d8286ad20032db4c842f26272e961 (PR266), one A10080GB PCIe, driver580.178.04, torch2.13.0+cu130, Transformers5.15.0. This evidence branch is based on main fd8926565217fdc675705cdf9d4477c1b0dbd57f; measurements use the explicitly named experimental runtime, not main. Frozen checkpoints: Qwen3-0.6B, Qwen3-4B, Gemma3-1B and KONI-Llama3.1-8B-Instruct-20241024; exact revisions in summary.json.
+
+BF16, four execution slots, context9216, prefill chunk256, prefix cache enabled. First collect serial128/2048/8192-token baselines; then mixed-length bursts of4 and8 requests, output32, greedy, ignore EOS. Validate HTTP readiness, completion counts, cancellation/disconnect recovery and empty-prompt rejection separately. All token comparisons below are independently recomputed from saved arrays. Throughput is not the correctness criterion.
+
+Only page size changes between configurations. Qwen0.6 runs64,256, then256,64 after other model pairs; each other model runs64 then256 once. Fresh server/process per arm, cached history retained within each arm. No competing GPU benchmark. Actual prefix-hit counters must be positive. Ten child exits and coordinator states checked; GPU compute-process list empty afterward.
+
+## Results
+
+|Model|Page|Concurrent equals own serial|Serial equals archived cache-off|Prefix-hit tokens|Mixed c8 wall(s)|
+|---|---:|---:|---:|---:|---:|
+|qwen06b|64|8/12|2/3|35072|1.498|
+|qwen06b|256|12/12|3/3|32768|1.445|
+|qwen4b|64|8/12|1/3|35072|1.961|
+|qwen4b|256|12/12|3/3|32768|1.956|
+|gemma|64|12/12|2/3|35072|1.761|
+|gemma|256|12/12|3/3|32768|1.811|
+|koni|64|7/12|2/3|35072|1.667|
+|koni|256|12/12|1/3|32768|1.781|
+|qwen06b|256|12/12|3/3|32768|1.513|
+|qwen06b|64|8/12|2/3|35072|1.465|
+
+Qwen0.6 page64 reproduces the archived symptom: four2K responses diverge at zero-based output index1, token374 versus594, and all three serial baselines equal the archived cache-enabled run. Page256 removes the discrepancy in this fixture. The reverse-order repeat checks reproducibility. Qwen4 and KONI also require separate within-route and cache-off comparisons; consult summary.json for each actual first divergence.
+
+Gemma page64 now has12/12 within-run matches, while its2K serial output differs from archived cache-off at index24, after the first EOS at index2. The archived cache-enabled experiment had8/12 matches. This changed observation must not be represented as a reproduced Gemma within-run failure or as general answer-quality improvement. Page256 comparisons remain scoped to these saved arrays.
+
+Current-runtime cold controls clarify the KONI historical mismatch: cold page64 and page256 match3/3, and page256 cache-on matches current cold3/3. Both current cold configurations differ from archived128/2K references. Thus that historical difference is not attributable to page256; its exact environmental/runtime cause remains unisolated. Page64 cache-on still differs from current cold at2K.
+
+All four models additionally complete current page256/cache-off controls:12/12 concurrent matches each, zero cache hits, and cache-on/cache-off serial baselines match3/3 each. Across the10cache-on arms and5cache-off controls, all15HTTP lifecycle checks pass; numerical comparisons are180concurrent responses with17mismatches, all in page64 cache-on arms (Qwen0.6 twice4, Qwen4 four, KONI five). These current controls use the same checkpoint, runtime and environment, rather than depending on archived comparisons.
+
+This supports evaluating page256/chunk256 as a workload-specific mitigation candidate. It does not establish a general numerical fix. Coarser pages reduce token reuse (32768 versus35072hit tokens in this fixture) and can change memory/recompute costs. Burst times are single measurements with uncontrolled execution order for three models; they are not stable speedup estimates. No production defaults or engine code were changed.
+
+## Reproduction
+
+Use validate.py from runtime a3497e57ac9d8286ad20032db4c842f26272e961 at opt_prime/examples/pd/http_matrix/validate.py, SHA2563d400c279f338fdba8e1f3f97d7f83da21271908116360279e1b0ca400de1798. The local diagnostic copies changed only the hard-coded page size and the import path to serve_batched. A portable equivalent is:
+
+```bash
+export RUNTIME=/path/to/runtime-checkout
+export PAGE=64 # repeat with256
+export COPY=/path/to/new-validation-copy.py
+python3 - <<'PYCODE'
+import os,pathlib,hashlib
+src=pathlib.Path(os.environ['RUNTIME'])/'opt_prime/examples/pd/http_matrix/validate.py'
+s=src.read_text()
+assert hashlib.sha256(src.read_bytes()).hexdigest()=='3d400c279f338fdba8e1f3f97d7f83da21271908116360279e1b0ca400de1798'
+assert s.count('        page_size=64,')==1
+s=s.replace('        page_size=64,',f"        page_size={int(os.environ['PAGE'])},")
+s=s.replace('sys.path.insert(0, str(Path(__file__).resolve().parents[1]))','')
+with open(os.environ['COPY'],'x') as f:f.write(s)
+PYCODE
+OMP_NUM_THREADS=4 MKL_NUM_THREADS=4 HF_HUB_OFFLINE=1 TRANSFORMERS_OFFLINE=1 PYTHONPATH="$RUNTIME/opt_prime:$RUNTIME/opt_prime/examples/pd" "$PYTHON" "$COPY" --model "$FROZEN_MODEL" --prefix-cache --output "$NEW_RESULT"
+```
+
+Set PYTHON to the torch/Transformers environment above and FROZEN_MODEL to the frozen checkpoint. Use an unused output path for every arm; keep originals. No HTTP module override. Raw result filenames/hashes and first divergences are in summary.json; site-specific commands/logs/configs remain local.
+
+## Remaining acceptance work
+
+#212 stays open. Generalization requires different partial-prefix lengths, shared/diverging prefixes, cache eviction/host reload, chunk/page combinations, all-model numerical controls and answer-quality checks. Aligning page and chunk sizes does not mathematically guarantee identical computation for arbitrary prompt histories. Engine-owned changes require the maintainer's implementation and regression review. Related #201; this is evidence, not a completed four-model GPU/RNGD/ATOM certification.
+
+## Additional prefix-length controls
+
+[Seed controls](seed-controls.md) extend page256/chunk256 to280/536-token initial prompts sharing one/two complete pages with2K requests. Four models,16runs:192/192 within-run token arrays and24/24 paired cacheoff/on serial references match. This adds prefix-length coverage; eviction/host reload is still untested here.
+
+## Device eviction and host reload
+
+[Pressure controls](pressure-controls.md) add four-model, one-slot A/B8K experiments with36device pages. All12arms pass pressure checks;72/72 within-arm and48/48 cross-mode response arrays match. Host arms each restore87blocks. Published fixture and verifier are included. Host-tier eviction and concurrent transfer remain outside this scope.
+
+## Full requested native GPU condition grid
+
+The [2026-10-30 full grid](full-grid-controls.md) adds eight matched page256 off/on arms: 192 conditions and 2,752 responses across all four models, with exact token/text checks and explicit limits.

experiments/cache-history-212/STATUS.md

@@ -0,0 +1,3 @@
+# Cache geometry evidence status
+
+The native GPU full condition grid is measured and independently verified; publication remains for maintainer review. Page64 history failures remain open. Heterogeneous cache and broader quality coverage are ongoing under #201. See full-grid-controls.md and the accompanying summary for scope and evidence.

experiments/cache-history-212/audit_full_grid.py

@@ -0,0 +1,52 @@
+import hashlib,json,math,pathlib,statistics,sys
+b=pathlib.Path(sys.argv[1]);q=json.loads((b/'cachefull212r1-gpu-queue.json').read_text())
+planned={(m,c) for m in ['qwen06b','qwen4b','gemma','koni'] for c in ['off','on']}
+keys=[(r['model'],r['cache']) for r in q['runs']]
+assert len(keys)==len(set(keys)) and set(keys)<=planned
+expected={(n,o,c) for n in [128,512,2048,8192] for o in [32,128] for c in [1,4,16]}
+report={'state':q['state'],'runtime_commit':q['runtime_commit'],'arms':{},'limits':'Single timing batch, native GPU, page256/chunk256; no PD, ATOM, default page64 or answer-quality certification.'}
+for run in q['runs']:
+ if 'exit_code' not in run:continue
+ m,c=run['model'],run['cache'];p=b/f'cachefull212r1-{m}-{c}.json';d=json.loads(p.read_text())
+ assert d['passed'] and d['mode']=='native'
+ assert d['physical_gpus']==1 and d['physical_rngd']==d['physical_atom']==0 and d['slots_per_decoder']==4 and d['context']==9216
+ assert d['producer_cache'] is False and d['decoder_cache']==(c=='on')
+ cfg=b/f'cachefull212r1-{m}-{c}-config.json';assert hashlib.sha256(cfg.read_bytes()).hexdigest()==run['config_sha256']
+ argv=json.loads(cfg.read_text())['servers'][0];assert argv[argv.index('--page-size')+1]==argv[argv.index('--chunk')+1]=='256'
+ assert ('--prefix-cache' in argv)==(c=='on')
+ assert len(d['warmups'])==8 and len(d['correctness_batches'])==len(d['batches'])==24
+ assert {tuple(map(int,r['id'].split('-')[-2:])) for r in d['warmups']}=={(n,o) for n in [128,512,2048,8192] for o in [32,128]}
+ assert set(d['references'])=={f'{n}:{o}' for n in [128,512,2048,8192] for o in [32,128]}
+ assert {(x['input_tokens'],x['output_tokens'],x['concurrency']) for x in d['correctness_batches']}==expected
+ assert {(x['input_tokens'],x['output_tokens_per_request'],x['concurrency']) for x in d['batches']}==expected
+ for n,ids in d['prompts'].items():assert len(ids)==int(n)
+ if c=='on':
+  refpath=b/f'cachefull212r1-{m}-off.json';off=json.loads(refpath.read_text())
+  assert off['prompts']==d['prompts'] and off['references']==d['references']
+  assert hashlib.sha256(refpath.read_bytes()).hexdigest()==run['reference_sha256']
+ mismatches=[]
+ for row in d['warmups']:
+  n,o=map(int,row['id'].split('-')[-2:]);ref=d['references'][f'{n}:{o}'];assert len(row['tokens'])==o
+  equal=row['tokens']==ref['tokens'];assert row['tokens_equal']==equal
+  if not equal:mismatches.append({'phase':'warmup','id':row['id']})
+ for batch in d['correctness_batches']:
+  n,o,cc=batch['input_tokens'],batch['output_tokens'],batch['concurrency'];ref=d['references'][f'{n}:{o}'];assert len(batch['rows'])==cc
+  for row in batch['rows']:
+   assert len(row['tokens'])==o;equal=row['tokens']==ref['tokens'];assert row['tokens_equal']==equal
+   if not equal:mismatches.append({'phase':'token','id':row['id']})
+ for batch in d['batches']:
+  n,o,cc=batch['input_tokens'],batch['output_tokens_per_request'],batch['concurrency'];ref=d['references'][f'{n}:{o}'];rows=batch['rows'];s=batch['summary'];assert len(rows)==cc
+  eq=[]
+  for row in rows:
+   assert row['output_tokens']==o and 0<row['first_sse_s']<=row['e2e_s']
+   equal=row['text']==ref['text'];assert row['text_equal']==equal;eq.append(equal)
+   if not equal:mismatches.append({'phase':'sse','id':row['id']})
+  assert s['all_text_equal']==all(eq) and s['output_tokens']==o*cc and s['requests']==cc
+  assert math.isclose(s['output_tok_s'],cc*o/s['wall_s'],rel_tol=1e-10)
+  for key,field in [('median_first_sse_s','first_sse_s'),('median_e2e_s','e2e_s'),('median_tpot_est_s','tpot_est_s')]:assert math.isclose(s[key],statistics.median(r[field] for r in rows),rel_tol=1e-10)
+ assert d['parity_passed']==(not mismatches) and run['exit_code']==int(bool(mismatches))
+ report['arms'][m+'/'+c]={'raw_sha256':hashlib.sha256(p.read_bytes()).hexdigest(),'conditions':24,'warmups':8,'token_checks':168,'sse_checks':168,'mismatches':mismatches,'parity_passed':not mismatches,'measurements':[{'input_tokens':x['input_tokens'],'output_tokens_per_request':x['output_tokens_per_request'],'concurrency':x['concurrency'],**x['summary']} for x in d['batches']]}
+if q['state']=='complete':assert set(keys)==planned and len(report['arms'])==8
+report['completed_arms']=len(report['arms'])
+(b/'cachefull212r1-summary.json').write_text(json.dumps(report,indent=2)+'\n')
+print({k:{kk:vv for kk,vv in v.items() if kk!='measurements'} for k,v in report['arms'].items()})

experiments/cache-history-212/full-grid-config.example.json

@@ -0,0 +1,39 @@
+{
+  "servers": [
+    [
+      "/path/to/python",
+      "/path/to/runtime/opt_prime/examples/pd/serve_batched.py",
+      "--model",
+      "/path/to/frozen/model",
+      "--device",
+      "cuda:0",
+      "--dtype",
+      "bfloat16",
+      "--slots",
+      "4",
+      "--max-context",
+      "9216",
+      "--max-output",
+      "128",
+      "--kv",
+      "paged",
+      "--page-size",
+      "256",
+      "--chunk",
+      "256",
+      "--port",
+      "{port}"
+    ]
+  ],
+  "metadata": {
+    "physical_gpus": 1,
+    "physical_rngd": 0,
+    "physical_atom": 0,
+    "slots_per_decoder": 4,
+    "producer_cache": false,
+    "decoder_cache": false,
+    "producer_chunk": 256,
+    "local_chunk": 256,
+    "context": 9216
+  }
+}

experiments/cache-history-212/full-grid-controls.md

@@ -0,0 +1,46 @@
+# Four-model native GPU cache condition grid
+
+- Date: 2026-10-30; related #212/#201 and PR279.
+- Runtime: `8f9f2a05178bcbb859c97dce2e5ee1bbc23eca3e` (PR266 experiment branch), not current main.
+- Hardware: one A100 80GB PCIe; driver 580.178.04. PyTorch 2.13.0+cu130, Transformers 5.15.0.
+
+## Setup and correctness
+
+Four selected models, each in fresh cache-off then cache-on server processes. Both use BF16, four decode slots, context 9216, page size 256 and chunk 256. Prompts have 128/512/2048/8192 tokens; outputs have 32/128 tokens; client concurrency is 1/4/16. Requests are greedy and ignore EOS. Each cell has one timed SSE batch after warmups and full-token checks. Repeated token-ID prompts are generated by the included fixture.
+
+All eight arms finished with exit 0: 192 conditions and 2,752 responses (64 warmups, 1,344 token checks, 1,344 SSE checks). Every cache-on response matches its fresh same-model, same-page-size cache-off reference. Full token arrays are checked for nonstream responses; exact rendered text and output length are checked for SSE responses. No semantic answer-quality claim follows from this fixture. Cache-hit counters are zero in every disabled arm and positive in every enabled arm.
+
+The independent auditor checks actual responses, exact condition/warmup sets, resource metadata, configuration/reference hashes and timing arithmetic. Twelve selfchecks include corrupted responses, duplicate warmups, duplicate/incomplete arms, wrong references/resources and incorrect throughput. Both the original child processes and coordinator exit 0; a fresh GPU compute query is empty before the next experiment starts.
+
+The initial attempt failed configuration validation because of an unsupported `page_size` metadata key, before a server was started. Removing that metadata key while retaining `--page-size 256` in server argv resolved the setup error. Initial queue/log/configs are retained separately as `cachefull212-*`; measured retry files use `cachefull212r1-*`.
+
+## Warm-prefix timing sample
+
+Input 8192, output 128, concurrency 4. These are single ordered samples with repeated warm prefixes, not stable or cold-prefill speedup estimates.
+
+| Model | Off tok/s | On tok/s | Off median first SSE (s) | On median first SSE (s) |
+|---|---:|---:|---:|---:|
+| Qwen/Qwen3-0.6B | 62.42 | 185.98 | 3.312 | 0.155 |
+| Qwen/Qwen3-4B | 40.59 | 134.29 | 5.474 | 0.247 |
+| unsloth/gemma-3-1b-it | 59.14 | 149.25 | 3.166 | 0.137 |
+| KISTI-KONI/KONI-Llama3.1-8B-Instruct-20241024 | 40.82 | 158.97 | 5.834 | 0.250 |
+
+All condition timings, source hashes, raw-result hashes and cache counters are in `full-grid-summary.json`. Model provenance includes the earlier complete weight-hash identity audit and its matched revisions. For this publication, all recorded file sizes and non-weight metadata hashes were rechecked; the weight shards were not rehashed during this grid. This distinction is retained in the summary.
+
+## Reproduction
+
+Use the recorded runtime and environments. `full_grid_http.py` is the measured fixture with only two publication guards: an explicit server config is required, and existing output paths are rejected before model import. `http267_diagnostics.py` is copied unchanged. Configure interpreter, runtime checkout and frozen model paths in the example JSON. For cache-on, add `--prefix-cache` to server argv and set `metadata.decoder_cache` to true. Keep page size 256 in both arms.
+
+```bash
+export PYTHONPATH="$RUNTIME/opt_prime:$RUNTIME/opt_prime/examples/pd"
+OMP_NUM_THREADS=4 MKL_NUM_THREADS=4 "$PYTHON" full_grid_http.py   --model "$FROZEN_MODEL" --mode native --server-config "$OFF_CONFIG"   --lengths 128 512 2048 8192 --outputs 32 128 --concurrency 1 4 16   --repeats 1 --request-timeout 180 --output "$NEW_OFF_RESULT"
+# Repeat with ON_CONFIG, a new output path and --reference "$NEW_OFF_RESULT".
+python audit_full_grid.py "$ARCHIVED_ARTIFACT_DIRECTORY"
+python selfcheck_full_grid.py "$ARCHIVED_ARTIFACT_DIRECTORY"
+```
+
+The archived directory contains `cachefull212r1-gpu-queue.json`, eight `cachefull212r1-{model}-{off,on}-config.json` files and corresponding raw JSON results. Model aliases are qwen06b/qwen4b/gemma/koni. The coordinator checked for idle GPU compute processes before every arm; preserve exclusive access when reproducing timings. Site-local paths and full raw responses are not published.
+
+## Limits
+
+This expands the page256/chunk256 mitigation evidence; it does not fix the original page64 history failures or change defaults. Prefixes are repeated, not a diverse workload. It does not establish NPU/heterogeneous cache behavior, host-capacity eviction, multi-slot reload faults, semantic quality or current-main support. The previous device-eviction and host-reload controls remain separate evidence. #212 and #201 remain open.