experiment: isolate Gemma native global-prefill causality discrepancy #225

Problem

Gemma RNGD→ATOM handoff has exact raw cache readback/native roundtrip but diverges from native ATOM prefill at the second generated token. GPU BF16/FP32 agree with the RNGD producer at128/2K. Native versus imported cache-bit agreement drops sharply after the first global attention layer.

Evidence and hypothesis

The installed Optimum0.11.0.post1 global AttentionOp passes is_bidirectional=True during text prefill when use_attention_mask/use_position_ids are both true; the compiled Gemma artifact has those settings. The existing causal-prefill compatibility patch fixes only SlidingWindowAttentionOp. Hypothesis: global text prefill remains noncausal and explains the remaining discrepancy. This is not yet proven by a hardware causality test.

Scope

Test unchanged prefixes with altered suffixes on the existing native Gemma artifact, recording per-layer KV changes. If global causality fails, isolate a process-local fix, rebuild into a separate artifact cache, and compare correctness and handoff. Preserve image-prefill semantics. Do not patch a shared SDK or production engine from this experiment.

Done when

Actual-device causality evidence confirms or rejects the hypothesis; any fix preserves text-prefix causality and is checked against independent references and handoff results. File an upstream SDK issue/PR with the reproduction if appropriate.

Owner

Codex; maintainer review. Related #223/#224, #207 and upstream optimum-rbln PR827.