[e6qu]

Two measurements of the model path after #83, and a logging addition to real_errors that the second needed.

False flags: the second development set's sample with the guards

The guards in #83 were written against the false flags of a sixteenth of the second development set (1,095 sentences, E2B). This reruns the same sample with them.

	Before the guards	With them
Flags	18 (16.4 per 1,000)	11 (10.0 per 1,000)
Real errors among them	4	4 (realitime, anyeone, a overflow, allowing to disable)
False	14	6, of which 1 (talking of to talking about) is answered by the last guard of #83, which this build predated

Left false: contrive to control and the thoughtless (Austen), continues our work from 2024H2 to on, Landed to Larded (both in the commonest tier of the word list, which can't rank them), and the unknown word sret (no model involved). One more, …, that allow latency spike detection to who, points at a real problem with a wrong fix. New false flags appeared where other versions reached the five compared in full once guarded candidates were gone.

Recall: where the model path loses the 34 pairs a perfect chooser catches

From the question log of the 34-pair run (12 caught: 7 by rules without a model, 5 by the model):

• No pair had the writer's version among the versions shown without E2B choosing it.
• Of the 22 missed, 17 were screened in and asked about without the writer's version among the five shown, and 5 got no version that parses.
• With the new LOG.estimates.tsv (every list of versions the one-pass estimate ranks): only 8 of 73 lists hold the writer's version at all, and 6 of those 8 rank in the first five. So the prefilter costs at most 2 pairs. The rest is upstream: without a locator, the writer's edit is not among the candidates verified in time.
• Showing fifteen versions instead of five (PREFILTER, measured and rejected): 7 of 34, none of them from the model.

A run with a wider candidate budget is in progress to size that upstream loss.

Code

examples/real_errors: with LOG=FILE, every list of versions the model estimates in one pass also goes to FILE.estimates.tsv, in the form examples/prefilter reads.

🤖 Generated with Claude Code

https://claude.ai/code/session_01TxRMoAtBqwUZXAvBmzVhC4