Repository navigation
test: Fix flaky cookie unquoting linear-performance test on PyPy - #3848
Merged
bdarnell merged 1 commit intoOct 26, 2026
Merged
Conversation
test_unquote_linear_performance failed intermittently on PyPy with a ratio just over its limit of 20. It was not timer noise: each sample is already at least 0.1s and the minimum of several. Three things were wrong: * assert_linear_scaling disabled the GC on every implementation. On PyPy that only stops major collections, so objects promoted out of the nursery pile up for the whole measurement (about 1MB per call here), and the larger input pays for touching fresh memory. This inflated the ratio of linear code to 20-90 depending on PYPY_GC_NURSERY. The GC is now only disabled on CPython. * Even without that, this test allocates heavily per input byte and its PyPy ratio depends on the nursery and cache sizes (11-24 measured), so its limit is raised to 40. The quadratic 6.4.1 implementation measures about 100 on CPython and 60-100 on PyPy. * The quick measurement timed n1 first, before PyPy's JIT had warmed up, which inflated it enough that the quadratic 6.4.1 implementation passed this test on PyPy. A warmup now precedes it. Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_0168zU4yXrAsAoY3KAWuiE8n
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
test_unquote_linear_performance failed intermittently on PyPy with a ratio just over its limit of 20. It was not timer noise: each sample is already at least 0.1s and the minimum of several. Three things were wrong:
assert_linear_scaling disabled the GC on every implementation. On PyPy that only stops major collections, so objects promoted out of the nursery pile up for the whole measurement (about 1MB per call here), and the larger input pays for touching fresh memory. This inflated the ratio of linear code to 20-90 depending on PYPY_GC_NURSERY. The GC is now only disabled on CPython.
Even without that, this test allocates heavily per input byte and its PyPy ratio depends on the nursery and cache sizes (11-24 measured), so its limit is raised to 40. The quadratic 6.4.1 implementation measures about 100 on CPython and 60-100 on PyPy.
The quick measurement timed n1 first, before PyPy's JIT had warmed up, which inflated it enough that the quadratic 6.4.1 implementation passed this test on PyPy. A warmup now precedes it.
Claude-Session: https://claude.ai/code/session_0168zU4yXrAsAoY3KAWuiE8n