Repository navigation
Benchmarks for common python I/O patterns #399
Description
As work happens on I/O pieces I've been building specialized micro-benchmarks (ex. gh-120754 Speed up open().read() pattern by reducing the number of system calls and others have gh-117151: IO performance improvement, increase io.DEFAULT_BUFFER_SIZE to 128k), it would be nice to have more general benchmarks to validate I/O performance for common cases.
Talking a little with people at PyConUS there was some interest in the tests, and a general desire for I/O tests not to be enabled by default, but to be a group which can be manually run.
General I/O shapes I'm hoping to add benchmarks for:
- read/write all of the byes of a file in a single call (including pathlib.Path.read_text, pathlib.Path.write_text)
- read/write many small files (ex.
.pycfiles, maybe justcompile_all?) - streaming bytes read/write (ex. to a pipe / console such as stdin/stdout/stderr, non-seekable devices)
- read/write a zipfile, tarfile (read + seek, write + seek, in particular buffering behavior)
- use
zipimport - Create a
zipapp - Multi-threaded write to stdout, stderr (ex. logging in a large application/codebase)
Note: With these aiming to stay at the Binary / Bytes IO layer as much as possible (not touch Text I/O for now)
Activity
jayzuccarelli commentedon Oct 27, 2026
I have a first slice of these working locally, covering whole-file read/write (including Path.read_bytes/write_bytes and read_text/write_text), streaming bytes through a pipe and a file (buffered and buffering=0), and creating/reading/deleting many small files. They sit in a new io group excluded from the default run, so they only run with -b io. Comparing 3.13 and 3.14, buffered streaming is about 2x faster while unbuffered is flat, so the gh-117151 buffer change shows up. Before I open a PR: do the grouping, naming and sizes fit what you had in mind? zipfile/tarfile, zipimport, zipapp and threaded stdout/stderr would follow separately.
cmaloney commentedon Oct 27, 2026
Nice! Definitely the set of tests matches what I had in mind. I don't have a strong preference around file layout, I haven't done a lot of work in this repository so far. Definitely makes sense to do a separate PR for each performance test, common infra ideally alongside the test that needs it. A lot easier to review / validate a perf test that is fully contained and if the common core makes sense.
Thanks for working on this!