Skip to content

Reduce per-byte stdio and GC handoff overhead in native file input #205

Closed

Description

@dimensionscape

Each _hx_std_file_read_char call in src/hx/libs/std/File.cpp validates the handle, enters a GC-free zone, calls fread for one byte, then exits the zone. The matching Haxe FileInput.readByte override uses this path. FILE already buffers disk reads, but the runtime still repeats register capture, GC handshakes and stdio calls for every byte.

I measured this against checkout 03a4e8d on Linux x86_64 through WSL, Haxe 4.3.3 and HXCPP_M64. Both native paths read and summed every byte of the same precreated 1 MiB regular file:

  • One _hx_std_file_read_char call per byte: 30.835 ms.
  • _hx_std_file_read into a reused 16 KiB buffer, with a loop summing its bytes: 0.278 ms.
  • Both checksums were 101711872.

This is a single warm-cache diagnostic. The buffered control changes call granularity and is not a drop-in runtime patch or a prediction of the speedup achievable while preserving the byte API.

Investigate a buffered fast path for byte-oriented file input, or a native batched scanner for operations such as readLine that otherwise consume the byte API. If the Haxe std needs to adopt an additional primitive, track that integration explicitly. Avoid relying on private FILE layouts across platforms.

A cached-byte path must still honor pending GC requests. Do not simply remove the GC-free zone around fread: it can block on refill. Preserve ordering with readBytes, seek, tell, eof and close, and check pipes, stdin, short reads and error reporting as well as regular files. Runtime-owned buffering also needs defined synchronization when handles are shared.

Measure byte reads, line parsing and mixed byte/bulk reads on Linux and Windows with allocation on another thread. Verify correctness under moving and generational collection. Related: #151 for repeatable benchmarks and #182 for the importance of safe points on fast paths.

Activity

  1. added a commit that references this issue on Oct 20, 2026
    4751c18
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions