Skip to content

perf(audio): validate streams without copying discarded PCM - #3194

Merged
richiemcilroy merged 1 commit into
CapSoftware:mainfrom
vedanth-jadhav:perf/validate-audio-without-pcm-copy
Oct 25, 2026
Merged

richiemcilroy merged 1 commit into
CapSoftware:mainfrom
vedanth-jadhav:perf/validate-audio-without-pcm-copy

Conversation

@vedanth-jadhav

@vedanth-jadhav vedanth-jadhav commented Oct 25, 2026 •

Copy link
Copy Markdown

Export-tail validation calls AudioStream::read_chunk(48_000) and immediately drops every returned PCM vector. This includes unused source tracks validated by ExportAudioSources before MP4 publication. Decoding and resampling are required to detect failures, but copying validated samples into a new output vector is unnecessary.

Extract the existing bounded sample-consumption loop into a private helper. Normal reads still copy samples into their output vector; validation supplies a no-op consumer. Both paths retain the same 48,000-frame checkpoint boundaries, cancellation checks, pending samples, decoder/resampler state machine, sticky failures and reported source positions. No public API or validation step is removed.

Release measurements

Baseline 4d718aa52, Linux x86_64, AMD EPYC 9V74, two-CPU quota, CPU 0 affinity, Rust 1.88.0, FFmpeg 7.1.5. Root release profile (opt-level=s, thin LTO, one codegen unit). Five alternating independent processes per version, 25 samples after three warmups; medians of process medians. Each fixture contains 60 seconds of seeded noise:

Source Validation before After Reduction
48 kHz mono PCM16 1.947275 ms 1.638139 ms 15.9%
48 kHz stereo PCM16 3.332021 ms 2.725514 ms 18.2%
44.1 kHz stereo PCM16 46.185466 ms 46.168234 ms 0.04%
48 kHz stereo AAC 23.470889 ms 23.359940 ms 0.47%

The resampling/AAC differences are too small to establish a useful CPU improvement in those cases. Validation after reading seven frames first improves 10.1%/18.0% for 48 kHz mono/stereo. Normal read_chunk controls vary from 1.1% slower (mono) to 3.5% faster (stereo); no read-path performance improvement is claimed. All final frame counts match across versions, including AAC's decoded tail. The benchmark times the full validate_to_end call with warm file cache; stream opening, prefix reads, post-call checks and destruction are outside timing. This is a source-validation result, not an end-to-end export speedup or GPU/capture claim. GPUI's separate level-3 release profile was not benchmarked.

A separate instrumented binary counts Rust allocation/reallocation calls only while validating, after opening the stream. PCM fixtures fall from 61 calls to zero; AAC falls from 62 to zero. Cumulative requested Rust allocation bytes fall from 11,712,000 for mono, 23,424,000 for stereo PCM, and 23,808,000 for AAC to zero. These totals sum allocations over the call; they are not peak-live memory figures. FFmpeg's C allocations, stream construction and process RSS are excluded; this is not a claim that decoding performs no allocations.

Correctness and validation

New differential tests compare validation with draining public chunks after partial reads across empty/one-frame inputs, mono/stereo/surround sources, resampling and EOF. An injected decoder failure with 0, 1, 47,999, 48,000, 48,007 and 96,001 pending frames must report the same final frame position and sticky error as normal reads. Existing exact sample-bit, cancellation/private-abort, decoder-tail, relocation/publication and missing-source tests pass.

  • cargo fmt --all --check
  • cargo check -p cap-audio --locked
  • cargo test -p cap-audio --locked: 90 library tests and one doctest pass; three existing benchmarks ignored.
  • cargo clippy -p cap-audio --all-targets --locked -- -D warnings
  • git diff --check

The existing vendored RNNoise C build emits its SSE/AVX tuning warning; Rust's strict Clippy gate passes. No unrelated build flags are changed.

Reproduction

Generate reproducible inputs with FFmpeg; use -threads 1 -filter_threads 1 to bound setup work:

ffmpeg -f lavfi -i 'anoisesrc=sample_rate=48000:seed=42:duration=60' -c:a pcm_s16le mono-48k.wav
ffmpeg -f lavfi -i 'anoisesrc=sample_rate=48000:seed=73:duration=60' -ac 2 -c:a pcm_s16le stereo-48k.wav
ffmpeg -f lavfi -i 'anoisesrc=sample_rate=44100:seed=17:duration=60' -ac 2 -c:a pcm_s16le stereo-44k.wav
ffmpeg -i stereo-48k.wav -c:a aac -b:a 128k stereo-aac.m4a
cargo build -p cap-audio --release --locked --example audio-validation-benchmark -j 2
taskset -c 0 target/release/examples/audio-validation-benchmark mono-48k.wav stereo-48k.wav stereo-44k.wav stereo-aac.m4a

Copy the example onto the baseline revision, save both binaries, and alternate five runs each. Output includes per-sample API timings, prefix lengths, normal-read controls, and decoded frame counts. Native FFmpeg/ALSA development libraries and libclang are required for this crate. The comparison script and external allocator probe are outside the production diff.

Publication validation: rebased onto ec15b9840; 91 scoped tests passed, plus formatting, check, and strict all-target Clippy. Benchmarks above were recorded before this rebase; the production paths changed by this PR remain as measured.

RetriggerView in GreptileConfidence Score: 5/5

The PR appears safe to merge.

Summary

AudioStream now shares a bounded sample loop between reads and validation. Validation still decodes and resamples audio but skips copying discarded samples.

  • Audio validation reaches the end without copying unused samples.

Reviews (1) · Last reviewed commit: "improve(audio): validate streams without..." · Reviewed by Greptile

@vercel

vercel Bot commented Oct 25, 2026

Copy link
Copy Markdown
Contributor

Someone is attempting to deploy a commit to the Cap Software Inc Team on Vercel.

A member of the Team first needs to authorize it.

@richiemcilroy
richiemcilroy merged commit a0118dc into CapSoftware:main Oct 25, 2026
48 of 51 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants