[Addin]

Summary

Windows capture currently copies every frame back to the CPU with a blocking Map(READ), then copies it again into an FFmpeg frame. This removes the second copy: the mapped staging texture is wrapped directly as an FFmpeg-backed frame, and Instant M4S recordings can encode straight from the captured D3D11 texture through Media Foundation. FFmpeg remains the fallback.

Changes

• scap-direct3d: staging-texture ring with in-use tracking. The blocking Map is the default; the bounded DO_NOT_WAIT deadline is opt-in with CAP_READBACK_BUDGET_MS.
• scap-ffmpeg: the mapped staging texture is an AVBufferRef-backed AVFrame; no per-frame pool or memcpy.
• win_fragmented_m4s: the MF texture path is opt-in for Instant recordings (CAP_MF_TEXTURE_INPUT=1), Instant-only, and falls back to FFmpeg on setup failure. A runtime MF failure finalizes the segments already written and disables the path for later recordings. Studio always uses FFmpeg.
• enc-mediafoundation / mediafoundation-ffmpeg: explicit-bitrate constructor, sequence_header(), sample_to_packet, non-fatal encoder-tail drain. The M4S path iterates MFTs, requests 2 s keyframe spacing, and reads the sequence header from GetOutputCurrentType. MP4/camera keep their previous MFT selection and keyframe behaviour.
• enc-ffmpeg: the M4S MF path outputs BT.709 limited range, tags the MF media types, and SegmentedPacketMuxer writes BT.709/limited-range colour tags. MP4/camera colour behaviour is unchanged.
• scap-ffmpeg/examples/cli.rs: --bench A/B readback harness.
• Removed the unused win_segmented* muxers and DiskSpaceCallback plumbing.

Performance

Measured on an HP Pavilion Gaming Notebook (i7-6700HQ, GTX 950M + HD 530, Windows 11, 1080p60).

Results summary

Resolution	Metric	Before	After	Improvement	Speedup
1280x720	p50	3.10 ms	2.64 ms	14.8%	1.17x
1600x900	p50	2.92 ms	2.03 ms	30.5%	1.44x
1920x1080	p50	4.80 ms	2.87 ms	40.2%	1.67x
1920x1080	p99	14.70 ms	6.37 ms	56.6%	2.31x

The 720p/900p rows are from the earlier opt-in deadline run; the 1080p rows are from the default blocking Map. Both runs had zero misses.

Instant mode

Metric	MF texture input	NVENC	libx264	vs NVENC	vs libx264
Total CPU	8.1%	11.2%	52.3%	27.7% lower (1.38x)	84.5% lower (6.46x)
Stop latency	70.6 ms	140.7 ms	89.0 ms	49.8% lower (1.99x)	20.7% lower (1.26x)

Readback A/B (--bench --seconds 30)

resolution	path	p50	p95	p99	max	frames / misses
1280×720	old	3.10 ms	3.66 ms	3.93 ms	4.21 ms	891 / 0
1280×720	new (deadline)	2.64 ms	3.08 ms	3.16 ms	3.38 ms	891 / 0
1600×900	old	2.92 ms	3.11 ms	3.25 ms	6.28 ms	891 / 0
1600×900	new (deadline)	2.03 ms	2.21 ms	2.26 ms	2.69 ms	891 / 0
1920×1080	old	4.80 ms	11.82 ms	14.70 ms	51.74 ms	1473 / 0
1920×1080	new (blocking Map)	2.87 ms	3.45 ms	6.37 ms	21.37 ms	1621 / 0

The 1080p rows are a fresh isolated A/B run on the default blocking path; 720p/900p used the opt-in 8 ms deadline. Neither run missed a frame. The deadline remains available with CAP_READBACK_BUDGET_MS.

Validation

• cargo fmt --all --check
• Windows MSVC cargo check for scap-direct3d, scap-ffmpeg, cap-enc-mediafoundation, cap-mediafoundation-ffmpeg, cap-recording
• Scoped Clippy with -D warnings
• scap-direct3d 6/6, cap-mediafoundation-ffmpeg 1/1, and the SegmentedPacketMuxer init/colour-tag test

Scope / follow-up

• AMD remains excluded from the MF path.
• The MP4 hardware muxer (H264StreamMuxer) is unchanged; this targets Instant/fragmented M4S.
• A continuous FFmpeg fallback inside one DASH init segment needs a re-mux design; the path stays opt-in until the real-device pass (AMD/Intel/NVIDIA/hybrid, ARM64, pause/resume, resize, crash recovery, 4K60).
• 4K and WARP performance were not measured.
• On Optimus, the recording executable needs the high-performance GPU preference for NVIDIA MFT activation.

Confidence Score: 5/5

The PR appears safe to merge with texture encoding remaining opt-in.

Summary

This PR removes a Windows frame copy and adds opt-in texture encoding for Instant recordings.

• Windows capture passes mapped pixels to FFmpeg without a second copy.
• Instant recordings can encode captured Windows textures through Media Foundation.
• Media Foundation packets can feed fragmented output with matching color tags.
• Unused Windows segmented muxers and callback hooks are removed.

_{Reviews (5) · Last reviewed commit: "fix(recording): address Windows capture ..." · Reviewed by Greptile}

[vercel]

@Addin is attempting to deploy a commit to the Cap Software Inc Team on Vercel.

A member of the Team first needs to authorize it.

[Addin]

@greptileai review

[Addin]

@greptileai review

[richiemcilroy]

Thanks @Addin, this is a lot of careful work and the measurements on the Pavilion are really useful. We can't merge it as is. These are the things we found:

1. Default readback now drops frames. With the 8 ms READBACK_BUDGET (crates/scap-ffmpeg/src/direct3d.rs:10, map_readback in crates/scap-direct3d/src/lib.rs), every FFmpeg path (all AMD users, the OOP muxer, and any MF fallback) duplicates frames under GPU load, spins the CPU for up to 4 ms per frame, and can drop tail frames at stop. Your own 1080p run shows 11 misses out of 881. Please make the waiting Map the default and the deadline opt-in.
2. Studio mode changes too, not only Instant. Studio recordings also go through start_media_foundation_encoder (win_fragmented_m4s.rs:631). If MF fails mid-recording, the rest of the video is discarded (:813) with no FFmpeg fallback, and the user is never told because desktop-gpui doesn't consume MuxerCrashed. We'd like the texture path limited to Instant, or behind a setting, until it's validated on more hardware, and a runtime failure should fall back rather than end the recording.
3. Colour. The MF video processor's colour-space setup (crates/enc-mediafoundation/src/video_processor.rs:99) leaves the output on BT.601, and SegmentedPacketMuxer writes no colour tags. The FFmpeg path produces tagged BT.709 limited range, so shares may play back with shifted colours. Please output BT.709 and tag the stream.
4. Stop errors. DrainIncomplete after 250 ms (crates/enc-mediafoundation/src/video/h264.rs:788-845) now fails WindowsMuxer and the camera MP4 stop with "Hardware encoder error", where main returned Ok. DRAIN is also sent before END_OF_STREAM. A slow drain shouldn't fail an otherwise good recording.
5. Encoder selection. The MP4 and camera MF encoders now try every hardware encoder, which changes the pick on hybrid laptops, and the 2 s keyframe interval may be ignored by some hardware encoders. Please keep the current selection for those paths unless that change is the goal, and say so in the description if it is.

Smaller things: the 285-line scap-ffmpeg/examples/cli.rs benchmark rewrite would be easier to review as its own PR, several new /// comments narrate the code (AGENTS.md asks for none), and CAP_READBACK_BLOCKING is read on every frame instead of once.

Before merging we'll also need CI to run on Windows and a real-device pass: AMD under GPU load, Intel iGPU, NVIDIA, a hybrid laptop, ARM64, pause/resume, window resize, crash recovery, and 4K60. Removing the unused win_segmented* muxers and DiskSpaceCallback is fine.

[Addin]

Thanks for the detailed review. Addressed in 270c880:

1. Readback — the waiting Map is the default again. The deadline is opt-in with CAP_READBACK_BUDGET_MS and is read once per process.
2. Texture path — restricted to Instant and now opt-in with CAP_MF_TEXTURE_INPUT=1; Studio always uses FFmpeg. A runtime MF failure disables the path for the rest of the process, finalizes the segments already written, and makes the next recording use FFmpeg. A continuous fallback inside the same DASH init.mp4 would need a re-mux design, so the path stays opt-in until it has had the wider hardware pass.
3. Colour — the M4S MF path now sets the video processor to BT.709 limited range, tags the MF media types, and SegmentedPacketMuxer writes BT.709/limited-range colour tags. The existing MP4/camera paths are unchanged.
4. Stop — END_OF_STREAM is sent before DRAIN; a slow or failed drain logs a warning and falls through to the normal shutdown instead of failing the recording.
5. Encoder selection — MP4/camera keep the previous single-MFT selection and keyframe behaviour. Only the M4S path iterates MFTs and requests the 2 s keyframe spacing.

Smaller items: the new /// comments that only narrated the code are gone, and the benchmark label now matches the blocking default. The benchmark harness is still in this PR; happy to split it into a follow-up if you prefer.

Validation: cargo fmt --check, Windows MSVC cargo check for scap-direct3d, scap-ffmpeg, cap-enc-mediafoundation, cap-mediafoundation-ffmpeg, and cap-recording, scoped Clippy with -D warnings, scap-direct3d 6/6 and cap-mediafoundation-ffmpeg 1/1, plus the SegmentedPacketMuxer init/colour-tag test. The real-device pass (AMD/Intel/NVIDIA/hybrid/ARM64, pause/resume, resize, crash recovery, 4K60) is still outstanding.

[Addin]

@greptileai review

[Addin]

@greptileai review