fix: Quad cells render to SurfaceView (#61) by compot-dev · Pull Request #63 · OpenIPC/decoder

compot-dev · 2026-10-12T17:06:07Z

Fixes #61.

Measurement (AG-216, Android 10, Quad with four cameras). A diagnostic build logged the allocations of each app thread (Debug.getThreadAllocSize, not part of this PR):

Thread	Allocations
`Quad-N-dec` ×4	7–10 KB/s each, mostly lock nodes from `poll(5 ms)`
`Quad-N-net` ×4	≈0
`quad-audio`	5 KB/s
main	0

That is about 45 KB/s in total, yet the process freed about 1.2 MB/s (9 young GCs per minute, ~8 MB each, 0 large objects). Every GC was triggered by one of the four JNISurfaceTexture threads. With a TextureView, each decoded frame raises a SurfaceTexture frame-available callback, and the framework delivers it on a native thread that it attaches to the VM for the call. With 4 × 25 fps that is about 100 attaches per second.

Change: Quad cells use SurfaceView. Frames go straight to the compositor, with no Java callbacks and one composition pass less. The black overlays and the AUDIO badge are normal views drawn above the surface, so they work as before. The single view keeps its TextureView, which pinch-zoom needs. Because SurfaceHolder callbacks are added rather than replaced, stop() removes the cell's callback.

Device results with the CI build will follow.

With a TextureView every decoded frame raises a SurfaceTexture frame-available callback, which the framework delivers on a native thread attached to the VM for each call ("JNISurfaceTexture"). With four streams this allocated about 1.2 MB/s of garbage and caused a blocking young GC every ~7 s. A SurfaceView hands frames straight to the compositor: no Java callbacks and one composition pass less. The single view keeps its TextureView, which pinch-zoom needs. SurfaceHolder callbacks are added, not replaced, so stop() removes the cell's callback.

compot-dev · 2026-10-12T17:06:33Z

/review

qodo-free-for-open-source-projects · 2026-10-12T17:08:16Z

Code Review by Qodo

🐞 Bugs (0) 📘 Rule violations (0) 📎 Requirement gaps (1) 🎨 UX issues (0) 🔗 Cross-repo conflicts (0) 📜 Skill insights (0)

1. Quad playback still allocates 📎 Requirement gap ➹ Performance

Description

The new SurfaceView path leaves QuadCell.startThreads() and the decoder/audio hot paths
unchanged, including the repeated timed queue poll. During four-stream playback, the supplied
measurement still records 7–10 KB/s per decoder thread and 5 KB/s in audio, so Quad playback does
not meet the allocation-free hot-path criterion.

Code

android/app/src/main/java/com/openipc/decoder/QuadCell.java[R106-109]

+        // SurfaceView, not TextureView (#61): with a TextureView every decoded frame raises a
+        // frame-available callback that the framework delivers by attaching a native thread to
+        // the VM, which allocated about 1 MB/s of garbage with four streams
+        view.getHolder().addCallback(holderCallback);

Evidence
The checklist requires four-stream Quad playback to have no per-frame hot-path allocations,
including decoder and audio buffers. The changed code only removes the TextureView callback
mechanism, while the Quad decode loop and packet handling remain in place; the PR's supplied
measurement still reports allocations from those decoder and audio paths.
Keep the Quad hot path allocation-free
android/app/src/main/java/com/openipc/decoder/QuadCell.java[106-109]
android/app/src/main/java/com/openipc/decoder/QuadCell.java[132-150]
android/app/src/main/java/com/openipc/decoder/QuadCell.java[184-205]

Agent prompt

The issue below was found during a code review. Follow the provided context and guidance below and implement a solution

## Issue description
Changing the Quad renderer to `SurfaceView` removes the framework frame-available callback allocations, but the decoder and audio paths still retain measured hot-path allocations during four-stream playback. Profile the updated path and remove or pool the remaining allocations so playback does not rely on repeated garbage collection.
## Fix Focus Areas
- android/app/src/main/java/com/openipc/decoder/QuadCell.java[106-109]
- android/app/src/main/java/com/openipc/decoder/QuadCell.java[132-150]
- android/app/src/main/java/com/openipc/decoder/QuadCell.java[184-205]
## Recommended Fix
Run allocation profiling during four-stream Quad playback after the `SurfaceView` change, then eliminate the remaining decoder, queue, and audio hot-path allocations by reusing buffers and avoiding per-iteration temporary objects. Verify that large video frames and audio buffers remain within the existing pool and that blocking young garbage collections no longer recur because of Quad playback.

ⓘ Copy this prompt and use it to remediate the issue with your preferred AI generation tools

2. ~~Quad video can stay black after a surface reset~~ ✓ Resolved

Description

surfaceCreated() stores the replacement surface but does not rebind or recreate the decoder
already configured with the old one. If a quad cell’s SurfaceView surface is destroyed and
recreated while its stream remains active, subsequent frames can continue going to the obsolete
surface; recovery then depends on a codec error or another decoder reset.

Code

android/app/src/main/java/com/openipc/decoder/QuadCell.java[R120-122]

+            // The surface belongs to the SurfaceView: it is used, never released, here
+            surface = holder.getSurface();
+            if (!threadsStarted) startThreads();

Evidence
The new callbacks clear and replace only QuadCell.surface. The decode path passes that field to
the manager only when no decoder exists, while the manager continues using an existing codec without
consulting the replacement surface. The activity also handles orientation and screen-size changes
without recreation, making surface lifecycle changes relevant to a running quad cell.
android/app/src/main/java/com/openipc/decoder/QuadCell.java[119-129]
android/app/src/main/java/com/openipc/decoder/QuadCell.java[224-232]
android/app/src/main/java/com/openipc/decoder/MediaCodecManager.java[103-141]
android/app/src/main/java/com/openipc/decoder/MediaCodecManager.java[168-199]
android/app/src/main/AndroidManifest.xml[29-29]

Agent prompt

The issue below was found during a code review. Follow the provided context and guidance below and implement a solution

## Issue description
A recreated quad-cell surface replaces `QuadCell.surface`, but an existing codec remains configured for the old surface.
## Fix Focus Areas
- android/app/src/main/java/com/openipc/decoder/QuadCell.java[119-129]
- android/app/src/main/java/com/openipc/decoder/QuadCell.java[224-232]
## Recommended Fix
Invalidate the existing decoder when its surface is destroyed, then recreate it against the current valid surface on the decode thread. Keep codec teardown off the UI callback thread and guard against a concurrent stop.

ⓘ Copy this prompt and use it to remediate the issue with your preferred AI generation tools

Tip of the day

💡 Did you know, you can commit Qodo's fix in one click with committable suggestions (GitHub & GitLab)

More tips ↗ | Customize Qodo ↗ | Qodo docs ↗

qodo-free-for-open-source-projects · 2026-10-12T17:08:17Z

+        // SurfaceView, not TextureView (#61): with a TextureView every decoded frame raises a
+        // frame-available callback that the framework delivers by attaching a native thread to
+        // the VM, which allocated about 1 MB/s of garbage with four streams
+        view.getHolder().addCallback(holderCallback);


1. Quad playback still allocates 📎 Requirement gap ➹ Performance

The new SurfaceView path leaves QuadCell.startThreads() and the decoder/audio hot paths unchanged, including the repeated timed queue poll. During four-stream playback, the supplied measurement still records 7–10 KB/s per decoder thread and 5 KB/s in audio, so Quad playback does not meet the allocation-free hot-path criterion.

Agent Prompt

## Issue description Changing the Quad renderer to `SurfaceView` removes the framework frame-available callback allocations, but the decoder and audio paths still retain measured hot-path allocations during four-stream playback. Profile the updated path and remove or pool the remaining allocations so playback does not rely on repeated garbage collection. ## Fix Focus Areas - android/app/src/main/java/com/openipc/decoder/QuadCell.java[106-109] - android/app/src/main/java/com/openipc/decoder/QuadCell.java[132-150] - android/app/src/main/java/com/openipc/decoder/QuadCell.java[184-205] ## Recommended Fix Run allocation profiling during four-stream Quad playback after the `SurfaceView` change, then eliminate the remaining decoder, queue, and audio hot-path allocations by reusing buffers and avoiding per-iteration temporary objects. Verify that large video frames and audio buffers remain within the existing pool and that blocking young garbage collections no longer recur because of Quad playback.

ⓘ Copy this prompt and use it to remediate the issue with your preferred AI generation tools

A recreated SurfaceView surface needs a decoder configured for it, and nothing may render into the old one after surfaceDestroyed() returns. The decode thread creates the new decoder once the next surface is there (Qodo).

compot-dev · 2026-10-12T17:22:18Z

Thanks.

Quad video can stay black after a surface reset: fixed in f8f4e98. surfaceDestroyed() now closes the cell's decoder, because nothing may render into the surface after it returns, and the decode thread creates a new decoder for the next surface. The wait is bounded by the 5 ms input dequeue.
Quad playback still allocates: declined. What is left after this change is small and comes from the platform, not from frame buffers:
- JDK lock nodes from the queue poll, plus MediaCodec's JNI ByteBuffer wrappers: about 45 KB/s in total, versus about 1.2 MB/s before;
- measured with 7 streams on the device: 1 young GC per minute (it was 9 per minute with 4 streams).
Removing the lock and JNI objects would mean replacing BlockingQueue and the MediaCodec buffer API, which is out of proportion to the gain. The wording in Quad: frequent blocking GC with four streams #61 is updated accordingly.

/review

compot-dev · 2026-10-12T17:31:06Z

Device test of the CI build f8f4e98 on the AG-216 (Android 10), Quad with four cameras:

	v2 (`TextureView`)	this PR (`SurfaceView`)
`JNISurfaceTexture` threads	4	0
logged young GCs per minute	9 (84 blocking-GC lines in 30 s)	0
CPU	≈134 %	≈108 %
queue overflow / NAL drops / codec errors (60 s)	0 / 0 / 0	0 / 0 / 0

3× Home → resume: same process, a stable 45 threads, 4 sessions and 4 decoders each time, no crash or ANR (one extra decoder start is the close/recreate on the surface reset). The black overlays and the AUDIO badge still draw above the video.

fix: Quad cells render to SurfaceView (#61)

qodo-free-for-open-source-projects Bot reviewed Oct 12, 2026

View reviewed changes

compot-dev mentioned this pull request Oct 12, 2026

feat: 8 camera slots, multi-view up to 3x3 (#13) #64

Merged

fix: close a Quad cell's decoder when its surface is destroyed

f8f4e98

A recreated SurfaceView surface needs a decoder configured for it, and nothing may render into the old one after surfaceDestroyed() returns. The decode thread creates the new decoder once the next surface is there (Qodo).

compot-dev merged commit 0a0dd40 into v2 Oct 12, 2026
4 checks passed

This was referenced Oct 12, 2026

fix: Escape also closes a menu opened by touch #65

Merged

ANR on AG-216: RenderThread blocked in eglSwapBuffers with 8 multi-view SurfaceViews #76

Closed

compot-dev deleted the fix/quad-surfaceview branch October 13, 2026 17:16

compot-dev added a commit that referenced this pull request Oct 15, 2026

Merge pull request #63 from OpenIPC/fix/quad-surfaceview

48e3472

fix: Quad cells render to SurfaceView (#61)

Navigation Menu

Repository navigation

fix: Quad cells render to SurfaceView (#61) - #63

fix: Quad cells render to SurfaceView (#61)#63
compot-dev merged 2 commits into
v2from
fix/quad-surfaceview

compot-dev commented Oct 12, 2026

Uh oh!

compot-dev commented Oct 12, 2026

Uh oh!

qodo-free-for-open-source-projects Bot commented Oct 12, 2026 •

edited

Loading

Uh oh!

qodo-free-for-open-source-projects Bot Oct 12, 2026

Uh oh!

Uh oh!

compot-dev commented Oct 12, 2026

Uh oh!

compot-dev commented Oct 12, 2026

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

1 participant

Conversation

compot-dev commented Oct 12, 2026

Uh oh!

compot-dev commented Oct 12, 2026

Uh oh!

qodo-free-for-open-source-projects Bot commented Oct 12, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Code Review by Qodo

Uh oh!

qodo-free-for-open-source-projects Bot Oct 12, 2026

Choose a reason for hiding this comment

Uh oh!

Uh oh!

compot-dev commented Oct 12, 2026

Uh oh!

compot-dev commented Oct 12, 2026

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

1 participant

qodo-free-for-open-source-projects Bot commented Oct 12, 2026 •

edited

Loading