feat: MJPEG over RTSP (RTP/JPEG, RFC 2435) in the single view (#119)

[compot-dev]

Part of #119 (single view first, as decided; multi-view later; HTTP MJPEG is #121).

• SDP: a video section with JPEG/90000 (or the static payload type 26 without an rtpmap) is accepted when no H.264/H.265 is offered — Sdp.videoJpeg.
• JpegAssembler (RFC 2435): collects the fragments of a frame (fragment offsets must follow each other, the marker bit ends it; a gap drops the frame, joining mid-frame waits for the next one) and rebuilds the JPEG headers — SOI, DQT from the in-band quantization table header (Q ≥ 128) or from Q with the standard tables, DRI for types 64–127, SOF0 (type 0 = 4:2:2, type 1 = 4:2:0), the standard Huffman tables (JPEG Annex K), SOS, scan, EOI.
• Single view: the decode thread decodes each JPEG with BitmapFactory (reusing the bitmap) and draws it over the TextureView's surface; the view's fit transform keeps the aspect ratio, updateResolution gets the size. A surface used with lockCanvas() is replaced before a MediaCodec stream follows. Status line: Connected • TCP • MJPEG • Opus • … (Opus was missing from that line before); /stats.json single.codec MJPEG; snapshots and video work.
• Multi-view: a cell that gets MJPEG shows Decoder: MJPEG in the single view only (no garbage into the H.264 decoder).
• Tests: JpegAssemblerTest with RTP fixtures cut from a real baseline JPEG (Q = 255 with tables, Q = 50) decoded back with ImageIO; missing fragment; joining mid-frame; Q tables. The same algorithm was checked in a prototype against a capture of .133 /stream=2: all 29 frames (1280×720, type 0, Q = 255) decode.
• READMEs (EN/RU/ZH): features line.

Device check: Aceline-1, a slot with rtsp://<.133>/stream=2 in single view — picture at 5 fps, Opus audio, status line, debug page.

[compot-dev]

/review

[qodo-free-for-open-source-projects]

Code Review by Qodo

🐞 Bugs (0) 📘 Rule violations (0) 📎 Requirement gaps (1) 🎨 UX issues (0) 🔗 Cross-repo conflicts (0) 📜 Skill insights (0)

1. Codec switches can lose the video surface ✓ Resolved

Description

onCodecChanged clears jpegDrawn and posts recreateVideoSurface() asynchronously, without
stopping the decode worker from creating a MediaCodec against the Canvas-used surface. If
H.264/H.265 frames arrive before that UI task runs, createDecoder() can use the old surface, which
replaceSurface() then releases.

Code

android/app/src/main/java/com/openipc/decoder/Decoder.java[R203-206]

+                } else if (jpegDrawn) {
+                    // A surface drawn with lockCanvas() cannot take a decoder any more
+                    jpegDrawn = false;
+                    runOnUiThread(() -> recreateVideoSurface());

Evidence

Surface recreation is merely queued, whereas the decode worker can immediately create a decoder;
creation takes the current surface, and replacement releases its predecessor.

android/app/src/main/java/com/openipc/decoder/Decoder.java[196-207]
android/app/src/main/java/com/openipc/decoder/Decoder.java[490-496]
android/app/src/main/java/com/openipc/decoder/Decoder.java[1563-1571]
android/app/src/main/java/com/openipc/decoder/Decoder.java[1660-1672]

Agent prompt

The issue below was found during a code review. Follow the provided context and guidance below and implement a solution

## Issue description
A JPEG-to-MediaCodec transition can configure the decoder on a surface that is about to be released.
## Fix Focus Areas
- android/app/src/main/java/com/openipc/decoder/Decoder.java[199-206]
- android/app/src/main/java/com/openipc/decoder/Decoder.java[1563-1571]
- android/app/src/main/java/com/openipc/decoder/Decoder.java[1662-1672]
## Recommended Fix
Keep non-JPEG decode and decoder creation paused during the transition. Replace the Canvas-used surface on the UI thread first, then release the decode worker with the new surface installed; guard against an intervening stream transition.

ⓘ Copy this prompt and use it to remediate the issue with your preferred AI generation tools

---

2. MJPEG never uses hardware decoding 📎 Requirement gap ➹ Performance

Description

drawJpeg always decodes assembled frames with BitmapFactory, while the MJPEG branch closes the
MediaCodec decoder instead of selecting a video/jpeg codec. Whenever an RTSP session advertises
MJPEG, rendering therefore bypasses available hardware JPEG decoders and relies exclusively on the
bitmap path.

Code

android/app/src/main/java/com/openipc/decoder/Decoder.java[R200-202]

+                if (jpeg) {
+                    // MJPEG is drawn by the app (#119): no MediaCodec may hold the surface
+                    closeDecoder();

Evidence

The RTSP MJPEG compliance requirement calls for MediaCodec video/jpeg decoding when available and
bitmap decoding otherwise. The changed MJPEG callback explicitly closes the decoder, and drawJpeg
decodes frames exclusively with BitmapFactory, so the hardware path is never attempted.

Decode assembled RTSP MJPEG frames
android/app/src/main/java/com/openipc/decoder/Decoder.java[199-206]
android/app/src/main/java/com/openipc/decoder/Decoder.java[1591-1605]

Agent prompt

The issue below was found during a code review. Follow the provided context and guidance below and implement a solution

## Issue description
RTSP MJPEG sessions unconditionally close the MediaCodec decoder and decode every frame through `BitmapFactory`, so devices with a usable hardware JPEG decoder cannot use it.
## Fix Focus Areas
- android/app/src/main/java/com/openipc/decoder/Decoder.java[199-206]
- android/app/src/main/java/com/openipc/decoder/Decoder.java[1591-1605]
## Recommended Fix
Detect and configure a `video/jpeg` MediaCodec when the device provides one, queue assembled JPEG frames through that decoder, and retain the existing `BitmapFactory` path as the fallback when hardware JPEG decoding is unavailable.

ⓘ Copy this prompt and use it to remediate the issue with your preferred AI generation tools

---

3. 2048-pixel JPEG frames fail to render ✓ Resolved

Description

assemble multiplies the RTP width and height bytes directly by eight, interpreting the encoded
zero for 2048 pixels as zero pixels. headers writes that zero into the JPEG frame header, so a
valid frame with either dimension equal to 2048 cannot be decoded correctly.

Code

android/app/src/main/java/com/openipc/decoder/JpegAssembler.java[R122-123]

+        int width = (buf[pos + 6] & 0xFF) * 8;
+        int height = (buf[pos + 7] & 0xFF) * 8;

Evidence

The parser passes a zero RTP dimension through unchanged, and the header writer places it directly
in SOF0; the tests exercise only 64×32 frames.

android/app/src/main/java/com/openipc/decoder/JpegAssembler.java[118-123]
android/app/src/main/java/com/openipc/decoder/JpegAssembler.java[230-244]
android/app/src/test/java/com/openipc/decoder/JpegAssemblerTest.java[25-29]

Agent prompt

The issue below was found during a code review. Follow the provided context and guidance below and implement a solution

## Issue description
RTP/JPEG's encoded zero dimension becomes an invalid zero-sized JPEG frame.
## Fix Focus Areas
- android/app/src/main/java/com/openipc/decoder/JpegAssembler.java[120-123]
- android/app/src/main/java/com/openipc/decoder/JpegAssembler.java[230-233]
## Recommended Fix
Interpret a zero width or height unit as 256 units before multiplying by eight, and add a test for a 2048-pixel dimension.

ⓘ Copy this prompt and use it to remediate the issue with your preferred AI generation tools

---

4. Frames with 16-bit tables decode incorrectly ✓ Resolved

Description

assemble ignores the quantization header's precision field and copies 16-bit table values as
though they were 8-bit entries. For an RTP/JPEG frame carrying 16-bit tables, headers splits those
bytes into incorrect 64-byte JPEG tables and references the resulting incorrect chroma table.

Code

android/app/src/main/java/com/openipc/decoder/JpegAssembler.java[R139-143]

+                int len = ((buf[pos + 2] & 0xFF) << 8) | (buf[pos + 3] & 0xFF);
+                pos += 4;
+                if (len < 64 || len % 64 != 0 || pos + len > end) return false;
+                tables = new byte[len];
+                System.arraycopy(buf, pos, tables, 0, len);

Evidence

Only the length is read from the four-byte table header. Header construction then unconditionally
divides the copied bytes into 64-byte tables and writes 8-bit DQT table identifiers.

android/app/src/main/java/com/openipc/decoder/JpegAssembler.java[135-144]
android/app/src/main/java/com/openipc/decoder/JpegAssembler.java[215-244]

Agent prompt

The issue below was found during a code review. Follow the provided context and guidance below and implement a solution

## Issue description
The RTP/JPEG table parser treats 16-bit quantizer elements as 8-bit tables.
## Fix Focus Areas
- android/app/src/main/java/com/openipc/decoder/JpegAssembler.java[137-144]
- android/app/src/main/java/com/openipc/decoder/JpegAssembler.java[220-244]
## Recommended Fix
Parse and validate the precision field, then emit DQT entries with the corresponding precision and element width. Add a fixture covering 16-bit table headers.

ⓘ Copy this prompt and use it to remediate the issue with your preferred AI generation tools

---

View review recommended (4)

5. Codec switches misinterpret queued frames ✓ Resolved

Description

startListener chooses JPEG drawing or NAL decoding from the current rtspClient.isJpeg() value
rather than from the dequeued frame's stream. If an RTSP reconnect changes codec while the shared
queue still contains frames from the prior connection, JPEGs enter the NAL decoder or NALs enter
BitmapFactory.

Code

android/app/src/main/java/com/openipc/decoder/Decoder.java[R1663-1667]

+                    if (buffer != null && rtspClient.isJpeg()) {
+                        drawJpeg(buffer);
+                        framePool.recycle(buffer);
+                    } else if (buffer != null) {
                    if (videoDecoder.decodeFrame(buffer, codecH265, 0)) {

Evidence

SDP updates the volatile codec flag without clearing the shared frame queue, while the worker
branches on that flag after polling; decodeFrame treats non-JPEG input as a NAL.

android/app/src/main/java/com/openipc/decoder/RtspClient.java[111-123]
android/app/src/main/java/com/openipc/decoder/RtspClient.java[153-160]
android/app/src/main/java/com/openipc/decoder/Decoder.java[1660-1672]
android/app/src/main/java/com/openipc/decoder/MediaCodecManager.java[330-344]

Agent prompt

The issue below was found during a code review. Follow the provided context and guidance below and implement a solution

## Issue description
Queued video frames are interpreted using the codec selected by a later SDP.
## Fix Focus Areas
- android/app/src/main/java/com/openipc/decoder/Decoder.java[1662-1672]
- android/app/src/main/java/com/openipc/decoder/RtspClient.java[114-123]
- android/app/src/main/java/com/openipc/decoder/RtspClient.java[153-160]
## Recommended Fix
Tag queued frames with their codec or stream generation and discard frames from an older generation before decoding. Coordinate the transition so a frame cannot be enqueued under the old codec after the new generation is installed.

ⓘ Copy this prompt and use it to remediate the issue with your preferred AI generation tools

---

6. MJPEG over UDP is cut off after 8 seconds ✓ Resolved

Description

singleUdpCheck measures decoded progress only with videoDecoder.framesOut(), which never
advances for MJPEG frames posted by drawJpeg. When a healthy UDP MJPEG stream receives more than
10 KB per check for eight consecutive checks, it calls reportUdpBroken(), and Auto or multicast
sessions mark UDP failed and reconnect over TCP.

Code

android/app/src/main/java/com/openipc/decoder/Decoder.java[R1663-1666]

+                    if (buffer != null && rtspClient.isJpeg()) {
+                        drawJpeg(buffer);
+                        framePool.recycle(buffer);
+                    } else if (buffer != null) {

Evidence

singleUdpCheck considers the stream stalled when bytes - lastSingleBytes > 10_000 while
videoDecoder.framesOut() equals its previous value. MJPEG frames take the drawJpeg path instead
of the MediaCodec path, so drawing them cannot advance that count; after the repeated checks,
reportUdpBroken() sets the condition that the RtspSession receive loop handles by marking UDP
failed and ending the UDP session for Auto or multicast.

android/app/src/main/java/com/openipc/decoder/Decoder.java[1714-1733]
android/app/src/main/java/com/openipc/decoder/RtspSession.java[672-682]
android/app/src/main/java/com/openipc/decoder/Decoder.java[1660-1672]
android/app/src/main/java/com/openipc/decoder/Decoder.java[1803-1807]

Agent prompt

The issue below was found during a code review. Follow the provided context and guidance below and implement a solution

## Issue description
The single-view UDP stall check uses only MediaCodec output counts, so successfully drawn MJPEG frames do not register as progress. A healthy UDP MJPEG stream can consequently be reported broken and reconnected over TCP.
## Fix Focus Areas
- android/app/src/main/java/com/openipc/decoder/Decoder.java[1591-1631]
- android/app/src/main/java/com/openipc/decoder/Decoder.java[1662-1665]
- android/app/src/main/java/com/openipc/decoder/Decoder.java[1714-1733]
## Recommended Fix
Add a `volatile int jpegFramesOut` counter and increment it in `drawJpeg` after each successful `unlockCanvasAndPost`. Have `singleUdpCheck` count posted JPEG frames as output progress, either by using `videoDecoder.framesOut() + jpegFramesOut` or by selecting the JPEG counter when `client.isJpeg()` is true. Reset the comparison baseline when the stream changes.

ⓘ Copy this prompt and use it to remediate the issue with your preferred AI generation tools

---

7. MJPEG cameras in multi-view reconnect forever ✓ Resolved

Description

QuadCell.onPacket returns false for every video packet when jpeg is set, so RtspSession never
treats the stream as usable. Because of that it never updates lastPacketMs and never reports
"connected". The watchdog then closes the socket after watchdogMs and reconnects, over and over,
pulling a high-bitrate MJPEG stream each time; with Auto over UDP the first-packet timeout also
marks UDP as failed.

Code

android/app/src/main/java/com/openipc/decoder/QuadCell.java[455]

+        if (jpeg) return false;

Evidence

RtspSession updates lastPacketMs and sends the "connected" status only when callback.onPacket
returns usable == true. The control thread's watchdog reconnects when `now - lastPacketMs >
watchdogMs`.

android/app/src/main/java/com/openipc/decoder/RtspSession.java[690-699]
android/app/src/main/java/com/openipc/decoder/RtspSession.java[267-273]
android/app/src/main/java/com/openipc/decoder/QuadCell.java[449-456]

Agent prompt

The issue below was found during a code review. Follow the provided context and guidance below and implement a solution

## Issue description
In multi-view, an MJPEG cell rejects every video packet (`return false`). The RtspSession watchdog then reconnects the cell in an endless loop, using a lot of bandwidth.
## Fix Focus Areas
- android/app/src/main/java/com/openipc/decoder/QuadCell.java[426-428]
- android/app/src/main/java/com/openipc/decoder/QuadCell.java[455-455]
## Recommended Fix
Preferred: when `onSdp` sees `sdp.videoJpeg`, stop or pause the cell's session (for example through the existing pause path) so it does not stream MJPEG it cannot show. At minimum, return `true` for MJPEG packets (discard them but count them as usable) so the watchdog does not keep reconnecting.

ⓘ Copy this prompt and use it to remediate the issue with your preferred AI generation tools

---

8. MJPEG streams that send tables only once fail ✓ Resolved

Description

assemble rejects a zero-length quantization header for Q values 128–254 instead of reusing tables
previously supplied for that Q, and it ignores the precision byte when parsing supplied tables. When
a camera sends static tables once and signals reuse on later frames, those frames never reach the
draw queue and the rejected packets feed the inactivity watchdog; frames containing 16-bit tables
are misread as two 8-bit tables.

Code

android/app/src/main/java/com/openipc/decoder/JpegAssembler.java[R137-144]

+            if (q >= 128) {
+                if (pos + 4 > end) return false;
+                int len = ((buf[pos + 2] & 0xFF) << 8) | (buf[pos + 3] & 0xFF);
+                pos += 4;
+                if (len < 64 || len % 64 != 0 || pos + len > end) return false;
+                tables = new byte[len];
+                System.arraycopy(buf, pos, tables, 0, len);
+                pos += len;

Evidence

The len < 64 check rejects Length 0 before a JPEG header can be constructed, and the assembler has
no stored tables to use on a later frame. A false return makes RtspSession treat the packet as
unusable, feeding the inactivity watchdog; the current fixture tests supply tables in their frame
and therefore do not cover reuse.

android/app/src/main/java/com/openipc/decoder/JpegAssembler.java[135-151]
android/app/src/main/java/com/openipc/decoder/RtspSession.java[690-699]
android/app/src/main/java/com/openipc/decoder/JpegAssembler.java[80-96]
android/app/src/test/java/com/openipc/decoder/JpegAssemblerTest.java[25-29]

Agent prompt

The issue below was found during a code review. Follow the provided context and guidance below and implement a solution

## Issue description
RTP/JPEG frames with Q in 128–254 can use a zero-length quantization header to reuse tables supplied earlier, but `JpegAssembler` rejects those frames. It also ignores the precision byte, so supplied 16-bit tables are parsed incorrectly.
## Fix Focus Areas
- android/app/src/main/java/com/openipc/decoder/JpegAssembler.java[135-151]
- android/app/src/main/java/com/openipc/decoder/JpegAssembler.java[92-96]
## Recommended Fix
Cache supplied quantization tables by Q identifier and reuse the matching tables when Length is 0 for Q in 128–254. If no matching tables are cached, skip collecting that frame without treating its packet as unusable; continue to reject Length 0 for Q = 255. Clear the cache at the appropriate stream boundary. Read the precision byte at `buf[pos+1]` and either emit 16-bit DQT entries (`Pq=1`, 128 bytes per table) when a precision bit is set or drop that frame rather than interpreting the data as 8-bit tables. Test consecutive frames that supply tables and then omit them.

ⓘ Copy this prompt and use it to remediate the issue with your preferred AI generation tools

---

Tip of the day

💡 Did you know, you can turn these tips off under Display > Comment parts

More tips ↗ | Customize Qodo ↗ | Qodo docs ↗

[qodo-free-for-open-source-projects]

2. Mjpeg never uses hardware decoding 📎 Requirement gap ➹ Performance

drawJpeg always decodes assembled frames with BitmapFactory, while the MJPEG branch closes the
MediaCodec decoder instead of selecting a video/jpeg codec. Whenever an RTSP session advertises
MJPEG, rendering therefore bypasses available hardware JPEG decoders and relies exclusively on the
bitmap path.

Agent Prompt

## Issue description
RTSP MJPEG sessions unconditionally close the MediaCodec decoder and decode every frame through `BitmapFactory`, so devices with a usable hardware JPEG decoder cannot use it.

## Fix Focus Areas
- android/app/src/main/java/com/openipc/decoder/Decoder.java[199-206]
- android/app/src/main/java/com/openipc/decoder/Decoder.java[1591-1605]

## Recommended Fix
Detect and configure a `video/jpeg` MediaCodec when the device provides one, queue assembled JPEG frames through that decoder, and retain the existing `BitmapFactory` path as the fallback when hardware JPEG decoding is unavailable.

ⓘ Copy this prompt and use it to remediate the issue with your preferred AI generation tools

[compot-dev]

Qodo review — decisions:

1. Codec switches can lose the video surface — fixed: after MJPEG, non-JPEG frames are dropped (surfacePending) until the UI thread has installed the new surface; no decoder can be created on the canvas-used one.
2. MJPEG never uses hardware decoding — deliberate for this first step (MJPEG over RTSP (RTP/JPEG, RFC 2435) — discussion #119 asked: software first or MediaCodec first). BitmapFactory works everywhere; a video/jpeg MediaCodec path (the AG-216 has OMX.allwinner.video.decoder.mjpeg) is a follow-up.
3. 2048-pixel JPEG frames — fixed: a size byte of 0 means 256 units; test.
4. 16-bit tables — fixed: the precision byte is read, table lengths checked (tablesFit), DQT written with Pq = 1 and 128 bytes for wide tables; test.
5. Codec switches misinterpret queued frames — fixed: a codec change in onSdp empties the frame queue before the new codec is set.
6. MJPEG over UDP is cut off after 8 seconds — fixed: drawn JPEG frames count as output in the single-view UDP check.
7. MJPEG cameras in multi-view reconnect forever — fixed: the cell is paused (MJPEG: single view only); until then its packets count as usable, so no watchdog loop.
8. Tables sent only once — fixed: in-band tables are kept per Q (128–254) and reused for a zero-length table header; test.

[qodo-free-for-open-source-projects]

Qodo review — decisions:

1. Codec switches can lose the video surface — fixed: after MJPEG, non-JPEG frames are dropped (surfacePending) until the UI thread has installed the new surface; no decoder can be created on the canvas-used one.
2. MJPEG never uses hardware decoding — deliberate for this first step (MJPEG over RTSP (RTP/JPEG, RFC 2435) — discussion #119 asked: software first or MediaCodec first). BitmapFactory works everywhere; a video/jpeg MediaCodec path (the AG-216 has OMX.allwinner.video.decoder.mjpeg) is a follow-up.
3. 2048-pixel JPEG frames — fixed: a size byte of 0 means 256 units; test.
4. 16-bit tables — fixed: the precision byte is read, table lengths checked (tablesFit), DQT written with Pq = 1 and 128 bytes for wide tables; test.
5. Codec switches misinterpret queued frames — fixed: a codec change in onSdp empties the frame queue before the new codec is set.
6. MJPEG over UDP is cut off after 8 seconds — fixed: drawn JPEG frames count as output in the single-view UDP check.
7. MJPEG cameras in multi-view reconnect forever — fixed: the cell is paused (MJPEG: single view only); until then its packets count as usable, so no watchdog loop.
8. Tables sent only once — fixed: in-band tables are kept per Q (128–254) and reused for a zero-length table header; test.

Thanks for the decisions. Based on the diff, findings 1, 3, 4, 5, 6, 7, and 8 are addressed as described. finding 2 remains intentionally unresolved: MJPEG uses the software BitmapFactory path, with hardware video/jpeg decoding deferred as follow-up work. The published review will remain unchanged until a new review is run.

[compot-dev]

Device check of the final build (Aceline-1, df65bb7): rtsp://<.133>/stream=2 in single view — status line Connected • TCP • MJPEG • Opus • 5.1 Mbps, picture 1280×720, /stats.json codec MJPEG, video true, no JPEG frames dropped; then the same slot back to .126 H.265 — hardware decoding works again on the recreated surface. Hardware MJPEG decoding is #132.