Repository navigation
audio: ffmpeg_dec: add MP3 and AAC-LC encoder mode (libshine, vo-aacenc) and capture topologies - #11414
Draft
lgirdwood wants to merge 4 commits into
Draft
audio: ffmpeg_dec: add MP3 and AAC-LC encoder mode (libshine, vo-aacenc) and capture topologies#11414lgirdwood wants to merge 4 commits into
lgirdwood wants to merge 4 commits into
Conversation
Add the SOF forks of libshine (fixed-point MP3 encoder) and vo-aacenc (VisualOn fixed-point AAC-LC encoder) under modules/audio/shine and modules/audio/vo-aacenc, pinned to their sof-hifi branches on thesofproject. shine (sof-hifi): ab594a0 shine: add Xtensa HiFi accelerated multipliers and optimize quantize loop a531d18 shine: work around clang/Xtensa backend miscompile in rate-control loop b917fc4 shine: precompute init tables to avoid runtime pow/sqrt on the DSP vo-aacenc (sof-hifi): c823d30 xtensa: optimize basic operators, band energy, and autocorrelation Signed-off-by: Liam Girdwood <liam.r.girdwood@linux.intel.com>
…capabilities Advertise SND_AUDIOCODEC_MP3 (guarded by !CONFIG_CADENCE_CODEC_MP3_ENC) and SND_AUDIOCODEC_AAC for SOF_IPC_STREAM_CAPTURE in IPC4_SOF_CODEC_INFO when CONFIG_FFMPEG_ENC_MP3 or CONFIG_FFMPEG_ENC_AAC is enabled. Signed-off-by: Liam Girdwood <liam.r.girdwood@linux.intel.com>
Add encoder mode (CONFIG_FFMPEG_DEC_ENCODE_MODE) to the ffmpeg_dec module, supporting: - Fixed-point MP3 encoding via libshine (CONFIG_FFMPEG_ENC_MP3, 1152-sample MPEG-1 Layer III frames). - Fixed-point AAC-LC encoding via vo-aacenc (CONFIG_FFMPEG_ENC_AAC, 1024-sample ADTS frames). Key details: - Driven via the DP sink/source .process interface with lazy encoder opening on the DP thread. - Selects the encoder codec and bit_rate from IPC4 snd_codec parameters when provided, rejecting unsupported or disabled codec IDs with -ENOTSUP, and falling back to Kconfig default (FFMPEG_ENC_DEFAULT_AAC or FFMPEG_ENC_DEFAULT_MP3) when no IPC codec ID is specified. - Cross-builds libshine and vo-aacenc archives in ffmpeg.cmake for both built-in (=y) and LLEXT (=m) builds. Signed-off-by: Liam Girdwood <liam.r.girdwood@linux.intel.com>
Add compressed audio capture (encode) topology support for the FFmpeg encoder widget (ffmpeg_enc): - include/components/ffmpeg_enc.conf: encoder widget class sharing the ffmpeg_dec UUID with DAPM widget type 'encoder'. - include/pipelines/cavs/compr-capture-ffmpeg.conf: DP encoder + aif_out host-copier capture pipeline. - include/pipelines/cavs/mixout-capture.conf: standalone timer-scheduled mixout pipeline for cross-pipeline LL->DP capture fan-out. - platform/intel/compr-capture-ffmpeg.conf & compr-default.conf: COMPR_FFMPEG_ENC capture PCM (id 51, pipeline id 91, mixout pipeline id 93) and per-instance stack/heap sizing (64 KiB stack, 256 KiB heap). - cavs-mixin-mixout-hda.conf & development/tplg-targets.cmake: mixin capture fan-out when COMPR_FFMPEG_ENC is enabled, plus sof-hda-generic-ffmpeg-mp3-enc-compr and sof-hda-generic-ffmpeg-aac-enc-compr development topology targets. Signed-off-by: Liam Girdwood <liam.r.girdwood@linux.intel.com>
lgirdwood
commented
Nov 6, 2026
lgirdwood
left a comment
lgirdwood
left a comment
Member
Author
There was a problem hiding this comment.
Lets also split the media format encoding logic into separate files like we do with decode.
Comment on lines
+179
to
+190
| #if defined(CONFIG_FFMPEG_ENC_DEFAULT_AAC) || \ | ||
| (defined(CONFIG_FFMPEG_ENC_AAC) && !defined(CONFIG_FFMPEG_ENC_MP3)) | ||
| e->type = FFMPEG_ENC_TYPE_AAC; | ||
| cd->codec = FFMPEG_DEC_CODEC_AAC; | ||
| #elif defined(CONFIG_FFMPEG_ENC_MP3) | ||
| e->type = FFMPEG_ENC_TYPE_MP3; | ||
| cd->codec = FFMPEG_DEC_CODEC_MP3; | ||
| #else | ||
| e->type = FFMPEG_ENC_TYPE_NONE; | ||
| cd->codec = FFMPEG_DEC_CODEC_NONE; | ||
| #endif | ||
| e->bit_rate = FFMPEG_ENC_DEFAULT_BITRATE; |
Member
Author
There was a problem hiding this comment.
The encoding type will be set at runtime not at build time. This mean we will have single binaries that can encode multiple formats
Contributor
There was a problem hiding this comment.
🟡 Changes recommended
Unresolved critical encoder defects can cause out-of-bounds reads, exhaust the MP3 capture heap, and carry prior-recording audio into restarted AAC streams.
8 open issues
What changed in this PR
This PR extends SOF’s FFmpeg module with a build-time encoder mode for MP3 and AAC-LC, adding compressed HDA capture alongside the existing decoder infrastructure.
Changes:
- Adds fixed-point encoding through pinned libshine and vo-aacenc dependencies.
- Introduces encoder configuration, processing callbacks, and capture capability reporting.
- Adds HDA capture routing, reusable topology classes, and development targets.
| File | Description |
|---|---|
west.yml |
Pins libshine and vo-aacenc dependencies. |
tools/topology/topology2/sof-hda-generic.conf |
Includes capture pipeline classes. |
tools/topology/topology2/platform/intel/compr-default.conf |
Defines encoder IDs and resource budgets. |
tools/topology/topology2/platform/intel/compr-capture-ffmpeg.conf |
Instantiates compressed capture and routing. |
tools/topology/topology2/include/pipelines/cavs/mixout-gain-host-copier-capture.conf |
Removes the class-level gain-to-copier route. |
tools/topology/topology2/include/pipelines/cavs/mixout-capture.conf |
Adds a standalone timer-scheduled mixout. |
tools/topology/topology2/include/pipelines/cavs/compr-capture-ffmpeg.conf |
Defines the encoder-to-host capture pipeline. |
tools/topology/topology2/include/components/ffmpeg_enc.conf |
Defines the encoder widget using the existing module UUID. |
tools/topology/topology2/development/tplg-targets.cmake |
Adds MP3 and AAC capture targets. |
tools/topology/topology2/cavs-mixin-mixout-hda.conf |
Adds conditional mixer-based capture fan-out. |
src/kernel/audio/base_fw.c |
Advertises MP3 and AAC capture capabilities. |
src/audio/ffmpeg_dec/llext/CMakeLists.txt |
Adds encoder library build dependencies. |
src/audio/ffmpeg_dec/Kconfig |
Adds encoder selections and build mode. |
src/audio/ffmpeg_dec/ffmpeg.cmake |
Cross-builds the encoder libraries. |
src/audio/ffmpeg_dec/ffmpeg_dec.h |
Declares encoder callbacks and shared copy helpers. |
src/audio/ffmpeg_dec/ffmpeg_dec.c |
Selects encoder callbacks and shares buffer helpers. |
src/audio/ffmpeg_dec/ffmpeg_dec-encode.c |
Implements PCM conversion and compressed encoding. |
src/audio/ffmpeg_dec/CMakeLists.txt |
Integrates encoder sources, headers, and libraries. |
Review effort: Balanced
Comment on lines
+319
to
+321
| cd->out_channels = source_get_channels(src); | ||
| cd->out_rate = source_get_rate(src); | ||
| e->frame_bytes = source_get_frame_bytes(src); |
Comment on lines
+327
to
+328
| AACENC_PARAM params = { 0 }; | ||
|
|
| # 48 KiB LL->DP input ring (3x 16 KiB) + ~32 KiB staging (pcm_in, out_stage, | ||
| # pcm_s16 / AVFrame) + ~48 KiB libshine / vo-aacenc state fits comfortably | ||
| # within 256 KiB. | ||
| COMPR_FFMPEG_ENC_HEAP 262144 |
Comment on lines
+400
to
+403
| add_custom_command( | ||
| OUTPUT "${SHINE_INSTALL}/lib/libshine.a" "${CMAKE_CURRENT_BINARY_DIR}/shine_cfgstub.o" | ||
| COMMAND sh "${CMAKE_CURRENT_BINARY_DIR}/shine-build.sh" | ||
| VERBATIM) |
Comment on lines
+487
to
+490
| add_custom_command( | ||
| OUTPUT "${VO_AACENC_INSTALL}/lib/libvo-aacenc.a" | ||
| COMMAND sh "${CMAKE_CURRENT_BINARY_DIR}/vo-aacenc-build.sh" | ||
| VERBATIM) |
Comment on lines
+201
to
+204
| case SND_AUDIOCODEC_AAC: | ||
| e->type = FFMPEG_ENC_TYPE_AAC; | ||
| cd->codec = FFMPEG_DEC_CODEC_AAC; | ||
| break; |
| #endif /* !CONFIG_FFMPEG_DEC_ENCODE_MODE */ | ||
|
|
||
| /* This defines the module operations */ | ||
| #if CONFIG_FFMPEG_DEC_ENCODE_MODE |
Comment on lines
+136
to
+143
| #if defined(CONFIG_FFMPEG_ENC_MP3) && !defined(CONFIG_CADENCE_CODEC_MP3_ENC) | ||
| /* MP3 encode (compress capture) via ffmpeg_dec (libshine), guarded so it is | ||
| * not advertised twice when the Cadence MP3 encoder is also present. | ||
| */ | ||
| codec_info.items[codec_info.count++] = | ||
| SET_CODEC_INFO_ITEM(SND_AUDIOCODEC_MP3, SOF_IPC_STREAM_CAPTURE); | ||
| #endif | ||
| #if defined(CONFIG_FFMPEG_ENC_AAC) |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
This PR (Phase 3 of the FFmpeg integration series, following #11308, #11334, and #11390) adds fixed-point audio encode support (
CONFIG_FFMPEG_DEC_ENCODE_MODE) and compressed capture topologies to theffmpeg_decmodule:west: manifest: add shine and vo-aacenc projectsshine(thesofproject/shine,sof-hifibranchab594a0) andvo-aacenc(thesofproject/vo-aacenc,sof-hifibranchc823d30) undermodules/audio/shineandmodules/audio/vo-aacenc.audio: base_fw: advertise FFmpeg MP3 and AAC compress capture encode capabilitiesSND_AUDIOCODEC_MP3(guarded by!CONFIG_CADENCE_CODEC_MP3_ENC) andSND_AUDIOCODEC_AACforSOF_IPC_STREAM_CAPTUREinIPC4_SOF_CODEC_INFOwhenCONFIG_FFMPEG_ENC_MP3/CONFIG_FFMPEG_ENC_AACare enabled.audio: ffmpeg_dec: add MP3 and AAC-LC encoder mode (libshine, vo-aacenc)src/audio/ffmpeg_dec/ffmpeg_dec-encode.cdriving fixed-point MP3 encoding (libshine, 1152-sample frames) and fixed-point AAC-LC ADTS encoding (vo-aacenc, 1024-sample frames) over the DP sink/source.processinterface.struct snd_codecparameters when provided, rejecting unsupported/disabled codec IDs with-ENOTSUP, and falls back to Kconfig defaults (CONFIG_FFMPEG_ENC_DEFAULT_AAC/CONFIG_FFMPEG_ENC_DEFAULT_MP3) when no IPC codec ID is specified.libshine(shine_ext) andvo-aacenc(vo_aacenc_ext) inffmpeg.cmakefor both built-in (=y) and LLEXT (=m) builds.topology2: add compressed audio capture pipelines for FFmpeg encodersffmpeg_enc.conf,compr-capture-ffmpeg.conf, andmixout-capture.confto tap the HDA analog capture BE via a standalone timer-scheduledmixoutinto the DPffmpeg_encpipeline (PCM ID 51, pipeline ID 91, mixout pipeline ID 93).sof-hda-generic-ffmpeg-mp3-enc-comprandsof-hda-generic-ffmpeg-aac-enc-comprdevelopment topology targets.