Repository navigation
[BUG][TGL]Dmesg shows "failed to reset HDA controller gctl 0x1" info when do aplay0,2(rt1308) stop/restart stress test. #1399
Description
Describe the bug
Speaker's playback is stopped at 267th time. Dmesg shows "failed to reset HDA controller gctl 0x1" info.
To Reproduce
1.Reboot system.
2.Run "sudo ./sof-test-playback.sh ".
frequency
Run 500 times and failed at 267th time .
Expected result
Failed info should not show when do speaker playback stress test.
Actual result
Speaker's playback is stopped at 267th time. Dmesg shows "failed to reset HDA controller gctl 0x1" info.
Test recipe:
Kernel: https://github.com/thesofproject/linux/tree/integration/soundwire-latest commit: 9f1bd5b
FW: https://github.com/thesofproject/sof/commits/master commit: 2ca6a4a
tplg: Same with TGL FW branch commit: sof-tgl-rt711-rt1308-nohdmi.tplg rename to sof-tgl-rt711-rt1308.tplg
Platform: TGL RVP, with codec ACL711 in SDW mode, ALC1308 in I2S mode
dmesg logger
[ 4240.194212] sof-audio-pci 0000:00:1f.3: pcm: trigger stream 2 dir 0 cmd 0
[ 4240.194218] sof-audio-pci 0000:00:1f.3: ipc tx: 0x60050000: GLB_STREAM_MSG: TRIG_STOP
[ 4240.194484] sof-audio-pci 0000:00:1f.3: ipc tx succeeded: 0x60050000: GLB_STREAM_MSG: TRIG_STOP
[ 4240.195068] sof-audio-pci 0000:00:1f.3: FW Poll Status: reg=0x140000 successful
[ 4240.195086] sof-audio-pci 0000:00:1f.3: ipc tx: 0x60030000: GLB_STREAM_MSG: PCM_FREE
[ 4240.195238] sof-audio-pci 0000:00:1f.3: ipc tx succeeded: 0x60030000: GLB_STREAM_MSG: PCM_FREE
[ 4240.195246] SSP2-Codec: ASoC: trigger BE SSP2-Codec cmd 0
[ 4240.195387] Speaker: ASoC: hw_free FE Speaker
[ 4240.195407] sof-audio-pci 0000:00:1f.3: pcm: free stream 2 dir 0
[ 4240.195420] SSP2-Codec: ASoC: hw_free BE SSP2-Codec
[ 4240.195479] SSP2-Codec: ASoC: close BE SSP2-Codec
[ 4240.195486] Speaker: ASoC: close FE Speaker
[ 4240.195489] sof-audio-pci 0000:00:1f.3: pcm: close stream 2 dir 0
[ 4240.195495] SSP2-Codec: ASoC: BE SSP2-Codec event 2 dir 0
[ 4240.396839] sof-audio-pci 0000:00:1f.3: snd_sof_runtime_idle
[ 4240.396889] Speaker: ASoC: BE playback disconnect check for SSP2-Codec
[ 4240.396894] Speaker: freed DSP playback path Speaker -> SSP2-Codec
[ 4240.955251] sof-audio-pci 0000:00:1f.3: ipc rx: 0x90020000: GLB_TRACE_MSG
[ 4240.955633] sof-audio-pci 0000:00:1f.3: ipc rx done: 0x90020000: GLB_TRACE_MSG
[ 4242.695588] sof-audio-pci 0000:00:1f.3: snd_sof_runtime_suspend
[ 4242.695593] sof-audio-pci 0000:00:1f.3: sof_suspend
[ 4242.696097] sof-audio-pci 0000:00:1f.3: FW Poll Status: reg=0x20140000 successful
[ 4242.736888] sof-audio-pci 0000:00:1f.3: sof_send_pm_ctx_ipc
[ 4242.736917] sof-audio-pci 0000:00:1f.3: ipc tx: 0x40010000: GLB_PM_MSG: CTX_SAVE
[ 4242.737568] sof-audio-pci 0000:00:1f.3: ipc tx succeeded: 0x40010000: GLB_PM_MSG: CTX_SAVE
[ 4242.737606] sof-audio-pci 0000:00:1f.3: FW Poll Status: reg=0xf010f0f successful
[ 4242.737620] sof-audio-pci 0000:00:1f.3: FW Poll Status: reg=0xf0f successful
[ 4242.737627] sof-audio-pci 0000:00:1f.3: DSP core(s) enabled? 0 : core_mask 1
[ 4242.737705] sof-audio-pci 0000:00:1f.3: Debug PCIR: 00000010 at 00000044
[ 4242.925914] sof-audio-pci 0000:00:1f.3: error: failed to reset HDA controller gctl 0x1
[ 4242.925922] sof-audio-pci 0000:00:1f.3: error: failed to reset controller during suspend
[ 4242.925924] sof-audio-pci 0000:00:1f.3: error: failed to power down DSP during suspend -5
[ 4242.925931] sof-audio-pci 0000:00:1f.3: can't suspend (snd_sof_runtime_suspend [snd_sof] returned -5)
[ 4244.495513] Speaker: ASoC: found 4 audio playback paths
[ 4244.495519] rt711_rt1308 rt711_rt1308: ASoC: find BE for widget SSP2.OUT
[ 4244.495521] rt711_rt1308 rt711_rt1308: ASoC: try BE : ALH2.OUT
[ 4244.495522] rt711_rt1308 rt711_rt1308: ASoC: try BE : (not set)
[ 4244.495524] rt711_rt1308 rt711_rt1308: ASoC: try BE : SSP2.OUT
[ 4244.495527] Speaker: connected new DPCM playback path Speaker -> SSP2-Codec
[ 4244.495539] Speaker: ASoC: found 1 new BE paths
[ 4244.495542] SSP2-Codec: ASoC: open playback BE SSP2-Codec
[ 4244.495549] Speaker: ASoC: open FE Speaker
[ 4244.495554] sof-audio-pci 0000:00:1f.3: pcm: open stream 2 dir 0
[ 4244.495557] sof-audio-pci 0000:00:1f.3: period min 192 max 16384 bytes
[ 4244.495559] sof-audio-pci 0000:00:1f.3: period count 2 max 16
[ 4244.495560] sof-audio-pci 0000:00:1f.3: buffer max 65536 bytes
[ 4244.495846] sof-audio-pci 0000:00:1f.3: rate_min: 48000 rate_max: 48000
[ 4244.495849] sof-audio-pci 0000:00:1f.3: channels_min: 2 channels_max: 2
[ 4244.495851] SSP2-Codec: ASoC: hw_params BE SSP2-Codec
[ 4244.495855] sof-audio-pci 0000:00:1f.3: rate_min: 48000 rate_max: 48000
[ 4244.495858] sof-audio-pci 0000:00:1f.3: channels_min: 2 channels_max: 2
[ 4244.495861] rt1308 i2c-10EC1308:00: bclk_ms is 1 and pre_div is 1 for iis 0
[ 4244.495863] rt1308 i2c-10EC1308:00: lrck is 48000Hz and pre_div is 1 for iis 0
[ 4244.495870] rt1308 i2c-10EC1308:00: Update DAI routes for rt1308-aif playback
[ 4244.495873] rt1308 i2c-10EC1308:00: Connecting DAI route AIF1 Playback -> AIF1RX
[ 4244.495876] sof-audio-pci 0000:00:1f.3: rate_min: 48000 rate_max: 48000
[ 4244.495877] sof-audio-pci 0000:00:1f.3: channels_min: 2 channels_max: 2
[ 4244.495879] sof-audio-pci 0000:00:1f.3: Update DAI routes for SSP2 Pin playback
[ 4244.495883] Speaker: ASoC: hw_params FE Speaker rate 48000 chan 2 fmt 2
[ 4244.495885] sof-audio-pci 0000:00:1f.3: Update DAI routes for Speaker 2 playback
[ 4244.495887] sof-audio-pci 0000:00:1f.3: Connecting DAI route Passthrough Playback 2 -> PCM2P
[ 4244.495889] sof-audio-pci 0000:00:1f.3: pcm: hw params stream 2 dir 0
[ 4244.495893] sof-audio-pci 0000:00:1f.3: generating page table for 000000009bb05176 size 0xffc0 pages 16
[ 4244.495960] sof-audio-pci 0000:00:1f.3: FW Poll Status: reg=0x40000 successful
[ 4244.497163] sof-audio-pci 0000:00:1f.3: error: stream reset failed
[ 4244.497174] sof-audio-pci 0000:00:1f.3: error: hdac prepare failed: ffffff92
[ 4244.497178] sof-audio-pci 0000:00:1f.3: error: platform hw params failed
[ 4244.497191] sof-audio-pci 0000:00:1f.3: ASoC: 0000:00:1f.3 hw params failed: -110
[ 4244.497198] Speaker: ASoC: hw_params FE failed -110
[ 4244.497203] SSP2-Codec: ASoC: hw_free BE SSP2-Codec
[ 4244.497205] Speaker: ASoC: hw_free FE Speaker
[ 4244.497207] sof-audio-pci 0000:00:1f.3: pcm: free stream 2 dir 0
[ 4244.497225] SSP2-Codec: ASoC: hw_free BE SSP2-Codec
[ 4244.497300] SSP2-Codec: ASoC: close BE SSP2-Codec
[ 4244.497305] Speaker: ASoC: close FE Speaker
[ 4244.497307] sof-audio-pci 0000:00:1f.3: pcm: close stream 2 dir 0
[ 4244.497327] SSP2-Codec: ASoC: BE SSP2-Codec event 2 dir 0
[ 4244.497352] Speaker: ASoC: BE playback disconnect check for SSP2-Codec
[ 4244.497353] Speaker: freed DSP playback path Speaker -> SSP2-Codec
aplay_error_267_dmesg.log
aplay_error_267_logger_all.log
sof-test-playback.txt
Activity
[-][BUG][TGL]Dmesg shows "failed to reset HDA controller gctl 0x1" info when do aplay0,2(rt1308)) stop/restart stress test.[/-][+][BUG][TGL]Dmesg shows "failed to reset HDA controller gctl 0x1" info when do aplay0,2(rt1308) stop/restart stress test.[/+]plbossart commentedon Oct 30, 2019
@sinahuang can you try to run this test in nocodec mode with topic/sof-dev. I'd like to make sure some of the quirks we added for SoundWire don't get in the way (we have additional reset assert/de-assert cycles).
sinahuang commentedon Oct 31, 2019
This issue can not be reproduced on BYT MinnowBoard in nocodec mode and APL UP2 in nocodec mode with topic/sof-dev.
Test for 500 times.
Test recipe:
Kernel: https://github.com/thesofproject/linux/commits/topic/sof-dev commit: f5e70ce
FW: https://github.com/thesofproject/sof/commits/master commit: 59c9338
Tplg: sof-byt-nocodec.tplg and sof-apl-nocodec.tplg
Platform: APL UP2 in nocodec mode , BYT MinnowBoard in nocodec mode.
plbossart commentedon Oct 31, 2019
@sinahuang I meant nocodec mode on TGL! The MinnowBoard has a DSP that's 4 generations older so it's not the same test at all.
sinahuang commentedon Nov 1, 2019
Verified the issue on TGL RVP in nocodec mode with topic/sof-dev, issue can't be reproduced.
Test for 1000 times.
Test recipe:
Kernel: https://github.com/thesofproject/linux/commits/topic/sof-dev commit: c265f25
FW: https://github.com/thesofproject/sof/commits/master commit: fe321d6
Tplg: Same with TGL FW branch commit: sof-tgl-rt711-rt1308-nohdmi.tplg rename to sof-tgl-rt711-rt1308.tplg
Platform:TGL RVP, with codec ACL711 in SDW mode, ALC1308 in I2S mode
plbossart commentedon Nov 1, 2019
Thanks @sinahuang, that's useful information.
One of the differences with SoundWire is the commit 10f5534
this introduces an additional cycle of DSP reset and that's precisely the register that can't be modified without a timeout.
can you retest with topic/sof-dev + 10f5534 on top?
Thanks!
sinahuang commentedon Nov 5, 2019
Verified the issue on TGL RVP in nocodec mode with topic/sof-dev + 10f5534, issue can't be reproduced.
Test for 1000 times.
Test recipe:
Kernel: https://github.com/thesofproject/linux/commits/topic/sof-dev commit: 4e76b4a +10f5534
FW: https://github.com/thesofproject/sof/commits/master commit: 0b7fd03
Tplg: Same with TGL FW branch commit: sof-tgl-rt711-rt1308-nohdmi.tplg rename to sof-tgl-rt711-rt1308.tplg
Platform:TGL RVP, with codec ACL711 SDW mode, ALC1308 in I2S mode.
plbossart commentedon Nov 5, 2019
@sinahuang ok, so if the nocodec path is fine, can you try w/ SoundWire again, but this time disable the HDAudio link with make menuconfig?
sinahuang commentedon Nov 7, 2019
Verified the issue on TGL RVP with codec in SDW mode after disabling HDAudio link, issue can't be reproduced.
Test for 1000 times
Test recipe:
Kernel: https://github.com/thesofproject/linux/tree/integration/soundwire-latest commit: 35dd388
FW: https://github.com/thesofproject/sof/commits/master commit: 9844920
Tplg: Same with TGL FW branch commit: sof-tgl-rt711-rt1308-nohdmi.tplg rename to sof-tgl-rt711-rt1308.tplg
Platform: TGL RVP with codec ACL711 in SDW mode, ALC1308 in I2S mode.
More info: HDAudio links always can be disabled when I compiled the kernel on TGL RVP with codec in SDW mode.
plbossart commentedon Nov 7, 2019
thanks @sinahuang
What this means is that we have a difference in the initialization of our hardware, and somehow the code that's executed with SOF_HDA_LINK interferes with the controller reset.
Maybe a problem with the i915/HDMI stuff.
That's not an easy one to fix. @mengdonglin @wenqingfu FYI, we need folks to look into this.
sinahuang commentedon Nov 8, 2019
HDAudio links always are disabled in TGL SDW test, for blocked by TGL i915 issue.
Issue still can be reproduced on TGL RVP in SDW mode when do multiple_pipeline-test stress test.
Test for 500 time, failed at 81th time.
dmesg logger
Kernel:https://github.com/thesofproject/linux/commits/integration/soundwire-latest commit: b12acfe
FW: https://github.com/thesofproject/sof/commits/master commit: 5211799
Tplg: Same with TGL FW branch commit: sof-tgl-rt711-rt1308-nohdmi.tplg rename to sof-tgl-rt711-rt1308.tplg
Platform:TGL RVP with onboard ACL711 in SDW mod and ALC1308 in I2S mode.
multipipeline_test_81_dmesg.log
Scripts:
multiplex_pipeline-test.txt
sinahuang commentedon Nov 12, 2019
Test the issue with legacy interrupt mode, issue still can be reproduced when doing multiple_pipeline-test stress test.
Test for 500 times, failed at 207th time.
Test recipe
Kernel:https://github.com/thesofproject/linux/commits/integration/soundwire-latest commit: 391b5f9
FW: https://github.com/thesofproject/sof/commits/master commit: fb068e6
Tplg: Same with TGL FW branch commit: sof-tgl-rt711-rt1308-nohdmi.tplg rename to sof-tgl-rt711-rt1308.tplg
Platform:TGL RVP with onboard ACL711 in SDW mod and ALC1308 in I2S mode.
dmesg.log
sinahuang commentedon Nov 13, 2019
Test the issue with legacy interrupt mode, issue still can be reproduced when doing multiple_pipeline-test stress test.
Test for 500 times, failed at 205th time.
Scripts:
multiplex_pipeline-test-tgl.txt
Test recipe
Kernel:https://github.com/thesofproject/linux/tree/integration/soundwire-latest commit: fb40b85 + PR1451
FW: https://github.com/thesofproject/sof/commits/master commit: 835d876
Tplg: Same with TGL FW branch commit: sof-tgl-rt711-rt1308-nohdmi.tplg rename to sof-tgl-rt711-rt1308.tplg
Platform:TGL RVP with onboard ACL711 in SDW mod and ALC1308 in I2S mode.
multipipeline_test_205_all_logger.log
multipipeline_test_205_logger_error.log
multipipeline_test_205_dmesg.log
sinahuang commentedon Nov 14, 2019
Verified this issue with MSI mode on TGL RVP, issue can't be reproduced when doing multiple_pipeline-test stress test.
I will also check this issue with legacy mode later.
Test for 500 times.
Test recipe
Kernel:https://github.com/thesofproject/linux/tree/integration/soundwire-latest commit: 413de1
FW: https://github.com/thesofproject/sof/commits/master commit: decd206
Tplg: Same with TGL FW branch commit: sof-tgl-rt711-rt1308-nohdmi.tplg rename to sof-tgl-rt711-rt1308.tplg
Platform:TGL RVP with onboard ACL711 in SDW mod and ALC1308 in I2S mode.
sinahuang commentedon Nov 15, 2019
Verified this issue with legacy mode on TGL RVP, issue still can be reproduced when doing multiple_pipeline-test stress test.
Test for 500 times, failed at 164th time.
Scripts:
multiplex_pipeline-test-tgl.txt
Test recipe
Kernel:https://github.com/thesofproject/linux/tree/integration/soundwire-latest commit: 1bb06ec
FW: https://github.com/thesofproject/sof/commits/master commit: f004d11
Tplg: Same with TGL FW branch commit: sof-tgl-rt711-rt1308-nohdmi.tplg rename to sof-tgl-rt711-rt1308.tplg
Platform:TGL RVP with onboard ACL711 in SDW mod and ALC1308 in I2S mode.
multipipeline_test_164_dmesg.log
multipipeline_test_164_all_logger.log
sinahuang commentedon Nov 27, 2019
Issue still can be reproduced when doing suspeand & resume without playback stress test , but after occured "error: failed to reset HDA controller gctl 0x1" , dmesg shows "error: ipc timed out for 0x40010000 size 76" error info.
Test for 500 times , failed at 127th time.
To Reproduce
1.Boot up system;
2.Run "sudo ./stress_s3.sh 300 1 0";
Scripts:
stress-s3.txt
dmesg error log
Test recipe
Kernel:https://github.com/thesofproject/linux/tree/integration/soundwire-latest commit: d6078f9
FW: https://github.com/thesofproject/sof/commits/master commit: d740a3e
Tplg: Same with TGL FW branch commit: sof-tgl-rt711-rt1308-nohdmi.tplg rename to sof-tgl-rt711-rt1308.tplg
Platform:TGL RVP with onboard ACL711 in SDW mod and ALC1308 in I2S mode.
test_127.log
s3_test_127_logger_all.log
plbossart commentedon Dec 19, 2019
@sinahuang can you please retest on CML and TGL with a more recent kernel version. Thanks!
sinahuang commentedon Dec 20, 2019
Check this issue on TGL-RVP with soundwire-latest, test for 1000 times original issue cannot be reproduced, so closed it.
Test recipe
Kernel:https://github.com/thesofproject/linux/tree/integration/soundwire-latest commit: 121c751
FW: https://github.com/thesofproject/sof/commits/master commit: f587993
Tplg: Same with TGL FW branch : sof-tgl-rt711-rt1308.tplg
Platform:TGL-U RVP with codec ALC711 in SDW mode and RT1308 in I2S mode