Repository navigation
An error occurred trying to start process 'llvm-symbolizer' #91975
Description
Looks like in some cases there is a failure to run llvm-symbolizer after a crash.
GC/API/Frozen/Frozen/Frozen.sh [FAIL]
apply_reg_state: ip and cfa unchanged; stopping here (ip=0xffea41ab8154)
waitpid() returned successfully (wstatus 00000000) WEXITSTATUS 0 WTERMSIG 0
/root/helix/work/workitem/e/GC/API/Frozen/Frozen/Frozen.sh: line 490: 6357 Aborted (core dumped) $LAUNCHER $ExePath "${CLRTestExecutionArguments[@]}"
Return code: 1
Raw output file: /root/helix/work/workitem/uploads/Reports/GC.API/Frozen/Frozen/Frozen.output.txt
Raw output:
BEGIN EXECUTION
/root/helix/work/correlation/corerun -p System.Reflection.Metadata.MetadataUpdater.IsSupported=false -p System.Runtime.Serialization.EnableUnsafeBinaryFormatterSerialization=true Frozen.dll ''
[createdump] Gathering state for process 6357 corerun
[createdump] Crashing thread 18ea signal 5 (0005)
[createdump] Writing crash report to file /home/helixbot/dotnetbuild/dumps/coredump.6357.dmp.crashreport.json
[createdump] Crash report successfully written
[createdump] Writing minidump with heap to file /home/helixbot/dotnetbuild/dumps/coredump.6357.dmp
[createdump] Written 713596928 bytes (174218 pages) to core file
[createdump] Target process is alive
[createdump] Dump successfully written in 1427ms
Expected: 100
Actual: 134
END EXECUTION - FAILED
Test failed. Trying to see if dump file was created in /home/helixbot/dotnetbuild/dumps since 9/12/2023 9:32:39 PM
Processing /home/helixbot/dotnetbuild/dumps/coredump.6357.dmp.crashreport.json
Printing stacktrace from '/home/helixbot/dotnetbuild/dumps/coredump.6357.dmp.crashreport.json'
Invoking llvm-symbolizer --pretty-print
Errors while running llvm-symbolizer --pretty-print
System.ComponentModel.Win32Exception (2): An error occurred trying to start process 'llvm-symbolizer' with working directory '/root/helix/work/workitem/e'. No such file or directory
at System.Diagnostics.Process.ForkAndExecProcess(ProcessStartInfo startInfo, String resolvedFilename, String[] argv, String[] envp, String cwd, Boolean setCredentials, UInt32 userId, UInt32 groupId, UInt32[] groups, Int32& stdinFd, Int32& stdoutFd, Int32& stderrFd, Boolean usesTerminal, Boolean throwOnNoExec)
at System.Diagnostics.Process.StartCore(ProcessStartInfo startInfo)
at System.Diagnostics.Process.Start()
at CoreclrTestLib.CoreclrTestWrapperLib.TryPrintStackTraceFromCrashReport(String crashReportJsonFile, TextWriter outputWriter)
Test Harness Exitcode is : 1
{
"ErrorMessage": "An error occurred trying to start process 'llvm-symbolizer' with working directory",
"BuildRetry": false,
"ErrorPattern": "",
"ExcludeConsoleLog": true
}Known issue validation
Build: 🔎 https://dev.azure.com/dnceng-public/public/_build/results?buildId=462152
Error message validated: An error occurred trying to start process 'llvm-symbolizer' with working directory
Result validation: ❌ Known issue did not match with the provided build.
Validation performed at: 11/7/2023 11:07:49 PM UTC
Report
Summary
| 24-Hour Hit Count | 7-Day Hit Count | 1-Month Count |
|---|---|---|
| 3 | 26 | 81 |
Metadata
Metadata
Assignees
Type
Projects
- Status
Activity
carlossanlop commentedon Nov 7, 2023
I hit this in my PR: #93906
kunalspathak commentedon Nov 7, 2023
@hoyosjs - do we know how we can address this problem?
hoyosjs commentedon Nov 8, 2023
I was gathering some data for this one. Things are looking like:
hoyosjs commentedon Nov 8, 2023
@carlossanlop that addresses the runtime queues/containers. I don't have the full list where it matters for libs.
1 remaining item
JulieLeeMSFT commentedon Jan 10, 2024
Removed the known build error label because it masks real crashes from PRs.
Example: https://github.com/dotnet/runtime/pull/96386/checks?check_run_id=20196770401
Juan will work on installing LLVM symbolizer on all test machines.
ericstj commentedon Jan 18, 2024
@hoyosjs just double checking that the expectation here is that this component needs to be added to the helix machines and is not already there nor is it expected to come with the test payload.
hoyosjs commentedon Jan 24, 2024
Oddly telemetry only reports this missing in macOS and ubuntu for inner loop scenarios. I can't see any other instances where it's missing except for one alpine group of containers (adding them in dotnet/dotnet-buildtools-prereqs-docker#966). Tracking getting in the queues in #91975.
carlossanlop commentedon Feb 26, 2024
@hoyosjs - Sharing some fresh hits from my closely related PR where I am forcing a crash in the common native System.IO.Compression.Brotli code to get some XUnitLogChecker results: #97135
Libraries Test Run checked coreclr linux_musl arm ReleaseExpand
v-wenyuxu commentedon May 27, 2024
Failed in: runtime-coreclr gcstress-extra 20240525.1
Failed tests:
Error message:
Stack trace:
More failures
dotnet-policy-service commentedon Apr 13, 2026
Tagging subscribers to this area: @dotnet/runtime-infrastructure
See info in area-owners.md if you want to be subscribed.