Skip to content

Enable Unix Helix crash dump capture - #7780

Merged
svick merged 5 commits into
dotnet:mainfrom
svick:test/helix-crash-capture
Oct 15, 2026
Merged

svick merged 5 commits into
dotnet:mainfrom
svick:test/helix-crash-capture

Conversation

@svick

@svick svick commented Oct 12, 2026 •

Copy link
Copy Markdown
Member

Summary

Enable .NET dumps, JSON crash reports, and dump-generation diagnostic logging in the Unix Helix runner when HELIX_DUMP_FOLDER is set. Document where to find the artifacts.

Motivation

While validating #7747, build 1629902 failed with an unrelated crash: the main Microsoft.ML.Tests process exited with SIGSEGV (139) on macOS x64 Debug. The main test process crashed while tests were still running, rather than during shutdown.

The console log ended just after an image-creation test started, but several tests were running concurrently. Helix uploaded no dump or crash report, despite the shell printing "core dumped", so there was no crashing-thread stack or process state to identify the faulting code. The available evidence was insufficient to determine the cause or establish whether the crash was related to the PR.

Enabling capture preserves evidence from future crashes instead of relying on the last test logged or waiting for a local reproduction. The JSON report provides thread stacks that can be inspected without a matching-OS debugger; the dump allows deeper analysis of memory, modules, and thread state with the matching runtime and native binaries. Writing them to Helix's dump directory makes them available as work-item artifacts, and diagnostic logging helps explain any capture failure.

Actual CI validation

Build 1632277 ran a temporary fatal native-memory-access probe on macOS x64 in both Debug and Release. Both processes exited with SIGSEGV (139), and both Helix work items uploaded a dump and readable JSON crash report. The reports identify the crashing thread and include DumpCaptureProbeTests.CaptureSegmentationFault plus native frames.

The probe and temporary test filtering were removed in a follow-up commit. The final diff contains only the runner configuration and documentation.

svick added 2 commits October 12, 2026 15:43
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

Copilot-Session: 7f213f15-6239-4a5b-b50e-de711637d646
Run only an intentional SIGSEGV probe in macOS x64 Helix work items. Remove this diagnostic commit after artifact capture is verified.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

Copilot-Session: 7f213f15-6239-4a5b-b50e-de711637d646
@codecov

codecov Bot commented Oct 12, 2026 •

Copy link
Copy Markdown

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 69.95%. Comparing base (2b30a85) to head (4e06579).
⚠️ Report is 9 commits behind head on main.

Additional details and impacted files
@@           Coverage Diff           @@
##             main    #7780   +/-   ##
=======================================
  Coverage   69.94%   69.95%           
=======================================
  Files        1419     1419           
  Lines      272603   272603           
  Branches    27562    27562           
=======================================
+ Hits       190680   190701   +21     
+ Misses      73728    73707   -21     
  Partials     8195     8195           
Flag Coverage Δ
Debug 69.95% <ø> (+<0.01%) ⬆️
production 64.14% <ø> (+<0.01%) ⬆️
test 89.69% <ø> (ø)

Flags with carried forward coverage won't be shown. Click here to find out more.
see 7 files with indirect coverage changes

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

svick added 2 commits October 12, 2026 18:23
raise(SIGSEGV) produced artifacts but the runtime resumed execution. Use an invalid native memset destination to verify actual process-crash capture.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

Copilot-Session: 7f213f15-6239-4a5b-b50e-de711637d646
macOS x64 Debug and Release in build 1632277 both exited with SIGSEGV (139) and uploaded a dump plus readable JSON crash report without signing.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

Copilot-Session: 7f213f15-6239-4a5b-b50e-de711637d646
@svick svick changed the title Validate Unix Helix crash dump capture Enable Unix Helix crash dump capture Oct 12, 2026
@svick
svick marked this pull request as ready for review October 14, 2026 16:34
@svick
svick requested review from matouskozak, rosebyte and vejbomar and a balanced review from Copilot October 14, 2026 16:34

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🟢 Approval recommended

The narrowly scoped configuration matches the documentation, with no identified blocking issues.

0 open findings

What changed in this PR

Adds crash evidence capture to ML.NET’s Unix Helix test runner to help diagnose native test-process crashes.

Changes:

  • Enables .NET dumps, JSON crash reports, and diagnostic logging when HELIX_DUMP_FOLDER is set.
  • Documents artifact retrieval and analysis.
File Description
eng/​testing/​runTests.sh Conditionally enables crash capture in Helix’s dump directory.
docs/​project-docs/​developer-guide.md Explains where to find artifacts and how to inspect them.

🧠 Review effort: Balanced


💡 Add a code-review agent skill or configure MCP servers for context-aware, tailored reviews. Learn more in the docs.

@matouskozak matouskozak left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM!

@svick
svick merged commit 6004134 into dotnet:main Oct 15, 2026
28 checks passed
@svick
svick deleted the test/helix-crash-capture branch October 15, 2026 15:10
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants