[Flaky Test] Microsoft.Build.UnitTests.ToolTask_Tests.ToolTaskCanChangeCanonicalErrorFormat #13949

Flaky test: ToolTask_Tests.ToolTaskCanChangeCanonicalErrorFormat

Assembly: Microsoft.Build.Utilities.UnitTests

Evidence

• Distinct sources: 6 (5 rolling main builds + 1 approved PR)
• PR numbers: #13855
• Rolling build IDs: 1431561, 1433982, 1433984, 1434111, 1435069
• Affected legs/TFMs: CoreOnWindows, FullOnWindows
• First seen: 2026-05-22 | Last seen: 2026-05-26
• Sample build: https://dev.azure.com/dnceng-public/public/_build/results?buildId=1431561

Sample error

Assert.Contains() Failure: Sub-string not found
String:    "C:\Windows\system32\cmd.exe /C type "C:\Users\clou"···
Not found: "CS0168"

Notes

The test fails intermittently on Windows when checking that a canonical error format string appears in the tool's output. This is likely a timing or process-output-buffering race condition.

Fix guidance

Preferred fix: identify the source of nondeterminism (e.g. output buffering, process exit timing) and make a minimal determinism fix in the test.

Fallback: quarantine via [ActiveIssue("https://github.com/dotnet/msbuild/issues/NNNN")] from Microsoft.DotNet.XUnitV3Extensions (namespace Xunit) — not [Fact(Skip=...)].

Generated by Flaky Test Triage and Fix · sonnet46 4.5M · ◷