Skip to content

Fix intermittent OSX CI failures in StartsWithNoMatch_StringComparison and EndsWithNoMatch_StringComparison - #125963

Merged
danmoseley merged 4 commits into
mainfrom
copilot/fix-osx-failure-string-tests
Apr 8, 2026
Merged

danmoseley merged 4 commits into
mainfrom
copilot/fix-osx-failure-string-tests

Conversation

Copilot AI commented Mar 23, 2026 •

Copy link
Copy Markdown
Contributor

Apple's NSString compare:options:range:locale: is non-deterministic for certain Unicode inputs (C1 control chars \u0080–\u009F), causing StartsWithNoMatch_StringComparison and EndsWithNoMatch_StringComparison to fail intermittently on macOS. The existing [ActiveIssue] only covered IsAppleMobile (iOS/tvOS), leaving macOS unguarded.

Changes

  • StartsWithNoMatch_StringComparison: Add [ActiveIssue(..., IsOSX)] referencing issue [Apple-mobile] System.Tests.StringTests.[Starts,Ends]WithNoMatch_StringComparison test failure #108832; replace four separate Assert.Equal calls with a foreach loop using Assert.True(expected == actual, message) that captures length, mismatchIndex, comp, char values at the mismatch index, and the expected/actual boolean values for easier debugging when re-enabled.

  • EndsWithNoMatch_StringComparison: Same fix — same root cause (same Apple native API path), same [ActiveIssue] gap, same diagnostic improvement applied consistently.

The diagnostic message format:

String.StartsWith: length=3, mismatchIndex=1, comp=CurrentCulture, chars=(2,3), expected=True, actual=False
Span.StartsWith:   length=3, mismatchIndex=1, comp=CurrentCulture, chars=(2,3), expected=True, actual=False
Original prompt

This section details on the original issue you should resolve

<issue_title>OSX failure on StringTests.StartsWithNoMatch_StringComparison</issue_title>
<issue_description>## Build Information
Build: https://dev.azure.com/dnceng-public/cbb18261-c48f-4abb-8651-8cdcb5474649/_build/results?buildId=940013
Build error leg or test failing: System.Tests.StringTests.StartsWithNoMatch_StringComparison
Pull request: #111877

Error Message

Fill the error message using step by step known issues guidance.

{
  "ErrorMessage": "[FAIL] System.Tests.StringTests.StartsWithNoMatch_StringComparison",
  "ErrorPattern": "",
  "BuildRetry": false,
  "ExcludeConsoleLog": false
}

Log

[17:27:34] dbug: 2025-02-04 17:27:34.201 Df System.Runtime.Tests[12401:3ab5fc] Failed tests:
[17:27:34] dbug: 2025-02-04 17:27:34.201 Df System.Runtime.Tests[12401:3ab5fc] 1) 	[FAIL] System.Tests.StringTests.StartsWithNoMatch_StringComparison   Test name: System.Tests.StringTests.StartsWithNoMatch_StringComparison
[17:27:34] dbug: Assembly:  [System.Runtime.Tests, Version=10.0.0.0, Culture=neutral, PublicKeyToken=cc7b13ffcd2ddd51]
[17:27:34] dbug: 2025-02-04 17:27:34.201 Df System.Runtime.Tests[12401:3ab5fc]    Exception messages: Assert.Equal() Failure: Values differ
[17:27:34] dbug: Expected: True
[17:27:34] dbug: Actual:   False   Exception stack traces:    at System.Tests.StringTests.StartsWithNoMatch_StringComparison()

Known issue validation

Build: 🔎 https://dev.azure.com/dnceng-public/public/_build/results?buildId=940013
Error message validated: [[FAIL] System.Tests.StringTests.StartsWithNoMatch_StringComparison]
Result validation: ❌ Known issue did not match with the provided build.
Validation performed at: 2/5/2025 4:19:32 PM UTC

Report

Build Definition Test Pull Request
1316114 dotnet/runtime System.Runtime.Tests.WorkItemExecution #125056

Summary

24-Hour Hit Count 7-Day Hit Count 1-Month Count
0 0 1

Comments on the Issue (you are @copilot in this section)

@tarekgh @matouskozak please triage. Thanks! @matouskozak Looks like duplicate of https://github.com//issues/108832 but with slightly different error message. fyi: @kotlarmilos @danmoseley > [!NOTE] > This analysis was generated by GitHub Copilot.

Analysis: Root cause identifiable from code inspection

A dump or repro isn't strictly necessary here — this is a wrong-return-value bug, not a crash, so a dump wouldn't add much. The root cause is identifiable from the code.

What the test does

The test (StartsWithNoMatch_StringComparison) creates strings from chars \u0001–\u0096 (control characters + basic Latin), introduces a single-char mismatch at each position, then asserts that culture-aware StartsWith returns consistent results between calls:

Assert.Equal(
    firstSpan.ToString().StartsWith(secondSpan.ToString(), StringComparison.CurrentCulture),
    firstSpan.StartsWith(secondSpan, StringComparison.CurrentCulture));

The failure (Expected: True, Actual: False) means the same underlying comparison returns different results on different invocations for the same input.

Where it goes wrong

On Apple mobile in hybrid globalization mode, culture-aware StartsWith routes through GlobalizationNative_StartsWithNative in pal_collation.m:

NSString *prefixStrComposed = RemoveWeightlessCharacters(
    prefixString.precomposedStringWithCanonicalMapping);
NSString *sourceStrComposed = RemoveWeightlessCharacters(
    sourceString.precomposedStringWithCanonicalMapping);

int32_t result = (int32_t)[sourceStrComposed compare:prefixStrComposed
                                    ...

</details>



<!-- START COPILOT CODING AGENT SUFFIX -->

- Fixes dotnet/runtime#112195

<!-- START COPILOT CODING AGENT TIPS -->
---

✨ Let Copilot coding agent [set things up for you](https://github.com/dotnet/runtime/issues/new?title=✨+Set+up+Copilot+instructions&body=Configure%20instructions%20for%20this%20repository%20as%20documented%20in%20%5BBest%20practices%20for%20Copilot%20coding%20agent%20in%20your%20repository%5D%28https://gh.io/copilot-coding-agent-tips%29%2E%0A%0A%3COnboard%20this%20repo%3E&assignees=copilot) — coding agent works faster and does higher quality work when set up for your repo.

@dotnet-policy-service

Copy link
Copy Markdown
Contributor

Tagging subscribers to this area: @dotnet/area-infrastructure-libraries
See info in area-owners.md if you want to be subscribed.

Copilot AI and others added 2 commits March 23, 2026 15:04
…put in StartsWithNoMatch_StringComparison

Co-authored-by: kotlarmilos <11523312+kotlarmilos@users.noreply.github.com>
Agent-Logs-Url: https://github.com/dotnet/runtime/sessions/ebe921f3-5e85-4102-be01-a6bd6fffada3
…improve diagnostic output

Co-authored-by: kotlarmilos <11523312+kotlarmilos@users.noreply.github.com>
Agent-Logs-Url: https://github.com/dotnet/runtime/sessions/ebe921f3-5e85-4102-be01-a6bd6fffada3
Copilot AI changed the title [WIP] Fix OSX failure on StringTests.StartsWithNoMatch_StringComparison Fix intermittent OSX CI failures in StartsWithNoMatch_StringComparison and EndsWithNoMatch_StringComparison Mar 23, 2026
Copilot AI requested a review from kotlarmilos March 23, 2026 15:10
@tarekgh tarekgh added the os-macos macOS aka OSX label Mar 23, 2026
@danmoseley
danmoseley requested review from Copilot and removed request for kotlarmilos March 28, 2026 14:53
@danmoseley
danmoseley marked this pull request as ready for review March 28, 2026 14:53

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Updates StringTests to avoid intermittent macOS test failures caused by non-deterministic Apple collation behavior for certain Unicode inputs, by expanding existing platform guards and improving failure diagnostics.

Changes:

  • Add [ActiveIssue(..., IsOSX)] to EndsWithNoMatch_StringComparison and StartsWithNoMatch_StringComparison to disable these tests on macOS (in addition to existing Apple mobile coverage).
  • Refactor repeated culture-comparison assertions into a loop and add richer diagnostic messages when the assertions fail.

@danmoseley

Copy link
Copy Markdown
Contributor

@copilot address this

…EndsWith assertions

Agent-Logs-Url: https://github.com/dotnet/runtime/sessions/dd927ad2-980d-4369-895a-0a96fe17f1ca

Co-authored-by: danmoseley <6385855+danmoseley@users.noreply.github.com>

Copilot AI commented Mar 28, 2026

Copy link
Copy Markdown
Contributor Author

@copilot address this

Done in 47fc187. All four Assert.True messages now include , expected={expected}, actual={actual} so the boolean values are visible in the failure output alongside the existing context (length, mismatchIndex, comp, chars).

Copilot AI requested a review from danmoseley March 28, 2026 15:33
@danmoseley
danmoseley requested a review from tarekgh April 8, 2026 19:09
@tarekgh tarekgh added this to the 11.0.0 milestone Apr 8, 2026

@tarekgh tarekgh left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM. Thanks @danmoseley!

@danmoseley
danmoseley merged commit a358478 into main Apr 8, 2026
94 of 97 checks passed
@danmoseley
danmoseley deleted the copilot/fix-osx-failure-string-tests branch April 8, 2026 20:38
@github-actions github-actions Bot locked and limited conversation to collaborators May 9, 2026
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

5 participants