Skip to content

[net11] Limit Mono, iOS 18.5, and MacCatalyst tests to release validation - #39550

Open
kubaflo wants to merge 2 commits into
mainfrom
kubaflo-net11-test-lanes
Open

kubaflo wants to merge 2 commits into
mainfrom
kubaflo-net11-test-lanes

Conversation

@kubaflo

@kubaflo kubaflo commented Oct 22, 2026

Copy link
Copy Markdown
Contributor

Note

Are you waiting for the changes in this PR to be merged?
It would be very helpful if you could test the resulting artifacts from this PR and let us know in a comment if this change resolves your issue. Thank you!

Description of Change

Intentionally targets net11.0, as requested. Reserve the following expanded coverage for release/* and the exact inflight/candidate branch:

  • Mono Android/iOS device-test build and run stages.
  • MacCatalyst device-test and UI-test stages, including the legacy device-test entry point.
  • iOS 18.5 UI-test jobs for both CoreCLR and NativeAOT.

Pull requests use their target branch; other builds use their source branch. Normalize both plain branch names and refs/heads/ names. Evaluate the shared allowlist at runtime because System.PullRequest.TargetBranch is unavailable during template expansion.

Regular net11.0 builds and PRs skip these lanes. inflight/current and other non-allowlisted branches also skip them, and BuildEverything cannot bypass the restriction. Existing iOS version selection is preserved on eligible branches: iOS 18.5 is included for non-PR builds or when BuildEverything is enabled. Latest-iOS, CoreCLR Android, and Windows coverage remains unchanged.

Unlike #39052, which proposes removing Mono coverage on main for .NET 12, this keeps the .NET 11 Mono smoke coverage available for release validation.

Updated the UI testing guide with the branch policy.

Validation

  • git diff --check - passed.
  • python3 -c "import pathlib, yaml; paths = list(pathlib.Path('eng/pipelines').rglob('*.yml')); [yaml.safe_load(p.read_text()) for p in paths]; print(f'Parsed {len(paths)} pipeline YAML files')" - passed, all 39 pipeline YAML files parsed.
  • Azure DevOps server-side previews for maui-pr-devicetests and maui-pr-uitests - compiled successfully. The UI preview included BuildEverything=true, BuildNativeAOT=true, and RunNativeAOT=true, confirming the iOS 18.5 conditions on both runtime variants without changing the latest-iOS jobs.
  • Pipeline previews validate template expansion, not runtime condition execution; device/UI execution is left to PR CI. No local UI automation was run.

Issues Fixed

N/A - CI configuration change requested directly; no linked issue.

Copilot AI balanced review requested due to automatic review settings October 22, 2026 10:59
@kubaflo
kubaflo temporarily deployed to copilot-pat-pool October 22, 2026 10:59 — with GitHub Actions Inactive
@github-actions

Copy link
Copy Markdown
Contributor

🚀 Dogfood this PR with:

⚠️ WARNING: Do not do this without first carefully reviewing the code of this PR to satisfy yourself it is safe.

curl -fsSL https://raw.githubusercontent.com/dotnet/maui/main/eng/scripts/get-maui-pr.sh | bash -s -- 39550

Or

  • Run remotely in PowerShell:
iex "& { $(irm https://raw.githubusercontent.com/dotnet/maui/main/eng/scripts/get-maui-pr.ps1) } 39550"

@kubaflo
kubaflo temporarily deployed to copilot-pat-pool October 22, 2026 11:00 — with GitHub Actions Inactive
@azure-pipelines

This comment has been minimized.

@kubaflo

This comment has been minimized.

@kubaflo

This comment has been minimized.

@azure-pipelines

This comment has been minimized.

1 similar comment
@azure-pipelines

This comment has been minimized.

@kubaflo
kubaflo temporarily deployed to copilot-pat-pool October 22, 2026 11:00 — with GitHub Actions Inactive
@kubaflo
kubaflo temporarily deployed to copilot-pat-pool October 22, 2026 11:02 — with GitHub Actions Inactive
@kubaflo
kubaflo temporarily deployed to copilot-pat-pool October 22, 2026 11:04 — with GitHub Actions Inactive
@kubaflo
kubaflo temporarily deployed to copilot-pat-pool October 22, 2026 11:04 — with GitHub Actions Inactive

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🔵 Needs a closer look

Reported template previews validate expansion but do not verify runtime coverage gates on eligible and excluded branches.

No open issues 🎉

What changed in this PR

This PR reserves expanded .NET 11 test coverage for release/* and the exact inflight/candidate branch, retaining those lanes for release validation rather than removing them.

Changes:

  • Adds a shared runtime allowlist using normalized PR target or build source branches.
  • Gates Mono device tests, MacCatalyst device/UI tests, and iOS 18.5 UI jobs without changing latest-iOS, CoreCLR Android, or Windows coverage.
  • Documents the branch policy and existing iOS version-selection behavior.
File Description
eng/​pipelines/​common/​variables.yml Defines normalized branch selection and the shared runtime allowlist.
eng/​pipelines/​common/​ui-tests.yml Gates iOS 18.5 jobs and MacCatalyst UI tests.
eng/​pipelines/​common/​device-tests.yml Gates legacy MacCatalyst device-test stages.
eng/​pipelines/​arcade/​stage-device-tests.yml Gates Mono build/run stages and MacCatalyst device tests.
docs/​UITesting-Guide.md Documents .NET 11 coverage restrictions and BuildEverything behavior.

Review effort: Balanced

@github-actions github-actions Bot added the s/agent-review-in-progress AI review is currently running for this PR label Oct 22, 2026
@kubaflo

kubaflo commented Oct 23, 2026

Copy link
Copy Markdown
Contributor Author

/rebase

@github-actions
github-actions Bot force-pushed the kubaflo-net11-test-lanes branch from 4cf0244 to 4027ee7 Compare October 23, 2026 10:03
Copilot AI balanced review requested due to automatic review settings October 23, 2026 10:03

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🔵 Needs a closer look

Template previews confirm expansion, but execution of the shared runtime gates on eligible and ineligible branches remains unverified.

No open issues 🎉

Review effort: Balanced

@kubaflo

This comment has been minimized.

@azure-pipelines

This comment has been minimized.

@github-actions github-actions Bot added s/agent-review-in-progress AI review is currently running for this PR and removed s/agent-review-in-progress AI review is currently running for this PR labels Oct 23, 2026
@MauiBot

MauiBot commented Oct 23, 2026

Copy link
Copy Markdown
Collaborator

PR Test Gate

@kubaflo — existing-test verification for commit 4027ee72.

Result SKIPPED Platform Android Commit 4027ee72


🧪 Gate analysis — click to expand

⚠ Skipped. No runnable PR tests were selected. The fix has not been verified by this gate.

No interpretable test-group summary was produced. No additional test execution is inferred.

PR head: 4027ee72.

Without-fix baseline: e6d22838.

@MauiBot MauiBot removed the s/agent-review-in-progress AI review is currently running for this PR label Oct 23, 2026
@kubaflo
kubaflo force-pushed the kubaflo-net11-test-lanes branch from 4027ee7 to f0051c1 Compare October 25, 2026 22:35
Copilot AI balanced review requested due to automatic review settings October 25, 2026 22:36
@kubaflo

This comment has been minimized.

@azure-pipelines

This comment has been minimized.

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🔵 Needs a closer look

Template previews succeeded, but runtime gating across eligible and excluded PR and non-PR branches still needs execution validation.

No open issues 🎉

Review effort: Balanced

@kubaflo

This comment has been minimized.

@azure-pipelines

This comment has been minimized.

@PureWeen PureWeen left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Adversarial review

Agent-assisted assessment by an independent reviewer of exact head f0051c18ed54c43bfc5686f68d18e21bae65affe.

No new actionable PR-caused defects survived verification and adversarial consensus. Reviewed the complete authoritative five-file diff, full changed files, current and legacy pipeline consumers, and relevant history.

The shared branch variables are declared in runtime evaluation order, and the affected entry points import them at pipeline scope. The eight added conditions retain succeeded() while restricting Mono build/run stages, MacCatalyst stages, and the explicit iOS 18.5 matrix jobs. The existing latest jobs remain unchanged; their generic simulator-target/UDID path was traced through the test runner rather than inferring the actual simulator version from --apiversion alone.

Evidence and limits: Exact-head YAML parsing and a deterministic branch-policy matrix were checked locally, including PR target versus non-PR source selection, qualified/unqualified branch names, allowed release/candidate branches, and excluded ordinary branches. These checks do not execute Azure runtime conditions. No Azure pipeline, device/UI test run, or independent server-side template preview was performed. The three existing reviews already note the missing executed runtime branch-matrix validation; that limitation is not a new finding. The complete prior-feedback inventory contained no inline review comments. Thread resolution status is unknown; no thread-state operations were performed.

Methodology: 3 independent reviewers with adversarial consensus, including retained-reviewer cross-examination of the only remaining documentation candidate; its claimed consequence was not established.

Scope: code review only. CI/check status and merge readiness were not evaluated; use the repository's CI-analysis workflow for that assessment.

kubaflo and others added 2 commits October 27, 2026 18:54
Gate Mono device tests, MacCatalyst tests, and the iOS 18.5 UI matrix on release branches or inflight/candidate, including PR target branches.

Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
Copilot-Session: a1da4931-4253-4b38-a31b-ada5592916d2
PR target branches are unavailable during template expansion. Apply the branch allowlist at runtime and gate iOS 18.5 jobs for both CoreCLR and NativeAOT without changing the existing matrix selection.

Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
Copilot-Session: a1da4931-4253-4b38-a31b-ada5592916d2
@kubaflo
kubaflo force-pushed the kubaflo-net11-test-lanes branch from f0051c1 to aed27b4 Compare October 27, 2026 17:54
Copilot AI balanced review requested due to automatic review settings October 27, 2026 17:54
@kubaflo

This comment has been minimized.

@azure-pipelines

This comment has been minimized.

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🔵 Needs a closer look

Template previews do not verify runtime gating, so execution evidence is still needed to confirm eligible branches retain coverage and excluded branches skip only the intended lanes.

No open issues 🎉

Review effort: Balanced

@PureWeen PureWeen left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Adversarial review

Agent-assisted assessment by an independent reviewer of exact head aed27b4106ec8316793fa3c300b7eaede397def0.

No new actionable PR-caused defects survived verification and adversarial consensus. All three reviewers independently examined the complete authoritative five-file, ten-hunk diff, full current-head files, relevant pipeline consumers and dependencies. This is a fresh assessment, not an inherited conclusion from the earlier head.

The shared runtime variables are declared in evaluation order and imported at pipeline scope by the affected entry points. All eight added conditions preserve succeeded(). The explicit iOS 18.5 jobs remain distinct from the ungated latest jobs, and BuildEverything does not bypass the branch allowlist. The title and description match this implementation.

Evidence and limits: Local YAML parsing, deterministic branch-policy simulations, source/history traces, and authoritative Azure expression/condition documentation were checked. Those simulations do not execute Azure's runtime expression engine. No Azure pipeline execution, independent server-side preview, build, Helix submission, or device/UI run was performed. Executed runtime coverage across eligible and excluded PR/non-PR branches therefore remains unverified; this limitation was already identified in prior feedback, not a new defect or a substitute for execution evidence.

The complete prior-feedback inventory contained five reviews, sixteen issue comments, and no inline review comments or replies. Existing runtime-validation observations were not re-raised as new findings. Thread resolution status is unknown; no thread-state operations were performed.

Methodology: 3 independent reviewers with adversarial consensus.

Scope: code review only. CI/check status and merge readiness were not evaluated; use the repository's CI-analysis workflow for that assessment.

@kubaflo
kubaflo changed the base branch from net11.0 to main November 2, 2026 10:33

This branch was previously deployed

1 inactive (outdated) deployment
copilot-pat-pool — 4cf02442 Deployed Oct 22, 2026 by kubaflo via conclusion #2452
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants