Skip to content

[run-tests trial 39487] [iOS/Mac] Fix ImageButton background rendering with oversized CornerRadius - #40449

Closed
kubaflo wants to merge 16 commits into
dotnet:mainfrom
kubaflo:run-tests-real-39487
Closed

kubaflo wants to merge 16 commits into
dotnet:mainfrom
kubaflo:run-tests-real-39487

Conversation

@kubaflo

@kubaflo kubaflo commented Nov 19, 2026

Copy link
Copy Markdown
Contributor

Warning

Do not merge. Do not review. Temporary trial PR for /review run-tests (#40305). It will be closed once the trial finishes.

This branch is #40305 plus the change from #39487, so maui-pr-uitests here uses the new selection parameters. GPT-6.1 Sol and Claude Opus 5.5 each chose UI tests for the same diff; only the union of their choices runs.

Copilot CI added 16 commits November 16, 2026 21:39
The agent picks platforms and UI test categories from the PR diff. A trusted
safe-output job validates the choice against the pipeline inventory, queues
maui-pr-uitests as a pull-request build so Azure Pipelines reports native
checks on the PR head, publishes a Skipped check for every platform and
category that was not selected, and posts a collapsible summary comment that
is updated when the run finishes.
The agent reads the collected context and uses GitHub read tools; it never
needs PR code on disk.
The previous expression also enabled the zero-test guard in the default
full run, which failed every job whose test step did not produce results.
The dispatcher always sets selectedRun for targeted runs.
Untyped template parameters receive the boolean as the string 'False',
which is truthy in a bare ${{ if }}.
…nly selection

- Hand the AI proposal to review-run-tests-dispatch.yml. Jobs in the
  copilot-pat-pool environment get an OIDC subject the Azure DevOps
  federated credential does not trust; the dispatch workflow runs on main
  without an environment, like review-trigger.yml.
- Queue only after GitHub's test-merge commit is confirmed to merge the
  selected head, and cancel any build whose source version differs.
- Evaluate #if guards per platform to know which categories have enabled
  tests (including the PR's own changed tests). Drop categories and
  platforms that would run zero tests, and pass platform-qualified
  categories (ios:Window) when a category only runs on some platforms.
- Pass the sample-build platforms to PowerShell as template-evaluated
  booleans instead of interpolating parameter text.
- Teach the rebalance-ui-test-categories skill the typed parameter form.
- Add regression tests for the full-run guard, the qualified matrix
  filter, the build-script inputs and the skill.
An agent can emit its proposal and then exit non-zero; dispatching both
the proposal and the fallback would queue two runs for one command.
The run-tests comment now lists the failing tests with the platforms they
failed on, read from the public Azure DevOps test results API. When the
maui-pr-uitests check completes again (failed jobs re-run), finished comments
for that commit are refreshed too, and the list notes that it may include
failures from earlier attempts.
The workflow posts run-tests comments as its bot; -CommentAuthor lets a maintainer refresh comments from a manual Invoke-RunTests trial with the same script.
- `/review run-tests --all` (and the workflow_dispatch `all` input) skips the
  AI and queues the full suite.
- PRs into release/* and candidate/* always get the full suite.
- maui-pr-uitests only applies a selection to pull-request builds. Branch,
  scheduled and manual builds of main, release/*, candidate/* and the other
  CI branches always run everything, whatever they were queued with.
- Comments for a required full suite say so instead of calling it an AI
  selection, and skipped sections point at `--all`.
/azp run maui-pr-uitests already runs the full suite, so the flag only duplicated it. PRs into release/* and candidate/* still get the full suite without an AI selection.
…at they pick

- copilot-review-run-tests (GPT-6.1 Sol) collects the context and selects, then
  hands over to the new copilot-review-run-tests-opus (Claude Opus 5.5), which
  selects from the same context with the same shared instructions and
  dispatches review-run-tests-dispatch.yml with both runs.
- The dispatcher validates each proposal on its own and runs the union per
  platform. A test runs only if at least one model selected it.
- No full-suite fallbacks: release/candidate PRs go through the models like any
  other PR, a missing proposal from both models queues nothing, and unknown
  categories run nothing instead of everything.
- Nothing is added for the models: the Material3 lane only runs when Material3
  is selected (in the pipeline too), and Android is never added for it.
- Only the declared proposal fields are read from agent output, and each run may
  only provide its own model's proposal. A proposal that exists but cannot be
  downloaded fails the dispatch instead of being dropped.
- The comment shows what each model chose per platform and both rationales.
…g with oversized CornerRadius

Copy of dotnet#39487 for a /review run-tests trial. Do not merge.
@kubaflo
kubaflo temporarily deployed to copilot-pat-pool November 19, 2026 18:51 — with GitHub Actions Inactive
@github-actions

Copy link
Copy Markdown
Contributor

🚀 Dogfood this PR with:

⚠️ WARNING: Do not do this without first carefully reviewing the code of this PR to satisfy yourself it is safe.

curl -fsSL https://raw.githubusercontent.com/dotnet/maui/main/eng/scripts/get-maui-pr.sh | bash -s -- 40449

Or

  • Run remotely in PowerShell:
iex "& { $(irm https://raw.githubusercontent.com/dotnet/maui/main/eng/scripts/get-maui-pr.ps1) } 40449"

@kubaflo
kubaflo temporarily deployed to copilot-pat-pool November 19, 2026 18:51 — with GitHub Actions Inactive
@azure-pipelines

This comment has been minimized.

@github-actions

Copy link
Copy Markdown
Contributor

Skill Validation Results

@kubaflo — new skill validation results are available based on this last commit: 77b00e3.
To request a fresh validation after new comments or commits, comment /evaluate-skills.

Overall Failed Static Failed LLM Needs attention Skills 0 Agents 0

❌ Skill Validation Results — 77b00e3 · [run-tests trial 39487] [iOS/Mac] Fix ImageButton background rendering with oversized CornerRadius · 2026-11-19T18:51:25Z

❌ Static Checks Failed

Skills: 0 | Eval specs linted:

⏭️ LLM Evaluation: Skipped

No changed skills with eval specs found.

🔍 Full results and investigation steps

@kubaflo
kubaflo temporarily deployed to copilot-pat-pool November 19, 2026 18:52 — with GitHub Actions Inactive
@kubaflo

kubaflo commented Nov 19, 2026 •

Copy link
Copy Markdown
Contributor Author

Selected UI Tests

@kubaflo — AI-selected UI tests (GPT-6.1 Sol + Claude Opus 5.5) for commit 77b00e3f.

Result FAILED Platforms iOS, MacCatalyst Categories 2 of 77 Commit 77b00e3f


🧪 Test selection — click to expand

❌ maui-pr-uitests #1690674 failed. See the failing checks below or the run for details.

GPT-6.1 Sol: Run Button and ImageButton UI tests on iOS and MacCatalyst for corner-radius clamping and background updates after resizing.
Claude Opus 5.5: iOS/MacCatalyst Button and ImageButton corner-radius clamping and arrange-time background layer update; runs Button and ImageButton UI tests on iOS and MacCatalyst.

🧪 Ran — 2 categories on iOS, MacCatalyst

Platform Categories
iOS Button, ImageButton
MacCatalyst Button, ImageButton
📊 Results — failed: 3 passed, 2 failed of 5 jobs
Job Result
Build UITests CoreCLR Sample App Build Sample App ✅ succeeded
iOS UITests CoreCLR Controls (vlatest) Button ❌ failed
iOS UITests CoreCLR Controls (vlatest) ImageButton ✅ succeeded
MacCatalyst UITests CoreCLR Controls Button ❌ failed
MacCatalyst UITests CoreCLR Controls ImageButton ✅ succeeded

Failed tests (1)

Test Platforms
ButtonFeatureTests.Button_SetBorderColorAndTextColor_VerifyVisualState iOS, MacCatalyst
⏭️ Skipped — 75 categories, 2 platforms

Platforms not run: Android, Windows

Categories not run: Accessibility, ActionSheet, ActivityIndicator, Animation, AppLinks, Border, BoxView, Brush, CarouselView, CollectionView1, CollectionView2, CollectionView3, CollectionView4, CollectionView5, CollectionView6, CollectionView7, Cells, CheckBox, ContextActions, CustomRenderers, DatePicker, Dispatcher, DisplayAlert, DisplayPrompt, DragAndDrop, Entry, Editor, Effects, Essentials, FlyoutPage, Focus, Fonts, Frame, Gestures, GraphicsView, Image, IndicatorView, InputTransparent, IsEnabled, IsVisible, Label, Layout, Lifecycle, ManualReview, Maps, Navigation, Page, Performance, Picker, ProgressBar, RadioButton, RefreshView, SafeAreaEdges, Shadow, ScrollView, SearchBar, Shape, Slider, SoftInput, Stepper, Switch, SwipeView, Shell1, Shell2, TabbedPage, TableView, TimePicker, TitleView, ToolbarItem, Triggers, ViewBaseTests, VisualStateManager, Window, WebView, Material3

Each skipped platform and category is listed as a Skipped maui-pr-uitests (skipped) … check on this commit. Use /azp run maui-pr-uitests for the full suite.

🤖 Why these tests — AI rationale (GPT-6.1 Sol + Claude Opus 5.5)

Each model chose independently; every test that runs was selected by at least one of them.

Platform GPT-6.1 Sol Claude Opus 5.5
Android — —
iOS Button, ImageButton Button, ImageButton
MacCatalyst Button, ImageButton Button, ImageButton
Windows — —

GPT-6.1 Sol — Run Button and ImageButton UI tests on iOS and MacCatalyst for corner-radius clamping and background updates after resizing.

  • iOS Button/ImageButton handlers change resize-time corner radii; MacCatalyst shares them.
  • LayerExtensions and ViewExtensions changes target button-stroke backgrounds, not base layout.
  • Device tests add no UI categories; omit Android, Windows, Material3 and full run.

Claude Opus 5.5 — iOS/MacCatalyst Button and ImageButton corner-radius clamping and arrange-time background layer update; runs Button and ImageButton UI tests on iOS and MacCatalyst.

  • ButtonHandler.iOS.cs, ImageButtonHandler.iOS.cs add PlatformArrange overrides re-applying CornerRadius → Button, ImageButton.
  • Platform/iOS/ButtonExtensions.cs, LayerExtensions.cs, ViewExtensions.cs (UpdateLayerBorder) change corner radius clamping for button strokes only → same categories.
  • All code is under iOS paths, so iOS + MacCatalyst only; Android/Windows untouched.
  • Device tests and PublicAPI files don't affect UI tests; no changed UI tests or category hints.

Generated by /review run-tests. A targeted run is a fast signal, not full-suite certification.

@kubaflo
kubaflo temporarily deployed to copilot-pat-pool November 19, 2026 18:54 — with GitHub Actions Inactive
@kubaflo
kubaflo temporarily deployed to copilot-pat-pool November 19, 2026 18:56 — with GitHub Actions Inactive
@github-actions github-actions Bot added area-ai-agents Copilot CLI agents, agent skills, AI-assisted development platform/ios platform/macos macOS / Mac Catalyst labels Nov 19, 2026
@kubaflo
kubaflo temporarily deployed to copilot-pat-pool November 19, 2026 18:56 — with GitHub Actions Inactive
@kubaflo

This comment has been minimized.

@github-actions github-actions Bot added the s/agent-review-in-progress AI review is currently running for this PR label Nov 19, 2026
@MauiBot

MauiBot commented Nov 19, 2026

Copy link
Copy Markdown
Collaborator

PR Test Gate

@kubaflo — existing-test verification; Setup did not capture a tested commit.

Result INCONCLUSIVE Platform iOS


🧪 Gate analysis — click to expand

⚠ Inconclusive. Setup, build, or execution did not produce a conclusive result. This does not prove the fix is incorrect.

No interpretable test-group summary was produced. No additional test execution is inferred.

Setup did not capture a verified PR head; no commit is claimed as tested.

No immutable without-fix baseline was prepared.

@MauiBot MauiBot removed the s/agent-review-in-progress AI review is currently running for this PR label Nov 19, 2026

@github-actions github-actions Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Expert Review — 8 findings

Reviewed commit 77b00e3.

See inline comments for details.

artifactName: 'uitest-snapshot-results-android-material3-$(System.StageName)-$(System.JobName)-$(System.JobAttempt)'

# SafeAreaEdges tests on Android API 36 with Pixel 3 XL
- ${{ if and(containsValue(parameters.selectedPlatforms, 'android'), or(eq(length(parameters.selectedCategories), 0), containsValue(parameters.selectedCategories, 'SafeAreaEdges'))) }}:

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

[major] Logic and Correctness Verification - Dedicated Android lanes reject the qualified categories produced by Merge-RunTestsSelections. Selecting Android Button + SafeAreaEdges and iOS Button produces Button and android:SafeAreaEdges, but this guard only accepts unqualified SafeAreaEdges. The ordinary Android matrix also explicitly excludes SafeAreaEdges, so the requested notch-device tests never run. The Shell Renderer gate and shard predicates likewise omit android:Shell1/android:Shell2, losing renderer coverage. Accept qualified identifiers in the dedicated gates/matrices and add selection-to-lane regression cases; the existing test only searches for a qualifier expression somewhere in each platform's matrix.

Undefined symbols are false, as in C#.
#>
param([Parameter(Mandatory)][string]$Expression, [Parameter(Mandatory)][AllowEmptyCollection()][string[]]$Symbols)
$tokens = @([regex]::Matches(($Expression -replace '//.*$', ''), '&&|\|\||!|\(|\)|[A-Za-z_][A-Za-z0-9_]*') | ForEach-Object Value)

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

[major] Input and Path Correctness - PR-head C# text flows from Get-RunTestsContext.ps1 through Get-RunTestsFileCategories into this parser without validating unsupported characters or complete token consumption. For the valid C# guard #if ANDROID == false, the tokenizer discards == and returns ANDROID's value, ignoring the remaining token. Thus an iOS test is classified as absent. If the changed test enables a category with no baseline tests, such as AppLinks, changedTestRunnable omits it and Resolve-RunTestsSelection drops even an explicitly selected iOS/AppLinks run. Support C# equality operators and validate complete parsing rather than treating an unsupported expression as evidence that tests cannot run.

} | Select-Object -First 40)
foreach ($file in $testFiles) {
$null = $file.path -match '^src/Controls/tests/TestCases\.(\w+)\.Tests/'
$raw = & gh api -H 'Accept: application/vnd.github.raw' "repos/$Repository/contents/$($file.path)?ref=$($pr.head.sha)" 2>$null

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

[major] Input and Path Correctness - The files API's filename is inserted directly into the contents URL without escaping its path segments. A valid changed test path ending in AppLinks#Regression.cs passes the filter, but #Regression.cs?ref=... becomes a URL fragment instead of the filename and pinned ref. The failed fetch is silently skipped on the next line. If that file introduces the only enabled iOS AppLinks test, changedTestRunnable remains empty and the dispatcher drops that category even when selected. URL-escape the filename segments before appending the separate ref query, and surface fetch failures instead of interpreting them as absent tests.

runtimeVariant: "CoreCLR"
useWindowsCV1: true
skipProvisioning: ${{ parameters.skipProvisioning }}
- ${{ if and(containsValue(parameters.selectedPlatforms, 'android'), or(eq(length(parameters.selectedCategories), 0), ne(replace(replace(join('', parameters.selectedCategories), 'Material3', ''), 'SafeAreaEdges', ''), ''))) }}:

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

[moderate] Logic and Correctness Verification - This stage guard examines the global category list rather than the categories eligible for its Android matrix. One model selecting only Android Material3 and the other only iOS Button produces [Material3, ios:Button] on [android, ios]. Removing Material3 still leaves ios:Button, so the ordinary Android stage is emitted, but neither entry matches any of its matrix rows. The resulting empty matrix has no CATEGORYGROUP for the testFilter and cannot execute a valid ordinary selected-test lane. Gate this stage on a nonempty platform-specific ordinary-category intersection, preserving the dedicated Material3 and SafeAreaEdges exclusions.

${{ if startsWith(categoryGroup, 'CollectionView') }}:
${{ categoryGroup }}:
CATEGORYGROUP: ${{ categoryGroup }}
${{ if or(eq(length(parameters.selectedCategories), 0), containsValue(parameters.selectedCategories, 'CollectionView'), containsValue(parameters.selectedCategories, categoryGroup)) }}:

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

[moderate] Build & MSBuild - The CV1 stage gate accepts a CollectionView substring, but its matrix rejects qualified Windows shards. Selecting CollectionView1 only on Windows and Button only on iOS produces windows:CollectionView1 and ios:Button. The Windows CV1 build/test stages are emitted, yet this predicate matches no shard, leaving an empty matrix without CATEGORYGROUP instead of running CollectionView1 under CV1. Accept windows:<shard> here and make the stage gate consider Windows selections rather than CollectionView categories belonging only to another platform.

# proposal and then exit non-zero; only hand over without one when nothing was dispatched.
run-tests-fallback:
needs: [pre_activation, activation, agent, run_selected_ui_tests]
if: ${{ !cancelled() && needs.pre_activation.outputs.context_ready == 'true' && needs.agent.result == 'failure' && needs.run_selected_ui_tests.result != 'success' }}

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

[moderate] Logic and Correctness Verification - A successful second-model invocation without run_selected_ui_tests strands a valid first-model proposal. The compiled workflow permits noop/missing_data output; then run_selected_ui_tests is skipped, while this fallback also skips because the agent job succeeded. Nothing dispatches GPT's available selection. The first-model workflow has the same condition and never starts the second model in that case. Base fallback handover on whether a handover succeeded, not solely on agent-process failure, and regenerate both lock files.


internal static void UpdateBackgroundLayerCornerRadius(this UIView control, int cornerRadius)
{
if (cornerRadius < 0 || control.Layer?.Sublayers is not { Length: > 0 } sublayers)

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

[moderate] Performance-Critical Path Optimization - Both new PlatformArrange overrides reach this lookup on each size-changing arrange when CornerRadius is nonnegative. CALayer.Sublayers marshals and copies the native array into a managed array, and Name lookups marshal strings. Size-animating or resizing Button/ImageButton controls now allocates and scans native layers on the arrange path, including solid-background buttons without a MauiBackgroundLayer. The pinned layer history already documents this marshaling cost. Track the background-layer reference with insertion/removal/replacement so radius updates do not repeatedly materialize the entire sublayer collection.

var resizedBounds = new Microsoft.Maui.Graphics.Rect(0, 0, 40, 40);
handler.PlatformArrange(resizedBounds);
var resizedRadius = (double)handler.PlatformView.Layer.CornerRadius;
handler.PlatformArrange(resizedBounds);

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

[moderate] Regression Prevention and Test Coverage - Repeating a same-size arrange and asserting that CornerRadius stays 20 does not test the negative branch of the new size-change guard: the assertion also passes if the mapper is invoked unconditionally. The Button and gradient variants have the same blind spot. That regression would rerun custom mapper hooks on every unchanged arrange, and ImageButton.MapCornerRadius also remaps Padding. Count mapper invocations across resize and same-size/position-only arrangements, following the existing Android ImageButton regression test, to protect the guard's actual behavior.

@kubaflo kubaflo closed this Nov 19, 2026
@kubaflo
kubaflo deleted the run-tests-real-39487 branch November 19, 2026 20:42
kubaflo added a commit that referenced this pull request Nov 20, 2026
…#40305)

<!-- Please let the below note in for people that find this PR -->
> [!NOTE]
> Are you waiting for the changes in this PR to be merged?
> It would be very helpful if you could [test the resulting
artifacts](https://github.com/dotnet/maui/wiki/Testing-PR-Builds) from
this PR and let us know in a comment if this change resolves your issue.
Thank you!

### Target Branch

- [x] I selected the applicable base branch using the [contribution
guide](https://github.com/dotnet/maui/blob/main/.github/CONTRIBUTING.md#choose-your-pr-base-before-starting).

CI and workflow infrastructure, so it targets `main`. The slash command
activates only once the workflow is on the default branch.

### Description of Change

Adds `/review run-tests`. Two AI models, **GPT-6.1 Sol** and **Claude
Opus 5.5**, each read the PR diff independently and pick the UI test
categories and platforms it can affect. `maui-pr-uitests` then runs
**only the union of their picks**. A CollectionView-only Android fix
runs the CollectionView shards on Android, not all 77 categories on 4
platforms.

Replaces #40253.

**When what runs**

| Trigger | What runs |
| --- | --- |
| `/review run-tests` on any PR (write access) | Only what the models
selected: the per-platform union of both picks |
| Both models choose the full suite (e.g. core `Element` changes) | Full
suite |
| Neither model produces a pick | Nothing is queued; the comment says so
|
| `/azp run maui-pr-uitests` | Full suite, unchanged |
| Branch, scheduled and manual builds (`main`, `release/*`,
`candidate/*`, …) | Full suite, unchanged. `ci-uitests.yml` only passes
a selection when `Build.SourceBranch` is `refs/pull/*`. |

Nothing is ever added that a model did not choose: no full-suite
fallback, unknown category names run nothing, Android and the Material3
lane run only when selected (also enforced in the pipeline template),
and only the five declared proposal fields are read.

**What the requester sees**

1. **Native checks in the PR's checks list.** `maui-pr-uitests` is
queued through the Build API as a *pull-request* build
(`reason=pullRequest` + `triggerInfo.pr.sourceSha`), so Azure Pipelines
reports the run and every job as checks on the PR head, exactly like
`/azp run`.
2. **Skipped checks for everything not selected.** One `maui-pr-uitests
(skipped) <Category>` check per category and one `maui-pr-uitests
(skipped) <Platform> UITests` check per platform, plus a
`maui-pr-uitests (test selection)` summary check.
3. **A summary comment in the PR Test Gate layout:** title, requester
line and Result / Platforms / Categories / Commit badges, with
everything else in one collapsed **Test selection** section: build
status, each model's summary, what ran, results with failed tests, the
skipped list, and **Why these tests** with a per-platform table of each
model's pick and both rationales. `review-run-tests-results.yml` updates
it in place when the `maui-pr-uitests` check completes, with a
half-hourly sweep as backup.

**How it works**

| Piece | Role |
| --- | --- |
| `copilot-review-run-tests.md` (gh-aw, GPT-6.1 Sol) | Pre-activation
authorizes an exact `/review run-tests` from a write+ user and collects
the diff, the category inventory from
`eng/pipelines/common/ui-tests.yml`, which categories have enabled tests
per platform, and which the PR's own tests enable. The read-only agent
(`checkout: false`) proposes platforms, categories, `full_run`, a
summary and a rationale. The safe-output job saves the proposal and
starts the Opus workflow. |
| `copilot-review-run-tests-opus.md` (gh-aw, Claude Opus 5.5, new) |
`workflow_dispatch` only. Verifies the source run (run-tests lock
workflow, default branch, same repo, under 3 hours old), reuses its
context artifact, and makes an independent proposal with the same shared
prompt (`shared/run-tests-selection.md`). Then it starts the dispatcher
with both run ids. If either agent fails, the chain still hands over. |
| `review-run-tests-dispatch.yml` | `workflow_dispatch` on `main`, with
**no environment**, so its OIDC subject is
`repo:dotnet/maui:ref:refs/heads/main` like `review-trigger.yml`.
Verifies both runs, downloads each proposal (a listed artifact that
fails to download fails the job), checks each proposal's model and
source run, and runs `Invoke-RunTests.ps1`. |
| `Invoke-RunTests.ps1` | Trusted. Validates each proposal against the
inventory, prunes each platform to categories with enabled tests there,
and merges the two per platform. When every selected platform runs the
same categories they are sent as plain names; otherwise every entry is
platform-qualified (`android:Button`, `ios:Label`), never a mix.
Re-checks the requester's permission, the PR head and its target branch,
waits for a verified test-merge commit, queues the pull-request build
and cancels it if Azure didn't apply the selection. Then publishes the
Skipped and selection checks and the comment. |
| `review-run-tests-results.yml` | Updates the comment when the
`azure-pipelines` `maui-pr-uitests` check completes; sweeps every 30
minutes as backup. |
| `eng/pipelines/**/ui-tests*.yml`, `ci-uitests.yml` | Optional
`selectedRun`, `selectedPlatforms`, `selectedCategories` parameters
filter stages and matrix entries at compile time. `CollectionView` and
`Shell` expand to their shards; `<platform>:<Category>` limits a
category to one platform, and every stage guard, including the dedicated
Shell renderer, SafeAreaEdges, Material3 and Windows CV1 lanes, handles
both the plain and the qualified form. The sample build gets
template-evaluated booleans, never parameter text. Defaults expand to
exactly `main`'s jobs. |
| `review-trigger.yml`, `ReviewCommandHelpers.ps1`, rebalance skill |
`/review run-tests` is excluded from `/review` and from recovery.
`rebalance-ui-test-categories` understands the typed
`categoryGroupsToTest` parameter. |

`/azp run`, `/review`, `/review gate` and `/review tests` are unchanged.
Device tests are out of scope.

**Model policy exception:** the repo's GPT-only automation policy
(`CopilotPipeline.Tests.ps1`, from #38040) gets one narrow exception:
exactly `claude-opus-5.5`, only in the run-tests selection files (the
Opus workflow and its lock file, the dispatcher, `RunTests.Common.ps1`
and the tests). Any other Claude/Anthropic model, or that model anywhere
else, still fails the test. Opus only proposes a selection; trusted code
validates it, and it can only add tests, never remove the other model's
picks.

**Security review notes:** the new Opus workflow uses only existing
secrets (the `COPILOT_PAT_0..9` pool via `copilot-pat-pool`); the
dispatcher uses the existing `AZDO_TRIGGER_TENANT_ID` /
`AZDO_TRIGGER_CLIENT_ID`. No new third-party actions. All actions are
pinned by SHA.

### Validation

**End to end in a fork (kubaflo/maui)** through the real GPT agent →
Opus agent → dispatcher path, on 8 current PRs. All 24 workflow runs
succeeded; each PR got the comment with both models' picks and the
Skipped checks. Azure is skipped there because the fork has no federated
identity.

**Real builds on dotnet/maui.** Other PRs' merge refs don't have the new
pipeline parameters until this merges, so each trial was a temporary
draft PR: this branch plus one current PR's change.
`Invoke-RunTests.ps1` ran locally with both models' proposals from the
fork run on the same diff, queued `maui-pr-uitests`, and posted the
comment. Every build got exactly the selected platforms and categories.

| Trial | Copy of | GPT-6.1 Sol | Claude Opus 5.5 | Ran | Result |
| --- | --- | --- | --- | --- | --- |
| #40442 | #40369 Windows title bar | `Window` | + `TitleView`,
`Lifecycle` | Windows, 3 categories | Passed in 39 min |
| #40443 | #40366 Shell modal stack | `Shell1/2` on iOS, Mac | same |
iOS + MacCatalyst | Passed |
| #40444 | #40363 Android EmptyView | CollectionView + CarouselView |
same | Android, 8 categories | Passed in 41 min |
| #40445 | #40344 Shell tab reselect | `Shell1/2`, `Navigation` | +
`TabbedPage` | All platforms, 4 categories | Running at time of writing
|
| #40446 | #40291 `DisplayPromptAsync` | Prompt, Alert, ActionSheet |
same | All platforms, 3 categories | 1 test failed on MacCatalyst |
| #40447 | #40234 `Element` / VSM threading | Full suite | Full suite |
Full suite | Running at time of writing |
| #40448 | #40385 CodeQL CI-only | Nothing | Nothing | Nothing (no
build) | Comment only |
| #40449 | #39487 Button corner radius | `Button`, `ImageButton` on iOS,
Mac | same | iOS + MacCatalyst | 1 Button test failed |

The failures are the same tests that failed in an earlier round on the
same copied changes, so they come from those PRs, not the selection. The
comments were refreshed from the real timelines and test results with
`Update-RunTestsResults.ps1`.

**Azure previews of this PR's merge ref:**
- Defaults: identical to `main` (13 jobs, 157 matrix entries).
- Android `Button`: 2 jobs, no Material3 lane. `Button` + `Material3`:
the Material3 lane is included.
- Qualified selections: `android:Material3` + `ios:Label` runs only the
Material3 lane on Android; `android:Shell1` also runs the Shell renderer
lane; `android:SafeAreaEdges` runs its dedicated lane;
`windows:CollectionView1` runs the CV1 lane; an Android-only
CollectionView pick never starts the Windows CV1 lane.
- `selectedPlatforms` containing `"x; Remove-Item"`: only template
booleans reach the build step.

**Tests:** `RunTests.Tests.ps1` has 39 Pester tests covering the
inventory and `#if` evaluator, pruning and qualification, the two-model
union (one model empty, neither, full only when chosen), AI-only rules,
the queue request and dropped-selection guard, the check plan, comment
rendering and updates, and static template checks. The `ReviewTrigger`,
`Recover-MissedReviewCommands` and `CopilotPipeline` suites pass. `gh aw
compile --strict` succeeds.

**Not verified before merge:**
- The slash command and the dispatcher's OIDC exchange on `dotnet/maui`
itself. GitHub only runs them from the default branch. After merge,
comment `/review run-tests` on a PR to confirm.
- The `check_run`-triggered comment update on `dotnet/maui` (the script
itself is verified; the half-hourly sweep is the backup).
- The `copilot-pat-pool` token selection in the new Opus workflow (the
fork used a single secret).

---------

Co-authored-by: Copilot CI <copilot-ci@microsoft.com>
HarishKumarSF4517 pushed a commit to HarishKumarSF4517/maui that referenced this pull request Nov 20, 2026
…dotnet#40305)

<!-- Please let the below note in for people that find this PR -->
> [!NOTE]
> Are you waiting for the changes in this PR to be merged?
> It would be very helpful if you could [test the resulting
artifacts](https://github.com/dotnet/maui/wiki/Testing-PR-Builds) from
this PR and let us know in a comment if this change resolves your issue.
Thank you!

### Target Branch

- [x] I selected the applicable base branch using the [contribution
guide](https://github.com/dotnet/maui/blob/main/.github/CONTRIBUTING.md#choose-your-pr-base-before-starting).

CI and workflow infrastructure, so it targets `main`. The slash command
activates only once the workflow is on the default branch.

### Description of Change

Adds `/review run-tests`. Two AI models, **GPT-6.1 Sol** and **Claude
Opus 5.5**, each read the PR diff independently and pick the UI test
categories and platforms it can affect. `maui-pr-uitests` then runs
**only the union of their picks**. A CollectionView-only Android fix
runs the CollectionView shards on Android, not all 77 categories on 4
platforms.

Replaces dotnet#40253.

**When what runs**

| Trigger | What runs |
| --- | --- |
| `/review run-tests` on any PR (write access) | Only what the models
selected: the per-platform union of both picks |
| Both models choose the full suite (e.g. core `Element` changes) | Full
suite |
| Neither model produces a pick | Nothing is queued; the comment says so
|
| `/azp run maui-pr-uitests` | Full suite, unchanged |
| Branch, scheduled and manual builds (`main`, `release/*`,
`candidate/*`, …) | Full suite, unchanged. `ci-uitests.yml` only passes
a selection when `Build.SourceBranch` is `refs/pull/*`. |

Nothing is ever added that a model did not choose: no full-suite
fallback, unknown category names run nothing, Android and the Material3
lane run only when selected (also enforced in the pipeline template),
and only the five declared proposal fields are read.

**What the requester sees**

1. **Native checks in the PR's checks list.** `maui-pr-uitests` is
queued through the Build API as a *pull-request* build
(`reason=pullRequest` + `triggerInfo.pr.sourceSha`), so Azure Pipelines
reports the run and every job as checks on the PR head, exactly like
`/azp run`.
2. **Skipped checks for everything not selected.** One `maui-pr-uitests
(skipped) <Category>` check per category and one `maui-pr-uitests
(skipped) <Platform> UITests` check per platform, plus a
`maui-pr-uitests (test selection)` summary check.
3. **A summary comment in the PR Test Gate layout:** title, requester
line and Result / Platforms / Categories / Commit badges, with
everything else in one collapsed **Test selection** section: build
status, each model's summary, what ran, results with failed tests, the
skipped list, and **Why these tests** with a per-platform table of each
model's pick and both rationales. `review-run-tests-results.yml` updates
it in place when the `maui-pr-uitests` check completes, with a
half-hourly sweep as backup.

**How it works**

| Piece | Role |
| --- | --- |
| `copilot-review-run-tests.md` (gh-aw, GPT-6.1 Sol) | Pre-activation
authorizes an exact `/review run-tests` from a write+ user and collects
the diff, the category inventory from
`eng/pipelines/common/ui-tests.yml`, which categories have enabled tests
per platform, and which the PR's own tests enable. The read-only agent
(`checkout: false`) proposes platforms, categories, `full_run`, a
summary and a rationale. The safe-output job saves the proposal and
starts the Opus workflow. |
| `copilot-review-run-tests-opus.md` (gh-aw, Claude Opus 5.5, new) |
`workflow_dispatch` only. Verifies the source run (run-tests lock
workflow, default branch, same repo, under 3 hours old), reuses its
context artifact, and makes an independent proposal with the same shared
prompt (`shared/run-tests-selection.md`). Then it starts the dispatcher
with both run ids. If either agent fails, the chain still hands over. |
| `review-run-tests-dispatch.yml` | `workflow_dispatch` on `main`, with
**no environment**, so its OIDC subject is
`repo:dotnet/maui:ref:refs/heads/main` like `review-trigger.yml`.
Verifies both runs, downloads each proposal (a listed artifact that
fails to download fails the job), checks each proposal's model and
source run, and runs `Invoke-RunTests.ps1`. |
| `Invoke-RunTests.ps1` | Trusted. Validates each proposal against the
inventory, prunes each platform to categories with enabled tests there,
and merges the two per platform. When every selected platform runs the
same categories they are sent as plain names; otherwise every entry is
platform-qualified (`android:Button`, `ios:Label`), never a mix.
Re-checks the requester's permission, the PR head and its target branch,
waits for a verified test-merge commit, queues the pull-request build
and cancels it if Azure didn't apply the selection. Then publishes the
Skipped and selection checks and the comment. |
| `review-run-tests-results.yml` | Updates the comment when the
`azure-pipelines` `maui-pr-uitests` check completes; sweeps every 30
minutes as backup. |
| `eng/pipelines/**/ui-tests*.yml`, `ci-uitests.yml` | Optional
`selectedRun`, `selectedPlatforms`, `selectedCategories` parameters
filter stages and matrix entries at compile time. `CollectionView` and
`Shell` expand to their shards; `<platform>:<Category>` limits a
category to one platform, and every stage guard, including the dedicated
Shell renderer, SafeAreaEdges, Material3 and Windows CV1 lanes, handles
both the plain and the qualified form. The sample build gets
template-evaluated booleans, never parameter text. Defaults expand to
exactly `main`'s jobs. |
| `review-trigger.yml`, `ReviewCommandHelpers.ps1`, rebalance skill |
`/review run-tests` is excluded from `/review` and from recovery.
`rebalance-ui-test-categories` understands the typed
`categoryGroupsToTest` parameter. |

`/azp run`, `/review`, `/review gate` and `/review tests` are unchanged.
Device tests are out of scope.

**Model policy exception:** the repo's GPT-only automation policy
(`CopilotPipeline.Tests.ps1`, from dotnet#38040) gets one narrow exception:
exactly `claude-opus-5.5`, only in the run-tests selection files (the
Opus workflow and its lock file, the dispatcher, `RunTests.Common.ps1`
and the tests). Any other Claude/Anthropic model, or that model anywhere
else, still fails the test. Opus only proposes a selection; trusted code
validates it, and it can only add tests, never remove the other model's
picks.

**Security review notes:** the new Opus workflow uses only existing
secrets (the `COPILOT_PAT_0..9` pool via `copilot-pat-pool`); the
dispatcher uses the existing `AZDO_TRIGGER_TENANT_ID` /
`AZDO_TRIGGER_CLIENT_ID`. No new third-party actions. All actions are
pinned by SHA.

### Validation

**End to end in a fork (kubaflo/maui)** through the real GPT agent →
Opus agent → dispatcher path, on 8 current PRs. All 24 workflow runs
succeeded; each PR got the comment with both models' picks and the
Skipped checks. Azure is skipped there because the fork has no federated
identity.

**Real builds on dotnet/maui.** Other PRs' merge refs don't have the new
pipeline parameters until this merges, so each trial was a temporary
draft PR: this branch plus one current PR's change.
`Invoke-RunTests.ps1` ran locally with both models' proposals from the
fork run on the same diff, queued `maui-pr-uitests`, and posted the
comment. Every build got exactly the selected platforms and categories.

| Trial | Copy of | GPT-6.1 Sol | Claude Opus 5.5 | Ran | Result |
| --- | --- | --- | --- | --- | --- |
| dotnet#40442 | dotnet#40369 Windows title bar | `Window` | + `TitleView`,
`Lifecycle` | Windows, 3 categories | Passed in 39 min |
| dotnet#40443 | dotnet#40366 Shell modal stack | `Shell1/2` on iOS, Mac | same |
iOS + MacCatalyst | Passed |
| dotnet#40444 | dotnet#40363 Android EmptyView | CollectionView + CarouselView |
same | Android, 8 categories | Passed in 41 min |
| dotnet#40445 | dotnet#40344 Shell tab reselect | `Shell1/2`, `Navigation` | +
`TabbedPage` | All platforms, 4 categories | Running at time of writing
|
| dotnet#40446 | dotnet#40291 `DisplayPromptAsync` | Prompt, Alert, ActionSheet |
same | All platforms, 3 categories | 1 test failed on MacCatalyst |
| dotnet#40447 | dotnet#40234 `Element` / VSM threading | Full suite | Full suite |
Full suite | Running at time of writing |
| dotnet#40448 | dotnet#40385 CodeQL CI-only | Nothing | Nothing | Nothing (no
build) | Comment only |
| dotnet#40449 | dotnet#39487 Button corner radius | `Button`, `ImageButton` on iOS,
Mac | same | iOS + MacCatalyst | 1 Button test failed |

The failures are the same tests that failed in an earlier round on the
same copied changes, so they come from those PRs, not the selection. The
comments were refreshed from the real timelines and test results with
`Update-RunTestsResults.ps1`.

**Azure previews of this PR's merge ref:**
- Defaults: identical to `main` (13 jobs, 157 matrix entries).
- Android `Button`: 2 jobs, no Material3 lane. `Button` + `Material3`:
the Material3 lane is included.
- Qualified selections: `android:Material3` + `ios:Label` runs only the
Material3 lane on Android; `android:Shell1` also runs the Shell renderer
lane; `android:SafeAreaEdges` runs its dedicated lane;
`windows:CollectionView1` runs the CV1 lane; an Android-only
CollectionView pick never starts the Windows CV1 lane.
- `selectedPlatforms` containing `"x; Remove-Item"`: only template
booleans reach the build step.

**Tests:** `RunTests.Tests.ps1` has 39 Pester tests covering the
inventory and `#if` evaluator, pruning and qualification, the two-model
union (one model empty, neither, full only when chosen), AI-only rules,
the queue request and dropped-selection guard, the check plan, comment
rendering and updates, and static template checks. The `ReviewTrigger`,
`Recover-MissedReviewCommands` and `CopilotPipeline` suites pass. `gh aw
compile --strict` succeeds.

**Not verified before merge:**
- The slash command and the dispatcher's OIDC exchange on `dotnet/maui`
itself. GitHub only runs them from the default branch. After merge,
comment `/review run-tests` on a PR to confirm.
- The `check_run`-triggered comment update on `dotnet/maui` (the script
itself is verified; the half-hourly sweep is the backup).
- The `copilot-pat-pool` token selection in the new Opus workflow (the
fork used a single secret).

---------

Co-authored-by: Copilot CI <copilot-ci@microsoft.com>

This branch was previously deployed

1 inactive deployment
copilot-pat-pool — 77b00e3f Deployed Nov 19, 2026 by kubaflo via conclusion #3147
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

area-ai-agents Copilot CLI agents, agent skills, AI-assisted development platform/ios platform/macos macOS / Mac Catalyst

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants