Repository navigation
Conversation
The agent picks platforms and UI test categories from the PR diff. A trusted safe-output job validates the choice against the pipeline inventory, queues maui-pr-uitests as a pull-request build so Azure Pipelines reports native checks on the PR head, publishes a Skipped check for every platform and category that was not selected, and posts a collapsible summary comment that is updated when the run finishes.
The agent reads the collected context and uses GitHub read tools; it never needs PR code on disk.
The previous expression also enabled the zero-test guard in the default full run, which failed every job whose test step did not produce results. The dispatcher always sets selectedRun for targeted runs.
Untyped template parameters receive the boolean as the string 'False',
which is truthy in a bare ${{ if }}.
…nly selection - Hand the AI proposal to review-run-tests-dispatch.yml. Jobs in the copilot-pat-pool environment get an OIDC subject the Azure DevOps federated credential does not trust; the dispatch workflow runs on main without an environment, like review-trigger.yml. - Queue only after GitHub's test-merge commit is confirmed to merge the selected head, and cancel any build whose source version differs. - Evaluate #if guards per platform to know which categories have enabled tests (including the PR's own changed tests). Drop categories and platforms that would run zero tests, and pass platform-qualified categories (ios:Window) when a category only runs on some platforms. - Pass the sample-build platforms to PowerShell as template-evaluated booleans instead of interpolating parameter text. - Teach the rebalance-ui-test-categories skill the typed parameter form. - Add regression tests for the full-run guard, the qualified matrix filter, the build-script inputs and the skill.
An agent can emit its proposal and then exit non-zero; dispatching both the proposal and the fallback would queue two runs for one command.
|
🚀 Dogfood this PR with:
curl -fsSL https://raw.githubusercontent.com/dotnet/maui/main/eng/scripts/get-maui-pr.sh | bash -s -- 40397Or
iex "& { $(irm https://raw.githubusercontent.com/dotnet/maui/main/eng/scripts/get-maui-pr.ps1) } 40397" |
This comment has been minimized.
This comment has been minimized.
Skill Validation Results
❌ Skill Validation Results —
|
Copy of dotnet#40366 for a /review run-tests trial. Do not merge.
17134c6 to
698d851
Compare
Selected UI TestsShell cross-tab modal navigation fix; runs both Shell UI test shards on iOS and MacCatalyst. ❌ maui-pr-uitests #1688769 failed. See the failing checks below or the run for details. ✅ Running — 2 categories on iOS, MacCatalyst
⏭️ Skipped — 75 categories, 2 platformsPlatforms not run: Android, Windows Categories not run: In this manual trial the Skipped 📊 Results — failed: 4 passed, 1 failed of 5 jobs
Failed tests (1)
🤖 Why these tests — AI rationale
Generated by /review run-tests. A targeted run is a fast signal, not full-suite certification. |
This comment has been minimized.
This comment has been minimized.
PR Test Gate
🧪 Gate analysis — click to expand⚠ Inconclusive. Setup, build, or execution did not produce a conclusive result. This does not prove the fix is incorrect. No interpretable test-group summary was produced. No additional test execution is inferred. Setup did not capture a verified PR head; no commit is claimed as tested. No immutable without-fix baseline was prepared. |
| env: | ||
| GH_TOKEN: ${{ github.token }} | ||
| PR_NUMBER: ${{ github.event.issue.number || inputs.pr_number }} | ||
| run: pwsh -NoProfile -File .github/scripts/Get-RunTestsContext.ps1 -PrNumber "$PR_NUMBER" -OutputDirectory "$RUNNER_TEMP/run-tests" |
There was a problem hiding this comment.
[moderate] Build & MSBuild - The collector invokes pwsh, but the generated pre_activation job runs on ubuntu-slim, which does not include PowerShell, and no preceding step installs it. An authorized /review run-tests fails before producing context; activation is skipped and the fallback cannot run because context_ready was never set. Use runs-on-slim: ubuntu-latest, as the existing issue-trace-regression workflow does for its PowerShell collector, and regenerate the lock.
| runtimeVariant: "CoreCLR" | ||
| useWindowsCV1: true | ||
| skipProvisioning: ${{ parameters.skipProvisioning }} | ||
| - ${{ if and(containsValue(parameters.selectedPlatforms, 'android'), or(eq(length(parameters.selectedCategories), 0), ne(replace(replace(join('', parameters.selectedCategories), 'Material3', ''), 'SafeAreaEdges', ''), ''))) }}: |
There was a problem hiding this comment.
[moderate] Build & MSBuild - This stage gate examines categories for every platform rather than entries runnable in the Android matrix. Selecting Android and Windows with Material3 and Window is normalized to Material3 plus windows:Window: Window tests are compiled out on Android. Removing Material3 still leaves windows:Window, so the ordinary Android stage is emitted, but neither value creates a matrix entry. Its test step consequently has no CATEGORYGROUP, despite the separate Material3 and Windows lanes being valid. Gate ordinary stages on their own matching matrix entries, excluding qualified categories belonging elsewhere.
| artifactName: 'uitest-snapshot-results-android-material3-$(System.StageName)-$(System.JobName)-$(System.JobAttempt)' | ||
|
|
||
| # SafeAreaEdges tests on Android API 36 with Pixel 3 XL | ||
| - ${{ if and(containsValue(parameters.selectedPlatforms, 'android'), or(eq(length(parameters.selectedCategories), 0), containsValue(parameters.selectedCategories, 'SafeAreaEdges'))) }}: |
There was a problem hiding this comment.
[moderate] Logic and Correctness Verification - The documented platform-qualified category contract is handled by ordinary matrices but not the dedicated lanes. With selectedPlatforms=[android] and selectedCategories=[android:SafeAreaEdges], the ordinary matrix excludes SafeAreaEdges and this gate also omits its dedicated stage, so the requested tests never run. Similarly, android:Shell1 omits ShellRenderer coverage, while windows:CollectionView1 enables the CV1 stage without creating a matrix entry. Recognize qualified identifiers in the dedicated gates and matrices, or normalize them consistently before evaluating either.
Warning
Do not merge. Do not review. Temporary trial PR for
/review run-tests(#40305). It will be closed once the trial finishes.This branch is #40305 plus the change from #40366, so
maui-pr-uitestshere uses the new selection parameters. The UI test selection comes from the AI agent's run on the same diff, and the trusted dispatcher queues only the selected categories and platforms.