Skip to content

Fix pipeline failures from run 20261113.16 and align .NET 11 GA pack pins - #12461

Merged
Amaury Levé (Evangelink) merged 3 commits into
mainfrom
dev/amauryleve/fix-pipeline-20261113-16
Nov 13, 2026
Merged

Amaury Levé (Evangelink) merged 3 commits into
mainfrom
dev/amauryleve/fix-pipeline-20261113-16

Conversation

@Evangelink

Copy link
Copy Markdown
Member

Pull request

Summary

Fixes the remaining failures from run 20261113.16 on main, and aligns the repository's .NET 10/11 pack pins with the GA runtimes now in global.json.

Failure in build 1679434 Resolution
Playwright EnablePlaywrightProperty_* TestContext TypeLoad (net11.0 all OSes, net462 Windows) Already fixed on main by #12447; nothing here.
Linux AssemblyOS_VSTest_IncludeNone (Assert.ContainsSingle, actual count 0) Commit 1 below.
Windows AzDO summary: "CI summary target framework provenance does not match" (non-fatal, "Reporting: degraded") Commit 2 below.
BuildXL DX0041/DX0064 (getmac.exe survivor) Known issue documented in docs/decisions/buildxl-build.md; the bounded retry passed. No change.

Commits:

  1. Fix AssemblyOS VSTest provenance race with testhost shutdown (acceptance test only). The generated consumer records provenance on ProcessExit, hashing assemblies lazily. vstest.console kills testhost VSTEST_TESTHOST_SHUTDOWN_TIMEOUT (default 100 ms) after closing the session, which truncated products.txt. The VSTest environment now sets a 60 s shutdown timeout for these generated runs.
  2. Map two-component .NET Framework display names to short TFMs (Microsoft.Testing.Platform). TargetFrameworkParser returned .NET Framework 4.8 unchanged because the runtime display name omits the zero patch, while the CI summary manifest side records net48, so the AzDO summary post-processor rejected the fragment. Such names now map to net{major}{minor}. User-visible side effect: terminal/report TFM labels for .NET Framework 4.7/4.8 now read net47/net48, matching other frameworks.
  3. Align .NET 10/11 pack pins with the GA global.json runtimes (build infrastructure). Update .NET SDK and runtime versions #12459/Update global.json #12460 moved global.json to runtimes 10.0.15/11.0.0, but eng/Net11FrameworkReferences.targets still pinned 10.0.14 and 11.0.0-rc.3.26522.148. The pinned 12.0.100-alpha SDK bundles prerelease rc.3 net11.0 packs, so the existing StartsWith('12.') guards never fired once the runtime left rc.3. The guards now also match 11.*, and both pins follow global.json. Windows SDK projection references use 10.* versions and remain excluded. Without this, FrameworkTargetingTests.NativeAotCompilerPack_MatchesBootstrappedRuntime fails on current main.

Design and decision record

Commit 3 extends the existing decision record: docs/decisions/net10-and-net11-builds.md, new section "CI follow-up: .NET 11 GA runtime pin". Commits 1 and 2 are narrow bug fixes to existing contracts (docs/decisions/assembly-os-conditions.md and the CI summary provenance check); no separate record is needed.

Not changed here: Bazel's independent SDK/runtime pins (MODULE.bazel, eng/bazel/*) still reference rc.3. They are a separate toolchain and should be bumped in a follow-up.

Demonstration

N/A for media: these are test-infrastructure, build-policy, and a short-TFM label change. The text transcripts in the validation section are the demonstration. The only rendered change is the TFM label text (.NET Framework 4.8 -> net48); no layout change.

Validation and regression proof

All runs on Windows x64 with the repository-local SDK 12.0.100-alpha.1.26556.115 (DOTNET_ROOT=.dotnet), Debug.

  • AssemblyOS (commit 1): MSTest.Acceptance.IntegrationTests.exe --filter "FullyQualifiedName~AssemblyOSConditionTests" after -pack: exit 0, 17 total, 0 failed.
  • TFM parser (commit 2):
    • Microsoft.Testing.Platform.UnitTests.exe --filter "FullyQualifiedName~TargetFrameworkParserTests" (net10.0): exit 0, 46/46 passed.
    • Microsoft.Testing.Extensions.UnitTests.exe --filter "FullyQualifiedName~CrashDump": exit 0, 220 passed, 4 skipped.
    • Full Microsoft.Testing.Extensions.UnitTests (net10.0): exit 0, 3849 passed, 26 skipped.
    • Full Microsoft.Testing.Platform.UnitTests (net10.0): exit 2, with 2 local-only failures in NamedPipeServerSecurityTests (Win32 1346/0x542, named-pipe impersonation on this machine; unrelated to the parser).
  • Pack pins (commit 3), on the rebased tree (current main + this branch): MSTest.Acceptance.IntegrationTests.exe --filter "FullyQualifiedName~FrameworkTargetingTests": exit 0, 4 total, 4 passed (NativeAotCompilerPack_MatchesBootstrappedRuntime and PackedPackages_UseMatchingFrameworkAssetsAndRuntime for net10.0 and net11.0; the packed consumers ran on .NET 10 and .NET 11).
  • eng\common\build.ps1 -restore with global.json from Update .NET SDK and runtime versions #12459 exits 1 (dotnet-install failed ... version '11.0.100', 404); with the 11.0.0 runtimes it exits 0. This confirms Update global.json #12460's fix and is the restore precondition for the targets change.

Not run: Linux/macOS locally, and a fresh full pipeline run (CI on this PR covers it; eng/** forces all tests). Evidence for commits 1 and 2 was collected before rebasing onto current main. The rebased commits carry the identical file changes, and the only intervening main commits touched unrelated files or global.json.

Test-value evidence

  • Commit 1 (AssemblyOS race): tests kept, production-equivalent race made deterministic by injecting a 2 s delay into the generated consumer's hash step.
    • Red (pre-fix test file + delay), --filter "FullyQualifiedName~Vstest_AssemblyOSCondition": exit 2, 2/2 failed with the CI assertion Assert.ContainsSingle(products.Where(...)), expected count 1, actual 0.
    • Green (fixed file + same delay), same filter: exit 0, 2/2 passed. Delay removed afterward; full class 17/17.
  • Commit 2 (TFM parser): tests kept, production change reverted.
    • Red: --filter "FullyQualifiedName~TargetFrameworkParserTests", exit 2, 46 total, 3 failed (ParseNETFramework rows for 4.7/4.8 and the new GetShortTargetFrameworkIncludingPlatform_WithSdkFrameworkDisplayName_MatchesDeclaredTarget net48 row, each returning .NET Framework 4.x instead of net4x).
    • Green: same filter, exit 0, 46/46 passed.
  • Commit 3 (pack pins): existing FrameworkTargetingTests are the regression coverage; the targets file was reverted while keeping everything else.
    • Red A (global.json from Update .NET SDK and runtime versions #12459, targets unchanged), --filter "FullyQualifiedName~FrameworkTargetingTests.NativeAotCompilerPack": exit 2, 2/2 failed. ILCompiler 10.0.14 vs expected 10.0.15; 11.0.0-rc.3.26522.148 vs 11.0.100.
    • Red B (global.json with 11.0.0, i.e. current main, targets unchanged): exit 2, 2/2 failed. 10.0.14 vs 10.0.15; 11.0.0-rc.3.26522.148 vs 11.0.0. This shows main is still red for this contract after Update global.json #12460.
    • Green (targets corrected): full FrameworkTargetingTests, exit 0, 4/4 passed.

The generated consumer records provenance from AppDomain.ProcessExit.
vstest.console only waits VSTEST_TESTHOST_SHUTDOWN_TIMEOUT (100 ms by
default) before killing testhost, which could interrupt the probe after
metadata.txt was written but before products.txt was populated, causing
AssemblyOS_VSTest_* to fail with an empty products match (build 1679434).

Give testhost enough time to finish the probe, materialize products
eagerly, and publish both files atomically with metadata.txt last so an
interrupted probe cannot leave complete-looking but partial evidence.

Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
TargetFrameworkAttribute.FrameworkDisplayName omits a zero patch for
.NET Framework 4.7/4.8 targets (".NET Framework 4.8"). The parser returned
such names unchanged, so CI summary fragments were named ".NET Framework 4.8"
while dotnet test recorded "net48" from the declared framework name. The
Azure DevOps summary post-processor then rejected the fragment with
"CI summary target framework provenance does not match" (seen on the
Windows Release test job for MSTestAdapter.PlatformServices.UnitTests net48).

Map these display names to the same short moniker as the declared target.

Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
#12459/#12460 moved global.json to runtimes 10.0.15 and 11.0.0, but
eng/Net11FrameworkReferences.targets still pinned 10.0.14 and
11.0.0-rc.3. The pinned .NET 12 alpha SDK bundles prerelease
11.0.0-rc.3 net11.0 packs, so the existing StartsWith('12.') guards
never fired once the runtime left rc.3.

Widen the net11 guards to also correct 11.x versions and pin them (and
the net10 Native AOT compiler) to the global.json runtimes, restoring
the FrameworkTargetingTests contract.

Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
Copilot AI balanced review requested due to automatic review settings November 13, 2026 15:12
@github-actions github-actions Bot added the state/needs-review Awaiting review from the team. label Nov 13, 2026

@github-actions github-actions Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🧪 Test quality review — PR #12461

Compared merge base e92578e3a4c076150c8b4174fbe04bd5b9df91d6 with head 673d7f8051876b68d0eb15e22e548df4bd7e5a9b. Evidence applies to this head, not every intermediate commit.

Test-value evidence

All entries are static predictions. This workflow did not run tests. The PR description records red/green runs, but I cannot match them to the exact revisions or per-case outcomes, so they are author claims here.

Contract Scenario / independent expectation Test and decisive assertion Counterfactual Status
TargetFrameworkParser: two-component .NET Framework names map to a short TFM .NET Framework 4.7 / 4.8 yield net47 / net48; the summary side records net48 for .NETFramework,Version=v4.8 New ParseNETFramework rows and GetShortTargetFrameworkIncludingPlatform_WithSdkFrameworkDisplayName_MatchesDeclaredTarget; Assert.AreEqual on the exact TFM Reverting the new branch returns the raw description, so the rows fail (static prediction) Likely detected (static)
3-part names (4.6.2, 4.8.1) are unchanged Existing mapping is preserved The new data rows plus existing rows The edit only adds an earlier return for names shorter than 6 extra characters, so 3-part names are unaffected Likely passes before and after (static)
AssemblyOS VSTest provenance race Provenance files are complete even when testhost is shut down early Vstest_AssemblyOSCondition_* checks products.txt via Assert.ContainsSingle, existing Without the env var or atomic write, a truncated products.txt gives a count of 0 (the CI failure). This is timing-dependent Unverified (executed evidence is not accessible)
eng/Net11FrameworkReferences.targets pack pins Pack versions follow the global.json runtimes Existing FrameworkTargetingTests.NativeAotCompilerPack_* Stale pins mismatch the expected ILCompiler version Static prediction only; not executed here
Decision doc Documentation only None None Not applicable

Unassessed: Bazel pins. The PR states they are out of scope.

🧵 Parallel safety

STATUS: PARTIAL. I did not launch the delegated specialist task for this run, so coverage is a limited static read only. Do not read this as clean.

  • AssemblyOSConditionTests adds an environment variable to the spawned vstest.console process only. It adds no shared mutable state.
  • WriteAtomically runs inside the generated consumer's own evidence directory. It writes a temp file, then moves it over the target (overwrite: true).
  • No [DoNotParallelize] or parallelization configuration changed.

Findings: none identified. Counts: Critical=0; High=0; Warning=0; Info=0.

Per-test assessment

Pseudo-mutation evidence below consists of static predictions, not executed mutation results.

Result Quality Test Notes How to improve
Pass A (90–100) new TargetFrameworkParserTests.GetShortTargetFrameworkIncludingPlatform_WithSdkFrameworkDisplayName_MatchesDeclaredTarget Exact TFM is asserted through both parser entry points for three data cases. None
Pass A (90–100) mod TargetFrameworkParserTests.ParseNETFramework Two new rows pin the zero-patch display-name mapping exactly. None
Pass B (80–89) mod AssemblyOSConditionTests.Vstest_AssemblyOSCondition_PreservesResultsReasonsAndLifecycle Env var change fixes the race; the existing assertions are unchanged and still protect the contract. None

Pseudo-mutation (static): removing the new two-component branch in TargetFrameworkParser is a Likely killed (inferred) change. The 4.8 and 4.7 rows would return .NET Framework 4.x instead of net4x.

No inline comments were published: no verified, actionable findings. Existing review threads were not inspected for duplicates, which is moot because nothing was queued.

<sub>This advisory review is heuristic and non-blocking.

Re-run the combined review with /review-tests, or only the parallel-safety specialist with /parallel-audit.</sub>

🤖 Automated content by GitHub Copilot. Generated by the Test Quality Review on PR (on open / sync) workflow. · copilot · auto · ~30 AIC · ⌖ ~1 AIC · ⊞ 8.1K

@microsoft-github-policy-service microsoft-github-policy-service Bot added state/needs-review Awaiting review from the team. and removed state/needs-review Awaiting review from the team. labels Nov 13, 2026
@github-actions github-actions Bot added state/needs-review Awaiting review from the team. and removed state/needs-review Awaiting review from the team. labels Nov 13, 2026

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🟢 Approval recommended

No active review issues are recorded for the provenance, framework-label normalization, and build-pack updates.

No open issues 🎉

What changed in this PR

Addresses remaining pipeline failures by stabilizing acceptance-test provenance, normalizing Microsoft.Testing.Platform framework labels, and aligning build pack pins with the runtime updates from #12459/#12460.

Changes:

  • Gives VSTest testhost more time to exit and publishes provenance files atomically, with metadata written last.
  • Maps two-component .NET Framework display names to short TFMs such as net48, with regression coverage.
  • Updates .NET 10/11 pack pins and broadens .NET 11 override guards while preserving Windows SDK exclusions.

Test-value evidence: The PR reports focused red/green checks for all three fixes. Execution was not independently verified here; Linux/macOS and a fresh full pipeline run remain outside the reported local validation.

File Description
test/​UnitTests/​Microsoft.Testing.Platform.UnitTests/​OutputDevice/​Terminal/​TargetFrameworkParserTests.cs Covers two-component framework names and agreement between display-name and declared-target parsing.
test/​IntegrationTests/​MSTest.Acceptance.IntegrationTests/​AssemblyOSConditionTests.cs Extends shutdown time and atomically publishes provenance evidence.
src/​Platform/​Microsoft.Testing.Platform/​OutputDevice/​TargetFrameworkParser.cs Converts two-component .NET Framework names to short TFMs.
eng/​Net11FrameworkReferences.targets Aligns runtime pack pins and handles bundled .NET 11 prerelease packs.
docs/​decisions/​net10-and-net11-builds.md Records the GA pin correction, validation results, and separate Bazel scope.

Review effort: Balanced

@microsoft-github-policy-service microsoft-github-policy-service Bot added state/needs-review Awaiting review from the team. and removed state/needs-review Awaiting review from the team. labels Nov 13, 2026
@github-actions github-actions Bot added state/approved Proposal approved; ready for implementation. and removed state/needs-review Awaiting review from the team. labels Nov 13, 2026
@Evangelink
Amaury Levé (Evangelink) merged commit 2cf92e4 into main Nov 13, 2026
42 checks passed
@Evangelink
Amaury Levé (Evangelink) deleted the dev/amauryleve/fix-pipeline-20261113-16 branch November 13, 2026 16:55
@github-actions github-actions Bot removed the state/approved Proposal approved; ready for implementation. label Nov 13, 2026
Amaury Levé (Evangelink) added a commit that referenced this pull request Nov 13, 2026
Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants