Repository navigation
Enable in-process MTP generation tasks for multithreaded MSBuild - #12307
Conversation
Opt configuration and self-registration generation into in-process -mt execution without requiring newer MSBuild hosts. Resolve file paths through item metadata, pass debugger activation explicitly, and add the private task-authoring analyzer with packaged red/green regression coverage. Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
There was a problem hiding this comment.
Note
🤖 Automated review by GitHub Copilot. Generated by the Expert Code Review workflow. To request a follow-up action, reply by tagging @copilot directly.
The approach is sound: it keeps the old-host-compatible embedded marker attribute, resolves paths through FullPath metadata, and replaces the process-environment read with an explicit LaunchDebugger input. I left four non-blocking comments.
Reviewer actions
- Recommended – Fix the
PublicAPIbaseline. Done when: previously shipped signatures are inPublicAPI.Shipped.txtand onlyLaunchDebuggeris inUnshipped. - Recommended – Replace or explain the
MT-sentinel-CWDassertion. Done when: the acceptance test depends only on output the test owns. - Recommended – Make the analyzer-version note consistent and track moving off the prerelease. Done when: the
Directory.Packages.propscomment and the decision record agree, and a tracked issue exists. - Recommended – Trim local-only artifacts from the decision record.
Change classification
- MSBuild task thread-safety migration for
ConfigurationFileTaskandTestingPlatformSelfRegisteredExtensions - New public task parameter
LaunchDebuggerand the targets change that sets it - Build-only analyzer dependency (prerelease)
Confidence at a glance
Build, dependencies & scripts
Dependency assessment: Microsoft.Build.TaskAuthoring.Analyzer 18.14.0-1.26559.118 is added as PrivateAssets=All, so it does not leak into the package. It is a prerelease pinned only because the stable version is unavailable on the configured feeds. See the inline comment.
Public API & compatibility
The behavior change is limited to path resolution. ItemSpec for these items is already normalized in the targets, so the change should be neutral for existing callers. The LaunchDebugger addition is additive, and a missing value defaults to false. The PublicAPI file seeding is questionable (see the inline comment).
Tests
The new unit cases cover resolved-path behavior. The acceptance test verifies in-process execution with a binlog check on TaskHost launches. I did not run any of it myself (read-only review). The MT-sentinel-CWD assertion is the main fragility.
There was a problem hiding this comment.
🧪 Test quality review — PR #12307
Compared merge base 8bdfc29f415696738d845d56aca5d438d0a068de with head 22b50c1dfd3cf33b2a606d1868592398c5db91e5. Evidence applies to this head, not every intermediate commit.
Test-value evidence
Static analysis only. No tests were run, so executed evidence is limited to the author's claims. Those claims are not independently verified here, and no execution records were accessible for review.
| Contract | Test / decisive assertion | Static prediction on the old behavior | Result |
|---|---|---|---|
Self-registered-extensions output is written to the resolved FullPath |
SelfRegisteredExtensions_UsesProjectResolvedFullPath; the only file key equals outputPath |
Old code writes to ItemSpec (obj/extensions.cs), so the key assertion would fail. Likely killed (inferred). |
Likely detects |
Configuration file task uses FullPath for source, project directory and output (copy and merge) |
ConfigurationFileTask_UsesProjectResolvedFullPaths; source lookup and output path assertions, plus JSON content |
Old code reads ItemSpec (testconfig.json), which is absent from the in-memory file system. The task would not produce the expected output. Likely killed (inferred). |
Likely detects |
Generation tasks run in-process under -mt |
GenerationTasks_PreserveProjectIsolationAndRunInProcess; counts 4 generation tasks and no "Launching task" messages, checks the DLL hash, and checks each project's own output |
Without the MSBuildMultiThreadableTask marker, the tasks launch in a TaskHost and the no-launch assertion fails. Likely killed (inferred). |
Likely detects (static) |
LaunchDebugger is now an explicit task input, replacing the constructor environment-variable read |
The acceptance test clears the env var. No test asserts the LaunchDebugger wiring. |
Not testable safely, because it would launch a debugger. | Unverified, evidence gap (not a defect) |
Analyzer package, PublicAPI inventory, docs |
Not behavioral | n/a | Not applicable |
The author reports an executed old-versus-new comparison in the PR body. It is not independently verified by this workflow.
🧵 Parallel safety
STATUS: CLEAN. Counts: Critical=0, High=0, Warning=0, Info=0.
| Test assembly | Scope | Workers | Analyzer coverage |
|---|---|---|---|
| Microsoft.Testing.Platform.Acceptance.IntegrationTests | MethodLevel | 0 (all cores) | Not run (read-only) |
| Microsoft.Testing.Platform.MSBuild.UnitTests | MethodLevel | 0 (all cores) | Not run (read-only) |
Both new test shapes use unique temp directories and an in-memory file system, so they share no mutable state.
Per-test assessment
Pseudo-mutation evidence below consists of static predictions, not executed mutation results.
| Result | Quality | Test | Notes | How to improve |
|---|---|---|---|---|
| Pass | A (90–100) | new MSBuildMultithreadingTests.GenerationTasks_PreserveProjectIsolationAndRunInProcess |
Checks task count, no TaskHost launch, DLL hash and per-project outputs. | None |
| Pass | A (90–100) | new MSBuildTests.ConfigurationFileTask_UsesProjectResolvedFullPaths |
Exact output path, file count and JSON contents are asserted for both copy and merge. | None |
| Pass | A (90–100) | new MSBuildTests.SelfRegisteredExtensions_UsesProjectResolvedFullPath |
Asserts the single output key, generated hook text and returned item. | None |
Pseudo-mutation evidence: reverting the FullPath use back to ItemSpec is a likely kill (inferred) for the two unit tests. Removing the thread-safety marker is a likely kill (inferred) for the acceptance test. The LaunchDebugger wiring is N/A / unverified.
No inline findings were published.
<sub>This advisory review is heuristic and non-blocking.
Re-run the combined review with /review-tests, or only the parallel-safety specialist with /parallel-audit.</sub>
🤖 Automated content by GitHub Copilot. Generated by the Test Quality Review on PR (on open / sync) workflow. · copilot · auto · ~59 AIC · ⌖ ~1 AIC · ⊞ 8.1K
There was a problem hiding this comment.
🟢 Approval recommended
No active review issues remain that require changes before approval.
No open issues 🎉
What changed in this PR
Enables MTP configuration-file and self-registered-extension generation to run in-process during multithreaded MSBuild builds, while preserving generated output and older-host compatibility.
Changes:
- Marks both generation tasks as thread-safe and uses project-resolved paths for file access.
- Makes debugger activation an explicit task input and adds a private task-authoring analyzer dependency.
- Adds resolved-path unit tests, packaged-consumer acceptance tests, API inventories, and compatibility documentation.
Test-value evidence: For revision 22b50c1, the PR reports 160 passing unit cases and two passing packaged cases, with six intended unit failures and one routing failure against original production sources. Raw execution artifacts remain local, so these results were not independently verified here.
| File | Description |
|---|---|
test/UnitTests/Microsoft.Testing.Platform.MSBuild.UnitTests/MSBuildTests.cs |
Adds resolved-path regression cases for both generation tasks. |
test/IntegrationTests/Microsoft.Testing.Platform.Acceptance.IntegrationTests/MSBuildTests.Multithreading.cs |
Tests packaged task routing, project isolation, and generated outputs. |
src/Platform/Microsoft.Testing.Platform.MSBuild/Tasks/TestingPlatformAutoRegisteredExtensions.cs |
Adds the thread-safety marker, resolved output path, and debugger input. |
src/Platform/Microsoft.Testing.Platform.MSBuild/Tasks/ConfigurationFileTask.cs |
Adds the thread-safety marker and resolved configuration paths. |
src/Platform/Microsoft.Testing.Platform.MSBuild/PublicAPI.Unshipped.txt |
Inventories existing task APIs and the new debugger property. |
src/Platform/Microsoft.Testing.Platform.MSBuild/PublicAPI.Shipped.txt |
Initializes the shipped API baseline. |
src/Platform/Microsoft.Testing.Platform.MSBuild/PACKAGE.md |
Documents multithreaded execution and older-host compatibility. |
src/Platform/Microsoft.Testing.Platform.MSBuild/MSBuildMultiThreadableTaskAttribute.cs |
Defines the embedded compatibility marker. |
src/Platform/Microsoft.Testing.Platform.MSBuild/Microsoft.Testing.Platform.MSBuild.csproj |
Adds the task-authoring analyzer as a private dependency. |
src/Platform/Microsoft.Testing.Platform.MSBuild/buildMultiTargeting/Microsoft.Testing.Platform.MSBuild.targets |
Passes debugger activation explicitly to the task. |
docs/decisions/msbuild-task-multithreading.md |
Records design decisions, comparison results, provenance, and validation limits. |
Directory.Packages.props |
Pins the task-authoring analyzer version. |
Review effort: Balanced
Seed shipped task APIs from the published preview and retain only LaunchDebugger as unshipped. Keep durable migration decisions and execution outcomes, clarify analyzer feed provenance, and explain the upstream strict-mode sentinel assertion without changing its checks. Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
There was a problem hiding this comment.
🧪 Test quality review — PR #12307
Compared merge base 8bdfc29f415696738d845d56aca5d438d0a068de with head 80ae839768b6890482996f534611da0efe0bd0fd. Evidence applies to this head, not every intermediate commit.
Test-value evidence
Static analysis only; no tests were run. The author's red/green comparison (PR body) was not independently verified, and no execution records were accessible.
| Contract | Test / decisive assertion | Static prediction on old behavior | Result |
|---|---|---|---|
Self-registered-extensions output is written to resolved FullPath |
SelfRegisteredExtensions_UsesProjectResolvedFullPath; the only file key equals outputPath |
Old code writes to ItemSpec (obj/extensions.cs), so the key assertion fails. Likely killed (inferred). |
Likely detects |
Configuration task uses FullPath for source, project dir and output (copy and merge) |
ConfigurationFileTask_UsesProjectResolvedFullPaths; output path, file count and JSON content |
Old code reads ItemSpec (testconfig.json), absent from the in-memory file system. Likely killed (inferred). |
Likely detects |
Generation tasks run in-process under -mt |
GenerationTasks_PreserveProjectIsolationAndRunInProcess; no Launching task messages, DLL hash, per-project outputs |
Without the MSBuildMultiThreadableTask marker, tasks launch in a TaskHost and the assertion fails. Likely killed (inferred). |
Likely detects (static) |
LaunchDebugger is an explicit task input replacing the constructor env-var read |
Acceptance test clears the env var; no assertion on the wiring | Cannot be tested safely (would launch a debugger). | Unverified, evidence gap (not a defect) |
Analyzer package, PublicAPI, docs, targets wiring |
Not behavioral | n/a | Not applicable |
🧵 Parallel safety
STATUS: CLEAN. Counts: Critical=0, High=0, Warning=0, Info=0.
| Test assembly | Scope | Workers | Analyzer coverage |
|---|---|---|---|
| Microsoft.Testing.Platform.Acceptance.IntegrationTests | MethodLevel | 4 | Not run (read-only) |
| Microsoft.Testing.Platform.MSBuild.UnitTests | MethodLevel | default | Not run (read-only) |
The acceptance asset uses a per-asset random directory and a GUID-named binlog; unit tests use an in-memory file system per test. Directory.Packages.props was the only configuration candidate and has no parallelization effect.
Per-test assessment
Pseudo-mutation evidence below consists of static predictions, not executed mutation results.
| Result | Quality | Test | Notes | How to improve |
|---|---|---|---|---|
| Pass | A (90–100) | new MSBuildMultithreadingTests.GenerationTasks_PreserveProjectIsolationAndRunInProcess |
Checks task count, no TaskHost launch, DLL hash and per-project outputs. | None |
| Pass | A (90–100) | new MSBuildTests.ConfigurationFileTask_UsesProjectResolvedFullPaths |
Exact output path, file count and JSON contents asserted for copy and merge. | None |
| Pass | A (90–100) | new MSBuildTests.SelfRegisteredExtensions_UsesProjectResolvedFullPath |
Asserts the single output key, generated hook text and returned item. | None |
Pseudo-mutation evidence: reverting FullPath use to ItemSpec is a likely kill (inferred) for both unit tests; removing the thread-safety marker is a likely kill (inferred) for the acceptance test; LaunchDebugger wiring is N/A / unverified.
No new inline findings. The MT-sentinel-CWD assertion fragility is already raised in an existing review thread, so it is not repeated here.
<sub>This advisory review is heuristic and non-blocking.
Re-run the combined review with /review-tests, or only the parallel-safety specialist with /parallel-audit.</sub>
🤖 Automated content by GitHub Copilot. Generated by the Test Quality Review on PR (on open / sync) workflow. · copilot · auto · ~58 AIC · ⌖ ~1 AIC · ⊞ 8.1K
This comment has been minimized.
This comment has been minimized.
|
Final test-triage resolution — Azure Pipelines build 1672831 Result: inconclusive (evidence incomplete: 2 artifact download failures; history collection hit Failing leg: Build › Windows Release › "Test affected changes" (exit code 2). Signature: Assessment (low–medium confidence):
Next step: Rerun the Windows Release leg. If
|
Pull request
Summary
Allow MTP configuration-file and self-registered-extension generation to run in-process during
-mtbuilds instead of requiring isolated TaskHosts. Preserve generated output and older-MSBuild compatibility.The tasks use project-resolved
FullPathmetadata and an embedded thread-safety marker. Debugger activation becomes an explicit task input rather than a constructor environment-variable read. Add the task-authoring analyzer privately to the task project and cover real packaged consumers.Design and decision record
Research, decisions, and revision-bound evidence.
Use MSBuild's attribute compatibility bridge instead of requiring
TaskEnvironmentor separate task assemblies. Existing task parameter types and constructors remain unchanged. The existing public/protected surface was checked against published package3.0.0-preview.26556.8and recorded inPublicAPI.Shipped.txt; only the new, default-falseLaunchDebuggerinput remains unshipped.Analyzer
18.13.0could not restore from the configured feeds, so use18.14.0-1.26559.118from the existing publicdotnet-toolsfeed. No new feed or runtime dependency is introduced. Other tasks are not marked thread-safe.Demonstration
Nonvisual build-routing change; binlogs and executable consumer assertions are clearer than screenshots. The same two-project build executes both generation tasks once per project:
-mtUpdated packaged consumers each exit 0 and emit only their own
hook=Copyorhook=Mergemarker. Configuration assertions associate each project with its own JSON and option defaults. The task DLL loaded by each invocation is checked against the packed DLL by SHA-256 and the consumer's resolved package folders.Validation and regression proof
The initial migration was validated at
22b50c1dfd3cf33b2a606d1868592398c5db91e5. Review corrections at80ae839768b6890482996f534611da0efe0bd0fdchange only API classification, documentation, and comments; production behavior and regression assertions are unchanged. The same 160 unit cases and two packaged cases were rerun successfully (exit 0, no skips). Repository SDK:12.0.100-alpha.1.26556.115, MSBuild18.14.0.55715.Commands from the worktree root, with the bootstrapped
.dotnetonPATHand asDOTNET_ROOT:GenerationTasks_PreserveProjectIsolationAndRunInProcesscovers multiprocess and strict multithreaded builds of twonet10.0consumers. The binlog contains only the strict-mode enable message forMT-sentinel-CWD;MSB4286,MSB4287, andMSB4288are promoted to errors. Fresh TRX execution gates and JUnit parsing confirm the reported counts.Actual older-host validation also passed (exit 0): from the retained
artifacts\mt-validation\legacy-msbuildprobe,C:\Program Files\dotnet\dotnet.exe msbuild Generation.proj -nologo -t:Build -m:2 -nr:false -v:normal -bl:legacy-host.binlog, using installed SDK10.0.114/ MSBuild18.0.36.52326. Both packed tasks generated the expected source and merged configuration.Known validation limit:
.\eng\build.ps1 -restore -build -pack -msbuildEngine dotnet -bl -properties '/p:SkipUapTargetFramework=true'failed (exit 1) at the final Windows package verifier: excluding UAP produces 64/66 expected adapter files and 88/105 framework files. Compilation and scoped task packing pass. This unrelated check was not suppressed. CI, desktop MSBuild, non-Windows execution, and performance gains are unverified.Raw TRX/JUnit/binlogs were retained locally, not uploaded; this is reported execution evidence rather than independently accessible CI proof. The linked decision record preserves durable rationale, reproducible test selections, observed outcomes, and verification limits.
Test-value evidence
Executed comparison: retain the same tests and assertions while using unchanged production sources at
8bdfc29f415696738d845d56aca5d438d0a068dein an isolated worktree. Rebuild and repack each variant with separate outputs and consumer caches; verify loaded package provenance.SelfRegisteredExtensions_UsesProjectResolvedFullPathConfigurationFileTask_UsesProjectResolvedFullPathsGenerationTasks_PreserveProjectIsolationAndRunInProcessThe same focused unit command executes 160 cases in each state: original production has 154 passed / 6 intended failures / 0 skipped (exit 2); updated production has 160 passed / 0 failed / 0 skipped (exit 0). The same acceptance command executes two cases: original production has 1 passed / 1 intended failure (exit 2); updated production has 2 passed (exit 0). Build/setup failures are not counted as red-phase proof.