Skip to content

[release/10.0.4xx] Serialize concurrent tool package installations - #56861

Open
baronfel wants to merge 2 commits into
release/10.0.4xxfrom
baronfel-backport-pr-51834
Open

baronfel wants to merge 2 commits into
release/10.0.4xxfrom
baronfel-backport-pr-51834

Conversation

@baronfel

@baronfel baronfel commented Oct 30, 2026 •

Copy link
Copy Markdown
Member

Backports #51834 to release/10.0.4xx.

Concurrent tool installs, updates, uninstalls, and shared-cache publication can race, causing sharing violations, partial package visibility, or rollback deleting state another operation has already accepted. This backport preserves the original fix's per-version installation locks, package-wide mutation locks, NuGet-compatible atomic publication, archive-identity validation, and contention diagnostics.

Differences from the .NET 11 implementation

The concurrency fix is essentially the same; the differences adapt it to the .NET 10 branch's existing infrastructure.

Area .NET 11 implementation This backport
Command execution Async install/update path, with a caller-provided cancellation token Retains .NET 10's synchronous command path
Lock cancellation Install/update passes its token into the package-wide mutation lock Lock helpers still support cancellation, but the synchronous command supplies no token; the five-minute timeout remains
Windows owner discovery CsWin32-generated Restart Manager bindings, conditionally compiled for Windows Hand-written P/Invoke bindings, matching the release branch's interop approach; guarded by a runtime Windows check
Tracing Activity spans around install, download, extraction, file movement, and runtime-config updates Retains the release branch's behavior without those newer tracing spans
Tests MSTest attributes and test-context cancellation tokens xUnit attributes, existing fixtures, and CancellationToken.None for test scheduling

The core guarantees are preserved: identical lock identities and polling/timeouts, package-wide serialization through commit or rollback, NuGet's canonical package lock, archive-identity validation, atomic completion markers, read-only cache reuse, and protection against rollback deleting published shared packages. Linux/macOS owner-discovery code and the new diagnostic messages are unchanged.

Other visible source differences—Newtonsoft.Json instead of System.Text.Json, the older runtime-graph path, and the absence of newer cached-version lookup APIs—were already differences between the branches, not functionality removed from this fix.

Follow-up correction from review

Commit e5d5f14 fixes a cleanup issue inherited from #51834: if metadata publication fails after a completion marker is visible, retain the archive and extracted package rather than deleting contents another install can use. The original exception still propagates. The standalone cherry-pick to release/11.0.1xx is #56862; its production patch is identical, with tests adapted to that branch's MSTest conventions.

The regression covers failures immediately before and after the atomic metadata move. With metadata present, a concurrent install accepts the package before the writer fails. With only the hash present, NuGet waits on the canonical package lock and repairs missing metadata after the writer fails. Both cases failed before the cleanup fix and pass afterward.

The separate NUGET_PACKAGES test-isolation review finding is not addressed by this correction.

Validation

  • Full build.cmd succeeded with zero warnings and zero errors for the initial backport.
  • File-lock owner tests: 2 passed, 1 Linux-only test skipped on Windows.
  • Tool package downloader tests after the cleanup correction: 61 passed, 1 platform-specific test skipped; the changed project and production references built successfully.
  • Global tool install, update, and uninstall command tests: 64 passed for the initial backport.
  • The package tests were rerun with the inherited NUGET_PACKAGES override removed so their per-test NuGet.config cache settings took effect; the first run's only failure was the cache override bypassing the test's isolated package directory.

Risk

Medium, as in the original change: this touches core package publication and global-tool mutation paths. The backport retains the original synchronization and publication behavior, with the cleanup correction above, while avoiding unrelated async, tracing, test-framework, or generated-interop migrations.

Tactics

Summary

Concurrent dotnet tool installs, updates, uninstalls, and shared-cache publication race with each other, causing sharing violations, partially visible packages, and rollbacks that delete state another operation already accepted. This backport of #51834 adds per-version installation locks, package-wide mutation locks (held through commit or rollback), NuGet-compatible atomic publication, archive-identity validation, and lock-contention diagnostics (including a file-lock owner finder for Windows/Linux/macOS). It adapts the .NET 11 change to the 10.0.4xx synchronous command path, hand-written Windows P/Invoke, and xUnit tests.

Customer Impact

Users running parallel tool operations (e.g. CI jobs or multiple shells installing/updating/uninstalling the same global tool or sharing a package cache) can hit sharing violations, partially published packages, or tools removed by a competing rollback. The affected scenario is concurrent tool management on release/10.0.4xx SDKs; serial use is unaffected. No workaround is documented in the PR beyond avoiding concurrent operations.

Regression?

Unknown — not enough information to determine origin. The PR describes a latent race condition rather than a regression from a specific PR or version.

Testing

  • New FileLockOwnerFinderTests (2 passed, 1 Linux-only skipped on Windows).
  • Expanded ToolPackageDownloaderTests (~737 added lines; 59 passed, 1 platform-specific skipped) and install/uninstall command tests (64 passed across global tool install/update/uninstall).
  • Full build.cmd succeeded with zero warnings/errors.
  • Package tests were rerun with the inherited NUGET_PACKAGES override removed after one failure caused by that override.
  • CI pipeline has been started; results are pending at the time of writing.

Risk

Medium. About 1,900 lines are added across core package publication and global-tool mutation paths, plus new platform-specific interop (Windows P/Invoke, Linux/macOS owner discovery), but most of that is localization (.xlf) and tests. The core logic mirrors the original .NET 11 fix, and the five-minute lock timeout remains. Risks include lock contention or timeouts in unusual environments and the branch-specific adaptations (synchronous path, no cancellation token passed to the mutation lock).

…51834)

Backport the tool package installation and mutation locks, atomic NuGet publication, lock-owner diagnostics, and regression coverage.

Adapt the change to synchronous tool commands, xUnit tests, and hand-written Windows Restart Manager interop on the release branch.

(cherry picked from commit a77934a)

Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
Copilot AI balanced review requested due to automatic review settings October 30, 2026 18:47
@baronfel

Copy link
Copy Markdown
Member Author

/tactics

@azure-pipelines

Copy link
Copy Markdown
Azure Pipelines:
Successfully started running 1 pipeline(s).
2 pipeline(s) were filtered out due to trigger conditions.
There may be pipelines that require an authorized user to comment /azp run to run.

@baronfel

Copy link
Copy Markdown
Member Author

cc @DamianEdwards for visibility

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🟡 Changes recommended

A metadata-publication failure can delete a package already accepted by a concurrent install, and the cache test can fail or target the wrong directory when NUGET_PACKAGES is set.

2 open issues
What changed in this PR

Backports #51834 to release/10.0.4xx to protect concurrent .NET tool operations and shared-cache publication, while retaining the branch’s synchronous command flow.

Changes:

  • Adds per-version installation locks and package-wide locks held through commit or rollback.
  • Validates archive identities and publishes completion markers atomically under NuGet’s package lock.
  • Adds contention diagnostics, localization entries, and regression tests for publication, rollback, and concurrent operations.
File Description
test/​Microsoft.DotNet.Tools.Tests.ComponentMocks/​ToolPackageDownloaderMock2.cs Aligns mock cleanup with published-package retention.
test/​Microsoft.DotNet.PackageInstall.Tests/​ToolPackageDownloaderTests.cs Tests locking, publication, cache reuse, and failure handling.
test/​Microsoft.DotNet.Cli.Utils.Tests/​FileLockOwnerFinderTests.cs Tests lock-owner discovery and fallback behavior.
test/​dotnet.Tests/​CommandTests/​Tool/​Uninstall/​ToolUninstallGlobalOrToolPathCommandTests.cs Tests uninstall serialization and installed-state reads.
test/​dotnet.Tests/​CommandTests/​Tool/​Install/​ToolInstallGlobalOrToolPathCommandTests.cs Tests serialization through shim-failure rollback.
src/​Cli/​Microsoft.DotNet.Cli.Utils/​FileLockOwnerFinder.Windows.cs Adds Restart Manager interop for owner discovery.
src/​Cli/​Microsoft.DotNet.Cli.Utils/​FileLockOwnerFinder.MacOS.cs Preserves generic diagnostics when ownership cannot be verified.
src/​Cli/​Microsoft.DotNet.Cli.Utils/​FileLockOwnerFinder.Linux.cs Identifies owners using process lock records.
src/​Cli/​Microsoft.DotNet.Cli.Utils/​FileLockOwnerFinder.cs Adds best-effort discovery and process-name sanitization.
src/​Cli/​dotnet/​xlf/​CliStrings.zh-Hant.xlf Adds pending Traditional Chinese contention resources.
src/​Cli/​dotnet/​xlf/​CliStrings.zh-Hans.xlf Adds pending Simplified Chinese contention resources.
src/​Cli/​dotnet/​xlf/​CliStrings.tr.xlf Adds pending Turkish contention resources.
src/​Cli/​dotnet/​xlf/​CliStrings.ru.xlf Adds pending Russian contention resources.
src/​Cli/​dotnet/​xlf/​CliStrings.pt-BR.xlf Adds pending Brazilian Portuguese contention resources.
src/​Cli/​dotnet/​xlf/​CliStrings.pl.xlf Adds pending Polish contention resources.
src/​Cli/​dotnet/​xlf/​CliStrings.ko.xlf Adds pending Korean contention resources.
src/​Cli/​dotnet/​xlf/​CliStrings.ja.xlf Adds pending Japanese contention resources.
src/​Cli/​dotnet/​xlf/​CliStrings.it.xlf Adds pending Italian contention resources.
src/​Cli/​dotnet/​xlf/​CliStrings.fr.xlf Adds pending French contention resources.
src/​Cli/​dotnet/​xlf/​CliStrings.es.xlf Adds pending Spanish contention resources.
src/​Cli/​dotnet/​xlf/​CliStrings.de.xlf Adds pending German contention resources.
src/​Cli/​dotnet/​xlf/​CliStrings.cs.xlf Adds pending Czech contention resources.
src/​Cli/​dotnet/​ToolPackage/​ToolPackageStoreAndQuery.cs Excludes lock metadata from package enumeration.
src/​Cli/​dotnet/​ToolPackage/​ToolPackageDownloaderBase.cs Adds lock helpers and cache-miss serialization.
src/​Cli/​dotnet/​ToolPackage/​ToolPackageDownloader.cs Adds NuGet locking, identity validation, and atomic markers.
src/​Cli/​dotnet/​Commands/​Tool/​Uninstall/​ToolUninstallGlobalOrToolPathCommand.cs Locks before reading and mutating installed state.
src/​Cli/​dotnet/​Commands/​Tool/​Install/​ToolInstallGlobalOrToolPathCommand.cs Serializes mutations and shortcuts exact-version lookup.
src/​Cli/​dotnet/​CliStrings.resx Defines waiting, owner, and timeout messages.

Review effort: Balanced

Comment on lines +1006 to +1007
string lockDirectory = Path.Combine(packagesRoot, ToolPackageStoreAndQuery.LockDirectory);
Directory.Delete(lockDirectory, recursive: true);
Delete failed extractions only while neither completion marker exists. Once published, retain the archive and extracted files so concurrent installs remain valid and NuGet can repair missing metadata under its canonical lock.

Cover failures before and after the metadata move with concurrent cache reuse and verify that both cases fail before the cleanup fix.

Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants