Repository navigation
Serialize concurrent tool package installations with file locks - #51834
Conversation
This comment was marked as outdated.
This comment was marked as outdated.
This comment was marked as outdated.
This comment was marked as outdated.
e779d23 to
137d0f7
Compare
This comment has been minimized.
This comment has been minimized.
This comment was marked as outdated.
This comment was marked as outdated.
Add succinct XML documentation describing why each lock and owner-discovery method exists, with platform and lifetime details in remarks. Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com> Copilot-Session: 815bde8e-0ac4-435d-8ca5-b0d0159ab079
|
Ok @marcpopMSFT I'm happy with this PR now and its ready for review. |
marcpopMSFT
left a comment
There was a problem hiding this comment.
🤖 Posted three review findings: incomplete coordination of cached tool reads, a read-only package-cache regression, and potentially incorrect lock-owner attribution.
Note
These comments were generated by AI. Findings were verified through source inspection; the Windows locking primitive was checked locally, but the reported scenarios were not reproduced end-to-end. No source files were changed.
Delay the NuGet completion marker until extraction succeeds and acquire shared-cache install locks only after a cache miss. This preserves read-only reuse of complete package caches while preventing readers from observing partial extraction. Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com> Copilot-Session: 815bde8e-0ac4-435d-8ca5-b0d0159ab079
Require Linux fdinfo evidence that the target descriptor owns the exclusive flock, and fall back to the generic contention message on macOS where open-file APIs cannot prove lock ownership. Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com> Copilot-Session: 815bde8e-0ac4-435d-8ca5-b0d0159ab079
Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com> Copilot-Session: 815bde8e-0ac4-435d-8ca5-b0d0159ab079
The XNU FWASLOCKED flag is sticky and produced false owner attribution when a process held an unrelated lock while merely opening the target. Keep the generic waiting message because macOS does not expose a reliable current BSD flock owner query. Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com> Copilot-Session: 815bde8e-0ac4-435d-8ca5-b0d0159ab079
|
/tactics |
marcpopMSFT
left a comment
There was a problem hiding this comment.
🤖 Two additional findings from the latest review: non-atomic completion-marker publication and deletion of published shared-cache content during later transaction rollback.
Note
These comments were generated by AI. The Windows marker-write race was reproduced in isolation; the rollback issue was verified through source inspection. No repository files were changed.
Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com> Copilot-Session: 815bde8e-0ac4-435d-8ca5-b0d0159ab079
Coordinate tool publication with NuGet's canonical package lock, validate downloaded identities, publish canonical completion metadata, and serialize package-wide install, update, and uninstall transactions through rollback. Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com> Copilot-Session: 815bde8e-0ac4-435d-8ca5-b0d0159ab079
Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com> Copilot-Session: 815bde8e-0ac4-435d-8ca5-b0d0159ab079
|
/backport to release/10.0.4xx (I think this will not succeed, but worth trying) |
|
Started backporting to |
|
@baronfel backporting to git am output$ git am --3way --empty=keep --ignore-whitespace --keep-non-patch changes.patch
Applying: Serialize concurrent tool package installation
Using index info to reconstruct a base tree...
M src/Cli/dotnet/CliStrings.resx
M src/Cli/dotnet/ToolPackage/ToolPackageDownloaderBase.cs
M src/Cli/dotnet/xlf/CliStrings.cs.xlf
M src/Cli/dotnet/xlf/CliStrings.de.xlf
M src/Cli/dotnet/xlf/CliStrings.es.xlf
M src/Cli/dotnet/xlf/CliStrings.fr.xlf
M src/Cli/dotnet/xlf/CliStrings.it.xlf
M src/Cli/dotnet/xlf/CliStrings.ja.xlf
M src/Cli/dotnet/xlf/CliStrings.ko.xlf
M src/Cli/dotnet/xlf/CliStrings.pl.xlf
M src/Cli/dotnet/xlf/CliStrings.pt-BR.xlf
M src/Cli/dotnet/xlf/CliStrings.ru.xlf
M src/Cli/dotnet/xlf/CliStrings.tr.xlf
M src/Cli/dotnet/xlf/CliStrings.zh-Hans.xlf
M src/Cli/dotnet/xlf/CliStrings.zh-Hant.xlf
Falling back to patching base and 3-way merge...
Auto-merging src/Cli/dotnet/CliStrings.resx
Auto-merging src/Cli/dotnet/ToolPackage/ToolPackageDownloaderBase.cs
CONFLICT (content): Merge conflict in src/Cli/dotnet/ToolPackage/ToolPackageDownloaderBase.cs
Auto-merging src/Cli/dotnet/xlf/CliStrings.cs.xlf
Auto-merging src/Cli/dotnet/xlf/CliStrings.de.xlf
Auto-merging src/Cli/dotnet/xlf/CliStrings.es.xlf
Auto-merging src/Cli/dotnet/xlf/CliStrings.fr.xlf
Auto-merging src/Cli/dotnet/xlf/CliStrings.it.xlf
Auto-merging src/Cli/dotnet/xlf/CliStrings.ja.xlf
Auto-merging src/Cli/dotnet/xlf/CliStrings.ko.xlf
Auto-merging src/Cli/dotnet/xlf/CliStrings.pl.xlf
Auto-merging src/Cli/dotnet/xlf/CliStrings.pt-BR.xlf
Auto-merging src/Cli/dotnet/xlf/CliStrings.ru.xlf
Auto-merging src/Cli/dotnet/xlf/CliStrings.tr.xlf
Auto-merging src/Cli/dotnet/xlf/CliStrings.zh-Hans.xlf
Auto-merging src/Cli/dotnet/xlf/CliStrings.zh-Hant.xlf
error: Failed to merge in the changes.
hint: Use 'git am --show-current-patch=diff' to see the failed patch
hint: When you have resolved this problem, run "git am --continue".
hint: If you prefer to skip this patch, run "git am --skip" instead.
hint: To restore the original branch and stop patching, run "git am --abort".
hint: Disable this message with "git config set advice.mergeConflict false"
Patch failed at 0001 Serialize concurrent tool package installation
Error: The process '/usr/bin/git' failed with exit code 128 |
Concurrent installs, updates, uninstalls, and shared-cache publication for the same .NET tool package could race, causing sharing violations, partial package visibility, or rollback deleting state another operation had already accepted.
Changes
FileShare.None..nupkglock so tool installation and ordinary NuGet writers cannot mutate the same package concurrently..sha512and NuGet.nupkg.metadatacompletion markers atomically after extraction succeeds, including NuGet-compatible content hashes for signed packages./proc/locksplus the matching descriptor's/proc/<pid>/fdinfolock record.libprocopen-file data and XNU's stickyFWASLOCKEDflag do not reliably identify the current BSDflockowner..locksmetadata directory from global tool package enumeration.Fixes #51831
Tactics
Summary
Concurrent
dotnet tool install,dotnet tool update,dotnet tool uninstall,dotnet tool exec, anddnxoperations could race while downloading, extracting, publishing, replacing, or rolling back the same tool package. This PR adds per-version package locks, interoperates with NuGet's canonical package lock and completion markers, and serializes package-wide global-tool mutations through transaction completion. Contention diagnostics include a verified owner where the platform exposes one and otherwise safely fall back to a generic message. Lock polling is cancellation-aware for callers that provide a token while retaining a timeout as the final bound for a hung operation.Customer Impact
Users and tools such as Aspire that run tool operations in parallel can hit sharing violations, incomplete package caches, inconsistent package-plus-shim state, or a package disappearing after another process accepted it. The reported Windows failure is tracked by #51831. Avoiding parallel operations or installing the tool once first remains a workaround for affected SDKs.
Regression?
No. This fixes concurrency failures in existing tool operations without changing command syntax or successful single-process behavior.
Testing
build.cmd.ToolPackageDownloaderTests: 61 total, 59 passed, 2 Windows-inapplicable tests skipped.Risk
Medium. The change touches the core tool package download and global mutation paths and introduces cross-process coordination. Risk is mitigated by preserving read-only cache fast paths, using NuGet's canonical lock and marker formats, keeping owner discovery best-effort, supporting cancellation during lock polling, timing out prolonged contention, and covering publication and transaction boundaries with focused concurrent tests.