-
-
Notifications
You must be signed in to change notification settings - Fork 10.6k
Cleanup partial download queue installations on error/interrupt #20966
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Conversation
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Pull Request Overview
This PR adds automatic cleanup of partial bottle installations when errors occur during the download and extraction process in RetryableDownload#fetch. The cleanup logic is extracted into a reusable method Keg#uninstall_ignore_interrupts_and_raise_failures! that is shared between RetryableDownload and FormulaInstaller.
Key Changes
- Added
cleanup_partial_installation_on_error!method to cleanup partial bottle installations in error scenarios - Extracted cleanup logic into a new
Keg#uninstall_ignore_interrupts_and_raise_failures!method for reusability - Refactored
FormulaInstallerto use the new shared cleanup method
Reviewed Changes
Copilot reviewed 3 out of 3 changed files in this pull request and generated 1 comment.
| File | Description |
|---|---|
| Library/Homebrew/retryable_download.rb | Added bottle_keg variable tracking and cleanup logic in rescue blocks; added cleanup_partial_installation_on_error! method |
| Library/Homebrew/keg.rb | Added new uninstall_ignore_interrupts_and_raise_failures! method and added type signature to existing uninstall method |
| Library/Homebrew/formula_installer.rb | Replaced inline cleanup code with call to new Keg#uninstall_ignore_interrupts_and_raise_failures! method |
Comments suppressed due to low confidence (1)
Library/Homebrew/retryable_download.rb:6
- Missing
require \"keg\"statement. The newcleanup_partial_installation_on_error!method at line 132 callsKeg.new(path), but the Keg class is not explicitly required in this file. Addrequire \"keg\"to the require statements at the top of the file.
require "bottle"
require "api/json_download"
require "utils/output"
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
An issue that remains is when the pour completes (i.e. promise is fulfilled) and then user interrupts. Since the job is done, there is no interrupt exception here. Also can happen when interrupt is done later, e.g. when I cancel at Not sure what is best way to show case reproduction, but here is my steps for above case run on this branch: ❯ brew list | grep qt
❯ brew install qt
==> Fetching downloads for: qt
✔︎ Bottle Manifest qt (6.9.3)
...
==> Installing qt dependency: md4c
==> Pouring md4c--0.5.2.arm64_tahoe.bottle.tar.gz
🍺 /opt/homebrew/Cellar/md4c/0.5.2: 20 files, 301.3KB
==> Installing qt dependency: qtbase
==> Pouring qtbase--6.9.3.arm64_tahoe.bottle.1.tar.gz
🍺 /opt/homebrew/Cellar/qtbase/6.9.3: 3,991 files, 58.9MB
==> Installing qt dependency: qtsvg
^C
❯ for formula in $(brew list --formulae); do
ls "$(brew --prefix)/Cellar/$formula"/*/INSTALL_RECEIPT.json >/dev/null
done
zsh: no matches found: /opt/homebrew/Cellar/abseil/*/INSTALL_RECEIPT.json
zsh: no matches found: /opt/homebrew/Cellar/gumbo-parser/*/INSTALL_RECEIPT.json
zsh: no matches found: /opt/homebrew/Cellar/hunspell/*/INSTALL_RECEIPT.json
zsh: no matches found: /opt/homebrew/Cellar/libmng/*/INSTALL_RECEIPT.json
...
❯ brew autoremove; echo $?
0
❯ ls $(brew --prefix)/Cellar/qtsvg/*/
Frameworks lib share sbom.spdx.json
❯ ls $(brew --prefix)/Cellar/qtbase/*/
bin Frameworks include lib share INSTALL_RECEIPT.json sbom.spdx.json |
|
Was looking into this a bit more and I am thinking (with current install flow) remove logic will need to be done around brew/Library/Homebrew/install.rb Lines 417 to 424 in 71358ca
As an interrupt can happen at any point from start of Only thing is we don't have all attempted pours available as a FormulaInstaller can create another FormulaInstaller for dependency. Could potentially track Bottle pours as part of download_queue. Also, there was a user who hit a similar issue (Homebrew/homebrew-core#252010) But it looks like they ended up with brew/Library/Homebrew/formula_installer.rb Lines 939 to 940 in 71358ca
brew/Library/Homebrew/formula_installer.rb Line 949 in 71358ca
|
|
@cho-m Thanks for reviewing. Unless I've misunderstood: this PR addresses at least one of the initial issues? Would it not be worth making ✅ and merging and then I can work on more cases in more PRs? |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Makes sense to me!
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Logically seems fine. Haven't figured out a good way to test this though due to concurrent timing. May require programmatically triggering interrupt during a download or adding sleeps to allow manual interrupt.
EDIT: I did try adding a timeout in directory unpack but not sure what signal is sent to all threads as ctrl+c still exits. Manually had to mkdir /opt/homebrew/Cellar/qtsvg/6.9.3 and tried sleep when moving /opt/homebrew/Cellar/qtsvg/6.9.3/lib
✔︎ Bottle qtwebengine (6.9.3)
^CBottle qtsvg (6.9.3)
❯ ls /opt/homebrew/Cellar/qtsvg/6.9.3
FrameworksThis will port the similar logic from `formula_installer.rb` that handles errors or manual user interrupts from producing partial installations. I noticed this code is almost identical (with some missing edge-cases) in `formula_installer.rb` to `Keg#uninstall` so I added a new `Keg#uninstall_ignore_interrupts_and_raise_failures!` method and made both `formula_installer.rb` and `retryable_download.rb` use it.
c6d84e1 to
d162130
Compare
This will port the similar logic from
formula_installer.rbthat handles errors or manual user interrupts from producing partial installations.I noticed this code is almost identical (with some missing edge-cases) in
formula_installer.rbtoKeg#uninstallso I added a newKeg#uninstall_ignore_interrupts_and_raise_failures!method and made bothformula_installer.rbandretryable_download.rbuse it.@cho-m can you try this out and see if it fixes the issue? I cannot reliably reproduce this so I've had to hack together something a little.