Wait for request stream to flush before returning resolution#2374
Merged
charliermarsh merged 1 commit intomainfrom Mar 12, 2024
Merged
Wait for request stream to flush before returning resolution#2374charliermarsh merged 1 commit intomainfrom
charliermarsh merged 1 commit intomainfrom
Conversation
charliermarsh
commented
Mar 12, 2024
|
|
||
| // Run the solver. | ||
| let resolve_fut = self.solve(&request_sink).fuse(); | ||
| let resolve_fut = self.solve(request_sink).fuse(); |
Member
Author
There was a problem hiding this comment.
Taking ownership means that request_sink is dropped as soon as the future completes.
Member
Author
|
No meaningful change, as expected: |
konstin
approved these changes
Mar 12, 2024
| Client(#[from] uv_client::Error), | ||
|
|
||
| #[error("The channel is closed, was there a panic?")] | ||
| #[error("The channel closed unexpectedly")] |
Member
There was a problem hiding this comment.
I added the panic message to all the broken channel cases because in my experience this error happens when there was a panic in another thread, so the error message you get is ~irrelevant, while scrolling up another thread spilled a first panic to stderr that has the actually relevant failure
BurntSushi
approved these changes
Mar 12, 2024
Member
Author
|
I also benchmarked on home-assistant to be safe (this branch is "faster" but I think it's just noise): |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
This is a more robust fix for #2300.
The basic issue is:
--no-deps, but the pre-fetch was causing us to attempt to build a package to get its dependencies. The resolution would then finish before the build completed.)Indexwill be marked as "waiting" for that response -- but it'll never come through.Index, to see if we should fetch or are waiting for that response, we'll end up waiting for it forever, since it looks like it's in-flight (but isn't). (In the linked issue, we had to build the source distribution for the install phase ofpip install, butsetuptoolswas in this bad state from the resolve phase.)This PR modifies the resolver to ensure that we flush the stream of requests before returning. Specifically, we now
joinrather thanselectbetween the resolution and request-handling futures.This could be wasteful, since we don't need those requests, but it at least ensures that every
.waitis followed by.done. In practice, I expect this not to have any significant effect on performance, since we end up using the pre-fetched distributions almost every time.Test Plan
I ran through the test plan from #2373, but ran the build 10 times and ensured it never crashed. (I reverted #2373, since that also fixes the issue in the proximate case, by never fetching
setuptoolsduring the resolve phase.)I also added logging to verify that requests are being handled after the resolution completes, as expected.
I also introduced an arbitrary error in
fetchto ensure that the error was immediately propagated.