-
Notifications
You must be signed in to change notification settings - Fork 2.7k
Do not lock the artifact-dir for check builds #16230
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
base: master
Are you sure you want to change the base?
Conversation
54eb64c to
1d7839e
Compare
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
This should be in check command test file maybe?
https://github.com/rust-lang/cargo/blob/master/tests/testsuite/check.rs
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
ah good idea. I was thinking about putting it in https://github.com/rust-lang/cargo/blob/master/tests/testsuite/artifact_dep.rs but noticed that file only has test for the unstable --artifact-dir flag.
master/tests/testsuite/check.rs seems like a good place for this
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Should we also add one test asserting that no files are generated to the artifact directory from cargo check?
(Maybe we already have one though)
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Added a test in da0dfd3
1d7839e to
6f322e8
Compare
| // directory, so just lock the entire thing for the duration of this | ||
| // compile. | ||
| let artifact_dir_lock = if is_on_nfs_mount(root.as_path_unlocked()) { | ||
| let artifact_dir_lock = if !must_take_artifact_dir_lock |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Instead of conditionally grabbing the lock, can we put the entire artifact layout in an Option? That has the potential to help us find bugs but unsure how messy that would be.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I looked into this and it got pretty hairy. Switching from T to Option<T> is not too bad but there quiet a few places during compilation where the paths are used but not written to.
For example, in build_runner::prepare we build up the root_output
cargo/src/cargo/core/compiler/build_runner/mod.rs
Lines 395 to 397 in 8c4a25c
| self.compilation | |
| .root_output | |
| .insert(kind, layout.artifact_dir().dest().to_path_buf()); |
and we use this in fill_env()
cargo/src/cargo/core/compiler/compilation.rs
Lines 308 to 311 in 8c4a25c
| search_path.extend(super::filter_dynamic_search_path( | |
| self.native_dirs.iter(), | |
| &self.root_output[&kind], | |
| )); |
| // We try to only lock the artifact-dir if we need to. | ||
| // For example, `cargo check` does not write any files to the artifact-dir so we don't need | ||
| // to lock it. | ||
| let must_take_artifact_dir_lock = match self.bcx.build_config.intent { |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Instead of intent, can we check if artifacts are produced? Do we know that?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I checked and I think it may be possible to use some of the logic from CompilationFiles output but we would likely need to duplicate it as we need to construct Layout before CompilationFiles.
Let me know if there is a better way to get this data.
What does this PR try to resolve?
This PR modifies the locking logic to avoid locking
artifact-dirfor check builds.Part of #4282
Note: This change is not behind
-Zbuild-dir-new-layoutor-Zfine-grain-lockingunstable flags.How to test and review this PR?
See the tests included in the PR.
You can run
cargo checkto verify.r? @epage