Skip to content

Do not lock the artifact-dir for check builds #16230

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
wants to merge 3 commits into
base: master
Choose a base branch
from
+64 −13
Open

Do not lock the artifact-dir for check builds #16230

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
3 commits
Select commit Hold shift + click to select a range
da0dfd3
test: Verify cargo check does not output files to artifact-dir
ranger-ross Nov 8, 2025
a1d20db
test: Added test for artifact-dir locking behavior
ranger-ross Nov 8, 2025
6f322e8
feat: Do not lock the artifact-dir for check builds
ranger-ross Nov 8, 2025
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
22 changes: 19 additions & 3 deletions src/cargo/core/compiler/build_runner/mod.rs
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -6,7 +6,7 @@ use std::sync::{Arc, Mutex};

use crate::core::PackageId;
use crate::core::compiler::compilation::{self, UnitOutput};
use crate::core::compiler::{self, Unit, artifact};
use crate::core::compiler::{self, Unit, UserIntent, artifact};
use crate::util::cache_lock::CacheLockMode;
use crate::util::errors::CargoResult;
use annotate_snippets::{Level, Message};
Expand Down Expand Up @@ -352,11 +352,27 @@ impl<'a, 'gctx> BuildRunner<'a, 'gctx> {
#[tracing::instrument(skip_all)]
pub fn prepare_units(&mut self) -> CargoResult<()> {
let dest = self.bcx.profiles.get_dir_name();
let host_layout = Layout::new(self.bcx.ws, None, &dest)?;
// We try to only lock the artifact-dir if we need to.
// For example, `cargo check` does not write any files to the artifact-dir so we don't need
// to lock it.
let must_take_artifact_dir_lock = match self.bcx.build_config.intent {
Copy link
Contributor

@epage epage Nov 8, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Instead of intent, can we check if artifacts are produced? Do we know that?

Copy link
Member Author

@ranger-ross ranger-ross Nov 9, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I checked and I think it may be possible to use some of the logic from CompilationFiles output but we would likely need to duplicate it as we need to construct Layout before CompilationFiles.

See https://github.com/rust-lang/cargo/blob/master/src/cargo/core/compiler/build_runner/compilation_files.rs#L405

Let me know if there is a better way to get this data.

Copy link
Contributor

@epage epage Nov 10, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Hmm, that can get messy. We could create an enum for the base path and calculate the full path later but unsure how well that will work out.

However, this does highlight a need for either this or conditionally grabbing the lock (#16230 (comment))

cargo/src/cargo/core/compiler/build_runner/compilation_files.rs

Lines 529 to 540 in 8c4a25c

if bcx.build_config.sbom && bcx.gctx.cli_unstable().sbom {
let sbom_files: Vec<_> = outputs
.iter()
.filter(|o| matches!(o.flavor, FileFlavor::Normal | FileFlavor::Linkable))
.map(|output| OutputFile {
path: Self::append_sbom_suffix(&output.path),
hardlink: output.hardlink.as_ref().map(Self::append_sbom_suffix),
export_path: output.export_path.as_ref().map(Self::append_sbom_suffix),
flavor: FileFlavor::Sbom,
})
.collect();
outputs.extend(sbom_files.into_iter());

We would have been writing out SBOMs without holding the lock. If we don't feel SBOMs are relevant for cargo check then making the layout optional (to catch problems like this) is sufficient. If not, then we need to somehow be aware of this for locking and then that starts to look like needing to resolve this item.

Copy link
Contributor

@epage epage Nov 10, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Note: if we change SBOMs, lets do that in its own PR and make a note against rust-lang/rfcs#3553

CC @arlosi

UserIntent::Check { .. } => false,
UserIntent::Build
| UserIntent::Test
| UserIntent::Doc { .. }
| UserIntent::Doctest
| UserIntent::Bench => true,
};
let host_layout = Layout::new(self.bcx.ws, None, &dest, must_take_artifact_dir_lock)?;
let mut targets = HashMap::new();
for kind in self.bcx.all_kinds.iter() {
if let CompileKind::Target(target) = *kind {
let layout = Layout::new(self.bcx.ws, Some(target), &dest)?;
let layout = Layout::new(
self.bcx.ws,
Some(target),
&dest,
must_take_artifact_dir_lock,
)?;
targets.insert(target, layout);
}
}
Expand Down
5 changes: 4 additions & 1 deletion src/cargo/core/compiler/layout.rs
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -127,6 +127,7 @@ impl Layout {
ws: &Workspace<'_>,
target: Option<CompileTarget>,
dest: &str,
must_take_artifact_dir_lock: bool,
) -> CargoResult<Layout> {
let is_new_layout = ws.gctx().cli_unstable().build_dir_new_layout;
let mut root = ws.target_dir();
Expand All @@ -153,7 +154,9 @@ impl Layout {
// For now we don't do any more finer-grained locking on the artifact
// directory, so just lock the entire thing for the duration of this
// compile.
let artifact_dir_lock = if is_on_nfs_mount(root.as_path_unlocked()) {
let artifact_dir_lock = if !must_take_artifact_dir_lock
Copy link
Contributor

@epage epage Nov 8, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Instead of conditionally grabbing the lock, can we put the entire artifact layout in an Option? That has the potential to help us find bugs but unsure how messy that would be.

Copy link
Member Author

@ranger-ross ranger-ross Nov 9, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I looked into this and it got pretty hairy. Switching from T to Option<T> is not too bad but there quiet a few places during compilation where the paths are used but not written to.

For example, in build_runner::prepare we build up the root_output

cargo/src/cargo/core/compiler/build_runner/mod.rs

Lines 395 to 397 in 8c4a25c

self.compilation
.root_output
.insert(kind, layout.artifact_dir().dest().to_path_buf());

and we use this in fill_env()

cargo/src/cargo/core/compiler/compilation.rs

Lines 308 to 311 in 8c4a25c

search_path.extend(super::filter_dynamic_search_path(
self.native_dirs.iter(),
&self.root_output[&kind],
));

Copy link
Contributor

@epage epage Nov 10, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Doesn't that make the case even stronger? We are trying to read from a location we do not write to and do not have locked.

|| is_on_nfs_mount(root.as_path_unlocked())
{
None
} else {
Some(dest.open_rw_exclusive_create(".cargo-lock", ws.gctx(), "artifact directory")?)
Expand Down
12 changes: 7 additions & 5 deletions src/cargo/ops/cargo_clean.rs
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -116,15 +116,17 @@ fn clean_specs(
let target_data = RustcTargetData::new(ws, &requested_kinds)?;
let (pkg_set, resolve) = ops::resolve_ws(ws, dry_run)?;
let prof_dir_name = profiles.get_dir_name();
let host_layout = Layout::new(ws, None, &prof_dir_name)?;
let host_layout = Layout::new(ws, None, &prof_dir_name, true)?;
// Convert requested kinds to a Vec of layouts.
let target_layouts: Vec<(CompileKind, Layout)> = requested_kinds
.into_iter()
.filter_map(|kind| match kind {
CompileKind::Target(target) => match Layout::new(ws, Some(target), &prof_dir_name) {
Ok(layout) => Some(Ok((kind, layout))),
Err(e) => Some(Err(e)),
},
CompileKind::Target(target) => {
match Layout::new(ws, Some(target), &prof_dir_name, true) {
Ok(layout) => Some(Ok((kind, layout))),
Err(e) => Some(Err(e)),
}
}
CompileKind::Host => None,
})
.collect::<CargoResult<_>>()?;
Expand Down
2 changes: 0 additions & 2 deletions tests/testsuite/build_dir.rs
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -1005,7 +1005,6 @@ fn template_workspace_path_hash_should_handle_symlink() {

p.root().join("target").assert_build_dir_layout(str![[r#"
[ROOT]/foo/target/CACHEDIR.TAG
[ROOT]/foo/target/debug/.cargo-lock

"#]]);

Expand Down Expand Up @@ -1044,7 +1043,6 @@ fn template_workspace_path_hash_should_handle_symlink() {

p.root().join("target").assert_build_dir_layout(str![[r#"
[ROOT]/foo/target/CACHEDIR.TAG
[ROOT]/foo/target/debug/.cargo-lock

"#]]);

Expand Down
2 changes: 0 additions & 2 deletions tests/testsuite/build_dir_legacy.rs
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -943,7 +943,6 @@ fn template_workspace_path_hash_should_handle_symlink() {

p.root().join("target").assert_build_dir_layout(str![[r#"
[ROOT]/foo/target/CACHEDIR.TAG
[ROOT]/foo/target/debug/.cargo-lock

"#]]);

Expand Down Expand Up @@ -978,7 +977,6 @@ fn template_workspace_path_hash_should_handle_symlink() {

p.root().join("target").assert_build_dir_layout(str![[r#"
[ROOT]/foo/target/CACHEDIR.TAG
[ROOT]/foo/target/debug/.cargo-lock

"#]]);

Expand Down
34 changes: 34 additions & 0 deletions tests/testsuite/check.rs
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -1684,3 +1684,37 @@ fn pkgid_querystring_works() {
"#]])
.run();
}

#[cargo_test]
fn check_build_should_not_output_files_to_artifact_dir() {
let p = project()
.file("src/main.rs", r#"fn main() { println!("Hello, World!") }"#)
.file(
".cargo/config.toml",
r#"
[build]
target-dir = "target-dir"
build-dir = "build-dir"
"#,
)
.build();

p.cargo("check").enable_mac_dsym().run();

p.root()
.join("target-dir")
.assert_build_dir_layout(str![[r#"
[ROOT]/foo/target-dir/CACHEDIR.TAG

"#]]);
}

#[cargo_test]
fn check_build_should_not_lock_artifact_dir() {
let p = project()
.file("src/main.rs", r#"fn main() { println!("Hello, World!") }"#)
.build();

p.cargo("check").enable_mac_dsym().run();
assert!(!p.root().join("target/debug/.cargo-lock").exists());
}