distcheck uses submodules of the repo, not the tarball

[lambdageek]

Summary

Discovered while working on #145121 - #145121 (comment)

Distcheck invokes make check which runs .../build/tmp/distcheck/src/bootstrap/bootstrap.py test --stage 2

when the inner bootstrap runs, it calls compute_src_directory where flags_src is None

rust/src/bootstrap/src/core/config/config.rs

Lines 473 to 475 in ffb9d94

	if let Some(src) = compute_src_directory(flags_src, &config.exec_ctx) {
	config.src = src;
	}

And compute_src_directory ends up executing git rev-parse --show-cdup with cwd in .../build/tmp/distcheck so it ends up setting src to the outer git checkout, not the current extracted tarball. From there, all the git_info for the submodules ends up finding them in the outer git checkout.

As a result, the LLVM build, for example, uses the source code from the surrounding git repository, not the source code from the extracted tarball (which has a subset of the LLVM projects):

Here is an example from CI, in this build the raw log has this:

2025-08-08T21:21:12.0222503Z running: cd "/checkout/obj/build/tmp/distcheck/build/x86_64-unknown-linux-gnu/llvm/build" && CMAKE_PREFIX_PATH="" DESTDIR="" LC_ALL="C" "cmake" "/checkout/src/llvm-project/llvm" "-G" "Ninja" "-DLLVM_ENABLE_ASSERTIONS=ON" "-DLLVM_UNREACHABLE_OPTIMIZE=OFF" "-DLLVM_ENABLE_PLUGINS=OFF" "-DLLVM_TARGETS_TO_BUILD=AArch64;AMDGPU;ARM;BPF;Hexagon;LoongArch;MSP430;Mips;NVPTX;PowerPC;RISCV;Sparc;SystemZ;WebAssembly;X86" "-DLLVM_EXPERIMENTAL_TARGETS_TO_BUILD=AVR;M68k;CSKY;Xtensa" "-DLLVM_INCLUDE_EXAMPLES=OFF" "-DLLVM_INCLUDE_DOCS=OFF" "-DLLVM_INCLUDE_BENCHMARKS=OFF" "-DLLVM_INCLUDE_TESTS=OFF" "-DLLVM_ENABLE_LIBEDIT=OFF" "-DLLVM_ENABLE_BINDINGS=OFF" "-DLLVM_ENABLE_Z3_SOLVER=OFF" "-DLLVM_PARALLEL_COMPILE_JOBS=4" "-DLLVM_TARGET_ARCH=x86_64" "-DLLVM_DEFAULT_TARGET_TRIPLE=x86_64-unknown-linux-gnu" "-DLLVM_ENABLE_WARNINGS=OFF" "-DLLVM_INSTALL_UTILS=ON" "-DLLVM_ENABLE_ZLIB=ON" "-DLLVM_ENABLE_LIBXML2=OFF" "-DLLVM_VERSION_SUFFIX=-rust-1.91.0-nightly" "-DCMAKE_INSTALL_MESSAGE=LAZY" "-DCMAKE_C_COMPILER_LAUNCHER=sccache" "-DCMAKE_CXX_COMPILER_LAUNCHER=sccache" "-DCMAKE_C_COMPILER=cc" "-DCMAKE_CXX_COMPILER=c++" "-DCMAKE_ASM_COMPILER=cc" "-DCMAKE_C_FLAGS=-ffunction-sections -fdata-sections -fPIC -m64" "-DCMAKE_CXX_FLAGS=-ffunction-sections -fdata-sections -fPIC -m64" "-DCMAKE_SHARED_LINKER_FLAGS= -Wl,-Bsymbolic -static-libstdc++" "-DCMAKE_MODULE_LINKER_FLAGS= -Wl,-Bsymbolic -static-libstdc++" "-DCMAKE_EXE_LINKER_FLAGS= -Wl,-Bsymbolic -static-libstdc++" "-DLLVM_ENABLE_ZSTD=OFF" "-DCMAKE_INSTALL_PREFIX=/checkout/obj/build/tmp/distcheck/build/x86_64-unknown-linux-gnu/llvm" "-DCMAKE_ASM_FLAGS= -ffunction-sections -fdata-sections -fPIC -m64" "-DCMAKE_BUILD_TYPE=Release"

we're in the build/tmp/distcheck build but the sources we're compiling are /checkout/src/llvm-project/llvm not build/tmp/distcheck/src/llvm-project

Command used

./x test distcheck

Expected behaviour

Distcheck checks the submodule sources from the tarball

Actual behaviour

Distcheck checks the submodule sources of the enclosing git repo.

Bootstrap configuration (bootstrap.toml)

profile = 'dist'

[llvm]
download-ci-llvm = false
targets = "AArch64;X86"

[gcc]

[build]

# Arguments passed to the `./configure` script, used during distcheck. You
# probably won't fill this in but rather it's filled in by the `./configure`
# script. Useful for debugging.
configure-args = []

[install]

[rust]
download-rustc = false

[dist]
compression-formats = ["gz"]

[target.x86_64-unknown-linux-gnu]

Operating system

All

HEAD

2886b36

Additional context

Fixing this issue will uncover a problem with the Tidy check gcc_submodule - the check ensures that the gcc submodule is up to date with respect to the codegen_gcc backend. However the tarball explicitly excludes src/gcc and as a result the tidy check will fail in the tarball.

One way to fix the check is to look for the src/gcc/notice.txt file that is generated in the tarball and skip the rest of the tidy check.

rust/src/bootstrap/src/core/build_steps/dist.rs

Lines 1119 to 1124 in ca77504

	if !builder.config.dry_run() {
	builder.create_dir(&plain_dst_src.join("src/gcc"));
	t!(std::fs::write(
	plain_dst_src.join("src/gcc/notice.txt"),
	"The GCC source code is not included due to unclear licensing implications\n"
	));

[lambdageek]

A suggestion for how to approach fixing this is in #145121 (comment)

modify it so that it extracts itself to some temporary directory, and then invoke the build from there, to minimize the chances that it will pick up something unrelated from the source checkout. And also we should pass --set llvm.download-ci-llvm = false to it explicitly, and not read configure_args from the external environment.