Confusing diagnostics for unstable features when building Rust with stable rustc

[catamorphism]

I'm trying to build Rust on a Tier 3 platform (riscv64-alpine-linux-musl; actually it's riscv64-unknown-musl that's listed in Tier 3, but I don't think that affects the problem). Since there are no snapshots for this platform, I installed rustc from the distro's package manager (apk) and created a config.toml file to force using the pre-installed rustc rather than downloading snapshots:

profile = "compiler"

[build]
rustc  = "/usr/bin/rustc"
cargo  = "/usr/bin/cargo"
rustfmt = "/usr/bin/rustfmt"
host   = ["riscv64-alpine-linux-musl"]
target = ["riscv64-alpine-linux-musl"]

[rust]
musl-root = "/usr"
debug     = true

[target.riscv64-alpine-linux-musl]
musl-libdir = "/usr/lib"

And the output of rustc --version -v:

# rustc --version -v
rustc 1.68.2 (9eb3afe9e 2023-03-27) (Alpine Linux)
binary: rustc
commit-hash: 9eb3afe9ebe9c7d2b84b71002d44f4a0edac95e0
commit-date: 2023-03-27
host: riscv64-alpine-linux-musl
release: 1.68.2
LLVM version: 15.0.7

I ran ./x.py build; when it got as far as building the core library, I got a series of errors that were very hard to make sense of until I realized that it's because core library code depends on unstable features and I was using a stable rustc to compile it:

building stage0 library artifacts (riscv64-alpine-linux-musl) 
  Downloaded adler v1.0.2
  Downloaded miniz_oxide v0.5.3
  Downloaded unicode-width v0.1.10
  Downloaded hashbrown v0.12.3
  Downloaded compiler_builtins v0.1.91
  Downloaded cc v1.0.77
  Downloaded addr2line v0.17.0
  Downloaded rustc-demangle v0.1.21
  Downloaded gimli v0.26.2
  Downloaded 9 crates (1.2 MB) in 1.45s
   Compiling cc v1.0.77
   Compiling compiler_builtins v0.1.91
   Compiling core v0.0.0 (/home/scratch/rust/library/core)
error: expected identifier, found keyword `move`
   --> library/core/src/ops/try_trait.rs:395:15
    |
395 |         const move |a, b| NeverShortCircuit(f(a, b))
    |               ^^^^ expected identifier, found keyword

error: expected one of `:`, `;`, or `=`, found `|`
   --> library/core/src/ops/try_trait.rs:395:20
    |
395 |         const move |a, b| NeverShortCircuit(f(a, b))
    |                    ^ expected one of `:`, `;`, or `=`

There were many more errors, but I think they all relate to unstable features.

It took me ages to figure out that (for example) the first error is due to the const_closures feature being disabled in stable.

I understand that the stable compiler won't allow using unstable features even with -Zunstable-options, but is it possible for the parser to recognize this code even so and reject it with something like "feature const_closures is not enabled" instead of the much more generic "expected identifier" error?

[workingjubilee]

Huh, usually Rust gets around this by setting RUSTC_BOOTSTRAP=1, and usually when we bootstrap it's with a beta compiler, which does need that set by x.py, and that would allow all the features. I wonder if RUSTC_BOOTSTRAP wasn't set?

[catamorphism]

Huh, usually Rust gets around this by setting RUSTC_BOOTSTRAP=1, and usually when we bootstrap it's with a beta compiler, which does need that set by x.py, and that would allow all the features. I wonder if RUSTC_BOOTSTRAP wasn't set?

It's set, but the command that's actually failing (it took some effort to figure out which executable was actually failing) is:

"/usr/bin/rustc" "--crate-name" "core" "--edition=2021" "library/core/src/lib.rs" "--error-format=json" "--json=diagnostic-rendered-ansi,artifacts,future-incompat" "--diagnostic-width=80" "--crate-type" "lib" "--emit=dep-info,metadata,link" "--cfg=bootstrap" "-Zunstable-options" "--check-cfg=values(bootstrap)"

And what's installed for me in /usr/bin/rustc is a stable compiler. I realized after the fact that I can't use it to bootstrap, but with a different error message I might have noticed that sooner.

[jyn514]

Forwards compatibility in diagnostics seems extremely hard. I think a much simpler and more helpful fix is to check the output of rustc --version in bootstrap and give a hard error if it's not either the current version or N-1.

[jyn514]

Note that the problem here is that you were trying to use 1.68 to build 1.70; it's unrelated to stable vs beta, the problem is that it was two versions behind instead of only one.

[jyn514]

Mentoring instructions: in src/bootstrap/config.rs, if build.rustc is set, run that process with --version. Compare that output to the contents of src/version using the semver crate; if the minor versions aren't either the same or 1 apart, give a hard error.

Note that this should also disallow e.g. building 1.68 with 1.69; we use deny-warnings in most cases and have no guarantees that unstable features will be the same between releases. Only building 1.68 with 1.67 or 1.68 should be allowed.

[jyn514]

btw @catamorphism I've noticed you running into a few different bootstrap bugs the past few days — reading through https://rustc-dev-guide.rust-lang.org/building/bootstrapping.html may or may not make it easier to diagnose these issues without having to spend ages interpreting the diagnostics. I'm also happy to chat about "how should this work" sorts of things on Zulip.

[catamorphism]

btw @catamorphism I've noticed you running into a few different bootstrap bugs the past few days — reading through https://rustc-dev-guide.rust-lang.org/building/bootstrapping.html may or may not make it easier to diagnose these issues without having to spend ages interpreting the diagnostics.

Yes, I've read that.

I'm also happy to chat about "how should this work" sorts of things on Zulip.

Thanks!