Add central execution context to bootstrap #141909

Shourya742 · 2025-06-02T13:06:20Z

This PR continues the effort toward command centralization as outlined in #126819. It introduces a centralized execution context through which all commands will be executed. Previously, centralization was limited to build methods; this PR extends it to the config module and updates the remaining methods accordingly.

Best reviewed commit by commit.

r? @Kobzol

rustbot · 2025-06-02T13:06:27Z

This PR modifies src/bootstrap/src/core/config.

If appropriate, please update CONFIG_CHANGE_HISTORY in src/bootstrap/src/utils/change_tracker.rs.

This PR changes how GCC is built. Consider updating src/bootstrap/download-ci-gcc-stamp.

This PR changes how LLVM is built. Consider updating src/bootstrap/download-ci-llvm-stamp.

Kobzol

This is awesome ❤️ I wanted to do this for a long time, and probably have like 3 unfinished branches with something like this locally 😆 Left some comments.

Kobzol · 2025-06-02T13:15:57Z

src/bootstrap/src/utils/execution_context.rs

+use crate::{BehaviorOnFailure, BootstrapCommand, CommandOutput, OutputMode, RefCell, exit};
+
+#[derive(Clone)]
+pub struct ExecutionContext {


Could you please add some doc comment on top of this that would explain what is it for? :)

Kobzol · 2025-06-02T13:21:56Z

src/bootstrap/src/core/config/config.rs

    /// Cache for determining path modifications
    pub path_modification_cache: Arc<Mutex<HashMap<Vec<&'static str>, PathFreshness>>>,
+
+    pub execution_context: ExecutionContext,


Just noting to myself: the execution context shouldn't be stored inside Config; I would rather put it into the Builder.

For the migration, storing it inside Config is of course the easiest. I wonder how hard it would be to put it inside the builder already in this PR?

src/bootstrap/src/utils/exec.rs

src/bootstrap/src/utils/execution_context.rs

src/bootstrap/src/utils/build_stamp.rs

src/bootstrap/src/core/download.rs

Kobzol · 2025-06-02T13:39:58Z

Btw you probably know this, but looking for usages of as_command_mut is a good source of figuring out what other old-school command invocations we'd ideally like to remove.

Kobzol · 2025-06-04T11:32:28Z

Looks like the command invocation from the error is swallowing errors. Not sure what/where fails.

bors · 2025-06-05T17:42:21Z

☔ The latest upstream changes (presumably #142070) made this pull request unmergeable. Please resolve the merge conflicts.

Kobzol

Left a few additional comments. I think that if Config::parse created the execuion context from the passed flags, rather than receiving it as a parameter, the diff of the PR could be reduced quite a lot, because the tests wouldn't need to change.

src/bootstrap/src/core/config/config.rs

src/bootstrap/src/core/download.rs

src/bootstrap/src/lib.rs

src/bootstrap/src/utils/channel.rs

…e methods via config

…ocation according to suggestions

…from the execution context, add getters and setters in the config, and update the tests and other relevant areas accordingly.

Kobzol · 2025-06-10T05:45:11Z

Thank you! I agree, let's land like this and we can make further refactorings in follow-up PRs.

@bors r+

bors · 2025-06-10T05:45:13Z

📌 Commit 51fbd14 has been approved by Kobzol

It is now in the queue for this repository.

tmiasko · 2025-06-11T16:34:46Z

src/bootstrap/src/core/config/config.rs

-            let current_branch = output_result(
-                helpers::git(Some(&self.src))
-                    .allow_failure()
-                    .run_always()
-                    .args(["symbolic-ref", "--short", "HEAD"])
-                    .as_command_mut(),
-            )
-            .map(|b| b.trim().to_owned());
+            let current_branch = helpers::git(Some(&self.src))
+                .allow_failure()
+                .run_always()
+                .args(["symbolic-ref", "--short", "HEAD"])
+                .run_capture(self);


When running ./x.py -vv build, I see following commands being executed, and then errors similar to those reported in #142350:

env -u GIT_ALTERNATE_OBJECT_DIRECTORIES -u GIT_DIR -u GIT_INDEX_FILE -u GIT_OBJECT_DIRECTORY -u GIT_WORK_TREE "git" "symbolic-ref" "--short" "HEAD" (failure_mode=Ignore) (created at src/bootstrap/src/core/config/config.rs:1393:34, executed at src/bootstrap/src/core/config/config.rs:1397:18) env -u GIT_ALTERNATE_OBJECT_DIRECTORIES -u GIT_DIR -u GIT_INDEX_FILE -u GIT_OBJECT_DIRECTORY -u GIT_WORK_TREE "git" "-c" "branch.master\n.remote=origin" "submodule" "update" "--init" "--recursive" "--depth=1" "--progress" "library/stdarch" (failure_mode=Ignore) error: invalid key (newline): branch.master

It looks like the new implementation no longer trims the trailing newline.

Oh, I missed that, sorry. #142374 should fix this.

rustbot assigned Kobzol Jun 2, 2025

Kobzol reviewed Jun 2, 2025

View reviewed changes

This comment has been minimized.

Sign in to view

Shourya742 force-pushed the 2025-06-01-add-execution-context branch from 8f7b375 to 8aed926 Compare June 3, 2025 18:54

This comment has been minimized.

Sign in to view

Shourya742 force-pushed the 2025-06-01-add-execution-context branch from a4b29b5 to 8b9bd93 Compare June 4, 2025 06:02

This comment has been minimized.

Sign in to view

Shourya742 added 4 commits June 8, 2025 12:19

Add execution context

3ab5e60

add execution context to bootstrap workflow

81ee86b

move git command to new execution context

def4488

moved render_tests in utils to new execution context

8bd8169

Shourya742 force-pushed the 2025-06-01-add-execution-context branch from 576f422 to 587c691 Compare June 8, 2025 08:20

Shourya742 requested a review from Kobzol June 9, 2025 12:05

Kobzol reviewed Jun 9, 2025

View reviewed changes

Shourya742 added 3 commits June 9, 2025 20:38

moved sanity command to exec context

81abbe1

moved curl to use new execution_context

2d1ca83

covert uname to new extext method

d544c2b

Shourya742 added 7 commits June 9, 2025 20:38

move all download.rs method to new execution context command invocation

bae39b8

moved git command to new exec context

746276c

move all config command invocation to new execution context invocation

98be2a0

move all commands to new execution context

50725f3

update dry_run value in exec_ctx and start forwarding exec_ctx verbos…

f3e1eb1

…e methods via config

remove execution context from flag module and correct the command inv…

e9ced50

…ocation according to suggestions

Initialize the execution context in parse_inner, start using dry run …

51fbd14

…from the execution context, add getters and setters in the config, and update the tests and other relevant areas accordingly.

Shourya742 force-pushed the 2025-06-01-add-execution-context branch from 587c691 to 51fbd14 Compare June 9, 2025 15:45

bors added S-waiting-on-bors Status: Waiting on bors to run and complete tests. Bors will change the label on completion. and removed S-waiting-on-review Status: Awaiting review from the assignee but also interested parties. labels Jun 10, 2025

This was referenced Jun 10, 2025

Rollup of 14 pull requests #142290

Closed

Rollup of 16 pull requests #142299

Merged

Kobzol mentioned this pull request Jun 10, 2025

Tracking Issue for consolidating command invocations in bootstrap #126819

Open

18 tasks

bors merged commit b9a578e into rust-lang:master Jun 10, 2025
10 checks passed

rustbot added this to the 1.89.0 milestone Jun 10, 2025

Shourya742 mentioned this pull request Jun 11, 2025

Add tracing import to execution context #142346

Merged

tmiasko reviewed Jun 11, 2025

View reviewed changes

Kobzol mentioned this pull request Jun 11, 2025

Weird errors when running ./x.py test? #142350

Closed

Uh oh!

Add central execution context to bootstrap #141909

Add central execution context to bootstrap #141909

Uh oh!

Conversation

Shourya742 commented Jun 2, 2025

Uh oh!

rustbot commented Jun 2, 2025

Uh oh!

Kobzol left a comment

Choose a reason for hiding this comment

Uh oh!

Kobzol Jun 2, 2025

Choose a reason for hiding this comment

Uh oh!

Kobzol Jun 2, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Kobzol commented Jun 2, 2025

Uh oh!

This comment has been minimized.

This comment has been minimized.

This comment has been minimized.

This comment has been minimized.

This comment has been minimized.

This comment has been minimized.

This comment has been minimized.

This comment has been minimized.

Kobzol commented Jun 4, 2025

Uh oh!

bors commented Jun 5, 2025

Uh oh!

Kobzol left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Kobzol commented Jun 10, 2025

Uh oh!

bors commented Jun 10, 2025

Uh oh!

Uh oh!

tmiasko Jun 11, 2025

Choose a reason for hiding this comment

Uh oh!

Kobzol Jun 11, 2025

Choose a reason for hiding this comment

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

6 participants

Kobzol Jun 2, 2025 •

edited

Loading