Reduce cargo overhead for incremental recompilation

[bjorn3]

Describe the problem you are trying to solve

When only a single small crate of a large crate graph is recompiled, cargo has a significant overhead trying to parse all Cargo.toml files and resolve all dependencies. On Bevy I measured an overhead of 28% when doing touch examples/game/breakout.rs; cargo build --example breakout after bevyengine/bevy#791.

Describe the solution you'd like

I would like the overhead of cargo to be reduced. This could be done by optimizing the relevant functions or by for example adding a binary cache for a specific cargo invocation that contains the build plan (I think it is called that way) of everything that cargo will do. This binary cache could be invalidated on any Cargo.toml change or change to the cargo arguments. In case of argument change it would be preferable to keep the old cache around though, as rust-analyzer may run cargo check and the user cargo run.

Notes

Profiles

By the way I noticed that the new resolver is a bit slower than the old one:

$ hyperfine --warmup 10 --runs 100 "touch examples/game/breakout.rs && cargo build --example breakout" "touch examples/game/breakout.rs && cargo build -Zfeatures=all -Zpackage-features --example breakout"
Benchmark #1: touch examples/game/breakout.rs && cargo build --example breakout
  Time (mean ± σ):     678.8 ms ±  21.5 ms    [User: 573.7 ms, System: 160.8 ms]
  Range (min … max):   668.1 ms … 812.3 ms    100 runs
 
  Warning: Statistical outliers were detected. Consider re-running this benchmark on a quiet PC without any interferences from other programs. It might help to use the '--warmup' or '--prepare' options.
 
Benchmark #2: touch examples/game/breakout.rs && cargo build -Zfeatures=all -Zpackage-features --example breakout
  Time (mean ± σ):     690.2 ms ±  20.4 ms    [User: 582.6 ms, System: 164.8 ms]
  Range (min … max):   683.3 ms … 891.4 ms    100 runs
 
  Warning: The first benchmarking run for this command was significantly slower than the rest (891.4 ms). This could be caused by (filesystem) caches that were not filled until after the first run. You should consider using the '--warmup' option to fill those caches before the actual benchmark. Alternatively, use the '--prepare' option to clear the caches before each timing run.
 
Summary
  'touch examples/game/breakout.rs && cargo build --example breakout' ran
    1.02 ± 0.04 times faster than 'touch examples/game/breakout.rs && cargo build -Zfeatures=all -Zpackage-features --example breakout'

cc bevyengine/bevy#791

[alexcrichton]

Thanks for the report! It's definitely the intention that Cargo is a speedy build system and doesn't have a lot of overhead. We've done a fair bit of performance work in the past, but unfortunately we don't have a great benchmark suite and don't track performance over time.

That being said, unfortunately "make Cargo faster" probably isn't the most actionable here. It's probably best to dig into profiles and knock out slow functions one-by-one.

[bjorn3]

or by for example adding a binary cache for a specific cargo invocation that contains the build plan (I think it is called that way) of everything that cargo will do. This binary cache could be invalidated on any Cargo.toml change or change to the cargo arguments. In case of argument change it would be preferable to keep the old cache around though, as rust-analyzer may run cargo check and the user cargo run.

What do you think about this idea?

[alexcrichton]

That sounds plausible, but I think it'd be good to dig in and see where the performance is going on a deeper level. It's not entirely obvious to me given the graphs where the time is going and whether that would help a whole lot.

[bjorn3]

You can inspect the flamegraph live at https://www.speedscope.app/#profileURL=https://gist.githubusercontent.com/bjorn3/562a8772b08ce58d0a7746a0f7c12b72/raw/e5646219a679b8fc3082c12578814e4b583ea3dd/bevy_breakout_fresh.collapsed

A non-trivial portion of the time is spent in the toml parsing code.

[alexcrichton]

I'd encourage local testing and strategies to try to measure the impact of possible optimizations, and PRs to make things faster are of course always welcome!

[bjorn3]

bjorn3@c7f58d3 saves about 7% by avoiding unnecessarily capturing a backtrace at the start of each job.

[bjorn3]

bjorn3@602a1bd saves another big chunk by avoiding spawning a new thread for each fresh job.

[alexcrichton]

Nice!

[epage]

Looks like some changes discussed in this thread have been merged at this point.

I'm going to close this and ask for more specific issues to be opened up in the future as a generic "reduce overhead" has no cohesion for discussion and no end goal.