shared target dir not fully working

[rphmeier]

cc #1657

I have a directory structure like this

project/
| Cargo.toml
| .cargo/
| |  config
|
| src/
| | main.rs
|
| subcrate1/
| | Cargo.toml
| | build.rs
| | src/
| | | lib.rs
|
| target/

.cargo/config:

[build]
target-dir = "./target" # it wouldn't build at all with just "target"

So we have a lot of dependencies that are shared between these crates, and it would greatly increase our travis testing times to not recompile all the dependencies for each subcrate. However, when I cd into a subcrate and run cargo build, it builds the dependencies into both the "shared" target directory and the local target directory. If I then go into another subcrate and type cargo build, it rebuilds all of the dependencies even though they are already compiled in the shared target dir.

[rphmeier]

Instead of trying to have a root-level .cargo/config, I have created a .cargo/config in each subcrate's directory which looks like this:

[build]
target-dir = "../target"

This solves the problem of the duplicate target directories, but it is still recompiling all dependencies for each crate. This caveat is mentioned in the PR which implemented the configuration, but if it doesn't skip recompiling I am not quite sure exactly what problem this feature solves.

Additionally it seems excessive to have to add a separate config file to each subcrate in order to prevent duplication.

[alexcrichton]

There was a bug with relative directory configurations fixed in #2703, so I wonder if a version of Cargo with that fix would help out? What version are you currently using?

[rphmeier]

I am on 0.10 nightly -- I will try the latest nightly.

[PAStheLoD]

Alas the problem still seem to persists on 0.12-nightly (7d79da0 2016-05-31).

We have 3 projects in subdirs, and all of them has a .cargo/config with the build.target-dir set to "../target".

It's actually even worse than having separate target dirs, because at least then incremental compilation would work.

[alexcrichton]

@PAStheLoD do you have a project I could poke around and see what's going on? It may be the case that features are being activated differently which are causing crates to get recompiled, but I'm not entirely sure.

[PAStheLoD]

It's not an open source project (so far) and I don't have a reduced test case for this.

However, is there any verbosity setting I could try (and could you tell me what to look for) to see what's going on?

[alexcrichton]

Ah yeah you can set RUST_LOG=cargo::ops::cargo_rustc::fingerprint and then gist the output. If you could gist the output of a run which should generate the relevant artifacts and then of a second run which should use those but doesn't end up doing so, that should be enough to diagnose.

[PAStheLoD]

After some cargo update and build build .. build, the following fingerprint errors informed me of the differing features:

DEBUG:cargo::ops::cargo_rustc::fingerprint: fingerprint at: /home/pas/projects/xxx/session-manager/../target/debug/.fingerprint/hyper-673d29e05f93a972/lib-hyper
INFO:cargo::ops::cargo_rustc::fingerprint: fingerprint error for hyper v0.9.6: features have changed: Some(["cookie", "default", "openssl", "openssl-verify", "ssl"]) != None
...
DEBUG:cargo::ops::cargo_rustc::fingerprint: fingerprint at: /home/pas/projects/xxx/session-manager/../target/debug/.fingerprint/cookie-be55a9dfee951f20/lib-cookie
INFO:cargo::ops::cargo_rustc::fingerprint: fingerprint error for cookie v0.2.4: features have changed: Some(["default", "openssl", "rustc-serialize", "secure"]) != None

versus

...
DEBUG:cargo::ops::cargo_rustc::fingerprint: fingerprint at: /home/pas/projects/xxx/payment/../target/debug/.fingerprint/hyper-673d29e05f93a972/lib-hyper
INFO:cargo::ops::cargo_rustc::fingerprint: fingerprint error for hyper v0.9.6: features have changed: None != Some(["cookie", "default", "openssl", "openssl-verify", "ssl"])
...
DEBUG:cargo::ops::cargo_rustc::fingerprint: fingerprint at: /home/pas/projects/xxx/payment/../target/debug/.fingerprint/cookie-be55a9dfee951f20/lib-cookie
INFO:cargo::ops::cargo_rustc::fingerprint: fingerprint error for cookie v0.2.4: features have changed: None != Some(["default", "openssl", "rustc-serialize", "secure"])
...

Both Cargo.toml have the same versions specified of the same dependencies. And the difference seems to be included due to the airbrake package.

pas@stranger:~/projects/xxx$ cd session-manager/
pas@stranger:~/projects/xxx/session-manager$ cargo build
pas@stranger:~/projects/xxx/session-manager$ cd ../payment/
pas@stranger:~/projects/xxx/payment$ cargo build
   Compiling cookie v0.2.4
   Compiling hyper v0.9.6
   Compiling xxx-payment v0.1.0 (file:///home/pas/projects/xxx/payment)
pas@stranger:~/projects/xxx/payment$ 
pas@stranger:~/projects/xxx/payment$ cd ../session-manager/
pas@stranger:~/projects/xxx/session-manager$ cargo build
   Compiling cookie v0.2.4
   Compiling hyper v0.9.6
   Compiling airbrake v0.1.0
   Compiling xxx-manager v0.1.0 (file:///home/pas/projects/xxx/session-manager)
pas@stranger:~/projects/xxx/session-manager$ 

So features don't seem to be included in the hash, hence the collision(s) and recompilations, right?

[alexcrichton]

This actually looks like everything is working as expected. The features of the hyper crate are changing, causing it to get recompiled. Currently features don't get mixed into the hash for a new file name, but rather only for determining if the artifact needs to be recompiled.

[PAStheLoD]

Ah, I suspected that it's not a bug, but a feature. Though this makes saving space and compilation time a bit harder, but maybe eventually cargo will be able to handle this too.

Thanks for the clarification anyway!

[alexcrichton]

Ok, sounds like everything is roughly working as expected then so I'm going to close, but thanks again for the report and diagnosis!