Support workspaces

[lu-zero]

• support building a specific package within the workspace
• support building all the library packages within the workspace

testcase: av-metrics

[gdesmott]

That's something we'd like to have in gst-plugins-rs as we currently have to build each plugin one by one which is sub optimal.

[lu-zero]

Great, we can discuss a little on what would be the expected behavior a little more.

[gdesmott]

Could we easily build all the Rust first in all workspaces (cargo build) and then generate the C libraries and pkg-config files once it's done? We'd be blocked on the full build to be done but that would be already much more efficient that the current serialization.

[lu-zero]

It won't be needed :) (I'll push a set about it soon)

[lu-zero]

Took a bit more than I wanted, if you could test it would be great

[gdesmott]

Cool! So I tested it with the gst plugins, it builds fine but seems to sequentially build each workspace one by one, instead of doing them all together as cargo build does. That's already what we are doing atm so we keep having the same issues:

• the same crates are being (re)build for each workspace, making the full build much slower (each plugin will rebuild the same gst crates for example).
• the total number of targets keep changing, leading to a weird/frustrating experience for users as they may hope it's almost done building while it's not.

[lu-zero]

Cool! So I tested it with the gst plugins, it builds fine but seems to sequentially build each workspace one by one, instead of doing them all together as cargo build does. That's already what we are doing atm so we keep having the same issues:

* the same crates are being (re)build for each workspace, making the full build much slower (each plugin will rebuild the same gst crates for example).

They are not:

       Fresh proc-macro-error v1.0.4
       Fresh futures-executor v0.3.15
       Fresh num-rational v0.4.0
       Fresh chrono v0.4.19
       Fresh thiserror v1.0.25
       Fresh gst-plugin-version-helper v0.6.0 (/home/lu_zero/Sources/gst-plugins-rs/version-helper)
       Fresh system-deps v3.1.1
       Fresh proc-macro-crate v1.0.0
   Compiling gst-plugin-lewton v0.6.0 (/home/lu_zero/Sources/gst-plugins-rs/audio/lewton)

Fresh means that the crate is a dependency and it is already built.

* the total number of targets keep changing, leading to a weird/frustrating experience for users as they may hope it's almost done building while it's not.

It builds and install a member at time. I need to figure out how to assemble a single workspace with all the information patched in and that might be quite a bit more involving or even not possible given the current structure of cargo.

[gdesmott]

Fresh means that the crate is a dependency and it is already built.

Strange, here is the full cbuild output when building all plugin.
cbuild.txt

Let's see how much time the gstreamer v0.17.0 crate is build:

$ grep Compiling ~/tmp/cbuild.txt | grep "gstreamer v" | wc -l
14
$ grep Fresh ~/tmp/cbuild.txt | grep "gstreamer v" | wc -l
10

Plugins are build with different gst features (actually gst versions). That would account for 5 builds out of the 14.

Also, using cbuild on a fresh target dir is significantly slower than build: 15m47 vs 6m36.
It may be only because of the better parallelization or cbuild is actually rebuilding stuffs.

It could also be related to #195? So may be best to focus on that first and improve the second point first (single workspace), if possible.

[lu-zero]

I cannot reproduce #195 so I'm a bit puzzled, Patching up the workspace the way I want would require to have upstream to let me have members_mut() I'm afraid. I'm experimenting about it right now.

Also, we'd have the constraint of having workspace-wide rustflags override. That might not be a problem though.