Consume libraries that we vendor fully by building separately and using a custom prefix path

[jkoritzinsky]

Today, we consume 2 libraries that we "fully vendor" (ie we build the libraries using their build infrastructure and don't just extract some files out to build): zlib-ng and brotli.

These projects we currently include using CMake's FetchContent module, which effectively adds them into the existing build tree. This works pretty well as we get a lot of control to poke and prod at the build to make it work in our repo. It also means that we don't need to build any projects ourselves.

However, it also means that we can't use any of CMake's logic around how it discovers dependencies, requiring some special handling for ie system zlib vs our zlib.

Additionally, it means that we can only build each dependency project once per our build. We need to build two copies of the static libraries when we're building CoreCLR: no-IPO for NativeAOT, and with-IPO for the single-file host.

The new approach has us manually building each of our vendored projects one time per required configuration and installing them into a fake "sysroot"-style layout, which we pass to CMake as a way for it to find the packages. This way, we can build multiple copies and we can remove custom logic around "use system x" vs "use vendored x".

[dotnet-policy-service]

Tagging subscribers to this area: @dotnet/area-system-io-compression
See info in area-owners.md if you want to be subscribed.

[am11]

In the current state, the non-msbuild workflow is broken:

$ src/native/libs/build-native.sh
...
CMake Error at System.IO.Compression.Native/CMakeLists.txt:3 (find_library):
  Could not find BROTLIDEC using the following names: brotlidec

(it's a regression compared to main)

[dotnet-policy-service]

Draft Pull Request was automatically closed for 30 days of inactivity. Please let us know if you'd like to reopen it.