Applying additional optimizations to clang (LTO, PGO, CSPGO, and BOLT)

[michaeleisel]

Comment:

There are a number of optimizations that could be done to the clang binary, and the libraries it loads, that aren't currently being done. LTO was specifically discussed here, but I think it'd be good to discuss all these different optimization topics together, as they're all along the same lines of increasing compilation time/complexity for the sake of better performance. Together, these optimizations can reduce the time clang takes for long compilations by 30% or more. LLVM 20's own releases include clang binaries that have these sorts of optimizations applied.

In the LTO issue, it was said that it'd need to use the same compiler/version for all downstream dependencies. I'm curious if someone could explain the connection.

[michaeleisel]

@isuruf I'm curious if you have any thoughts on this. In #155 you said that "LTO implies we need to use the same compiler/compiler version for all downstream packages which we are not going to enforce". However, I don't see how using LTO here (or any of the other optimizations mentioned) would create any sort of restrictions on downstream packages. LTO would only affect the performance of clang, not its behavior.

[isuruf]

However, I don't see how using LTO here (or any of the other optimizations mentioned) would create any sort of restrictions on downstream packages.

Using LTO results in LLVM/GIMPLE bitcode in archives (.a files) in the case of clang/gcc. These are not binary compatible across compiler versions.

[michaeleisel]

Got it. Why isn't clang built as a monolithic binary? Then LTO and other optimization strategies would become more feasible

[h-vetinari]

It's not about clang, but about the stuff we build with it. We cannot rebuild thousands of packages for every new clang version, so artefacts produced by clang must not have binary compatibility issues across versions.

(also, building a monolith is both undesirable from a packaging POV, as well as impossible due to current CI constraints)

[michaeleisel]

This wouldn’t affect what clang outputs, these would just be optimizations internal to the clang binary itself. However, I understand if there are other constraints.

[isuruf]

Why isn't clang built as a monolithic binary?

What do you mean by this?

[h-vetinari]

@isuruf, since you downvoted my comment, care to correct what I said wrong?

[michaeleisel]

@isuruf By "monolithic" I mean that the clang binary would generally contain everything (e.g. libLLVM.so's code would be in it and there would be no libLLVM.so dependency for it). In doing this, we'd decouple clang from the static libraries it currently depends on, meaning we'd be able to use LTO etc. without breaking anything. It also simplifies PGO, BOLT, etc. by only having to optimize a single binary target, and not all of its .so file dependencies as well. However, I believe it would still be totally feasible to do LTO, BOLT, PGO, etc. with the current .so file dependency tree as long as it didn't depend on the shared .a conda package, assuming that the inability to inline across .so boundaries doesn't remove too many optimization opportunities (I'm working on doing this for a private fork).

[isuruf]

@michaeleisel the issue is that we use clang as a binary and as a library with libclang*.a. Downstream consumers of libclang*.a would need to match the compiler settings if we enable LTO, PGO settings.

One other option would be to enable LLVM_ENABLE_FATLTO as well which worksaround this problem by embedding the LLVM/GIMPLE bitcode into the COFF format's data section.

@h-vetinari this has nothing to do with packages built by clang. Just the consumers of libclang*.a.

[michaeleisel]

@isuruf yeah I'm wondering if it would be feasible to build a different set of static libraries specifically for clang, or to change it more fundamentally (e.g., not depend on a separate libclang.so at all and build everything as one big optimized binary)

[michaeleisel]

A smaller step though would just be to do post-linking optimizations, i.e. BOLT. This wouldn't require any changes to the current build except post-link. The only issue I can see with that is that so much of the code (and thus I imagine the time spent during compilation) is in libLLVM.so, which is in llvmdev-feedstock and thus maybe tricky to optimize as part of this. Like, how would libLLVM get profiled for BOLT except by running it with clang? But clang needs it to be built first. So there's a chicken and egg issue maybe.