Improve Velox build to use less memory

[majetideepak]

Recently, we had to reduce the build parallelism since CI was running out of memory. #8592
This is an indication that Velox build is consuming more memory over time.
This is expected as we add more source code, but I believe there are many opportunities to improve
existing code and reduce the memory footprint.

Some factors I can think of

1. Remove unnecessary headers.
2. Remove duplicate linking. I believe this is also a blocker to build a dynamic Velox library.
3. Limit the symbols being exported.

I want to brainstorm other approaches, and tools to identify such issues and improve the build.

CC: @assignUser, @kgpai, @czentgr, @pedroerp, @mbasmanova

[kgpai]

@majetideepak : Thanks for bringing this up. This is something we are aware about and are working towards.

Short term: In the very short term, I will send a PR shortly that reduces memory footprint for ArthmeticRegistrations.cpp. Compiling this has been quite memory intensive and one relatively easy fix is to split into smaller files. @assignUser did some analysis using clang-build-analyzer , which I am going to continue.

Long term: A more long term thing we are planning to do with is try and use conbench to store build metrics. Currently we plan to setup a nightly job that builds from scratch and gets metrics on memory/space/cpu etc and upload them to conbench. Again we plan to use clang-build-analyzer for this. This will give us metrics on which TU's are most at fault and that can help us drive down memory footprint etc.

I will keep you folks in the loop should you be interested.

[kgpai]

@majetideepak We can still use all the help we can with this effort. If you have bandwidth it would be great if you can pitch in.
To start off with it might help to install and run clang-build-analyser (https://github.com/aras-p/ClangBuildAnalyzer) and see potential hotspots. Feel free to refactor ArthmeticRegistrations.cpp.

[majetideepak]

#9693 refactors ArithmeticRegistrations.cpp.

[pedroerp]

@kgpai @assignUser how is the cmake modernization effort going? I suppose building shared libraries will significantly reduce linking time and memory consumption, no?

[kgpai]

@pedroerp To an extent yes, The problem we are seeing here happens during the compilation phase and not linking . The TLU here ArithmeticFunctionRegistration pulls all the arithmetic functions which are all templatized and compiles them leading to high memory usage. The prudent thing to do is to split it up into 2 smaller files .

Coming back to your original question on CMake modernization, the effort is still going on though in the short term we are focusing on migrating off our current CI. We are hoping to make much more rapid progress there after moving to Github actions.

[majetideepak]

Based on the discussion from the Velox monthly yesterday, I see the following next steps.

1. Add memory consumed to the build stats tracker.
2. Split high memory-consuming translation units.
3. Decouple Velox components and remove duplicate symbols. This is needed to build shared libraries as well.
4. Finalize public API for Velox modules and export only those symbols.