This is related to pytorch/pytorch#39968 which got resolved in pytorch/pytorch#49050 by splitting the library into smaller ones
I am also including the Nixpkgs issue for more details: NixOS/nixpkgs#239237
In Nixpkgs we used -Xfatbin=-compress-all first (same as pytorch), but we are again hitting the 2G limit. Using -mcmodel=large does not seem to help, so I guess the only way is to split the magma library into smaller ones too (same approach as pytorch)
On x86_64-linux:
/nix/store/81mi7m3k3wsiz9rrrg636sx21psj20hc-glibc-2.40-66/lib/crti.o: in function `_init':
(.init+0xb): relocation truncated to fit: R_X86_64_REX_GOTPCRELX against undefined symbol `__gmon_start__'
CMakeFiles/magma.dir/control/get_batched_crossover.cpp.o: in function `magma_use_zgeqrf_batched_fused_update':
get_batched_crossover.cpp:(.text+0x35a): relocation truncated to fit: R_X86_64_REX_GOTPCRELX against symbol `zgeqrf_panel_decision_a100' defined in .bss section in CMakeFiles/magma.dir/control/get_batched_crossover.cpp.o
CMakeFiles/magma.dir/control/get_batched_crossover.cpp.o: in function `magma_use_cgeqrf_batched_fused_update':
get_batched_crossover.cpp:(.text+0x45a): relocation truncated to fit: R_X86_64_REX_GOTPCRELX against symbol `cgeqrf_panel_decision_a100' defined in .bss section in CMakeFiles/magma.dir/control/get_batched_crossover.cpp.o
CMakeFiles/magma.dir/control/get_batched_crossover.cpp.o: in function `magma_use_dgeqrf_batched_fused_update':
get_batched_crossover.cpp:(.text+0x55a): relocation truncated to fit: R_X86_64_REX_GOTPCRELX against symbol `dgeqrf_panel_decision_a100' defined in .bss section in CMakeFiles/magma.dir/control/get_batched_crossover.cpp.o
CMakeFiles/magma.dir/control/get_batched_crossover.cpp.o: in function `magma_use_sgeqrf_batched_fused_update':
get_batched_crossover.cpp:(.text+0x65a): relocation truncated to fit: R_X86_64_REX_GOTPCRELX against symbol `sgeqrf_panel_decision_a100' defined in .bss section in CMakeFiles/magma.dir/control/get_batched_crossover.cpp.o
CMakeFiles/magma.dir/control/get_batched_crossover.cpp.o: in function `__static_initialization_and_destruction_0()':
get_batched_crossover.cpp:(.text.startup+0x4f8): relocation truncated to fit: R_X86_64_REX_GOTPCRELX against symbol `sgeqrf_panel_decision_mi100' defined in .bss section in CMakeFiles/magma.dir/control/get_batched_crossover.cpp.o
get_batched_crossover.cpp:(.text.startup+0x554): relocation truncated to fit: R_X86_64_REX_GOTPCRELX against symbol `std::vector<std::vector<int, std::allocator<int> >, std::allocator<std::vector<int, std::allocator<int> > > >::~vector()' defined in .text._ZNSt6vectorIS_IiSaIiEESaIS1_EED2Ev[_ZNSt6vectorIS_IiSaIiEESaIS1_EED5Ev] section in CMakeFiles/magma.dir/control/get_batched_crossover.cpp.o
get_batched_crossover.cpp:(.text.startup+0x55b): relocation truncated to fit: R_X86_64_PC32 against symbol `__dso_handle' defined in .data.rel.local section in /nix/store/dmpy95dhb2m473l4nyxgmcp4cnnial8w-gfortran-14-20241116/lib/gcc/x86_64-unknown-linux-gnu/14.2.1/crtbeginS.o
get_batched_crossover.cpp:(.text.startup+0x854): relocation truncated to fit: R_X86_64_REX_GOTPCRELX against symbol `dgeqrf_panel_decision_mi100' defined in .bss section in CMakeFiles/magma.dir/control/get_batched_crossover.cpp.o
get_batched_crossover.cpp:(.text.startup+0x8a0): relocation truncated to fit: R_X86_64_REX_GOTPCRELX against symbol `dgeqrf_panel_decision_mi100' defined in .bss section in CMakeFiles/magma.dir/control/get_batched_crossover.cpp.o
get_batched_crossover.cpp:(.text.startup+0x8a7): additional relocation overflows omitted from the output
lib/libmagma.so: PC-relative offset overflow in GOT PLT entry for `_Z38magmablas_zapply_vector_kernel_batchediiP7double2iPS0_ii'
collect2: error: ld returned 1 exit status
ninja: build stopped: subcommand failed.
On aarch64-linux:
CMakeFiles/magma.dir/control/get_batched_crossover.cpp.o:(.eh_frame+0x1c): relocation truncated to fit: R_AARCH64_PREL32 against `.text._ZNSt6vectorIS_IiSaIiEESaIS1_EED2Ev'
CMakeFiles/magma.dir/control/get_batched_crossover.cpp.o:(.eh_frame+0x6c): relocation truncated to fit: R_AARCH64_PREL32 against `.text.startup'
CMakeFiles/magma.dir/control/get_batched_crossover.cpp.o:(.eh_frame+0x9c): relocation truncated to fit: R_AARCH64_PREL32 against `.text'
CMakeFiles/magma.dir/control/get_batched_crossover.cpp.o:(.eh_frame+0xb0): relocation truncated to fit: R_AARCH64_PREL32 against `.text'
CMakeFiles/magma.dir/control/get_batched_crossover.cpp.o:(.eh_frame+0xc4): relocation truncated to fit: R_AARCH64_PREL32 against `.text'
CMakeFiles/magma.dir/control/get_batched_crossover.cpp.o:(.eh_frame+0xd8): relocation truncated to fit: R_AARCH64_PREL32 against `.text'
CMakeFiles/magma.dir/control/get_batched_crossover.cpp.o:(.eh_frame+0xec): relocation truncated to fit: R_AARCH64_PREL32 against `.text'
CMakeFiles/magma.dir/control/get_batched_crossover.cpp.o:(.eh_frame+0x100): relocation truncated to fit: R_AARCH64_PREL32 against `.text'
CMakeFiles/magma.dir/control/get_batched_crossover.cpp.o:(.eh_frame+0x114): relocation truncated to fit: R_AARCH64_PREL32 against `.text'
CMakeFiles/magma.dir/control/get_batched_crossover.cpp.o:(.eh_frame+0x128): relocation truncated to fit: R_AARCH64_PREL32 against `.text'
CMakeFiles/magma.dir/control/get_batched_crossover.cpp.o:(.eh_frame+0x13c): additional relocation overflows omitted from the output
collect2: error: ld returned 1 exit status
ninja: build stopped: subcommand failed.
This is related to pytorch/pytorch#39968 which got resolved in pytorch/pytorch#49050 by splitting the library into smaller ones
I am also including the Nixpkgs issue for more details: NixOS/nixpkgs#239237
In Nixpkgs we used
-Xfatbin=-compress-allfirst (same as pytorch), but we are again hitting the 2G limit. Using-mcmodel=largedoes not seem to help, so I guess the only way is to split the magma library into smaller ones too (same approach as pytorch)On
x86_64-linux:On
aarch64-linux: