Skip to content

Conversation

@Crivella
Copy link
Contributor

@Crivella Crivella commented Nov 12, 2024

@boegel
Copy link
Member

boegel commented Mar 4, 2025

@boegelbot please test @ jsc-zen3
CORE_CNT=16
EB_BRANCH=5.0.x
EB_ARGS="--installpath /tmp/$USER/pr21833 --include-easyblocks-from-pr 3373"

@easybuilders easybuilders deleted a comment from boegelbot Mar 4, 2025
@boegelbot
Copy link
Collaborator

@boegel: Request for testing this PR well received on jsczen3l1.int.jsc-zen3.fz-juelich.de

PR test command 'if [[ "5.0.x" != 'develop' ]]; then EB_BRANCH="5.0.x" ./easybuild_develop.sh 2> /dev/null 1>&2; EB_PREFIX=/home/boegelbot/easybuild/"5.0.x" source init_env_easybuild_develop.sh; fi; EB_PR=21833 EB_ARGS="--installpath /tmp/$USER/pr21833 --include-easyblocks-from-pr 3373" EB_CONTAINER= EB_REPO=easybuild-easyconfigs EB_BRANCH="5.0.x" /opt/software/slurm/bin/sbatch --job-name test_PR_21833 --ntasks="16" ~/boegelbot/eb_from_pr_upload_jsc-zen3.sh' executed!

  • exit code: 0
  • output:
Submitted batch job 5864

Test results coming soon (I hope)...

Details

- notification for comment with ID 2699045110 processed

Message to humans: this is just bookkeeping information for me,
it is of no use to you (unless you think I have a bug, which I don't).

@boegelbot
Copy link
Collaborator

Test report by @boegelbot
Using easyblocks from PR(s) easybuilders/easybuild-easyblocks#3373
FAILED
Failed during parsing of the easyconfigs, so no ecs were built (2 easyconfigs in total)
jsczen3c3.int.jsc-zen3.fz-juelich.de - Linux Rocky Linux 9.5, x86_64, AMD EPYC-Milan Processor (zen3), Python 3.9.21
See https://gist.github.com/boegelbot/fab6814b2e95abeed36808e1b3f55ccb for a full test report.

@boegel
Copy link
Member

boegel commented Mar 5, 2025

@boegelbot please test @ jsc-zen3
CORE_CNT=16
EB_BRANCH=5.0.x
EB_ARGS="--installpath /tmp/$USER/pr21833 --include-easyblocks-from-pr 3373"

@github-actions github-actions bot added the update label Mar 5, 2025
@github-actions
Copy link

github-actions bot commented Mar 5, 2025

Updated software LLVM-18.1.8.eb

Diff against LLVM-14.0.6-GCCcore-13.3.0-llvmlite.eb

easybuild/easyconfigs/l/LLVM/LLVM-14.0.6-GCCcore-13.3.0-llvmlite.eb

diff --git a/easybuild/easyconfigs/l/LLVM/LLVM-14.0.6-GCCcore-13.3.0-llvmlite.eb b/easybuild/easyconfigs/l/LLVM/LLVM-18.1.8.eb
index 093c727cec..6fb7511cd7 100644
--- a/easybuild/easyconfigs/l/LLVM/LLVM-14.0.6-GCCcore-13.3.0-llvmlite.eb
+++ b/easybuild/easyconfigs/l/LLVM/LLVM-18.1.8.eb
@@ -1,6 +1,7 @@
 name = 'LLVM'
-version = '14.0.6'
-versionsuffix = '-llvmlite'
+version = '18.1.8'
+
+local_gcc_version = '13.3.0'
 
 homepage = "https://llvm.org/"
 description = """The LLVM Core libraries provide a modern source- and target-independent
@@ -9,63 +10,67 @@ description = """The LLVM Core libraries provide a modern source- and target-ind
  specified code representation known as the LLVM intermediate representation
  ("LLVM IR"). The LLVM Core libraries are well documented, and it is
  particularly easy to invent your own language (or port an existing compiler)
- to use LLVM as an optimizer and code generator.
-
- This version include patches for llvmlite / numba."""
+ to use LLVM as an optimizer and code generator."""
 
-toolchain = {'name': 'GCCcore', 'version': '13.3.0'}
-toolchainopts = {'cstd': 'gnu11', 'pic': True}
+toolchain = SYSTEM
 
 source_urls = ['https://github.com/llvm/llvm-project/releases/download/llvmorg-%(version)s/']
-sources = ['llvm-project-%(version)s.src.tar.xz']
+sources = [
+    'llvm-project-%(version)s.src.tar.xz',
+]
 patches = [
-    'LLVM-14.0.6-clear-gotoffsetmap.patch',
-    'LLVM-14.0.6-svml.patch',
-    'LLVM-14.0.x_fix-print-changed-dot-cfg-test.patch',
+    'LLVM-18.1.8_envintest.patch',
+    'LLVM-18.1.8_libomptarget_tests.patch',
+    'LLVM-19.1.7_clang_rpathwrap_test.patch',
 ]
 checksums = [
-    {'llvm-project-14.0.6.src.tar.xz': '8b3cfd7bc695bd6cea0f37f53f0981f34f87496e79e2529874fd03a2f9dd3a8a'},
-    {'LLVM-14.0.6-clear-gotoffsetmap.patch': 'c048afdddcf54c7213018d06f709f61274af5b90b8dcd97a632be4fe53750a51'},
-    {'LLVM-14.0.6-svml.patch': '5776de38e7b663fe9d3ae0a218e380dd59186c565ae277cb7e50785b253640c0'},
-    {'LLVM-14.0.x_fix-print-changed-dot-cfg-test.patch':
-     'd21abda1ecc5d15f1734d529a1332eea2c5f429281fb62969b9ee88acf3516f3'},
+    {'llvm-project-18.1.8.src.tar.xz': '0b58557a6d32ceee97c8d533a59b9212d87e0fc4d2833924eb6c611247db2f2a'},
+    {'LLVM-18.1.8_envintest.patch': '8e25dfab8a29a860717b4bd2d8cdd0e795433766d7fffbda32d06a2bde47058d'},
+    {'LLVM-18.1.8_libomptarget_tests.patch': '858669446358d24936e2c85fa2bdc0b9e77427dc4a6f2aaa9c6d8e28638041c8'},
+    {'LLVM-19.1.7_clang_rpathwrap_test.patch': '5ee6a87ec8ff1c8b736ffe0513aa2098bd2b83a1ffc647a1ad2cf966f567e8a1'},
 ]
 
-# Patches from https://github.com/numba/llvmlite/raw/v0.41.1/conda-recipes/ but
-# renamed to follow EasyBuild conventions.
+local_gcc_compiler = ('GCCcore', local_gcc_version)
+
 builddependencies = [
-    ('binutils', '2.42'),
-    ('CMake', '3.29.3'),
-    ('Python', '3.12.3'),
-    ('lit', '18.1.8'),
-    ('git', '2.45.1'),
+    ('GCCcore', local_gcc_version, '', SYSTEM),
+    # ('binutils', '2.42', '', SYSTEM),
+    ('CMake', '3.29.3', '', local_gcc_compiler),
+    ('psutil', '6.0.0', '', local_gcc_compiler),  # Needed to enable test timeout in lit
+    ('Python', '3.12.3', '', local_gcc_compiler),
+    ('lit', '18.1.8', '', local_gcc_compiler),
+    ('git', '2.45.1', '', local_gcc_compiler),
 ]
 
 dependencies = [
-    ('ncurses', '6.5'),
-    ('zlib', '1.3.1'),
+    ('ncurses', '6.5', '', SYSTEM),
+    ('zlib', '1.3.1', '', SYSTEM),
 ]
 
 build_shared_libs = True
 
-minimal = True
-
-bootstrap = False
-full_llvm = False
-build_clang_extras = False
-build_runtimes = False
-build_lld = False
-build_lldb = False
-build_bolt = False
-build_openmp = False
-build_openmp_offload = False
-build_openmp_tools = False
-usepolly = False
+bootstrap = True
+full_llvm = True
+build_clang_extras = True
+build_runtimes = True
+build_lld = True
+build_lldb = True
+build_bolt = True
+build_openmp = True
+build_openmp_offload = True
+build_openmp_tools = True
+usepolly = True
 
+# Need to reconcile full_llvm and python_bindings as loading python will cause a conflict between system
+# ncurses/zlib and the GCC ones
 python_bindings = False
 
+# build_targets = ['all']
+# disable_werror = True
+
 skip_all_tests = False
 skip_sanitizer_tests = False
-test_suite_max_failed = 0
+test_suite_max_failed = 150  # Could increase depending on build targets
+test_suite_timeout_single = 5 * 60
 
 moduleclass = 'compiler'
Diff against LLVM-14.0.6-GCCcore-13.2.0-llvmlite.eb

easybuild/easyconfigs/l/LLVM/LLVM-14.0.6-GCCcore-13.2.0-llvmlite.eb

diff --git a/easybuild/easyconfigs/l/LLVM/LLVM-14.0.6-GCCcore-13.2.0-llvmlite.eb b/easybuild/easyconfigs/l/LLVM/LLVM-18.1.8.eb
index cce51f9755..6fb7511cd7 100644
--- a/easybuild/easyconfigs/l/LLVM/LLVM-14.0.6-GCCcore-13.2.0-llvmlite.eb
+++ b/easybuild/easyconfigs/l/LLVM/LLVM-18.1.8.eb
@@ -1,6 +1,7 @@
 name = 'LLVM'
-version = '14.0.6'
-versionsuffix = '-llvmlite'
+version = '18.1.8'
+
+local_gcc_version = '13.3.0'
 
 homepage = "https://llvm.org/"
 description = """The LLVM Core libraries provide a modern source- and target-independent
@@ -9,63 +10,67 @@ description = """The LLVM Core libraries provide a modern source- and target-ind
  specified code representation known as the LLVM intermediate representation
  ("LLVM IR"). The LLVM Core libraries are well documented, and it is
  particularly easy to invent your own language (or port an existing compiler)
- to use LLVM as an optimizer and code generator.
-
- This version include patches for llvmlite / numba."""
+ to use LLVM as an optimizer and code generator."""
 
-toolchain = {'name': 'GCCcore', 'version': '13.2.0'}
-toolchainopts = {'cstd': 'gnu++11', 'pic': True}
+toolchain = SYSTEM
 
 source_urls = ['https://github.com/llvm/llvm-project/releases/download/llvmorg-%(version)s/']
-sources = ['llvm-project-%(version)s.src.tar.xz']
+sources = [
+    'llvm-project-%(version)s.src.tar.xz',
+]
 patches = [
-    'LLVM-14.0.6-clear-gotoffsetmap.patch',
-    'LLVM-14.0.6-svml.patch',
-    'LLVM-14.0.x_fix-print-changed-dot-cfg-test.patch',
+    'LLVM-18.1.8_envintest.patch',
+    'LLVM-18.1.8_libomptarget_tests.patch',
+    'LLVM-19.1.7_clang_rpathwrap_test.patch',
 ]
 checksums = [
-    {'llvm-project-14.0.6.src.tar.xz': '8b3cfd7bc695bd6cea0f37f53f0981f34f87496e79e2529874fd03a2f9dd3a8a'},
-    {'LLVM-14.0.6-clear-gotoffsetmap.patch': 'c048afdddcf54c7213018d06f709f61274af5b90b8dcd97a632be4fe53750a51'},
-    {'LLVM-14.0.6-svml.patch': '5776de38e7b663fe9d3ae0a218e380dd59186c565ae277cb7e50785b253640c0'},
-    {'LLVM-14.0.x_fix-print-changed-dot-cfg-test.patch':
-     'd21abda1ecc5d15f1734d529a1332eea2c5f429281fb62969b9ee88acf3516f3'},
+    {'llvm-project-18.1.8.src.tar.xz': '0b58557a6d32ceee97c8d533a59b9212d87e0fc4d2833924eb6c611247db2f2a'},
+    {'LLVM-18.1.8_envintest.patch': '8e25dfab8a29a860717b4bd2d8cdd0e795433766d7fffbda32d06a2bde47058d'},
+    {'LLVM-18.1.8_libomptarget_tests.patch': '858669446358d24936e2c85fa2bdc0b9e77427dc4a6f2aaa9c6d8e28638041c8'},
+    {'LLVM-19.1.7_clang_rpathwrap_test.patch': '5ee6a87ec8ff1c8b736ffe0513aa2098bd2b83a1ffc647a1ad2cf966f567e8a1'},
 ]
 
-# Patches from https://github.com/numba/llvmlite/raw/v0.41.1/conda-recipes/ but
-# renamed to follow EasyBuild conventions.
+local_gcc_compiler = ('GCCcore', local_gcc_version)
+
 builddependencies = [
-    ('binutils', '2.40'),
-    ('CMake', '3.27.6'),
-    ('Python', '3.11.5'),
-    ('lit', '18.1.7'),
-    ('git', '2.42.0'),
+    ('GCCcore', local_gcc_version, '', SYSTEM),
+    # ('binutils', '2.42', '', SYSTEM),
+    ('CMake', '3.29.3', '', local_gcc_compiler),
+    ('psutil', '6.0.0', '', local_gcc_compiler),  # Needed to enable test timeout in lit
+    ('Python', '3.12.3', '', local_gcc_compiler),
+    ('lit', '18.1.8', '', local_gcc_compiler),
+    ('git', '2.45.1', '', local_gcc_compiler),
 ]
 
 dependencies = [
-    ('ncurses', '6.4'),
-    ('zlib', '1.2.13'),
+    ('ncurses', '6.5', '', SYSTEM),
+    ('zlib', '1.3.1', '', SYSTEM),
 ]
 
 build_shared_libs = True
 
-minimal = True
-
-bootstrap = False
-full_llvm = False
-build_clang_extras = False
-build_runtimes = False
-build_lld = False
-build_lldb = False
-build_bolt = False
-build_openmp = False
-build_openmp_offload = False
-build_openmp_tools = False
-usepolly = False
+bootstrap = True
+full_llvm = True
+build_clang_extras = True
+build_runtimes = True
+build_lld = True
+build_lldb = True
+build_bolt = True
+build_openmp = True
+build_openmp_offload = True
+build_openmp_tools = True
+usepolly = True
 
+# Need to reconcile full_llvm and python_bindings as loading python will cause a conflict between system
+# ncurses/zlib and the GCC ones
 python_bindings = False
 
+# build_targets = ['all']
+# disable_werror = True
+
 skip_all_tests = False
 skip_sanitizer_tests = False
-test_suite_max_failed = 0
+test_suite_max_failed = 150  # Could increase depending on build targets
+test_suite_timeout_single = 5 * 60
 
 moduleclass = 'compiler'
Diff against LLVM-18.1.8-GCCcore-13.3.0-minimal.eb

easybuild/easyconfigs/l/LLVM/LLVM-18.1.8-GCCcore-13.3.0-minimal.eb

diff --git a/easybuild/easyconfigs/l/LLVM/LLVM-18.1.8-GCCcore-13.3.0-minimal.eb b/easybuild/easyconfigs/l/LLVM/LLVM-18.1.8.eb
index c9a25fa7e2..6fb7511cd7 100644
--- a/easybuild/easyconfigs/l/LLVM/LLVM-18.1.8-GCCcore-13.3.0-minimal.eb
+++ b/easybuild/easyconfigs/l/LLVM/LLVM-18.1.8.eb
@@ -1,6 +1,7 @@
 name = 'LLVM'
 version = '18.1.8'
-versionsuffix = '-minimal'
+
+local_gcc_version = '13.3.0'
 
 homepage = "https://llvm.org/"
 description = """The LLVM Core libraries provide a modern source- and target-independent
@@ -11,48 +12,65 @@ description = """The LLVM Core libraries provide a modern source- and target-ind
  particularly easy to invent your own language (or port an existing compiler)
  to use LLVM as an optimizer and code generator."""
 
-toolchain = {'name': 'GCCcore', 'version': '13.3.0'}
-toolchainopts = {'cstd': 'gnu++11', 'pic': True}
+toolchain = SYSTEM
 
 source_urls = ['https://github.com/llvm/llvm-project/releases/download/llvmorg-%(version)s/']
 sources = [
-    'llvm-project-%(version)s.src.tar.xz'
+    'llvm-project-%(version)s.src.tar.xz',
+]
+patches = [
+    'LLVM-18.1.8_envintest.patch',
+    'LLVM-18.1.8_libomptarget_tests.patch',
+    'LLVM-19.1.7_clang_rpathwrap_test.patch',
+]
+checksums = [
+    {'llvm-project-18.1.8.src.tar.xz': '0b58557a6d32ceee97c8d533a59b9212d87e0fc4d2833924eb6c611247db2f2a'},
+    {'LLVM-18.1.8_envintest.patch': '8e25dfab8a29a860717b4bd2d8cdd0e795433766d7fffbda32d06a2bde47058d'},
+    {'LLVM-18.1.8_libomptarget_tests.patch': '858669446358d24936e2c85fa2bdc0b9e77427dc4a6f2aaa9c6d8e28638041c8'},
+    {'LLVM-19.1.7_clang_rpathwrap_test.patch': '5ee6a87ec8ff1c8b736ffe0513aa2098bd2b83a1ffc647a1ad2cf966f567e8a1'},
 ]
-checksums = ['0b58557a6d32ceee97c8d533a59b9212d87e0fc4d2833924eb6c611247db2f2a']
+
+local_gcc_compiler = ('GCCcore', local_gcc_version)
 
 builddependencies = [
-    ('binutils', '2.42'),
-    ('CMake', '3.29.3'),
-    ('Python', '3.12.3'),
-    ('lit', '18.1.8'),
-    ('git', '2.45.1'),
+    ('GCCcore', local_gcc_version, '', SYSTEM),
+    # ('binutils', '2.42', '', SYSTEM),
+    ('CMake', '3.29.3', '', local_gcc_compiler),
+    ('psutil', '6.0.0', '', local_gcc_compiler),  # Needed to enable test timeout in lit
+    ('Python', '3.12.3', '', local_gcc_compiler),
+    ('lit', '18.1.8', '', local_gcc_compiler),
+    ('git', '2.45.1', '', local_gcc_compiler),
 ]
 
 dependencies = [
-    ('ncurses', '6.5'),
-    ('zlib', '1.3.1'),
+    ('ncurses', '6.5', '', SYSTEM),
+    ('zlib', '1.3.1', '', SYSTEM),
 ]
 
 build_shared_libs = True
 
-minimal = True
-
-bootstrap = False
-full_llvm = False
-build_clang_extras = False
-build_runtimes = False
-build_lld = False
-build_lldb = False
-build_bolt = False
-build_openmp = False
-build_openmp_offload = False
-build_openmp_tools = False
-usepolly = False
+bootstrap = True
+full_llvm = True
+build_clang_extras = True
+build_runtimes = True
+build_lld = True
+build_lldb = True
+build_bolt = True
+build_openmp = True
+build_openmp_offload = True
+build_openmp_tools = True
+usepolly = True
 
+# Need to reconcile full_llvm and python_bindings as loading python will cause a conflict between system
+# ncurses/zlib and the GCC ones
 python_bindings = False
 
+# build_targets = ['all']
+# disable_werror = True
+
 skip_all_tests = False
 skip_sanitizer_tests = False
-test_suite_max_failed = 0
+test_suite_max_failed = 150  # Could increase depending on build targets
+test_suite_timeout_single = 5 * 60
 
 moduleclass = 'compiler'

@boegelbot
Copy link
Collaborator

@boegel: Request for testing this PR well received on jsczen3l1.int.jsc-zen3.fz-juelich.de

PR test command 'if [[ "5.0.x" != 'develop' ]]; then EB_BRANCH="5.0.x" ./easybuild_develop.sh 2> /dev/null 1>&2; EB_PREFIX=/home/boegelbot/easybuild/"5.0.x" source init_env_easybuild_develop.sh; fi; EB_PR=21833 EB_ARGS="--installpath /tmp/$USER/pr21833 --include-easyblocks-from-pr 3373" EB_CONTAINER= EB_REPO=easybuild-easyconfigs EB_BRANCH="5.0.x" /opt/software/slurm/bin/sbatch --job-name test_PR_21833 --ntasks="16" ~/boegelbot/eb_from_pr_upload_jsc-zen3.sh' executed!

  • exit code: 0
  • output:
Submitted batch job 5867

Test results coming soon (I hope)...

Details

- notification for comment with ID 2700091219 processed

Message to humans: this is just bookkeeping information for me,
it is of no use to you (unless you think I have a bug, which I don't).

@boegelbot
Copy link
Collaborator

Test report by @boegelbot
Using easyblocks from PR(s) easybuilders/easybuild-easyblocks#3373
FAILED
Build succeeded for 1 out of 2 (2 easyconfigs in total)
jsczen3c2.int.jsc-zen3.fz-juelich.de - Linux Rocky Linux 9.5, x86_64, AMD EPYC-Milan Processor (zen3), Python 3.9.21
See https://gist.github.com/boegelbot/fb8724b1024f8ac4caf9eb4feee19667 for a full test report.

@boegel
Copy link
Member

boegel commented Mar 5, 2025

@boegelbot please test @ jsc-zen3
CORE_CNT=16
EB_BRANCH=5.0.x
EB_ARGS="--installpath /tmp/$USER/pr21833 --include-easyblocks-from-pr 3373"

@boegelbot
Copy link
Collaborator

@boegel: Request for testing this PR well received on jsczen3l1.int.jsc-zen3.fz-juelich.de

PR test command 'if [[ "5.0.x" != 'develop' ]]; then EB_BRANCH="5.0.x" ./easybuild_develop.sh 2> /dev/null 1>&2; EB_PREFIX=/home/boegelbot/easybuild/"5.0.x" source init_env_easybuild_develop.sh; fi; EB_PR=21833 EB_ARGS="--installpath /tmp/$USER/pr21833 --include-easyblocks-from-pr 3373" EB_CONTAINER= EB_REPO=easybuild-easyconfigs EB_BRANCH="5.0.x" /opt/software/slurm/bin/sbatch --job-name test_PR_21833 --ntasks="16" ~/boegelbot/eb_from_pr_upload_jsc-zen3.sh' executed!

  • exit code: 0
  • output:
Submitted batch job 5877

Test results coming soon (I hope)...

Details

- notification for comment with ID 2701606152 processed

Message to humans: this is just bookkeeping information for me,
it is of no use to you (unless you think I have a bug, which I don't).

@boegelbot
Copy link
Collaborator

Test report by @boegelbot
Using easyblocks from PR(s) easybuilders/easybuild-easyblocks#3373
FAILED
Build succeeded for 1 out of 2 (2 easyconfigs in total)
jsczen3c1.int.jsc-zen3.fz-juelich.de - Linux Rocky Linux 9.5, x86_64, AMD EPYC-Milan Processor (zen3), Python 3.9.21
See https://gist.github.com/boegelbot/649070d6033f8202df77fa79b3faffd7 for a full test report.

@boegel boegel removed the status in EasyBuild v5.0 Mar 7, 2025
@boegel boegel added this to the 5.0.0 milestone Mar 7, 2025
@Crivella Crivella force-pushed the feature-LLVM18_system branch from 3af6dd1 to 135d44b Compare March 10, 2025 09:28
@Crivella
Copy link
Contributor Author

Crivella commented Mar 10, 2025

Additional tests will most likely require the fix in

to ensure the test suite does not fail with error previously observed

@Crivella
Copy link
Contributor Author

The problem in the test suite with the clang rpath wrappers should be solved by applying the patch i had made for 19.1.7

@boegel boegel modified the milestones: 5.0.0, release after 5.0.0 Mar 18, 2025
@boegel boegel changed the base branch from 5.0.x to develop April 9, 2025 13:30
@boegel
Copy link
Member

boegel commented Apr 9, 2025

@Crivella I changed to target branch in this PR from 5.0.x to develop, you should synchronize your PR branch with current develop branch (which has received a massive update after the release of EasyBuild v5.0.0), for example using eb --sync-pr-with-develop.

@Crivella Crivella force-pushed the feature-LLVM18_system branch from 0372e3a to e499fff Compare April 9, 2025 13:42
@Crivella
Copy link
Contributor Author

Crivella commented Apr 9, 2025

@boegel done

@boegel boegel modified the milestones: next release (5.1.0), 5.x May 23, 2025
@Crivella
Copy link
Contributor Author

Crivella commented Jun 16, 2025

Closing this in favor of the GCCcore version of the LLVM PRs.
Having a proper SYSTEM verion might require a few changes to the easyblocks in order to not have any actual dependencies like zlib and ncurses tied to it

@Crivella Crivella closed this Jun 16, 2025
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

Projects

No open projects
Status: No status

Development

Successfully merging this pull request may close these issues.

3 participants