Deprecate the Clang Easyblock in favor of the LLVM one for newer (>=18.1.6) versions of Clang #3746

Crivella · 2025-05-28T16:06:09Z

The LLVM Easyblock should replace the Clang one as it supports building Clang + other subproject and has been extensively reworked to ensure the proper testing, interplay between projects, and newer subproject/project structure introduced in LLVM >= 19, in particular related to offloading

edit: need to make sure that Clang 18.1.8 easyconfigs no longer use the Clang easyblock, so requires merge of:

Use LLVM easyblock for Clang 18 easyconfigs easybuild-easyconfigs#23055

easybuild/easyblocks/c/clang.py

Thyre · 2025-06-04T15:29:07Z

Hm, uploading a test report doesn't work at the point where this aborts. Works as expected though:

$ eb --include-easyblocks-from-pr=3746 ./LLVM-20.1.5-GCCcore-13.3.0-amdgpu.eb --rebuild --installpath /tmp --upload-test-report
== Temporary log file in case of crash /tmp/eb-dbzi0nvz/easybuild-l73_0e22.log
== easyblock clang.py included from PR #3746
== found valid index for /home/jreuter/.local/easybuild/easyconfigs, so using it...
== found valid index for /home/jreuter/.local/easybuild/easyconfigs, so using it...

WARNING: Use of 10 unknown easyconfig parameters detected in LLVM-20.1.5-GCCcore-13.3.0-amdgpu.eb: build_bolt, build_clang_extras, build_openmp, build_openmp_offload, build_openmp_tools, build_runtimes, full_llvm, test_suite_ignore_patterns, test_suite_max_failed, test_suite_timeout_single
If these are just local variables please rename them to start with 'local_', or try using --fix-deprecated-easyconfigs to do this automatically.
For more information, see https://docs.easybuild.io/easyconfig-files-local-variables/ .

== processing EasyBuild easyconfig /home/jreuter/Sources/EasyBuild/Custom/LLVM-20.1.5-GCCcore-13.3.0-amdgpu.eb
ERROR: Failed to get application instance for LLVM (easyblock: EB_Clang): The Clang EasyBlock has been deprecated and does not support LLVM versions >= 19. Please use the 'LLVM' EasyBlock instead, which supports building Clang as well as other LLVM projects.

LGTM.

Thyre · 2025-06-04T15:30:33Z

@boegelbot please test @ jsc-zen3
EB_ARGS="Clang-18.1.8-GCCcore-13.3.0.eb --installpath /tmp/$USER/pr3746"

Crivella · 2025-06-04T15:32:14Z

Main thing I am not sure is if we want to merge this before or after we start migrating testing all software that uses Clang as a dependency with LLVM instead

boegelbot · 2025-06-04T15:33:09Z

@Thyre: Request for testing this PR well received on jsczen3l1.int.jsc-zen3.fz-juelich.de

PR test command 'if [[ develop != 'develop' ]]; then EB_BRANCH=develop ./easybuild_develop.sh 2> /dev/null 1>&2; EB_PREFIX=/home/boegelbot/easybuild/develop source init_env_easybuild_develop.sh; fi; EB_PR=3746 EB_ARGS="Clang-18.1.8-GCCcore-13.3.0.eb --installpath /tmp/$USER/pr3746" EB_REPO=easybuild-easyblocks EB_BRANCH=develop /opt/software/slurm/bin/sbatch --job-name test_PR_3746 --ntasks=8 ~/boegelbot/eb_from_pr_upload_jsc-zen3.sh' executed!

exit code: 0
output:

Submitted batch job 6604

Test results coming soon (I hope)...

- notification for comment with ID 2940475387 processed

Message to humans: this is just bookkeeping information for me,
it is of no use to you (unless you think I have a bug, which I don't).

Thyre · 2025-06-04T15:38:37Z

Main thing I am not sure is if we want to merge this before or after we start migrating testing all software that uses Clang as a dependency with LLVM instead

I think we can merge this independently. For 2024a, I would really recommend to either keep Clang, or switch to LLVM 18.1.8 to have the same version. But yeah, certainly needs testing, since some options differ between Clang and LLVM at the moment.

boegelbot · 2025-06-04T20:35:52Z

Test report by @boegelbot

Overview of tested easyconfigs (in order)

SUCCESS Clang-18.1.8-GCCcore-13.3.0.eb

Build succeeded for 1 out of 1 (1 easyconfigs in total)
jsczen3c1.int.jsc-zen3.fz-juelich.de - Linux Rocky Linux 9.5, x86_64, AMD EPYC-Milan Processor (zen3), Python 3.9.21
See https://gist.github.com/boegelbot/e4248df8acf2ad73bd212a84c4b53b2a for a full test report.

Crivella · 2025-06-06T09:14:28Z

This https://github.com/easybuilders/easybuild-easyblocks/compare/3faa418fd4784e8b1ffbc4ce0729dd8efcad9e39..aac5947b5809c99e02276d387951baf3451e924c got lost in a rebase gonna reapply the change sorry

Flamefire · 2025-06-06T11:31:39Z

I would go as far as using 18.1.6 as the cutoff which is the first version that can be used with the LLVM easyblock. I'm not even sure why we have this seemingly arbitrary version there. As far as I can tell it would work fine with LLVM 18 already and in fact actually fixes a bug: The Clang easyblock for LLVM 18 installs all GPU architectures instead of the selected ones. See easybuilders/easybuild-easyconfigs#23055 which can be considered a companion PR for this one
@Crivella Do you remember why you(?) added that restriction?

Crivella · 2025-06-06T11:35:45Z

I put 19 as there was already an EConfig with Clang 18.1.8 and i did not want to alter the already present behavior.
But if we are retesting and fixing that EC than i do not see a problem with putting the cutoff at 18.1.6

Flamefire · 2025-06-11T08:26:05Z

@Crivella Do you remember why you(?) added that restriction?

Sorry that was a bit misleading in this context. I was referring to the arbitrary looking restriction to a specific patch version in the LLVM easyblock. As far as I can tell it would work fine with LLVM 18 already

So I did a test comparing Clang 18.1.0 & 18.1.6 built with the LLVM easyblock against the ones built with the clang easyblock

It works and results only in a couple addition files in the LLVM version:

Only in /tmp/install/software/Clang/18.1.0-GCCcore-13.3.0-CUDA-12.6.0-LLVM/bin: count.
Only in /tmp/install/software/Clang/18.1.0-GCCcore-13.3.0-CUDA-12.6.0-LLVM/bin: FileCheck.
Only in /tmp/install/software/Clang/18.1.0-GCCcore-13.3.0-CUDA-12.6.0-LLVM/bin: lli-child-target.
Only in /tmp/install/software/Clang/18.1.0-GCCcore-13.3.0-CUDA-12.6.0-LLVM/bin: llvm-jitlink-executor.
Only in /tmp/install/software/Clang/18.1.0-GCCcore-13.3.0-CUDA-12.6.0-LLVM/bin: llvm-PerfectShuffle.
Only in /tmp/install/software/Clang/18.1.0-GCCcore-13.3.0-CUDA-12.6.0-LLVM/bin: not.
Only in /tmp/install/software/Clang/18.1.0-GCCcore-13.3.0-CUDA-12.6.0-LLVM/bin: obj2yaml.
Only in /tmp/install/software/Clang/18.1.0-GCCcore-13.3.0-CUDA-12.6.0-LLVM/bin: split-file.
Only in /tmp/install/software/Clang/18.1.0-GCCcore-13.3.0-CUDA-12.6.0-LLVM/bin: UnicodeNameMappingGenerator.
Only in /tmp/install/software/Clang/18.1.0-GCCcore-13.3.0-CUDA-12.6.0-LLVM/bin: yaml2obj.
Only in /tmp/install/software/Clang/18.1.0-GCCcore-13.3.0-CUDA-12.6.0-LLVM/bin: yaml-bench.
Only in /tmp/install/software/Clang/18.1.0-GCCcore-13.3.0-CUDA-12.6.0-LLVM/include: x86_64-pc-linux.
Only in /tmp/install/software/Clang/18.1.0-GCCcore-13.3.0-CUDA-12.6.0-LLVM/include: x86_64-pc-linux-gnu.
Only in /tmp/install/software/Clang/18.1.0-GCCcore-13.3.0-CUDA-12.6.0-LLVM/lib/clang/18/lib: x86_64-pc-linux.
Only in /tmp/install/software/Clang/18.1.0-GCCcore-13.3.0-CUDA-12.6.0-LLVM/lib/clang/18/lib: x86_64-pc-linux-gnu.
Only in /tmp/install/software/Clang/18.1.0-GCCcore-13.3.0-CUDA-12.6.0-LLVM/lib: libLLVM-18rc.so.
Only in /tmp/install/software/Clang/18.1.0-GCCcore-13.3.0-CUDA-12.6.0-LLVM/lib: libLLVM.so.
Only in /tmp/install/software/Clang/18.1.0-GCCcore-13.3.0-CUDA-12.6.0-LLVM/lib: libLLVM.so.18.1rc.
Only in /tmp/install/software/Clang/18.1.0-GCCcore-13.3.0-CUDA-12.6.0-LLVM/lib: libMLIR.so.
Only in /tmp/install/software/Clang/18.1.0-GCCcore-13.3.0-CUDA-12.6.0-LLVM/lib: libMLIR.so.18.1rc.
Only in /tmp/install/software/Clang/18.1.0-GCCcore-13.3.0-CUDA-12.6.0-LLVM/lib: x86_64-pc-linux.
Only in /tmp/install/software/Clang/18.1.0-GCCcore-13.3.0-CUDA-12.6.0-LLVM/lib: x86_64-pc-linux-gnu.
Only in /tmp/install/software/Clang/18.1.0-GCCcore-13.3.0-CUDA-12.6.0-LLVM/lib64/clang/18/lib: x86_64-pc-linux.
Only in /tmp/install/software/Clang/18.1.0-GCCcore-13.3.0-CUDA-12.6.0-LLVM/lib64/clang/18/lib: x86_64-pc-linux-gnu.
Only in /tmp/install/software/Clang/18.1.0-GCCcore-13.3.0-CUDA-12.6.0-LLVM/lib64: libLLVM-18rc.so.
Only in /tmp/install/software/Clang/18.1.0-GCCcore-13.3.0-CUDA-12.6.0-LLVM/lib64: libLLVM.so.
Only in /tmp/install/software/Clang/18.1.0-GCCcore-13.3.0-CUDA-12.6.0-LLVM/lib64: libLLVM.so.18.1rc.
Only in /tmp/install/software/Clang/18.1.0-GCCcore-13.3.0-CUDA-12.6.0-LLVM/lib64: libMLIR.so.
Only in /tmp/install/software/Clang/18.1.0-GCCcore-13.3.0-CUDA-12.6.0-LLVM/lib64: libMLIR.so.18.1rc.
Only in /tmp/install/software/Clang/18.1.0-GCCcore-13.3.0-CUDA-12.6.0-LLVM/lib64: x86_64-pc-linux.
Only in /tmp/install/software/Clang/18.1.0-GCCcore-13.3.0-CUDA-12.6.0-LLVM/lib64: x86_64-pc-linux-gnu.
Only in /tmp/install/software/Clang/18.1.0-GCCcore-13.3.0-CUDA-12.6.0-Clang/share: gdb.

So IMO the best would be to allow building at least all Clang 18.x with LLVM and optionally lower this deprecation warning and update the easyconfigs.
If we consider the additional files an issue then we should not change the 18.1.8 easyconfigs and change the deprecation to >18.1.8 with a comment mentioning the existing easyconfigs. But I'd still allow using the LLVM easyblock with 18.0 as the above shows it does work.

What do you (and @boegel ?) think about that?

Crivella · 2025-06-11T08:30:42Z

Sorry that was a bit misleading in this context. I was referring to the arbitrary looking restriction to a specific patch version in the LLVM easyblock. As far as I can tell it would work fine with LLVM 18 already

Mostly because i think 18.1.6 was the latest version when i started working on this and that was what i was heavily testing at the beginning.

boegel · 2025-07-04T06:35:44Z

Based on test reports in easybuilders/easybuild-easyconfigs#23055, it seems like this will require more work, so this won't make it in for EasyBuild v5.1.1

Crivella · 2025-08-04T11:41:43Z

@boegel I think following the merge of

Use LLVM easyblock for Clang 18 easyconfigs easybuild-easyconfigs#23055

This is ready for review/merge

boegel · 2025-08-12T18:08:35Z

@boegelbot please test @ jsc-zen3
EB_ARGS="Clang-16.0.6-GCCcore-12.3.0.eb Clang-18.1.8-GCCcore-13.3.0.eb --installpath /tmp/$USER/pr3746"
CORE_CNT=16

boegelbot · 2025-08-12T18:13:10Z

@boegel: Request for testing this PR well received on jsczen3l1.int.jsc-zen3.fz-juelich.de

PR test command 'if [[ develop != 'develop' ]]; then EB_BRANCH=develop ./easybuild_develop.sh 2> /dev/null 1>&2; EB_PREFIX=/home/boegelbot/easybuild/develop source init_env_easybuild_develop.sh; fi; EB_PR=3746 EB_ARGS="Clang-16.0.6-GCCcore-12.3.0.eb Clang-18.1.8-GCCcore-13.3.0.eb --installpath /tmp/$USER/pr3746" EB_REPO=easybuild-easyblocks EB_BRANCH=develop /opt/software/slurm/bin/sbatch --job-name test_PR_3746 --ntasks="16" ~/boegelbot/eb_from_pr_upload_jsc-zen3.sh' executed!

exit code: 0
output:

Submitted batch job 7564

Test results coming soon (I hope)...

- notification for comment with ID 3180447972 processed

Message to humans: this is just bookkeeping information for me,
it is of no use to you (unless you think I have a bug, which I don't).

boegelbot · 2025-08-12T23:07:34Z

Test report by @boegelbot

Overview of tested easyconfigs (in order)

SUCCESS Clang-16.0.6-GCCcore-12.3.0.eb
SUCCESS Clang-18.1.8-GCCcore-13.3.0.eb

Build succeeded for 2 out of 2 (2 easyconfigs in total)
jsczen3c1.int.jsc-zen3.fz-juelich.de - Linux Rocky Linux 9.6, x86_64, AMD EPYC-Milan Processor (zen3), Python 3.9.21
See https://gist.github.com/boegelbot/b3428706203a70dfc7fe0ade27776e15 for a full test report.

Crivella added 2 commits May 28, 2025 18:02

Added version check for the Clang Easyblock

75e548a

Improved version check

aac5947

Thyre reviewed Jun 3, 2025

View reviewed changes

easybuild/easyblocks/c/clang.py Show resolved Hide resolved

Crivella added the change label Jun 5, 2025

Crivella added this to the next release (5.1.1?) milestone Jun 5, 2025

Crivella force-pushed the fix-deprecate_clang_eb branch from 3faa418 to aac5947 Compare June 5, 2025 11:56

Reapply space lost in rebase

8965084

Changed cutoff limit to 18.1.6

816d494

boegel changed the title ~~Deprecates the Clang Easyblock in Favor of the LLVM one for newer (>=19) versions of LLVM~~ Deprecates the Clang Easyblock in Favor of the LLVM one for newer (>=18.1.6) versions of LLVM Jun 6, 2025

Crivella mentioned this pull request Jun 16, 2025

Add easyconfig Clang-18.1.8-GCCcore-13.2.0.eb easybuilders/easybuild-easyconfigs#22626

Closed

boegel changed the title ~~Deprecates the Clang Easyblock in Favor of the LLVM one for newer (>=18.1.6) versions of LLVM~~ Deprecate the Clang Easyblock in Favor of the LLVM one for newer (>=18.1.6) versions of LLVM Jun 18, 2025

boegel changed the title ~~Deprecate the Clang Easyblock in Favor of the LLVM one for newer (>=18.1.6) versions of LLVM~~ Deprecate the Clang Easyblock in favor of the LLVM one for newer (>=18.1.6) versions of LLVM Jul 2, 2025

boegel mentioned this pull request Jul 4, 2025

Use LLVM easyblock for Clang 18 easyconfigs easybuilders/easybuild-easyconfigs#23055

Merged

1 task

boegel modified the milestones: 5.1.1, release after 5.1.1 Jul 4, 2025

boegel approved these changes Aug 13, 2025

View reviewed changes

boegel merged commit 3da0965 into easybuilders:develop Aug 13, 2025
17 checks passed

boegel changed the title ~~Deprecate the Clang Easyblock in favor of the LLVM one for newer (>=18.1.6) versions of LLVM~~ Deprecate the Clang Easyblock in favor of the LLVM one for newer (>=18.1.6) versions of Clang Sep 26, 2025

Deprecate the Clang Easyblock in favor of the LLVM one for newer (>=18.1.6) versions of Clang #3746

Deprecate the Clang Easyblock in favor of the LLVM one for newer (>=18.1.6) versions of Clang #3746

Uh oh!

Conversation

Crivella commented May 28, 2025 • edited by boegel Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

Uh oh!

Thyre commented Jun 4, 2025

Uh oh!

Thyre commented Jun 4, 2025

Uh oh!

Crivella commented Jun 4, 2025

Uh oh!

boegelbot commented Jun 4, 2025

Uh oh!

Thyre commented Jun 4, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

boegelbot commented Jun 4, 2025

Overview of tested easyconfigs (in order)

Uh oh!

Crivella commented Jun 6, 2025

Uh oh!

Flamefire commented Jun 6, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

Crivella commented Jun 6, 2025

Uh oh!

Flamefire commented Jun 11, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

Crivella commented Jun 11, 2025

Uh oh!

boegel commented Jul 4, 2025

Uh oh!

Crivella commented Aug 4, 2025

Uh oh!

boegel commented Aug 12, 2025

Uh oh!

boegelbot commented Aug 12, 2025

Uh oh!

boegelbot commented Aug 12, 2025

Overview of tested easyconfigs (in order)

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

5 participants

Crivella commented May 28, 2025 •

edited by boegel

Loading

Thyre commented Jun 4, 2025 •

edited

Loading

Flamefire commented Jun 6, 2025 •

edited

Loading

Flamefire commented Jun 11, 2025 •

edited

Loading