enhance Bundle generic easyblock to also run test step for each bundle component

[ocaisa]

No description provided.

[boegel]

Not opposing this, but it's a big change, we'll suddenly start running make test for a whole bunch of components in existing easyconfigs that use Bundle easyblock?

[ocaisa]

Without this there is no way to check installations. There is an extensive test suite for Flux (see easybuilders/easybuild-easyconfigs#22979) and without this it can't be run.

I also don't think it does trigger for existing easyconfigs unless they actually want it. At least for things like ConfigureMake/CMakeMake you need something like

        'runtest': 'check',

for the step to run. Something else indeed probably happens for custom easyblocks.

[ocaisa]

@boegelbot please test @ jsc-zen3
EB_ARGS="Xvfb-21.1.14-GCCcore-13.3.0.eb ollama-0.6.0-GCCcore-13.3.0.eb bsddb3-6.2.9-GCCcore-11.3.0.eb R-INLA-24.01.18-foss-2023a.eb Wayland-1.23.0-GCCcore-13.3.0.eb"

[boegelbot]

@ocaisa: Request for testing this PR well received on jsczen3l1.int.jsc-zen3.fz-juelich.de

PR test command 'if [[ develop != 'develop' ]]; then EB_BRANCH=develop ./easybuild_develop.sh 2> /dev/null 1>&2; EB_PREFIX=/home/boegelbot/easybuild/develop source init_env_easybuild_develop.sh; fi; EB_PR=3748 EB_ARGS="Xvfb-21.1.14-GCCcore-13.3.0.eb ollama-0.6.0-GCCcore-13.3.0.eb bsddb3-6.2.9-GCCcore-11.3.0.eb R-INLA-24.01.18-foss-2023a.eb Wayland-1.23.0-GCCcore-13.3.0.eb" EB_REPO=easybuild-easyblocks EB_BRANCH=develop /opt/software/slurm/bin/sbatch --job-name test_PR_3748 --ntasks=8 ~/boegelbot/eb_from_pr_upload_jsc-zen3.sh' executed!

• exit code: 0
• output:

Submitted batch job 6529

Test results coming soon (I hope)...

- notification for comment with ID 2922706113 processed

Message to humans: this is just bookkeeping information for me,
it is of no use to you (unless you think I have a bug, which I don't).

[branfosj]

% eb --include-easyblocks-from-pr 3748 Xvfb-21.1.14-GCCcore-13.3.0.eb
[snip]
== installing bundle component mkfontscale v1.2.3 (1/10)...
  >> running shell command:
        /rds/projects/b/branfosj-bear-apps/sources/generic/eb_v5.1.1.dev0/ConfigureMake/config.guess
        [started at: 2025-05-30 16:31:43]
        [working dir: /dev/shm/branfosj/build-up-EL8/Xvfb/21.1.14/GCCcore-13.3.0/mkfontscale-1.2.3]
        [output and state saved to /dev/shm/branfosj/tmp-up-EL8/eb-cqrfjt8y/run-shell-cmd-output/configguess-1nvb3t28]
  >> command completed: exit 0, ran in < 1s
  >> running shell command:
        ./configure --prefix=/rds/projects/2017/branfosj-rse/easybuild/tmp/software/Xvfb/21.1.14-GCCcore-13.3.0  --build=x86_64-pc-linux-gnu  --host=x86_64-pc-linux-gnu
        [started at: 2025-05-30 16:31:44]
        [working dir: /dev/shm/branfosj/build-up-EL8/Xvfb/21.1.14/GCCcore-13.3.0/mkfontscale-1.2.3]
        [output and state saved to /dev/shm/branfosj/tmp-up-EL8/eb-cqrfjt8y/run-shell-cmd-output/configure-fi4du_g3]
  >> command completed: exit 0, ran in 00h00m33s
  >> running shell command:
        make  -j 8
        [started at: 2025-05-30 16:32:18]
        [working dir: /dev/shm/branfosj/build-up-EL8/Xvfb/21.1.14/GCCcore-13.3.0/mkfontscale-1.2.3]
        [output and state saved to /dev/shm/branfosj/tmp-up-EL8/eb-cqrfjt8y/run-shell-cmd-output/make-9l5op229]
  >> command completed: exit 0, ran in 00h00m02s
  >> running shell command:
        make install
        [started at: 2025-05-30 16:32:21]
        [working dir: /dev/shm/branfosj/build-up-EL8/Xvfb/21.1.14/GCCcore-13.3.0/mkfontscale-1.2.3]
        [output and state saved to /dev/shm/branfosj/tmp-up-EL8/eb-cqrfjt8y/run-shell-cmd-output/make-noos5et3]
  >> command completed: exit 0, ran in < 1s
== installing bundle component mkfontdir v1.0.7 (2/10)...
[snip]

No sign of the test_step in that at all.

[boegelbot]

Test report by @boegelbot

Overview of tested easyconfigs (in order)

• SUCCESS Xvfb-21.1.14-GCCcore-13.3.0.eb
• SUCCESS ollama-0.6.0-GCCcore-13.3.0.eb
• SUCCESS bsddb3-6.2.9-GCCcore-11.3.0.eb
• SUCCESS R-INLA-24.01.18-foss-2023a.eb
• SUCCESS Wayland-1.23.0-GCCcore-13.3.0.eb

Build succeeded for 5 out of 5 (5 easyconfigs in total)
jsczen3c2.int.jsc-zen3.fz-juelich.de - Linux Rocky Linux 9.5, x86_64, AMD EPYC-Milan Processor (zen3), Python 3.9.21
See https://gist.github.com/boegelbot/07df846762c9e3b904e994133d4cd943 for a full test report.

[ocaisa]

That set was pretty representative of the variety of easyblocks that are used as bundle components.

[boegel]

lgtm

[Flamefire]

This makes some ECs fail now where the test step wasn't correct.

Just as a heads up, we'll need to fix the ECs