Skip to content

Conversation

@Flamefire
Copy link
Contributor

@Flamefire Flamefire commented Nov 21, 2025

@boegel
Copy link
Member

boegel commented Dec 13, 2025

@Flamefire This will need to be re-worked on top of:

@Flamefire
Copy link
Contributor Author

Shall I wait until that is merged and rebase then?

@boegel
Copy link
Member

boegel commented Dec 13, 2025

Shall I wait until that is merged and rebase then?

Yes, makes sense.

@boegel
Copy link
Member

boegel commented Dec 14, 2025

@Flamefire #4016 is merged...

@boegel
Copy link
Member

boegel commented Dec 15, 2025

@boegelbot please test @ jsc-zen3
CORE_CNT=16
EB_ARGS="--installpath /tmp/$USER/pr3995 bamtofastq-1.4.1-GCCcore-12.3.0.eb bcrypt-4.0.1-GCCcore-12.3.0.eb chopper-0.9.0-GCCcore-12.3.0.eb cramino-0.14.5-GCC-12.3.0.eb cramjam-2.9.0-GCCcore-12.3.0.eb cryptography-41.0.1-GCCcore-12.3.0.eb elfx86exts-0.6.2-GCC-12.3.0.eb Evcxr-REPL-0.14.2-GCCcore-12.2.0-Rust-1.65.0.eb jiter-0.4.1-GCCcore-12.3.0.eb juliaup-1.17.9-GCCcore-12.3.0.eb jupyter-server-2.7.2-GCCcore-12.3.0.eb JupyterLab-4.0.3-GCCcore-12.2.0.eb kyber-0.4.0-GCC-12.3.0.eb Longshot-0.4.5-GCCcore-11.3.0.eb Longshot-1.0.0-GCCcore-12.3.0.eb maturin-1.1.0-GCCcore-12.3.0.eb maturin-1.4.0-GCCcore-12.3.0-Rust-1.75.0.eb maturin-1.7.8-GCCcore-12.3.0-Rust-1.81.0.eb mfqe-0.5.0-GCC-12.3.0.eb modkit-0.3.3-GCCcore-12.3.0.eb orjson-3.9.15-GCCcore-12.3.0.eb phasius-0.2.0-GCC-12.3.0.eb poetry-1.7.1-GCCcore-12.3.0.eb pydantic-2.5.3-GCCcore-12.2.0.eb pydantic-2.5.3-GCCcore-12.3.0.eb pydantic-2.6.4-GCCcore-13.2.0.eb pydantic-2.7.4-GCCcore-13.2.0.eb python-jellyfish-1.1.0-GCCcore-12.3.0.eb SKA2-0.3.7-GCCcore-12.3.0.eb skani-0.2.2-GCCcore-12.3.0.eb smafa-0.8.0-GCC-12.3.0.eb Spyder-5.5.1-GCCcore-12.3.0.eb tiktoken-0.6.0-GCCcore-12.3.0.eb tokenizers-0.13.3-GCCcore-12.2.0.eb tokenizers-0.15.2-GCCcore-12.3.0.eb Python-bundle-PyPI-2024.06-GCCcore-13.3.0.eb "

@boegelbot
Copy link

@boegel: Request for testing this PR well received on jsczen3l1.int.jsc-zen3.fz-juelich.de

PR test command 'if [[ develop != 'develop' ]]; then EB_BRANCH=develop ./easybuild_develop.sh 2> /dev/null 1>&2; EB_PREFIX=/home/boegelbot/easybuild/develop source init_env_easybuild_develop.sh; fi; EB_PR=3995 EB_ARGS="--installpath /tmp/$USER/pr3995 bamtofastq-1.4.1-GCCcore-12.3.0.eb bcrypt-4.0.1-GCCcore-12.3.0.eb chopper-0.9.0-GCCcore-12.3.0.eb cramino-0.14.5-GCC-12.3.0.eb cramjam-2.9.0-GCCcore-12.3.0.eb cryptography-41.0.1-GCCcore-12.3.0.eb elfx86exts-0.6.2-GCC-12.3.0.eb Evcxr-REPL-0.14.2-GCCcore-12.2.0-Rust-1.65.0.eb jiter-0.4.1-GCCcore-12.3.0.eb juliaup-1.17.9-GCCcore-12.3.0.eb jupyter-server-2.7.2-GCCcore-12.3.0.eb JupyterLab-4.0.3-GCCcore-12.2.0.eb kyber-0.4.0-GCC-12.3.0.eb Longshot-0.4.5-GCCcore-11.3.0.eb Longshot-1.0.0-GCCcore-12.3.0.eb maturin-1.1.0-GCCcore-12.3.0.eb maturin-1.4.0-GCCcore-12.3.0-Rust-1.75.0.eb maturin-1.7.8-GCCcore-12.3.0-Rust-1.81.0.eb mfqe-0.5.0-GCC-12.3.0.eb modkit-0.3.3-GCCcore-12.3.0.eb orjson-3.9.15-GCCcore-12.3.0.eb phasius-0.2.0-GCC-12.3.0.eb poetry-1.7.1-GCCcore-12.3.0.eb pydantic-2.5.3-GCCcore-12.2.0.eb pydantic-2.5.3-GCCcore-12.3.0.eb pydantic-2.6.4-GCCcore-13.2.0.eb pydantic-2.7.4-GCCcore-13.2.0.eb python-jellyfish-1.1.0-GCCcore-12.3.0.eb SKA2-0.3.7-GCCcore-12.3.0.eb skani-0.2.2-GCCcore-12.3.0.eb smafa-0.8.0-GCC-12.3.0.eb Spyder-5.5.1-GCCcore-12.3.0.eb tiktoken-0.6.0-GCCcore-12.3.0.eb tokenizers-0.13.3-GCCcore-12.2.0.eb tokenizers-0.15.2-GCCcore-12.3.0.eb Python-bundle-PyPI-2024.06-GCCcore-13.3.0.eb " EB_CONTAINER= EB_REPO=easybuild-easyblocks EB_BRANCH=develop /opt/software/slurm/bin/sbatch --job-name test_PR_3995 --ntasks="16" ~/boegelbot/eb_from_pr_upload_jsc-zen3.sh' executed!

  • exit code: 0
  • output:
Submitted batch job 9187

Test results coming soon (I hope)...

Details

- notification for comment with ID 3657191421 processed

Message to humans: this is just bookkeeping information for me,
it is of no use to you (unless you think I have a bug, which I don't).

@boegel
Copy link
Member

boegel commented Dec 15, 2025

@Flamefire Looks like HERRO-0.1.0_20240808-foss-2023a.eb fails with this updated Cargo easyblock?

Build step fails with:

error: failed to parse lock file at: /tmp/vsc40023/easybuild_build/HERRO/0.1.0_20240808/foss-2023a/herro-deccc46823615577f6fcda4ec9469f592fc3ffcb/Cargo.lock

Caused by:
  lock file version `2` was found, but this version of Cargo does not understand this lock file, perhaps Cargo needs to be updated?

@boegel
Copy link
Member

boegel commented Dec 15, 2025

There's also trouble with other easyconfigs:

For poetry-1.8.3-GCCcore-13.3.0.eb:

ERROR: Installation of poetry-1.8.3-GCCcore-13.3.0.eb failed: Extension 'six' cannot have crates. You can add them to the top-level config parameters when using e.g.the Cargo or CargoPythonBundle easyblock.

For pydantic-2.9.1-GCCcore-13.3.0.eb:

ERROR: Installation of pydantic-2.9.1-GCCcore-13.3.0.eb failed: Extension 'annotated_types' cannot have crates. You can add them to the top-level config parameters when using e.g.the Cargo or CargoPythonBundle easyblock.

@Flamefire It seems like this needs more work, so this won't make it in for EasyBuild v5.2.0...

@boegelbot
Copy link

Test report by @boegelbot

Overview of tested easyconfigs (in order)

Build succeeded for 19 out of 36 (total: 1 hour 5 mins 29 secs) (36 easyconfigs in total)
jsczen3c2.int.jsc-zen3.fz-juelich.de - Linux Rocky Linux 9.6, x86_64, AMD EPYC-Milan Processor (zen3), Python 3.9.21
See https://gist.github.com/boegelbot/593ab131ec81464c230ef82184d98389 for a full test report.

@Flamefire
Copy link
Contributor Author

There's also trouble with other easyconfigs:

For poetry-1.8.3-GCCcore-13.3.0.eb:

ERROR: Installation of poetry-1.8.3-GCCcore-13.3.0.eb failed: Extension 'six' cannot have crates. You can add them to the top-level config parameters when using e.g.the Cargo or CargoPythonBundle easyblock.

For pydantic-2.9.1-GCCcore-13.3.0.eb:

ERROR: Installation of pydantic-2.9.1-GCCcore-13.3.0.eb failed: Extension 'annotated_types' cannot have crates. You can add them to the top-level config parameters when using e.g.the Cargo or CargoPythonBundle easyblock.

Those errors are real, somehow. They only get detected through the recent change in #3993, prior to that the crates specified in those easyconfigs are silently ignored unless there is something happening that I'm missing.
In this case the crates are actually from the top-level EC and "only" copied to extensions which they should not. This so far only caused errors when sources of an extension was a dict and the extension derived from Cargo, like CargoPythonPackage

So that needs to be fixed first.

lock file version 2 was found, but this version of Cargo does not understand this lock file, perhaps Cargo needs to be updated?

That is weird. Then I'll need to recheck which cargo version has which lockfile version

@Flamefire Flamefire force-pushed the 20251121090749_new_pr_cargo branch from 806ad6d to 3f95cfa Compare December 18, 2025 16:52
@Flamefire Flamefire force-pushed the 20251121090749_new_pr_cargo branch from a13f9bf to d491dee Compare December 19, 2025 13:46
@Flamefire
Copy link
Contributor Author

lock file version 2 was found, but this version of Cargo does not understand this lock file, perhaps Cargo needs to be updated?

That is weird. Then I'll need to recheck which cargo version has which lockfile version

Found it: Versions 1 and 2 were internal versions only, not reflected in the lockfile, so version = 1/2 must not be added.

Also fixed an issue where cargo might output warnings that confuse the code expecting a path.

I started a job with 40 random easyconfigs using Rust as a dependency and the ECs you used above. That will take a while but should succeed now.

@Flamefire
Copy link
Contributor Author

Test report by @Flamefire

Overview of tested easyconfigs (in order)

  • SUCCESS bamtofastq-1.4.1-GCCcore-12.3.0.eb

  • SUCCESS bcrypt-4.0.1-GCCcore-12.3.0.eb

  • SUCCESS bcrypt-4.3.0-GCCcore-14.3.0.eb

  • SUCCESS chopper-0.9.0-GCCcore-12.3.0.eb

  • SUCCESS cramino-0.14.5-GCC-12.3.0.eb

  • SUCCESS cryptography-41.0.1-GCCcore-12.3.0.eb

  • SUCCESS cryptography-42.0.8-GCCcore-13.3.0.eb

  • SUCCESS cryptography-44.0.2-GCCcore-14.2.0.eb

  • SUCCESS cryptography-45.0.5-GCCcore-14.3.0.eb

  • SUCCESS DeltaLake-0.25.5-gfbf-2023b.eb

  • SUCCESS elfx86exts-0.6.2-GCC-12.3.0.eb

  • SUCCESS Evcxr-REPL-0.14.2-GCCcore-12.2.0-Rust-1.65.0.eb

  • SUCCESS gsutil-5.10-GCCcore-11.2.0.eb

  • SUCCESS HERRO-0.1.0_20240808-foss-2023a.eb

  • SUCCESS isONclust3-0.3.0-GCCcore-13.3.0.eb

  • SUCCESS juliaup-1.17.9-GCCcore-12.3.0.eb

  • SUCCESS jupyter-collaboration-2.1.1-GCCcore-13.2.0.eb

  • SUCCESS kyber-0.4.0-GCC-12.3.0.eb

  • SUCCESS libavif-1.0.4-GCCcore-12.3.0.eb

  • FAIL librsvg-2.58.0-GCCcore-13.2.0.eb (build issue)
    (partial log available at https://gist.github.com/Flamefire/537dbc7f58a92ec8323b8cce6b23ed40)

  • FAIL librsvg-2.60.0-GCCcore-13.3.0.eb (build issue)
    (partial log available at https://gist.github.com/Flamefire/3160204ffc60638bf435358c41731eda)

  • FAIL librsvg-2.61.0-GCCcore-14.3.0.eb (build issue)
    (partial log available at https://gist.github.com/Flamefire/cf4f1773e7fbe07652a9b221416859b3)

  • SUCCESS Longshot-0.4.5-GCCcore-11.3.0.eb

  • SUCCESS Longshot-1.0.0-GCCcore-12.3.0.eb

  • SUCCESS maturin-1.1.0-GCCcore-12.3.0.eb

  • SUCCESS maturin-1.3.2-GCCcore-11.3.0-Rust-1.65.0.eb

  • SUCCESS maturin-1.4.0-GCCcore-12.3.0-Rust-1.75.0.eb

  • SUCCESS maturin-1.7.8-GCCcore-12.3.0-Rust-1.81.0.eb

  • SUCCESS mfqe-0.5.0-GCC-12.3.0.eb

  • SUCCESS modkit-0.3.3-GCCcore-12.3.0.eb

  • SUCCESS orjson-3.9.15-GCCcore-12.3.0.eb

  • SUCCESS phasius-0.2.0-GCC-12.3.0.eb

  • SUCCESS poetry-1.7.1-GCCcore-12.3.0.eb

  • SUCCESS poetry-1.8.3-GCCcore-13.3.0.eb

  • SUCCESS poetry-2.1.2-GCCcore-14.2.0.eb

  • SUCCESS poetry-2.1.3-GCCcore-14.3.0.eb

  • FAIL polars-0.15.6-foss-2022a.eb (build issue)
    (partial log available at https://gist.github.com/Flamefire/d0c0d08e5f8b638710c400a0974adb14)

  • SUCCESS pydantic-2.5.3-GCCcore-12.3.0.eb

  • SUCCESS pydantic-2.6.4-GCCcore-13.2.0.eb

  • SUCCESS pydantic-2.7.4-GCCcore-13.2.0.eb

  • FAIL Python-3.10.8-GCCcore-12.2.0.eb (build issue)
    (partial log available at https://gist.github.com/Flamefire/8f0e380cc79a62a995739be90839ffe3)

  • SUCCESS Python-bundle-PyPI-2024.06-GCCcore-13.3.0.eb

  • SUCCESS Python-bundle-PyPI-2025.07-GCCcore-14.3.0.eb

  • SUCCESS python-jellyfish-1.1.0-GCCcore-12.3.0.eb

  • SUCCESS rav1e-0.8.1-GCCcore-14.3.0.eb

  • FAIL retworkx-0.9.0-foss-2021a.eb (build issue)
    (partial log available at https://gist.github.com/Flamefire/1fe4d18c707d47606c810a9df9fbf885)

  • SUCCESS retworkx-0.9.0-foss-2023a.eb

  • SUCCESS rustworkx-0.12.1-foss-2022a.eb

  • SUCCESS rustworkx-0.15.1-gfbf-2023a.eb

  • SUCCESS rustworkx-0.16.0-gfbf-2024a.eb

  • SUCCESS SKA2-0.3.7-GCCcore-12.3.0.eb

  • SUCCESS skani-0.2.2-GCCcore-12.3.0.eb

  • SUCCESS smafa-0.8.0-GCC-12.3.0.eb

  • SUCCESS tiktoken-0.6.0-GCCcore-12.3.0.eb

  • SUCCESS tiktoken-0.7.0-GCCcore-12.3.0.eb

  • SUCCESS timm-0.6.13-foss-2022a-CUDA-11.7.0.eb

  • SUCCESS tokenizers-0.13.3-GCCcore-12.2.0.eb

  • SUCCESS tokenizers-0.15.2-GCCcore-12.3.0.eb

  • SUCCESS tokenizers-0.19.1-GCCcore-13.2.0.eb

  • SUCCESS uv-0.4.20-GCCcore-13.3.0.eb

  • SUCCESS vLLM-0.4.0-foss-2023a-CUDA-12.1.1.eb

  • SUCCESS yacrd-1.0.0-foss-2023a.eb

  • SUCCESS cramjam-2.9.0-GCCcore-12.3.0.eb

  • SUCCESS jiter-0.4.1-GCCcore-12.3.0.eb

  • SUCCESS JupyterLab-4.0.3-GCCcore-12.2.0.eb

  • SUCCESS jupyter-server-2.7.2-GCCcore-12.3.0.eb

  • SUCCESS pydantic-2.5.3-GCCcore-12.2.0.eb

  • FAIL Qiskit-2.0.2-foss-2023a.eb (build issue)
    (partial log available at https://gist.github.com/Flamefire/0ff203ea5f6664f3abcf4ea76cb516d1)

  • SUCCESS Spyder-5.5.1-GCCcore-12.3.0.eb

Build succeeded for 62 out of 69 (total: 19 hours 9 mins 11 secs) (75 easyconfigs in total)
i7095 - Linux Rocky Linux 9.6, x86_64, AMD EPYC 7702 64-Core Processor (zen2), Python 3.9.21
See https://gist.github.com/Flamefire/8df3a3453f13b50ed09a326319c8abfb for a full test report.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants