Skip to content

no longer experimental or removed #298

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
boegel merged 2 commits into from
Mar 17, 2025
Merged

no longer experimental or removed #298

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
2 commits
Select commit Hold shift + click to select a range
6f86f79
no longer experimental
branfosj Mar 6, 2025
9bc9c9e
clean up known issues for installing extensions in parallel: skipping…
boegel Mar 17, 2025
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
4 changes: 0 additions & 4 deletions docs/easystack-files.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -3,10 +3,6 @@
This documentation covers aspects of specifying a software stack to
install with Easybuild with *easystack files*.

**Note: this is an**
[experimental feature][experimental_features]. **Some of the mentioned functionality may be subject to
change or be prone to errors.**

!!! note
Some breaking changes were made to the experimental easystack support in EasyBuild v4.7.0.

Expand Down
12 changes: 2 additions & 10 deletions docs/experimental-features.md
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,23 +1,15 @@
# Experimental features {: #experimental_features }

First introduced in EasyBuild v2.1.0 (see
[EasyBuild v2.1.0 Release Notes][release_notes_v2.1.0]), experimental features can only be used by enabling the
`--experimental` configuration option.
Experimental features can only be used by enabling the `--experimental` configuration option.

An experimental feature indicates to users that these features may
change significantly in a future release and should be used only for
testing, not (yet) for production usage.

Currently enabled experimental features include:

- support for easyconfig files in YAML syntax (see
[Writing easyconfig files in YAML syntax][easyconfig_yeb_format])
- support for generating container recipes & images (see
- support for generating container recipes & images (see
[Generating container recipes & images][containers])
- support for using easystack files (see
[Easystack files][easystack])
- support for trying to update versions of the dependencies of an
easyconfig based on what is available in the robot path (see
[Output of `eb --help`][basic_usage_help])
- support for installing extensions in parallel (see
[Installing extensions in parallel][installing_extensions_in_parallel])
38 changes: 6 additions & 32 deletions docs/installing-extensions-in-parallel.md
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,13 +1,10 @@
# Installing extensions in parallel *(experimental!)* {: #installing_extensions_in_parallel }
# Installing extensions in parallel {: #installing_extensions_in_parallel }

Since EasyBuild v4.5.0 (initial) support is available for installing
extensions in parallel, that is to run the installation command for
multiple extensions that are ready to install at the same time in the
background, to exploit the availability of multiple cores.
Since EasyBuild v4.5.0 support is available for installing R libraries
(extensions in an easyconfig) in parallel, that is to run the installation
command for multiple extensions that are ready to install at the same time
in the background, to exploit the availability of multiple cores.

**Note: this is an**
[experimental feature][experimental_features]. **Some of the mentioned functionality may be subject to
change or be prone to errors.**

## Configuration {: #installing_extensions_in_parallel_configuration }

Expand Down Expand Up @@ -42,7 +39,7 @@ easyblock.

## Caveats and Known issues {: #installing_extensions_in_parallel_caveats_known_issues }

*(last update: EasyBuild v4.5.0)*
*(last update: EasyBuild v5.0.0)*

There are a couple of caveats and known issues with installing
extensions in parallel, which we hope to resolve in future EasyBuild
Expand All @@ -64,29 +61,6 @@ the corresponding easyblocks that are used to install those extensions
required dependencies and starting the installation command
asynchronously.

### List of extensions must be self-contained {: #installing_extensions_in_parallel_caveats_known_issues_exts_list }

The EasyBuild framework currently enforces that all required
dependencies for a particular extension are included in the list of
extensions that is being installed (specified via the `exts_list`
easyconfig parameter). Extensions provided by dependencies are not taken
into account (yet).

As a result, trying to install an
[R-bundle-Bioconductor](https://github.com/easybuilders/easybuild-easyconfigs/tree/main/easybuild/easyconfigs/r/R-bundle-Bioconductor)
easyconfig with `parallel-extensions-install` enabled will result in an
error reporting missing required dependencies.

### Skipping of installed extensions is still done sequentially {: #installing_extensions_in_parallel_caveats_known_issues_skip }

Checking which extensions are already installed for which the
installation can be skipped when `--skip` (see
[Installing additional extensions using `-k`/`-skip`][partial_installation_skip]) is being used
is still done sequentially.

Additional work is needed to also run these checks in parallel across
the cores that are available to EasyBuild.

### Sanity check for extensions is still run sequentially {: #installing_extensions_in_parallel_caveats_known_issues_sanity_check }

The sanity checks that are done for each extension are still done
Expand Down
Loading