easybuilders · boegel · Nov 23, 2022 · Oct 21, 2022 · Oct 21, 2022 · Oct 21, 2022
diff --git a/docs/Easystack-files.rst b/docs/Easystack-files.rst
@@ -7,6 +7,13 @@ This documentation covers aspects of specifying a software stack to install with
 
 **Note: this is an** :ref:`experimental feature <experimental_features>`. **Some of the mentioned functionality may be subject to change or be prone to errors.**
 
+.. note::
+   Some breaking changes were made to the experimental easystack support in EasyBuild v4.7.0.
+
+   Easystack files must now use the ``easyconfigs`` top-level key to list *easyconfig filenames*,
+   as opposed to the ``software`` top-level key and separate subkeys like ``version``, ``versionsuffix`` and
+   ``toolchain`` to specify aspects of an easyconfig file that were supported before.
+
 .. contents::
     :depth: 3
     :backlinks: none
@@ -32,89 +39,161 @@ To build software with *Easystack*, type:
 
 where ``example.yaml`` is the file with specifications that you just created (more about this in the next section).
 
+
 .. _easystack_structure:
 
 Structure of an easystack file
 ------------------------------
 
 Easystack files are written in `YAML syntax <https://learnxinyminutes.com/docs/yaml>`_.
 
-General options, which should be applied to each software (for example ``robot``), are defined at the top of the file.
+Essentially, an easystack file lists the easyconfig files you want to install, which are specified under the
+`easyconfigs` key.
+
+For example:
+
+.. code:: yaml
 
-Afterwards, particular software specifications follow.
+  easyconfigs:
+    - PyTorch-1.12.0-foss-2022a-CUDA-11.7.0.eb
+    - OpenFOAM-v2206-foss-2022a.eb
 
-It is mandatory to specify basic software-related keywords: *software name*, *toolchains* and *versions*.
+.. note::
+   You must use '``-``' to list the easyconfigs in an easystack file.
 
-- *software name*: Name of the software.
-- *toolchains*: Names and versions of compiler :ref:`toolchains`.
-- *versions*: Versions of software. If multiple entries are provided, EasyBuild will install all of them. 
-Can be in form of a list or consecutive line entries (see example). 
+In addition, you can specify additional configuration options via the ``options`` subkey
+which will only apply to the installation of a particular easyconfig file.
 
-**General structure of YAML-formatted easystack:**
+For example:
 
 .. code:: yaml
 
-  software:
-    <software_name>:
-      toolchains:
-        <toolchain name and version>:
-            <software_version>:
-              versionsuffix: '<-example>'
+  easyconfigs:
+    - PyTorch-1.12.0-foss-2022a-CUDA-11.7.0.eb:
+      options:
+        from-pr: 15924
+        debug: True
+    - Hypre-2.25.0-foss-2022a.eb:
+    - OpenFOAM-v2206-foss-2022a.eb:
+      options:
+        installpath: /my/custom/installpath
+
+.. note::
+   You need to take care with some values in YAML, especially integers, booleans, etc.
+
+   If the specified value definitely must be a string value, you should use quotes (``'...'``) to avoid
+   that the YAML parser automatically converts the value to be of a specific type.
+
+   In other cases, the automatic conversion is useful, like for the ``True`` used above,
+   since ``debug`` is a boolean configuration option.
+
+The configuration options that are valid for the ``eb`` command can be used (see ``eb --help``),
+but the ``-`` or ``--`` prefixes that are commonly used on the command line are omitted in easystack files.
+
+Using the example easystack file above would be equivalent to running:
+
+.. code::
+
+  eb PyTorch-1.12.0-foss-2022a-CUDA-11.7.0.eb --from-pr 15924 --debug
+  eb Hypre-2.25.0-foss-2022a.eb
+  eb OpenFOAM-v2206-foss-2022a.eb --installpath /my/custom/installpath
+
+.. note::
+   Whenever configuration options are *not* specified (as is the case for ``Hypre`` in the example easystack file above),
+   you are still allowed to use '``:``' after the easyconfig filename: there is no difference in behaviour in ending with or without '``:``'.
+
+Specifying short options in an easystack file is allowed, for example:
+
+.. code:: yaml
+
+  easyconfigs:
+    - OpenFOAM-v2206-foss-2022a.eb:
+      options:
+        D: True
+
+This is not recommended however, as short options are more difficult to interpret by humans.
+
+
+.. _easystack_combining_options:
+
+Combining command line options with options in an easystack file
+----------------------------------------------------------------
 
-**Example of YAML-formatted easystack:**
+When building software with an easystack file, you can still add additional options on the command line as well.
+These apply to *all* items in the easystack file. For example, if you have an easystack file named 
+``my_easystack.yaml``
 
 .. code:: yaml
 
-  software:
-    Bioconductor:
-      toolchains:
-        foss-2020a:
-          versions:
-            3.11:
-    EasyBuild:
-      toolchains:
-        SYSTEM:
-          versions: [4.3.1]
-    GROMACS:
-      toolchains:
-        foss-2020a:
-          versions:
-            2020.1:
-            2020.3:
-        fosscuda-2020a:
-          versions: [2020.1]
-    OpenFOAM:
-      toolchains:
-        foss-2020a:
-          versions: [8, v2006]
-    R:
-      toolchains:
-        foss-2020a:
-          versions: [4.0.0]
-
-To install the software specified in this *easystack file* named '``myeasystack.yaml``', run:
+  easyconfigs:
+    - PyTorch-1.12.0-foss-2022a-CUDA-11.7.0.eb:
+      options:
+        from-pr: 15924
+        debug: True
+    - OpenFOAM-v2206-foss-2022a.eb:
+
+and you run with
+
+.. code::
+
+  eb --experimental --easystack my_easystack.yaml --dry-run
+
+this will have the same effect as running
 
 .. code::
 
-  eb --easystack myeasystack.yaml
+  eb PyTorch-1.12.0-foss-2022a-CUDA-11.7.0.eb --dry-run --from-pr 15924 --debug
+  eb OpenFOAM-v2206-foss-2022a.eb --dry-run --installpath /my/custom/installpath
+
+Note that options specified on the command line are placed *before* the easyconfig-specific options in the easystack file.
+EasyBuild will always respect the argument that was put *last*.
+
+For example:
+
+.. code::
+
+  eb PyTorch-1.12.0-foss-2022a-CUDA-11.7.0.eb --dry-run --disable-dry-run
+
+will effectively run *without* enabling dry run mode, since ``--disable-dry-run`` is specified after ``--dry-run``.
 
-This is equivalent to running:
+Since easyconfig-specific configuration options specified in the easystack file are put *last*,
+they take priority over the the ones on the command line, if the same configuration option is specified in both.
+
+For example, running:
 
 .. code::
 
-  eb Bioconductor-3.11-foss-2020a.eb EasyBuild-4.3.1-SYSTEM.eb GROMACS-2020.1-foss-2020a.eb GROMACS-2020.3-foss-2020a.eb GROMACS-2020.1-fosscuda-2020a.eb OpenFOAM-8-foss-2020a.eb OpenFOAM-v2006-foss-2020a.eb R-4.0.0-foss-2020a.eb
+  eb --experimental --easystack my_easystack.yaml --disable-debug
+
+will effectively cause ``PyTorch-1.12.0-foss-2022a-CUDA-11.7.0.eb`` to be installed with debug logging enabled,
+while ``OpenFOAM-v2206-foss-2022a.eb`` will be effectively installed *without* debug logging.
+
 
 To be developed
 ---------------
 
-Optionally, more advanced keywords can be specified: *easybuild_version*, *robot*, *from_pr*, *versionsuffix*, *include-labels*, *exclude-labels*.
-
-- *easybuild_version:* if present, EasyBuild will check if the easystack file was intended for the current version of EasyBuild.
-- *robot:* enables dependency resolution; see :ref:`Using_the_EasyBuild_command_line` for more details.
-- *from_pr:* easyconfig files that are added or modified by a
-  particular pull request to the easybuild-easyconfigs GitHub repository
-  can be used (regardless of whether the pull request is merged or not).
-  (see :ref:`Integration_with_GitHub` for more details).
-- *versionsuffix:* additional suffix for software version (placed after toolchain name)
-- *include-labels:* only include this software when EasyBuild is configured with one of the specified labels
-- *exclude-labels:* **do not** include this software when EasyBuild is configured with one of the specified labels
+In the future, we are planning to support additional also global options specified in the easystack file. For example:
+
+.. code:: yaml
+
+  options:
+    robot: True
+
+  easyconfigs:
+  - PyTorch-1.12.0-foss-2022a-CUDA-11.7.0.eb
+  - OpenFOAM-v2206-foss-2022a.eb
+
+would installed both ``PyTorch-1.12.0-foss-2022a-CUDA-11.7.0.eb`` and ``OpenFOAM-v2206-foss-2022a.eb`` using ``--robot``
+(see `issue #4105 <https://github.com/easybuilders/easybuild-framework/issues/4105>`_).
+
+Additionally, we plan to support specifying for which EasyBuild version an easystack file was intended,
+which can be helpful in more accurately recreating a certain software stack
+(see `issue #4106 <https://github.com/easybuilders/easybuild-framework/issues/4106>`_).
+
+In the future, the ``--easystack`` option will probably be dropped, and EasyBuild will automatically detect
+the use of easystack files (see `issue #4104 <https://github.com/easybuilders/easybuild-framework/issues/4104>`_).
+
+Finally, we plan to support specifying *labels*, which would make it more easy to install only a certain subset of
+the items listed in an easystack file. For example, by labelling all GPU-capable software with a ``gpu`` label,
+one could easily choose to *not* build anything labeled ``gpu`` on a CPU node
+(see `issue #3512 <https://github.com/easybuilders/easybuild-framework/issues/3512>`_).