Skip to content

Commit 39b2908

Browse files
boegelboegel
committed
Merge pull request #166 from boegel/minimal_toolchains
document experimental support for minimal toolchains
2 parents 8d2af9c + a0f8f0e commit 39b2908

5 files changed

Lines changed: 105 additions & 4 deletions

File tree

‎docs/Changelog.rst‎

Lines changed: 4 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -3,6 +3,10 @@
33
Changelog for EasyBuild documentation
44
-------------------------------------
55

6+
* **release 20151108.01** (`November 8th 2015`):
7+
8+
* document (experimental) support for using minimal toolchains (see :ref:`minimal_toolchains`)
9+
610
* **release 20151028.01** (`October 28th 2015`): document extended dry run mechanism (see :ref:`extended_dry_run`)
711
* **release 20151021.01** (`October 21st 2015`):
812

‎docs/Experimental_features.rst‎

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -13,3 +13,4 @@ Currently enabled experimental features include:
1313
* Cray support (see https://github.com/hpcugent/easybuild/wiki/EasyBuild-on-Cray)
1414
* :ref:`packaging_support`
1515
* support for easyconfig files in YAML syntax (see :ref:`easyconfig_yeb_format`)
16+
* support for using minimal toolchains for dependencies (see :ref:`minimal_toolchains`)

‎docs/Manipulating_dependencies.rst‎

Lines changed: 90 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -6,6 +6,10 @@ Manipulating dependencies
66
A couple of different ways are available to manipulate the list of dependencies that are specified for the software
77
packages being installed.
88

9+
.. contents::
10+
:depth: 3
11+
:backlinks: none
12+
913
.. _filter_deps:
1014

1115
Filtering out dependencies using ``--filter-deps``
@@ -56,6 +60,7 @@ For example (note the preceding '``.``' in the last part of the module names for
5660
* [ ] $CFGS/s/Szip/Szip-2.1-intel-2015a.eb (module: Szip/.2.1-intel-2015a)
5761
* [ ] $CFGS/h/HDF5/HDF5-1.8.13-intel-2015a.eb (module: HDF5/1.8.13-intel-2015a)
5862

63+
5964
.. note::
6065
Using Lmod (version >= 5.7.5), hidden modules can be made visible in the output of '``module avail``' using the
6166
``--show-hidden`` option.
@@ -73,3 +78,88 @@ For example (note the preceding '``.``' in the last part of the module names for
7378

7479
Use "module spider" to find all possible modules.
7580
Use "module keyword key1 key2 ..." to search for all possible modules matching any of the "keys".
81+
82+
83+
.. _minimal_toolchains:
84+
85+
Using minimal toolchains for dependencies
86+
-----------------------------------------
87+
88+
.. note:: This is an **experimental** feature, see :ref:`experimental_features`.
89+
90+
By default, EasyBuild will try to resolve dependencies using the same toolchain as the one that is used for the
91+
software being installed, unless a specific toolchain is specified for the dependency itself
92+
(see :ref:`dependency_specs`).
93+
94+
Using the ``--minimal-toolchains`` configuration option, you can instruct EasyBuild to consider subtoolchains
95+
as well for dependencies. This can be useful to refrain from having to frequently hardcode specific toolchains in order
96+
to avoid having the same dependency version installed with multiple toolchains that are compatible with each other.
97+
Although hardcoding toolchain for dependencies will work fine, it severly limits the power of other EasyBuild features,
98+
like ``--try-toolchain`` for example.
99+
100+
When instructed to use minimal toolchains, EasyBuild will check whether an easyconfig file is available (in the robot
101+
search path, see :ref:`robot_search_path`) for that dependency using the different subtoolchains of the toolchain
102+
specified for the 'parent' software. Subtoolchains are considered 'bottom-up', i.e. starting with the most minimal
103+
subtoolchain (typically a compiler-only toolchain), and then climbing up towards the toolchain that is specified for
104+
the software being installed.
105+
106+
Note that if a specific toolchain is specified for a particular dependency, EasyBuild will stick to using it, even
107+
when instructed to use minimal toolchains.
108+
109+
.. _minimal_toolchains_dummy:
110+
111+
Considering ``dummy`` as minimal toolchain
112+
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
113+
114+
The :ref:`dummy_toolchain` is only considered as the most minimal subtoolchain if the
115+
``--add-dummy-to-minimal-toolchains`` configuration option is enabled.
116+
By default, this configuration option is *disabled*.
117+
118+
.. _minimal_toolchains_existing_modules:
119+
120+
Taking existing modules into account
121+
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
122+
123+
You can instruct EasyBuild to take existing modules into account when determining which subtoolchain should be used
124+
for each of the dependencies, using the ``--use-existing-modules`` configuration option.
125+
126+
By default existing modules are ignored, meaning that the EasyBuild dependency resolution mechanism will pick a
127+
minimal toolchain for each dependency solely based on the available easyconfig files (if the ``--minimal-toolchains``
128+
configuration option is enabled, that is).
129+
130+
With ``--use-existing-modules`` enabled, EasyBuild will first check whether modules exist for the dependencies that were
131+
built with any of the subtoolchains. If so, the module using the most minimal toolchain will determine the toolchain
132+
being used. If not, the toolchain to use will be determined based on the available easyconfig files.
133+
134+
.. _minimal_toolchains_example:
135+
136+
Example
137+
~~~~~~~
138+
139+
Consider the following (partial) easyconfig file for Python v2.7.9 with the ``foss/2015b`` toolchain::
140+
141+
name = 'Python'
142+
version = '2.7.9'
143+
144+
toolchain = {'name': 'foss', 'version': '2015b'}
145+
146+
dependencies = [
147+
('zlib', '1.2.8'),
148+
]
149+
150+
When the ``--minimal-toolchains`` configuration option is enabled, EasyBuild will also consider the subtoolchains
151+
``GCC/4.9.3`` and ``gompi/2015b`` of the ``foss/2015b`` toolchain (in that order) as potential minimal toolchains
152+
when determining the toolchain to use for dependencies.
153+
154+
So, for the zlib v1.2.8 dependency included in the example above, the following scenarios are possible:
155+
156+
* without the use of ``--minimal-toolchains``, EasyBuild will only consider the ``foss/2015b`` toolchain for zlib,
157+
even if other zlib easyconfigs using a compatible toolchain are available
158+
* if (only) ``--minimal-toolchains`` is enabled, EasyBuild will search for an easyconfig file for
159+
zlib v1.2.8 using the ``GCC/4.9.3`` toolchain; if no such easyconfig file is found, it will continue searching
160+
using the ``gompi/2015b`` toolchain, and finally the ``foss/2015b`` toolchain
161+
* if ``--add-dummy-to-minimal-toolchains`` is also enabled, EasyBuild will try locating an easyconfig file for
162+
zlib v1.2.8 that uses the ``dummy`` toolchain prior to consider the ``GCC/4.9.3`` toolchain
163+
* additionally, with ``--use-existing-modules`` enabled, EasyBuild will first check whether a ``zlib`` module for
164+
version 1.2.8 built with the (sub)toolchains being considered exists; if not, it will search for an easyconfig file
165+
for zlib as outlined above

‎docs/Writing_easyconfig_files.rst‎

Lines changed: 8 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -169,8 +169,14 @@ For each of the specified (build) dependencies, the corresponding module will be
169169
defined by EasyBuild. For the *runtime* dependencies, ``module load`` statements will be included in the generated
170170
module file.
171171
172-
.. note:: By default, EasyBuild will try to resolve dependencies using the same toolchain as specified for the software being installed.
173-
A different toolchain can be specified on a per-dependency level (cfr. the ``CMake`` build dependency in the example above).
172+
.. note:: By default, EasyBuild will try to resolve dependencies using the same toolchain as specified for the
173+
software being installed.
174+
175+
A different toolchain can be specified on a per-dependency level (cfr. the ``CMake`` build dependency in the
176+
example above).
177+
178+
Alternatively, you can instruct EasyBuild to use the most minimal (sub)toolchain when resolving dependencies,
179+
see :ref:`minimal_toolchains`.
174180
175181
Loading of modules for dependencies with a ``dummy`` toolchain
176182
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^

‎docs/conf.py‎

Lines changed: 2 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -41,9 +41,9 @@
4141
# other places throughout the built documents.
4242
#
4343
# The short X.Y version.
44-
version = '2.3.0' # this is meant to reference the version of EasyBuild
44+
version = '2.4.0dev' # this is meant to reference the version of EasyBuild
4545
# The full version, including alpha/beta/rc tags.
46-
release = '20151028.01' # this is meant to reference the version of the documentation itself
46+
release = '20151108.01' # this is meant to reference the version of the documentation itself
4747

4848
# There are two options for replacing |today|: either, you set today to some
4949
# non-false value, then it is used:

0 commit comments

Comments
 (0)