Always use the "dummy-toolchain" version of Autotools

[riccardomurri]

There is no point in building Autotools with different toolchains,
since they're m4+shell scripts and do not depend in any way from the
toolchain machinery. Therefore, use a "dummy-toolchain" version of
Autotools wherever they are referenced.

[hpcugentbot]

Automatic reply from Jenkins: Can I test this?

[boegel]

Oh my... :-)

With this change, you're enforcing that people install modules using dummy.

Also, are you sure not binaries are involved here, and that nothing is being compiled?

easybuilders/easybuild-framework#1306 will make this change sort of useless, in the sense that just having a dummy easyconfig for Autotools available is enough to have it being picked up and used.

There's no need to hardwire Autotools to being dummy.

[boegel]

@riccardomurri: please revert 9d15e99, never remove easyconfigs files...

Other easyconfig files that are not included in this repository (and where we have no control over) may still require them to be around.

[boegel]

Jenkins: ok to test

[riccardomurri]

Hi @boegel:

With this change, you're enforcing that people install modules using dummy

...and why is that bad?

are you sure not binaries are involved here, and that nothing is being compiled?

I'm pretty sure, but my point is that even if binaries were involved, it's binaries that do not depend on the tools in the toolchain. I cannot care less what compiler or optimization flags were used for building m4 -- I would rather have m4 compiled for 386 than have 5 different binaries of the same M4 sources, one per toolchain. I think it's just confusing as it clutters the module files space.

easybuilders/easybuild-framework#1306 will make this change sort of useles

Agreed, but that's a rather large PR, which might take quite some time to debug fully and merge...

There's no need to hardwire Autotools to being dummy.

Hmmm.... if I don't do that, I would still end up with a different Autotools modulefile per each toolchain, won't I?

[hpcugentbot]

Easyconfigs unit test suite FAILed.

See https://jenkins1.ugent.be/job/easybuild-easyconfigs-pr-builder/4924/console for more details.

Please fix the reported issues by pushing additional commits to the branch corresponding with this pull request; contact @boegel if you're not sure what to do.

[hpcugentbot]

Easyconfigs unit test suite FAILed.

See https://jenkins1.ugent.be/job/easybuild-easyconfigs-pr-builder/4925/console for more details.

Please fix the reported issues by pushing additional commits to the branch corresponding with this pull request; contact @boegel if you're not sure what to do.

[boegel]

@riccardomurri:

With this change, you're enforcing that people install modules using dummy

...and why is that bad?

Because you're forcing something onto people. Not using the system compiler means that you're pretty sure it will work, since you're not at the mercy of whatever GCC version the system provides.

are you sure not binaries are involved here, and that nothing is being compiled?

I'm pretty sure, but my point is that even if binaries were involved, it's binaries that do not depend on the tools in the toolchain. I cannot care less what compiler or optimization flags were used for building m4 -- I would rather have m4 compiled for 386 than have 5 different binaries of the same M4 sources, one per toolchain. I think it's just confusing as it clutters the module files space.

It's not so much about for which target M4 and co are being built, but having control over which compiler version is used to build it.

easybuilders/easybuild-framework#1306 will make this change sort of useless

Agreed, but that's a rather large PR, which might take quite some time to debug fully and merge...

The plan is to get it included for EasyBuild v2.4.0 (it's the only major change blocking the release now, actually). @ocaisa and myself are setting up a conf call to discuss this tomorrow.

Also, it's not that big (and it'll likely get smaller if it gets cleaned up a bit). Take a look at easybuilders/easybuild-framework#1388, that's big (and we've had worse).
It's all about being willing to spend time on it.

There's no need to hardwire Autotools to being dummy.

Hmmm.... if I don't do that, I would still end up with a different Autotools modulefile per each toolchain, won't I?

Not if easybuilders/easybuild-framework#1388 is merged, you'll be able to specify that the robot should consider easyconfigs that use a dummy toolchain first, before trying to use other easyconfigs that use a non-dummy toolchain. That's the whole point of that PR.

[riccardomurri]

Never mind, if easybuilders/easybuild-framework#1388 is merged soon(ish), I agree this one becomes redundant, and all we need is to make sure there are "dummy-toolchain" versions of std software like m4, bison, flex, etc. I'm ok with closing this one as "won't fix".

However, for the sake of clarifying a point, let me reply once again:

...and why is that bad?

Because you're forcing something onto people.

But also the opposite is true: if you're not specifying the dummy toolchain, then M4, autotools, etc will be built for every toolchain that is installed. Which is needlessly wasteful of build time, disk space, and most important clutters the module file space with several versions of tools which are really toolchain-independent.

Not using the system compiler means that you're pretty sure it will work, since you're not at the mercy of whatever GCC version the system provides.

I would agree if we were speaking of some niche software that only gets compiled at a handful of HPC sites. But this issue is about M4 and autotools (and potentially, bison, flex, etc.): M4 1.4 was released in 1996 (!!!) and has seen only bugfixes since. I will pay you a beer next time we meet if you are able to find a Linux distribution that cannot compile M4 :-). And the rest of the Autotools suite is really only PERL + SH, so it doesn't depend on the compiler either.

In the event, my point is just this: software that does not depend on anything the toolchain provides (compiler, MPI, BLAS/LAPACK, FFT) should be toolchain-independent.

[boegel]

To pick this up again...

The points you're making are valid, but the age of M4 is also a concern. What if the compiler provided by the OS is too recent (think GCC 5/6)?

Anyway, with support for --minimal-toolchains being merged in now, and the idea to (maybe) enable it by default in EB v3.0 (whenever that happens), we'll probably have this discussion again in the future.

The question is what the default setting should be. In my view, it should never be to use the system compiler when we can avoid it.

If --minimal-toolchains is enabled by default, you would certainly get less builds of M4. If you allow it to be built with dummy/system toolchain, you would only get the one.

W.r.t. cluttering the module tree: I'm not sure that's a problem that should be solved on the EB-side; Lmod is going to have support for making modules invisible soon-ish (i.e. hide them from users, but still having them around).

[migueldiascosta]

@riccardomurri, this PR is being closed for the following reason(s): no activity for > 6 months.
Please don't hesitate to reopen this PR or add a comment if you feel this contribution is still relevant.
For more information on our policy w.r.t. closing PRs, see https://easybuild.readthedocs.io/en/latest/Contributing.html#why-a-pull-request-may-be-closed-by-a-maintainer