Making the bootstrap script work offline#1880
Making the bootstrap script work offline#1880ocaisa wants to merge 5 commits intoeasybuilders:developfrom
Conversation
Allow people to place the patched `distribute` tarball in `$EASYBUILD_BOOTSTRAP_SOURCEPATH`
| if EASYBUILD_BOOTSTRAP_SOURCEPATH is not None: | ||
| # Assume we're offline and the file should be available | ||
| debug("Looking for distribute tarball in EASYBUILD_BOOTSTRAP_SOURCEPATH") | ||
| sys.argv.append('--download-base=file://' + os.path.abspath(EASYBUILD_BOOTSTRAP_SOURCEPATH) + '/') |
There was a problem hiding this comment.
@ocaisa currently, not all tarballs must be provided in $EASYBUILD_BOOTSTRAP_SOURCEPATH, but here you're requiring that they are
How about making this conditional on having the expected tarball there?
There was a problem hiding this comment.
I thought about that but then I said if you were aiming for an offline install then there's no difference, if the file is not there it will fail to get it anyway
There was a problem hiding this comment.
the$EASYBUILD_BOOTSTRAP_SOURCEPATH option is not intended exclusively for offline installation, but also for testing the bootstrap mechanism under certain conditions (e.g. with a develop tarball for framework, or with an updated easyblock for EasyBuild itself, etc.)...
|
@ocaisa this is only a partial fix, right? Since you need to make sure |
|
Actually it's a fix, in that it's already stated that |
|
@ocaisa How does this PR add support to the bootstrap script for picking up a vsc-install tarball? It doesn't yet, right? |
|
No, that's an install requirement of EasyBuild as I understand things, otherwise you would have implemented it :) |
|
Well, no, the Enhancing the bootstrap script to also support picking up a vsc-install tarball makes sense. |
|
So is setuptools, if your underlying python doesn't have setuptools then the final installation of eb also won't work, it will complain about not being able to load |
|
@ocaisa OK, then that should be taken care of too |
|
distribute is not included in the installation directory of the bootstrapped EB, but even if it was that's a problem when you want to update EB |
|
You'd be forced into re-bootstrapping for every EB install |
|
@ocaisa Is it worth following up on this, or shall we just close it? |
|
(tldr) Up to you, I don't mind. There is no meat to this PR really, just a discussion. Allowing for an offline installation can be required (that was my experience on an OpenPower system which had no internet access), this change was what made it possible for me. It would have to be updated based on the current status of the bootstrap script and it's requirements, I don't have the need for that any more so I'm reluctant to commit to trying to make it happen. |
|
@ocaisa Do you happen to remember how this is different from the offline support that is already there in the bootstrap script? Cfr. https://easybuild.readthedocs.io/en/latest/Installation.html#offline-bootstrapping-using-supplied-source-tarballs. There was a fix fairly recently to ensure it fully works offline: #2517 |
|
So I think basically comes back to whether we need Stage0 to be offline or not, currently it (still) tries to download the patched distribute (see https://github.com/easybuilders/easybuild-framework/blob/develop/easybuild/scripts/bootstrap_eb.py#L436) and will fail if we are offline. That makes setuptools a hard requirement for an offline install...and maybe that is ok? If you do want fully offline capabilities this fix is potentially valid but it means that you can only upgrade EB by rebootstrapping (since your system python has no setuptools). |
|
Indeed, having setuptools as a hard requirement for an offline install is probably required looking back at my comment above:
|
|
@boegel You will understand this better than me. Currently, in Stage3 setuptools gets pulled into the installation when the bootscript triggers Stage0. Now if I supply an EasyBuild easyconfig to the bootstrapped eb to do an upgrade ( |
|
With the work that's going to remove the setuptools dependency this is no longer relevant. |
2 participants
Allow people to place the patched
distributetarball in$EASYBUILD_BOOTSTRAP_SOURCEPATH