CI: Cache GMT remote files for testings #3056
Merged
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Recently, the nightly CI jobs fail on Linux and Windows, but succeed on macOS.
The CI jobs fail because the Linux and Windows agents cann't connect to
any external websites, thus GMT can't download any remote files from the GMT server.
It's not clear if it's a temporary issue of Azure Pipelines.
Currently, the only workaround is using the cache mechanism of Azure Pipelines.
The macOS agents download remote files to ${HOME}/.gmt and save them to
the CI cache, then Linux and Windows agents can restore the cache when
needed.
The only problem is that the Azure Pipelines caches are immutable.
When the remote files are changed, we have to manually update the cache key
to regenerate the caches.
PaulWessel
approved these changes
Apr 10, 2020
5 tasks
5 tasks
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Recently, the nightly CI jobs fail on Linux and Windows, but succeed on macOS.
The CI jobs fail because the Linux and Windows agents cann't connect to
any external websites, thus GMT can't download any remote files from the GMT server.
It's unclear if it's a temporary issue of Azure Pipelines.
Currently, the only workaround is using the cache mechanism of Azure Pipelines.
The macOS agents download remote files to ${HOME}/.gmt and save them to
the CI cache, then Linux and Windows agents can restore the cache when
needed.
The only problem is that the Azure Pipelines caches are immutable.
When the remote files are changed, we have to manually update the cache key (i.e. update the key 20200409 to a new date) to regenerate the caches.