Added GHA job for uploading velox deps image to S3 #127
Conversation
Avinash-Raj
requested review from
mattgara,
misiugodfrey,
paul-aiyedun and
simoneves
simoneves
left a comment
simoneves
left a comment
There was a problem hiding this comment.
LGTM but need ARM version and also verify creds to write to bucket.
We might also consider a versioning system, or at least a history system, whereby we rename the existing file as old/filename_DATE before pushing the new one.
|
|
||
| jobs: | ||
| build-and-upload-deps: | ||
| runs-on: linux-amd64-cpu4 |
There was a problem hiding this comment.
Presumably we need an ARM version of this too? Are there any cheap CPU-only ARM GHA runners?
| AWS_ACCESS_KEY_ID: ${{ secrets.AWS_ACCESS_KEY_ID }} | ||
| AWS_SECRET_ACCESS_KEY: ${{ secrets.AWS_SECRET_ACCESS_KEY }} | ||
| S3_BUCKET_NAME: ${{ vars.S3_BUCKET_NAME }} | ||
| S3_BUCKET_REGION: ${{ vars.S3_BUCKET_REGION }} |
There was a problem hiding this comment.
IIUC, these creds are read-only? Do we need a different set of creds for this process?
| # and uploads it to s3://rapidsai-velox-testing/<subdir>/<filename> | ||
| # | ||
|
|
||
| validate_docker_image() { |
There was a problem hiding this comment.
Much of this file seems common with the fetch version. Can we factor out the common parts?
mattgara
left a comment
mattgara
left a comment
There was a problem hiding this comment.
Could you please elaborate on how we choose which version of the deps images to build?
That to me seems like a critical point, I don't think we want to be building every single commit (since many of them will have no deps updates.)
This will (presumably?) be a manually-triggered job, just for convenience so that one of us doesn't have to do it manually. I guess we'll do it whenever the build fails due to a deps mismatch? There's no other clear indicator that I can think of. |
No description provided.