Skip to content

Enable nightly builds via GitHub Actions #3467

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Closed
ramilbakhshyiev wants to merge 3 commits into from
Closed

Enable nightly builds via GitHub Actions #3467

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
3 commits
Select commit Hold shift + click to select a range
1d83d65
Provide ANACONDA_API_TOKEN secret to GitHub Actions build_conda action
May 21, 2024
4cd518b
Fix CUDA 11.4.4 builds under CircleCI
May 21, 2024
0c5a893
Enable nightly builds via GitHub Actions (#3467)
ramilbakhshyiev May 21, 2024
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
8 changes: 4 additions & 4 deletions .github/actions/build_conda/action.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -44,7 +44,7 @@ runs:
if: inputs.label != ''
shell: ${{ steps.choose_shell.outputs.shell }}
env:
PACKAGE_TYPE: inputs.label
PACKAGE_TYPE: ${{ inputs.label }}
run: |
conda install -y -q anaconda-client
conda config --set anaconda_upload yes
Expand All @@ -59,7 +59,7 @@ runs:
shell: ${{ steps.choose_shell.outputs.shell }}
working-directory: conda
env:
PACKAGE_TYPE: inputs.label
PACKAGE_TYPE: ${{ inputs.label }}
run: |
conda build faiss --user pytorch --label ${{ inputs.label }} -c pytorch
- name: Conda build (GPU)
Expand All @@ -74,7 +74,7 @@ runs:
shell: ${{ steps.choose_shell.outputs.shell }}
working-directory: conda
env:
PACKAGE_TYPE: inputs.label
PACKAGE_TYPE: ${{ inputs.label }}
run: |
conda build faiss-gpu --variants '{ "cudatoolkit": "${{ inputs.cuda }}", "c_compiler_version": "${{ inputs.compiler_version }}", "cxx_compiler_version": "${{ inputs.compiler_version }}" }' \
--user pytorch --label ${{ inputs.label }} -c pytorch -c nvidia/label/cuda-${{ inputs.cuda }} -c nvidia
Expand All @@ -90,7 +90,7 @@ runs:
shell: ${{ steps.choose_shell.outputs.shell }}
working-directory: conda
env:
PACKAGE_TYPE: inputs.label
PACKAGE_TYPE: ${{ inputs.label }}
run: |
conda build faiss-gpu-raft --variants '{ "cudatoolkit": "${{ inputs.cuda }}", "c_compiler_version": "${{ inputs.compiler_version }}", "cxx_compiler_version": "${{ inputs.compiler_version }}" }' \
--user pytorch --label ${{ inputs.label }} -c pytorch -c nvidia/label/cuda-${{ inputs.cuda }} -c nvidia -c rapidsai -c rapidsai-nightly -c conda-forge
1 change: 1 addition & 0 deletions .github/workflows/build.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -121,6 +121,7 @@ jobs:
runs-on: 4-core-ubuntu-gpu-t4
env:
CUDA_ARCHS: "60-real;61-real;62-real;70-real;72-real;75-real;80;86-real"
FAISS_FLATTEN_CONDA_INCLUDES: "1"
steps:
- name: Checkout
uses: actions/checkout@v4
Expand Down
138 changes: 138 additions & 0 deletions .github/workflows/nightly.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,138 @@
name: Nightly
on:
schedule:
- cron: '1 10 * * *'
env:
OMP_NUM_THREADS: '10'
MKL_THREADING_LAYER: GNU
jobs:
linux-x86_64-nightly:
name: Linux x86_64 nightlies
runs-on: ubuntu-latest
steps:
- name: Checkout
uses: actions/checkout@v4
with:
fetch-depth: 0
fetch-tags: true
- uses: ./.github/actions/build_conda
env:
ANACONDA_API_TOKEN: ${{ secrets.ANACONDA_API_TOKEN }}
with:
label: nightly
linux-x86_64-GPU-CUDA-11-4-4-nightly:
name: Linux x86_64 GPU nightlies (CUDA 11.4.4)
runs-on: 4-core-ubuntu-gpu-t4
env:
CUDA_ARCHS: "60-real;61-real;62-real;70-real;72-real;75-real;80;86-real"
steps:
- name: Checkout
uses: actions/checkout@v4
with:
fetch-depth: 0
fetch-tags: true
- uses: ./.github/actions/build_conda
env:
ANACONDA_API_TOKEN: ${{ secrets.ANACONDA_API_TOKEN }}
with:
label: nightly
cuda: "11.4.4"
compiler_version: "11.2"
linux-x86_64-GPU-RAFT-CUDA11-8-0-nightly:
name: Linux x86_64 GPU w/ RAFT nightlies (CUDA 11.8.0)
runs-on: 4-core-ubuntu-gpu-t4
env:
CUDA_ARCHS: "70-real;72-real;75-real;80;86-real"
steps:
- name: Checkout
uses: actions/checkout@v4
with:
fetch-depth: 0
fetch-tags: true
- uses: ./.github/actions/build_conda
env:
ANACONDA_API_TOKEN: ${{ secrets.ANACONDA_API_TOKEN }}
with:
label: nightly
raft: "ON"
cuda: "11.8.0"
compiler_version: "11.2"
linux-x86_64-GPU-CUDA-12-1-1-nightly:
name: Linux x86_64 GPU nightlies (CUDA 12.1.1)
runs-on: 4-core-ubuntu-gpu-t4
env:
CUDA_ARCHS: "70-real;72-real;75-real;80;86-real"
steps:
- name: Checkout
uses: actions/checkout@v4
with:
fetch-depth: 0
fetch-tags: true
- uses: ./.github/actions/build_conda
env:
ANACONDA_API_TOKEN: ${{ secrets.ANACONDA_API_TOKEN }}
with:
label: nightly
cuda: "12.1.1"
compiler_version: "11.2"
linux-x86_64-GPU-RAFT-CUDA12-1-1-nightly:
name: Linux x86_64 GPU w/ RAFT nightlies (CUDA 12.1.1)
runs-on: 4-core-ubuntu-gpu-t4
env:
CUDA_ARCHS: "70-real;72-real;75-real;80;86-real"
steps:
- name: Checkout
uses: actions/checkout@v4
with:
fetch-depth: 0
fetch-tags: true
- uses: ./.github/actions/build_conda
env:
ANACONDA_API_TOKEN: ${{ secrets.ANACONDA_API_TOKEN }}
with:
label: nightly
raft: "ON"
cuda: "12.1.1"
compiler_version: "11.2"
windows-x86_64-nightly:
name: Windows x86_64 nightlies
runs-on: windows-2019
steps:
- name: Checkout
uses: actions/checkout@v4
with:
fetch-depth: 0
fetch-tags: true
- uses: ./.github/actions/build_conda
env:
ANACONDA_API_TOKEN: ${{ secrets.ANACONDA_API_TOKEN }}
with:
label: nightly
osx-arm64-nightly:
name: OSX arm64 nightlies
runs-on: macos-14
steps:
- name: Checkout
uses: actions/checkout@v4
with:
fetch-depth: 0
fetch-tags: true
- uses: ./.github/actions/build_conda
env:
ANACONDA_API_TOKEN: ${{ secrets.ANACONDA_API_TOKEN }}
with:
label: nightly
linux-arm64-nightly:
name: Linux arm64 nightlies
runs-on: 2-core-ubuntu-arm
steps:
- name: Checkout
uses: actions/checkout@v4
with:
fetch-depth: 0
fetch-tags: true
- uses: ./.github/actions/build_conda
env:
ANACONDA_API_TOKEN: ${{ secrets.ANACONDA_API_TOKEN }}
with:
label: nightly
4 changes: 1 addition & 3 deletions conda/faiss-gpu/meta.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -43,9 +43,7 @@ outputs:
- {{ pin_compatible('libfaiss', exact=True) }}
script_env:
- CUDA_ARCHS
{% if cudatoolkit == '11.4.4' %}
- FAISS_FLATTEN_CONDA_INCLUDES=1
{% endif %}
- FAISS_FLATTEN_CONDA_INCLUDES
requirements:
build:
- {{ compiler('cxx') }}
Expand Down