Skip to content

Commit c55fa59

Browse files
dantegddantegd
committed
tighten wheel size limits, expand CI-skipping logic, other small build changes (rapidsai#7738)
Proposes a batch of miscellaneous build / packaging / CI changes. ## Changes ### Tightens wheel size limits Contributes to rapidsai/build-planning#219 CUDA 13 wheels can be as much as 50% smaller than the equivalent CUDA 12 wheels, because of some new compression features in `nvcc`. To ensure surprising package-size growth is caught in CI, this PR tightens the limits in the following ways: * setting different limits based on CUDA major version where package size differs significantly * setting all limits to `{compressed_size} + 10Mi`, rounded to the nearest 5Mi ### Expands CI-skipping logic Contributes to rapidsai/build-planning#243 Tries to avoid unnecessary CI runs by making the CI-skipping rules finer-grained. For example, PRs that only touch `.pre-commit-config.yaml` should now not require any runners with GPUs 😁 ### Removes reliance on `pypi.nvidia.com` Contributes to rapidsai/build-planning#241 ```shell git grep -i -E 'pypi\.nvidia\.com' git grep -i -E 'rapids\-init\-pip' ``` And removed/updated all relevant references. This project does not need any wheels from `pypi.nvidia.com` at build-time or runtime, it can safely avoid searching that index. ### Enforces PEP 639 license metadata in `pyproject.toml` Contributes to rapidsai/pre-commit-hooks#95 Authors: - James Lamb (https://github.com/jameslamb) Approvers: - Bradley Dice (https://github.com/bdice) URL: rapidsai#7738
1 parent d072ff7 commit c55fa59

12 files changed

Lines changed: 170 additions & 37 deletions

‎.github/workflows/pr.yaml‎

Lines changed: 119 additions & 9 deletions
Original file line numberDiff line numberDiff line change
@@ -61,37 +61,146 @@ jobs:
6161
uses: rapidsai/shared-workflows/.github/workflows/changed-files.yaml@main
6262
with:
6363
files_yaml: |
64+
build_docs:
65+
- '**'
66+
- '!.devcontainer/**'
67+
- '!.git-blame-ignore-revs'
68+
- '!.github/CODEOWNERS'
69+
- '!.github/ISSUE_TEMPLATE/**'
70+
- '!.github/copy-pr-bot.yaml'
71+
- '!.github/labeler.yml'
72+
- '!.github/ops-bot.yaml'
73+
- '!.github/release.yml'
74+
- '!.github/workflows/new-issues-to-triage-projects.yml'
75+
- '!.github/workflows/pr_assign_author.yml'
76+
- '!.github/workflows/pr_issue_status_automation.yml'
77+
- '!.github/workflows/trigger-breaking-change-alert.yaml'
78+
- '!.pre-commit-config.yaml'
79+
- '!ci/build_wheel*.sh'
80+
- '!ci/check_style.sh'
81+
- '!ci/release/update-version.sh'
82+
- '!ci/test_notebooks.sh'
83+
- '!ci/test_python*.sh'
84+
- '!ci/test_wheel*.sh'
85+
- '!codecov.yml'
6486
test_cpp:
6587
- '**'
6688
- '!.devcontainer/**'
89+
- '!.git-blame-ignore-revs'
90+
- '!.github/CODEOWNERS'
91+
- '!.github/ISSUE_TEMPLATE/**'
92+
- '!.github/copy-pr-bot.yaml'
93+
- '!.github/labeler.yml'
94+
- '!.github/ops-bot.yaml'
95+
- '!.github/release.yml'
96+
- '!.github/workflows/new-issues-to-triage-projects.yml'
97+
- '!.github/workflows/pr_assign_author.yml'
98+
- '!.github/workflows/pr_issue_status_automation.yml'
99+
- '!.github/workflows/trigger-breaking-change-alert.yaml'
67100
- '!.pre-commit-config.yaml'
101+
- '!BUILD.md'
68102
- '!CONTRIBUTING.md'
69103
- '!README.md'
104+
- '!ci/build_docs.sh'
105+
- '!ci/build_python.sh'
106+
- '!ci/build_wheel*.sh'
107+
- '!ci/check_style.sh'
108+
- '!ci/notebooks/**'
70109
- '!ci/release/update-version.sh'
110+
- '!ci/test_notebooks.sh'
111+
- '!ci/test_python*.sh'
112+
- '!ci/test_wheel*.sh'
71113
- '!docs/**'
72114
- '!img/**'
73115
- '!notebooks/**'
74116
- '!python/**'
75117
- '!thirdparty/LICENSES/**'
118+
- '!wiki/**'
76119
test_notebooks:
77120
- '**'
78121
- '!.devcontainer/**'
122+
- '!.git-blame-ignore-revs'
123+
- '!.github/CODEOWNERS'
124+
- '!.github/ISSUE_TEMPLATE/**'
125+
- '!.github/copy-pr-bot.yaml'
126+
- '!.github/labeler.yml'
127+
- '!.github/ops-bot.yaml'
128+
- '!.github/release.yml'
129+
- '!.github/workflows/new-issues-to-triage-projects.yml'
130+
- '!.github/workflows/pr_assign_author.yml'
131+
- '!.github/workflows/pr_issue_status_automation.yml'
132+
- '!.github/workflows/trigger-breaking-change-alert.yaml'
79133
- '!.pre-commit-config.yaml'
134+
- '!BUILD.md'
80135
- '!CONTRIBUTING.md'
81136
- '!README.md'
137+
- '!ci/check_style.sh'
82138
- '!ci/release/update-version.sh'
83139
- '!thirdparty/LICENSES/**'
84-
test_python:
140+
- '!wiki/**'
141+
test_python_conda:
85142
- '**'
86143
- '!.devcontainer/**'
144+
- '!.git-blame-ignore-revs'
145+
- '!.github/CODEOWNERS'
146+
- '!.github/ISSUE_TEMPLATE/**'
147+
- '!.github/copy-pr-bot.yaml'
148+
- '!.github/labeler.yml'
149+
- '!.github/ops-bot.yaml'
150+
- '!.github/release.yml'
151+
- '!.github/workflows/new-issues-to-triage-projects.yml'
152+
- '!.github/workflows/pr_assign_author.yml'
153+
- '!.github/workflows/pr_issue_status_automation.yml'
154+
- '!.github/workflows/trigger-breaking-change-alert.yaml'
87155
- '!.pre-commit-config.yaml'
156+
- '!BUILD.md'
88157
- '!CONTRIBUTING.md'
89158
- '!README.md'
159+
- '!ci/build_wheel*.sh'
160+
- '!ci/check_style.sh'
90161
- '!ci/release/update-version.sh'
162+
- '!ci/run_clang_tidy.sh'
163+
- '!ci/run_ctests.sh'
164+
- '!ci/test_wheel*.sh'
165+
- '!ci/validate_wheel.sh'
91166
- '!docs/**'
92167
- '!img/**'
93168
- '!notebooks/**'
94169
- '!thirdparty/LICENSES/**'
170+
- '!wiki/**'
171+
test_python_wheels:
172+
- '**'
173+
- '!.devcontainer/**'
174+
- '!.git-blame-ignore-revs'
175+
- '!.github/CODEOWNERS'
176+
- '!.github/ISSUE_TEMPLATE/**'
177+
- '!.github/copy-pr-bot.yaml'
178+
- '!.github/labeler.yml'
179+
- '!.github/ops-bot.yaml'
180+
- '!.github/release.yml'
181+
- '!.github/workflows/new-issues-to-triage-projects.yml'
182+
- '!.github/workflows/pr_assign_author.yml'
183+
- '!.github/workflows/pr_issue_status_automation.yml'
184+
- '!.github/workflows/trigger-breaking-change-alert.yaml'
185+
- '!.pre-commit-config.yaml'
186+
- '!BUILD.md'
187+
- '!CONTRIBUTING.md'
188+
- '!README.md'
189+
- '!ci/build_cpp.sh'
190+
- '!ci/build_python.sh'
191+
- '!ci/check_style.sh'
192+
- '!ci/notebooks/**'
193+
- '!ci/release/update-version.sh'
194+
- '!ci/run_clang_tidy.sh'
195+
- '!ci/run_tests.sh'
196+
- '!ci/test_cpp.sh'
197+
- '!ci/test_notebooks.sh'
198+
- '!ci/test_python*.sh'
199+
- '!docs/**'
200+
- '!img/**'
201+
- '!notebooks/**'
202+
- '!thirdparty/LICENSES/**'
203+
- '!wiki/**'
95204
checks:
96205
secrets: inherit
97206
needs: telemetry-setup
@@ -145,7 +254,7 @@ jobs:
145254
needs: [conda-python-build, changed-files]
146255
secrets: inherit
147256
uses: rapidsai/shared-workflows/.github/workflows/conda-python-tests.yaml@main
148-
if: fromJSON(needs.changed-files.outputs.changed_file_groups).test_python
257+
if: fromJSON(needs.changed-files.outputs.changed_file_groups).test_python_conda
149258
with:
150259
build_type: pull-request
151260
script: "ci/test_python_singlegpu.sh"
@@ -154,7 +263,7 @@ jobs:
154263
needs: [conda-python-build, changed-files]
155264
secrets: inherit
156265
uses: rapidsai/shared-workflows/.github/workflows/conda-python-tests.yaml@main
157-
if: fromJSON(needs.changed-files.outputs.changed_file_groups).test_python
266+
if: fromJSON(needs.changed-files.outputs.changed_file_groups).test_python_conda
158267
with:
159268
# Select the amd64 entry with the highest CUDA and Python version
160269
matrix_filter: map(select(.ARCH=="amd64")) | [max_by([.CUDA_VER,.PY_VER]|map(split(".")|map(tonumber)))]
@@ -165,7 +274,7 @@ jobs:
165274
needs: [conda-python-build, changed-files]
166275
secrets: inherit
167276
uses: rapidsai/shared-workflows/.github/workflows/conda-python-tests.yaml@main
168-
if: fromJSON(needs.changed-files.outputs.changed_file_groups).test_python
277+
if: fromJSON(needs.changed-files.outputs.changed_file_groups).test_python_conda
169278
with:
170279
build_type: pull-request
171280
script: "ci/test_python_dask.sh"
@@ -174,7 +283,7 @@ jobs:
174283
needs: [conda-python-build, changed-files]
175284
secrets: inherit
176285
uses: rapidsai/shared-workflows/.github/workflows/conda-python-tests.yaml@main
177-
if: fromJSON(needs.changed-files.outputs.changed_file_groups).test_python
286+
if: fromJSON(needs.changed-files.outputs.changed_file_groups).test_python_conda
178287
with:
179288
build_type: pull-request
180289
script: "ci/test_python_scikit_learn_tests.sh"
@@ -196,7 +305,7 @@ jobs:
196305
needs: [conda-python-build, changed-files]
197306
secrets: inherit
198307
uses: rapidsai/shared-workflows/.github/workflows/conda-python-tests.yaml@main
199-
if: fromJSON(needs.changed-files.outputs.changed_file_groups).test_python
308+
if: fromJSON(needs.changed-files.outputs.changed_file_groups).test_python_conda
200309
with:
201310
build_type: pull-request
202311
script: "ci/test_python_cuml_accel_upstream.sh"
@@ -215,9 +324,10 @@ jobs:
215324
container_image: "rapidsai/ci-conda:26.04-latest"
216325
script: "ci/test_notebooks.sh"
217326
docs-build:
218-
needs: conda-python-build
327+
needs: [conda-python-build, changed-files]
219328
secrets: inherit
220329
uses: rapidsai/shared-workflows/.github/workflows/custom-job.yaml@main
330+
if: fromJSON(needs.changed-files.outputs.changed_file_groups).build_docs
221331
with:
222332
build_type: pull-request
223333
node_type: "gpu-l4-latest-1"
@@ -255,7 +365,7 @@ jobs:
255365
needs: [wheel-build-cuml, changed-files]
256366
secrets: inherit
257367
uses: rapidsai/shared-workflows/.github/workflows/wheels-test.yaml@main
258-
if: fromJSON(needs.changed-files.outputs.changed_file_groups).test_python
368+
if: fromJSON(needs.changed-files.outputs.changed_file_groups).test_python_wheels
259369
with:
260370
build_type: pull-request
261371
script: ci/test_wheel.sh
@@ -264,7 +374,7 @@ jobs:
264374
needs: [wheel-build-cuml, changed-files]
265375
secrets: inherit
266376
uses: rapidsai/shared-workflows/.github/workflows/wheels-test.yaml@main
267-
if: fromJSON(needs.changed-files.outputs.changed_file_groups).test_python
377+
if: fromJSON(needs.changed-files.outputs.changed_file_groups).test_python_wheels
268378
with:
269379
build_type: pull-request
270380
script: ci/test_wheel_dask.sh

‎.pre-commit-config.yaml‎

Lines changed: 8 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -1,5 +1,5 @@
11
---
2-
# SPDX-FileCopyrightText: Copyright (c) 2023-2025, NVIDIA CORPORATION.
2+
# SPDX-FileCopyrightText: Copyright (c) 2023-2026, NVIDIA CORPORATION.
33
# SPDX-License-Identifier: Apache-2.0
44

55
repos:
@@ -107,7 +107,7 @@ repos:
107107
args: ['format', '--cleanup']
108108
additional_dependencies: ['pyyaml>=6.0', 'packaging>=21.0']
109109
- repo: https://github.com/rapidsai/pre-commit-hooks
110-
rev: v1.1.0
110+
rev: v1.3.3
111111
hooks:
112112
- id: verify-copyright
113113
name: verify-copyright-cuml
@@ -164,6 +164,12 @@ repos:
164164
- id: verify-alpha-spec
165165
- id: verify-codeowners
166166
args: [--fix, --project-prefix=cuml]
167+
- id: verify-pyproject-license
168+
# ignore the top-level pyproject.toml, which doesn't
169+
# have or need a [project] table
170+
exclude: |
171+
(?x)
172+
^pyproject[.]toml$
167173
- repo: https://github.com/rapidsai/dependency-file-generator
168174
rev: v1.20.0
169175
hooks:

‎ci/build_wheel.sh‎

Lines changed: 3 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -9,6 +9,8 @@ package_dir=$2
99

1010
source rapids-configure-sccache
1111
source rapids-date-string
12+
RAPIDS_INIT_PIP_REMOVE_NVIDIA_INDEX="true"
13+
export RAPIDS_INIT_PIP_REMOVE_NVIDIA_INDEX
1214
source rapids-init-pip
1315

1416
export SCCACHE_S3_PREPROCESSOR_CACHE_KEY_PREFIX="${package_name}/${RAPIDS_CONDA_ARCH}/cuda${RAPIDS_CUDA_VERSION%%.*}/wheel/preprocessor-cache"
@@ -32,7 +34,7 @@ RAPIDS_PIP_WHEEL_ARGS=(
3234
#
3335
# Passing '--build-constraint' and '--no-build-isolation` together results in an error from 'pip',
3436
# but we want to keep environment variable PIP_CONSTRAINT set unconditionally.
35-
# PIP_NO_BUILD_ISOLATION=0 means "add --no-build-isolation" (ref: https://github.com/pypa/pip/issues/573
37+
# PIP_NO_BUILD_ISOLATION=0 means "add --no-build-isolation" (ref: https://github.com/pypa/pip/issues/5735)
3638
if [[ "${PIP_NO_BUILD_ISOLATION:-}" != "0" ]]; then
3739
RAPIDS_PIP_WHEEL_ARGS+=(--build-constraint="${PIP_CONSTRAINT}")
3840
fi

‎ci/build_wheel_cuml.sh‎

Lines changed: 2 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -4,6 +4,8 @@
44

55
set -euo pipefail
66

7+
RAPIDS_INIT_PIP_REMOVE_NVIDIA_INDEX="true"
8+
export RAPIDS_INIT_PIP_REMOVE_NVIDIA_INDEX
79
source rapids-init-pip
810

911
package_name="cuml"

‎ci/build_wheel_libcuml.sh‎

Lines changed: 3 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -1,9 +1,11 @@
11
#!/bin/bash
2-
# SPDX-FileCopyrightText: Copyright (c) 2025, NVIDIA CORPORATION.
2+
# SPDX-FileCopyrightText: Copyright (c) 2025-2026, NVIDIA CORPORATION.
33
# SPDX-License-Identifier: Apache-2.0
44

55
set -euo pipefail
66

7+
RAPIDS_INIT_PIP_REMOVE_NVIDIA_INDEX="true"
8+
export RAPIDS_INIT_PIP_REMOVE_NVIDIA_INDEX
79
source rapids-init-pip
810

911
package_name="libcuml"

‎ci/test_wheel.sh‎

Lines changed: 3 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -1,9 +1,11 @@
11
#!/bin/bash
2-
# SPDX-FileCopyrightText: Copyright (c) 2023-2025, NVIDIA CORPORATION.
2+
# SPDX-FileCopyrightText: Copyright (c) 2023-2026, NVIDIA CORPORATION.
33
# SPDX-License-Identifier: Apache-2.0
44

55
set -euo pipefail
66

7+
RAPIDS_INIT_PIP_REMOVE_NVIDIA_INDEX="true"
8+
export RAPIDS_INIT_PIP_REMOVE_NVIDIA_INDEX
79
source rapids-init-pip
810

911
RAPIDS_PY_CUDA_SUFFIX="$(rapids-wheel-ctk-name-gen "${RAPIDS_CUDA_VERSION}")"

‎ci/test_wheel_dask.sh‎

Lines changed: 3 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -1,9 +1,11 @@
11
#!/bin/bash
2-
# SPDX-FileCopyrightText: Copyright (c) 2025, NVIDIA CORPORATION.
2+
# SPDX-FileCopyrightText: Copyright (c) 2025-2026, NVIDIA CORPORATION.
33
# SPDX-License-Identifier: Apache-2.0
44

55
set -euo pipefail
66

7+
RAPIDS_INIT_PIP_REMOVE_NVIDIA_INDEX="true"
8+
export RAPIDS_INIT_PIP_REMOVE_NVIDIA_INDEX
79
source rapids-init-pip
810

911
RAPIDS_PY_CUDA_SUFFIX="$(rapids-wheel-ctk-name-gen "${RAPIDS_CUDA_VERSION}")"

‎ci/test_wheel_integrations.sh‎

Lines changed: 2 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -4,6 +4,8 @@
44

55
set -euo pipefail
66

7+
RAPIDS_INIT_PIP_REMOVE_NVIDIA_INDEX="true"
8+
export RAPIDS_INIT_PIP_REMOVE_NVIDIA_INDEX
79
source rapids-init-pip
810

911
RAPIDS_PY_CUDA_SUFFIX="$(rapids-wheel-ctk-name-gen "${RAPIDS_CUDA_VERSION}")"

‎ci/validate_wheel.sh‎

Lines changed: 24 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -1,18 +1,40 @@
11
#!/bin/bash
2-
# SPDX-FileCopyrightText: Copyright (c) 2024-2025, NVIDIA CORPORATION.
2+
# SPDX-FileCopyrightText: Copyright (c) 2024-2026, NVIDIA CORPORATION.
33
# SPDX-License-Identifier: Apache-2.0
44

55
set -euo pipefail
66

77
package_dir=$1
88
wheel_dir_relative_path=$2
99

10+
RAPIDS_CUDA_MAJOR="${RAPIDS_CUDA_VERSION%%.*}"
11+
1012
cd "${package_dir}"
1113

1214
rapids-logger "validate packages with 'pydistcheck'"
1315

16+
PYDISTCHECK_ARGS=(
17+
--inspect
18+
)
19+
20+
# PyPI hard limit is 1GiB, but try to keep these as small as possible
21+
if [[ "${package_dir}" == "python/libcuml" ]]; then
22+
if [[ "${RAPIDS_CUDA_MAJOR}" == "12" ]]; then
23+
PYDISTCHECK_ARGS+=(
24+
--max-allowed-size-compressed '475Mi'
25+
)
26+
else
27+
PYDISTCHECK_ARGS+=(
28+
--max-allowed-size-compressed '275Mi'
29+
)
30+
fi
31+
elif [[ "${package_dir}" != "python/cuml" ]]; then
32+
rapids-echo-stderr "unrecognized package_dir: '${package_dir}'"
33+
exit 1
34+
fi
35+
1436
pydistcheck \
15-
--inspect \
37+
"${PYDISTCHECK_ARGS[@]}" \
1638
"$(echo "${wheel_dir_relative_path}"/*.whl)"
1739

1840
rapids-logger "validate packages with 'twine'"

0 commit comments

Comments
 (0)