wheels CI: test mix of `cuda-toolkit` version in CI

[jameslamb]

Description

RAPIDS projects attempt to support a wide range of minor versions within each major CUDA version.

They should test that support in wheels CI by pinning to specific cuda-toolkit package versions.

Benefits of this work

• improves test coverage of RAPIDS compatibility claims
• reduces the risk of runtime issues with other projects that pin back to older-than-latest CTKs
  • *like torch: 'torch' wheels not compatible with 'cuda-toolkit' (CUDA 12 only) #255
• side benefit: makes it easier for RAPIDS projects to add more lower-bound testing (say, of cupy or numpy) in the future
  • *(because this will be the first time some projects have used

Acceptance Criteria

• RAPIDS wheels are tested against multiple versions of the cuda-toolkit metapackage per each supported major CTK version

Approach

Test in cuVS: rapidsai/cuvs#1855

We probably want something like pip constraints that constraint cuda-toolkit to whatever {major}.{minor} is set in the RAPIDS_CUDA_VERSION environment variable, similar to how cuda-version is set for conda environments:

  cuda_version:
    specific:
      - output_types: conda
        matrices:
          - matrix:
              cuda: "12.2"
            packages:
              - cuda-version=12.2

(rapidsai/cudf - dependencies.yaml)

• gha-tools changes to accommodate this
  • rapids-generate-pip-constraints updates (rapids-generate-pip-constraints: stop special-casing 'latest' gha-tools#247)
• shared-workflows changes to accommodate this (if any) (wheels-build: build on CUDA 13.0 shared-workflows#510)
• library repos
  • cucim (build wheels with CUDA 13.0.x, test wheels against mix of CTK versions cucim#1054)
  • cudf (enforce a floor on libnvjitlink, build wheels with CUDA 13.0.x, test wheels against mix of CTK versions cudf#21671)
  • cugraph
    • ensure 'torch' CUDA wheels are installed in CI, remove unused dependencies cugraph#5453
    • enforce a floor on libnvjitlink, build wheels with CUDA 13.0.x, test wheels against mix of CTK versions cugraph#5457
  • cugraph-gnn
    • libwholegraph: declare nvidia-nccl dependency for CUDA 13 wheels cugraph-gnn#428
    • make PyTorch installation in conda test jobs stricter cugraph-gnn#427
    • CI: restore arm64 conda tests, re-use run_* scripts in test_* scripts cugraph-gnn#429
    • ensure 'torch' CUDA wheels are installed in CI, test that 'torch' is an optional dependency cugraph-gnn#425
    • WIP: wheels CI: stricter torch index selection, test oldest versions of dependencies cugraph-gnn#413 - handled by other PRs above
    • one more with the testing changes (wheels: build with CUDA 13.0, test against mix of CTK versions, make 'torch-geometric' fully optional for 'cugraph-pyg' cugraph-gnn#434)
  • cuml (build wheels with CUDA 13.0.x, test wheels against mix of CTK versions cuml#7907)
  • cuopt (build wheels with CUDA 13.0.x, test wheels against mix of CTK versions NVIDIA/cuopt#973)
  • cuvs (enforce a floor on libnvjitlink, build wheels with CUDA 13.0.x, test wheels against mix of CTK versions cuvs#1862)
  • cuxfilter (build wheels with CUDA 13.0.x, test wheels against mix of CTK versions cuxfilter#777)
  • dask-cuda (test wheels with mix of CTK versions dask-cuda#1632)
  • kvikio (build wheels with CUDA 13.0.x, test wheels against mix of CTK versions kvikio#942)
  • nvforest (build wheels with CUDA 13.0.x, test wheels against mix of CTK versions, drop CUDA math libraries dependencies nvforest#87)
  • nx-cugraph (test wheels with mix of CTK versions nx-cugraph#248)
  • raft (build wheels with CUDA 13.0.x, test wheels against mix of CTK versions raft#2971)
  • rapidsmpf (build wheels with CUDA 13.0.x, test wheels against mix of CTK versions rapidsmpf#919)
  • rmm
    • ensure 'torch' CUDA wheels are installed in CI rmm#2279
    • build wheels with CUDA 13.0.x, test wheels against mix of CTK versions rmm#2270
  • ucxx (build wheels with CUDA 13.0.x, test wheels against mix of CTK versions ucxx#604)
• update rapids-reviser examples used when we change the set of RAPIDS-supported CUDA versions

Notes

A few things have all led up to this:

• Testing against oldest python dependencies  #81 introduced a pattern for RAPIDS CI to pull in the oldest versions of Python dependencies, test lower bounds
• wheels: adapt to new CTK wheel naming scheme and layouts, use cuda-toolkit metapackage #213 introduced use of the cuda-toolkit wheels for pinning CTK components like cuBLAS and nvJitLink in RAPIDS wheels.
• Some recent work on using JIT-LTO for some RAPIDS kernels exposed a bug of the form "built against CTK 13.1, fails to run with CTK 13.0" ([BUG]: libcugraph 26.04 nightly wheels fail to load with older nvidia-nvjitlink installed (CUDA 13 only) cugraph#5443)