Skip to content

Add nvidia-cuda-{nvrtc, nvcc} as a dependency for cuDF wheels#18686

Merged
rapids-bot[bot] merged 7 commits intorapidsai:branch-25.06from
brandon-b-miller:require-jit-wheels
May 12, 2025
Merged

Add nvidia-cuda-{nvrtc, nvcc} as a dependency for cuDF wheels#18686
rapids-bot[bot] merged 7 commits intorapidsai:branch-25.06from
brandon-b-miller:require-jit-wheels

Conversation

@brandon-b-miller
Copy link
Copy Markdown
Contributor

With the merge of NVIDIA/numba-cuda#155 we need to depend on these two wheels if we want numba-cuda to be able to find the runtime libraries it needs in the final cuDF environment.

@brandon-b-miller brandon-b-miller requested a review from a team as a code owner May 6, 2025 19:47
@brandon-b-miller brandon-b-miller requested a review from bdice May 6, 2025 19:47
@jakirkham jakirkham added improvement Improvement / enhancement to an existing function non-breaking Non-breaking change labels May 6, 2025
@jakirkham
Copy link
Copy Markdown
Member

Seeing the following error on CI:

numba.cuda.cudadrv.driver.LinkerError: [222] Call to cuLinkAddData results in CUDA_ERROR_UNSUPPORTED_PTX_VERSION
ptxas application ptx input, line 9; fatal   : Unsupported .version 8.8; current version is '8.7'

Looks like pynvjitlink is enabled in this code

numba_config.CUDA_ENABLE_PYNVJITLINK = True

Not sure what else we are missing

@brandon-b-miller
Copy link
Copy Markdown
Contributor Author

brandon-b-miller commented May 6, 2025

These errors

ptxas application ptx input, line 9; fatal   : Unsupported .version 8.8; current version is '8.7'

Indicate the PTX is coming from the CUDA 12.9 NVVM library. The version of the nvvm wheel that ends up on the user system cant be newer than the driver it seems. We're probably getting the latest now so that's why it shows up.

@brandon-b-miller
Copy link
Copy Markdown
Contributor Author

reproduced using local CI, following up.

@vyasr
Copy link
Copy Markdown
Contributor

vyasr commented May 7, 2025

After NVIDIA/numba-cuda#155 are these hard dependencies of numba-cuda? If so, why aren't they part of the dependency list there? If not and that PR is only adding optional support (i.e. the wheels are used if available but other sources are also OK) then should we figure out what we want the final state to be for cudf? I'm fine with making these hard dependencies if we want to say that cudf will always get these libraries from wheels going forward, I just want to make sure that we are aware of the choices we are making if they are not forced upon us by numba-cuda.

@brandon-b-miller brandon-b-miller requested a review from a team as a code owner May 7, 2025 22:14
@github-actions github-actions bot added the Python Affects Python cuDF API. label May 7, 2025
@github-actions github-actions bot removed the Python Affects Python cuDF API. label May 8, 2025
@brandon-b-miller
Copy link
Copy Markdown
Contributor Author

are these hard dependencies of numba-cuda?

No, they are not. The current strategy is for the traditional numba-cuda not to depend on CUDA wheels and to be able to leverage system packages where necessary. However in the future there may be a one liner cuDF can use to install numba-cuda with cuda wheel support so that cuDF doesn't have to manage the dependencies explicitly.

@github-actions github-actions bot added the Python Affects Python cuDF API. label May 8, 2025
@github-actions github-actions bot removed the Python Affects Python cuDF API. label May 12, 2025
@GPUtester GPUtester moved this to In Progress in cuDF Python May 12, 2025
@brandon-b-miller
Copy link
Copy Markdown
Contributor Author

/merge

@rapids-bot rapids-bot bot merged commit 3248bcc into rapidsai:branch-25.06 May 12, 2025
127 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

improvement Improvement / enhancement to an existing function non-breaking Non-breaking change

Projects

Archived in project

Development

Successfully merging this pull request may close these issues.

5 participants