Skip to content

WIP: Prevent path conflict in builds#5821

Closed
AyodeAwe wants to merge 2 commits intorapidsai:branch-24.08from
AyodeAwe:jawe/ops-2621
Closed

WIP: Prevent path conflict in builds#5821
AyodeAwe wants to merge 2 commits intorapidsai:branch-24.08from
AyodeAwe:jawe/ops-2621

Conversation

@AyodeAwe
Copy link
Contributor

@AyodeAwe AyodeAwe commented Mar 25, 2024

Contributes to rapidsai/build-planning#54

This will make builds fail when there are path conflicts

@AyodeAwe AyodeAwe requested a review from a team as a code owner March 25, 2024 19:35
@AyodeAwe AyodeAwe added non-breaking Non-breaking change improvement Improvement / enhancement to an existing function ci labels Mar 25, 2024
@AyodeAwe AyodeAwe changed the base branch from branch-24.04 to branch-24.06 April 3, 2024 18:39
@jameslamb jameslamb changed the title Prevent path conflict in builds WIP: Prevent path conflict in builds Apr 22, 2024
@jameslamb jameslamb marked this pull request as draft April 22, 2024 23:11
@jameslamb
Copy link
Member

jameslamb commented Apr 23, 2024

This is currently failing with the following conflicts.

(I've included just 1 example of each type below)

(CUDA 11.8 build log) (CUDA 12.2 build log)

  1. fmt and spdlog issues (conflicting packages: conda-forge/fmt, librmm, libraft-headers-only)

Described in rapidsai/rmm#1508 (comment).

  1. (CUDA 11.x only) CUDA toolkit packages packaged differently (conflicting packages: nvidia/cuda-{component}, conda-forge/cudatoolkit)
This transaction has incompatible packages due to a shared path.
  packages: nvidia/linux-64::libcublas-11.11.3.6-0, conda-forge/linux-64::cudatoolkit-11.8.0-h4ba93d1_13
  path: 'lib/libcublas.so.11'

And similar for all these:

lib/libcublas.so.11.11.3.6
lib/libcublasLt.so.11
lib/libcublasLt.so.11.11.3.6
lib/libnvblas.so.11
lib/libnvblas.so.11.11.3.6
lib/libcufft.so.10
lib/libcufft.so.10.9.0.58
lib/libcufftw.so.10
lib/libcufftw.so.10.9.0.58
lib/libcurand.so.10
lib/libcurand.so.10.3.0.86
lib/libcusolver.so.11
lib/libcusolver.so.11.4.1.48
lib/libcusolverMg.so.11
lib/libcusolverMg.so.11.4.1.48
lib/libcusparse.so.11
lib/libcusparse.so.11.7.5.86
lib/libcublas.so
lib/libcublasLt.so
lib/libnvblas.so
lib/libcufft.so
lib/libcufftw.so
lib/libcurand.so
lib/libcusolver.so
lib/libcusolverMg.so
lib/libcusparse.so
  1. (CUDA 11.x only) lib-{component} and lib-{component}-dev from the nvidia channel
This transaction has incompatible packages due to a shared path.
  packages: nvidia/linux-64::libcusparse-11.7.5.86-0, nvidia/linux-64::libcusparse-dev-11.7.5.86-0, conda-forge/linux-64::cudatoolkit-11.8.0-h4ba93d1_13
 path: 'lib/libcusparse.so.11.7.5.86'
  1. cub *(conflicting packages: librmm and libraft-headers-only)
This transaction has incompatible packages due to a shared path.
 packages: rapidsai-nightly/linux-64::librmm-24.04.00a38-cuda11_240326_ga98931b9_38, rapidsai-nightly/linux-64::libraft-headers-only-24.04.00a93-cuda11_240326_g9637b3c2_93
 path: 'include/rapids/cub/agent/agent_radix_sort_upsweep.cuh'
  1. libcudacxx (conflicting packages: librmm and libraft-headers-only)
This transaction has incompatible packages due to a shared path.
 packages: rapidsai-nightly/linux-64::librmm-24.04.00a38-cuda11_240326_ga98931b9_38, rapidsai-nightly/linux-64::libraft-headers-only-24.04.00a93-cuda11_240326_g9637b3c2_93
  path: 'include/rapids/libcudacxx/cuda/std/cassert'
  1. thrust (conflicting packages: librmm and libraft-headers-only)
This transaction has incompatible packages due to a shared path.
 packages: rapidsai-nightly/linux-64::librmm-24.04.00a38-cuda11_240326_ga98931b9_38, rapidsai-nightly/linux-64::libraft-headers-only-24.04.00a93-cuda11_240326_g9637b3c2_93
 path: 'include/rapids/thrust/addressof.h'
  1. rmm (conflicting packages: librmm and libraft-headers-only)
This transaction has incompatible packages due to a shared path.
  packages: rapidsai-nightly/linux-64::librmm-24.04.00a38-cuda11_240326_ga98931b9_38, rapidsai-nightly/linux-64::libraft-headers-only-24.04.00a93-cuda11_240326_g9637b3c2_93
  path: 'include/rmm/mr/device/arena_memory_resource.hpp'

@jameslamb jameslamb changed the base branch from branch-24.06 to branch-24.08 May 21, 2024 15:49
@jameslamb
Copy link
Member

This work is paused, in favor of pursuing a better long-term solution in the future. Closing this PR for now.

Subscribe to rapidsai/build-planning#54 and rapidsai/build-planning#56 for updates.

@jameslamb jameslamb closed this Jul 18, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

ci improvement Improvement / enhancement to an existing function non-breaking Non-breaking change

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants