Skip to content

use libkvikio wheels in wheel builds#16778

Merged
rapids-bot[bot] merged 3 commits intorapidsai:branch-24.10from
jameslamb:use-libkvikio-wheels
Sep 10, 2024
Merged

use libkvikio wheels in wheel builds#16778
rapids-bot[bot] merged 3 commits intorapidsai:branch-24.10from
jameslamb:use-libkvikio-wheels

Conversation

@jameslamb
Copy link
Member

@jameslamb jameslamb commented Sep 9, 2024 •
edited
Loading

Description

Follow-up to #15483.
Contributes to rapidsai/build-planning#33.

Adds a build-time dependency on libkvikio wheels for libcudf wheels (per #15483 (comment)).

With this change, CPM is no longer used to download and install the kvikio headers.

Before:

  -- Found cuFile: /usr/local/cuda/lib64/libcufile.so
  -- CPM: Adding package [email protected] (branch-24.10)

(recent build link from branch-24.10)

After:

  -- KvikIO: Found cuFile Batch API: TRUE
  -- KvikIO: Found cuFile Stream API: TRUE
  -- CPM: Using local package [email protected]

(build link from this PR)

Notes for Reviewers

This removes kvikio headers/CMake files from libcudf wheels

Cuts around 0.8 MB (23 files) out of libcudf wheels.

As of this PR, these would no longer be vendored in libcudf wheels:

    0  09-08-2024 06:17   libcudf/include/kvikio/
    0  09-08-2024 06:17   libcudf/include/kvikio/shim/
 6356  09-08-2024 06:17   libcudf/include/kvikio/batch.hpp
 3812  09-08-2024 06:17   libcudf/include/kvikio/buffer.hpp
10499  09-08-2024 06:17   libcudf/include/kvikio/utils.hpp
 1399  09-08-2024 06:17   libcudf/include/kvikio/cufile_config.hpp
33385  09-08-2024 06:17   libcudf/include/kvikio/file_handle.hpp
 7299  09-08-2024 06:17   libcudf/include/kvikio/driver.hpp
 9678  09-08-2024 06:17   libcudf/include/kvikio/defaults.hpp
 5352  09-08-2024 06:17   libcudf/include/kvikio/stream.hpp
 6002  09-08-2024 06:17   libcudf/include/kvikio/error.hpp
 4501  09-08-2024 06:17   libcudf/include/kvikio/bounce_buffer.hpp
 3197  09-08-2024 06:17   libcudf/include/kvikio/parallel_operation.hpp
 9864  09-08-2024 06:17   libcudf/include/kvikio/posix_io.hpp
  717  09-08-2024 06:17   libcudf/include/kvikio/version_config.hpp
 4529  09-08-2024 06:17   libcudf/include/kvikio/shim/cuda.hpp
 3331  09-08-2024 06:17   libcudf/include/kvikio/shim/utils.hpp
 4055  09-08-2024 06:17   libcudf/include/kvikio/shim/cufile_h_wrapper.hpp
 2242  09-08-2024 06:17   libcudf/include/kvikio/shim/cuda_h_wrapper.hpp
 7510  09-08-2024 06:17   libcudf/include/kvikio/shim/cufile.hpp
    0  09-08-2024 06:17   libcudf/lib64/cmake/kvikio/
 5031  09-08-2024 06:17   libcudf/lib64/cmake/kvikio/kvikio-targets.cmake
 3681  09-08-2024 06:17   libcudf/lib64/cmake/kvikio/kvikio-config-version.cmake
 6915  09-08-2024 06:17   libcudf/lib64/cmake/kvikio/kvikio-config.cmake
 1529  09-08-2024 06:17   libcudf/lib64/cmake/kvikio/kvikio-dependencies.cmake
 3851  09-08-2024 06:17   libcudf/lib64/cmake/kvikio/FindcuFile.cmake

This is safe because kvikio is a PRIVATE dependency of libcudf.

cudf/cpp/CMakeLists.txt

Lines 796 to 802 in 150f1b1

# Specify the target module library dependencies
target_link_libraries(
cudf
PUBLIC CCCL::CCCL rmm::rmm $<BUILD_LOCAL_INTERFACE:BS::thread_pool>
PRIVATE $<BUILD_LOCAL_INTERFACE:nvtx3::nvtx3-cpp> cuco::cuco ZLIB::ZLIB nvcomp::nvcomp
kvikio::kvikio $<TARGET_NAME_IF_EXISTS:cuFile_interface> nanoarrow
)

Checklist

  • I am familiar with the Contributing Guidelines.
  • New or existing tests cover these changes.
  • The documentation is up to date with these changes.

@jameslamb jameslamb added 2 - In Progress Currently a work in progress improvement Improvement / enhancement to an existing function non-breaking Non-breaking change labels Sep 9, 2024
@jameslamb jameslamb changed the title WIP: use libkvikio in wheel builds WIP: use libkvikio wheels in wheel builds Sep 9, 2024
@github-actions github-actions bot added the Python Affects Python cuDF API. label Sep 9, 2024
@jameslamb jameslamb mentioned this pull request Sep 9, 2024
Merged
3 tasks
@jameslamb jameslamb changed the title WIP: use libkvikio wheels in wheel builds use libkvikio wheels in wheel builds Sep 9, 2024
@jameslamb jameslamb marked this pull request as ready for review September 9, 2024 22:11
@jameslamb jameslamb requested a review from a team as a code owner September 9, 2024 22:11
@jameslamb jameslamb requested review from bdice and msarahan September 9, 2024 22:11
@jameslamb jameslamb added 3 - Ready for Review Ready for review by team and removed 2 - In Progress Currently a work in progress labels Sep 10, 2024
bdice
bdice requested changes Sep 10, 2024
View reviewed changes
Copy link
Contributor

@bdice bdice left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Caught something on further inspection. I'll push a fix.

@bdice
Copy link
Contributor

bdice commented Sep 10, 2024

/merge

@rapids-bot rapids-bot bot merged commit 6dd5689 into rapidsai:branch-24.10 Sep 10, 2024
@jameslamb jameslamb deleted the use-libkvikio-wheels branch September 11, 2024 19:28
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@bdice bdice bdice approved these changes

@msarahan msarahan Awaiting requested review from msarahan

Assignees

No one assigned

Labels

3 - Ready for Review Ready for review by team improvement Improvement / enhancement to an existing function non-breaking Non-breaking change Python Affects Python cuDF API.

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

2 participants

@jameslamb @bdice