Solve forward merger conflicts of 6435#6501
Merged
AyodeAwe merged 51 commits intorapidsai:branch-25.06from Apr 2, 2025
Merged
Conversation
Reduce the UMAP logging verbosity. Avoids printing potentially large arrays.
…ai#6329) PRs being backported: - [x] rapidsai#6234 - [x] rapidsai#6306 - [x] rapidsai#6320 - [x] rapidsai#6319 - [x] rapidsai#6327 - [x] rapidsai#6333 - [x] rapidsai#6142 - [x] rapidsai#6223 - [x] rapidsai#6235 - [x] rapidsai#6317 - [x] rapidsai#6331 - [x] rapidsai#6326 - [x] rapidsai#6332 - [x] rapidsai#6347 - [x] rapidsai#6348 - [x] rapidsai#6337 - [x] rapidsai#6355 - [x] rapidsai#6354 - [x] rapidsai#6322 - [x] rapidsai#6353 - [x] rapidsai#6359 - [x] rapidsai#6364 - [x] rapidsai#6363 - [x] [FIL BATCH_TREE_REORG fix for SM90, 100 and 120](rapidsai@a3e419a) --------- Co-authored-by: William Hicks <whicks@nvidia.com>
PR removes a wrong `click` library option that was present in the CLI functionality.
Due to a bug in the import code, experimental FIL was previously not making use of the `align_bytes` argument correctly. The effect was not just a failure to take advantage of cache line boundaries but a severe pessimization in which padding nodes were inserted in the forest structure at highly non-optimal places. This PR corrects this, resulting in a substantial performance improvement. It also introduces the `layered` layout type, in which nodes of the same depth are stored together. This allows for a moderate performance improvement in some models. It also allows CPU FIL to intelligently set the number of threads rather than accepting the highly non-optimal default. This provides a significant performance improvement for small batch size. Authors: - William Hicks (https://github.com/wphicks) Approvers: - Philip Hyunsu Cho (https://github.com/hcho3) - Dante Gama Dessavre (https://github.com/dantegd) - https://github.com/jakirkham URL: rapidsai#6397
`shellcheck` is a fast, static analysis tool for shell scripts. It's good at
flagging up unused variables, unintentional glob expansions, and other potential
execution and security headaches that arise from the wonders of `bash` (and
other shlangs).
This PR adds a `pre-commit` hook to run `shellcheck` on all of the `sh-lang`
files in the `ci/` directory, and the changes requested by `shellcheck` to make
the existing files pass the check.
xref: rapidsai/build-planning#135
Authors:
- Gil Forsyth (https://github.com/gforsyth)
Approvers:
- Dante Gama Dessavre (https://github.com/dantegd)
- James Lamb (https://github.com/jameslamb)
URL: rapidsai#6246
… RF (rapidsai#6387) If both results are NaNs, pass the test rather than attempting to `ASSERT_NEAR` on NaN values. Authors: - William Hicks (https://github.com/wphicks) - Jim Crist-Harif (https://github.com/jcrist) - Simon Adorf (https://github.com/csadorf) Approvers: - Simon Adorf (https://github.com/csadorf) - Dante Gama Dessavre (https://github.com/dantegd) URL: rapidsai#6387
This fixes `test_accuracy_score` to still work when `cudf.pandas` is active. The failure had gone unnoticed since `cudf.pandas` builds are optional currently and have been flakey long enough that I've stopped inspecting them when they're red :/. More motivation to fix our test issues and make that test run non-optional. Authors: - Jim Crist-Harif (https://github.com/jcrist) Approvers: - Jake Awe (https://github.com/AyodeAwe) - Dante Gama Dessavre (https://github.com/dantegd) URL: rapidsai#6439
…apidsai#6447) This PR adds a filter to skip CUDA 11.4 jobs on PRs as a precursor to enabling them in shared-workflows. Once the 11.4 issues are fixed, this matrix filter should be removed so 11.4 gets tested on PRs. xref: rapidsai/build-planning#164 Authors: - Gil Forsyth (https://github.com/gforsyth) Approvers: - James Lamb (https://github.com/jameslamb) URL: rapidsai#6447
AFAICT these are no longer failing. Some of the disabled tests were reenabled a while ago, but these were missed. After this PR, everything disabled due to rapidsai#5441 has been reenabled. Authors: - Jim Crist-Harif (https://github.com/jcrist) Approvers: - Simon Adorf (https://github.com/csadorf) URL: rapidsai#6446
Clarifies supported platforms on cuML docs homepage. Authors: - Bradley Dice (https://github.com/bdice) - Tim Head (https://github.com/betatim) Approvers: - Simon Adorf (https://github.com/csadorf) URL: rapidsai#6452
This PR builds benchmarks in devcontainers. This closes rapidsai#6443. Authors: - Bradley Dice (https://github.com/bdice) Approvers: - Kyle Edwards (https://github.com/KyleFromNVIDIA) URL: rapidsai#6450
This modifies the way GPU<>CPU conversion is done for random forests to make sure hyper-parameters are preserved and that some fitted attributes are copied over explicitly. Also adds a few more fitted attributes to match the scikit-learn API. Authors: - Tim Head (https://github.com/betatim) - Simon Adorf (https://github.com/csadorf) Approvers: - Simon Adorf (https://github.com/csadorf) URL: rapidsai#6425
I noticed that `dependencies.yaml` had a dependency on an `x86_64` compiler on `aarch64`. This was a typo, not an attempt at cross-compilation. Authors: - Bradley Dice (https://github.com/bdice) - James Lamb (https://github.com/jameslamb) Approvers: - Tim Head (https://github.com/betatim) - James Lamb (https://github.com/jameslamb) - Jim Crist-Harif (https://github.com/jcrist) URL: rapidsai#6456
Provide docs for cuml.accel command line feature
Follow up to rapidsai#6142 (comment) This cleans up some leftovers from the deprecation cycle for `KMeans`'s `init="auto"` Authors: - Tim Head (https://github.com/betatim) - Jim Crist-Harif (https://github.com/jcrist) Approvers: - Jim Crist-Harif (https://github.com/jcrist) URL: rapidsai#6445
The number of embeddings fitted during the training step (`tail` and `tail_embeddings`) was set incorrectly when processing the transformation step. This impacted negative sampling and in turn the trustworthiness score. Authors: - Victor Lafargue (https://github.com/viclafargue) Approvers: - Jim Crist-Harif (https://github.com/jcrist) - William Hicks (https://github.com/wphicks) URL: rapidsai#6449
Fixes rapidsai#6403 This project publishes a conda package, `cuml-cpu`, which does what it sounds like... allows the use of cuML on systems without a GPU. This proposes some updates to packaging for `cuml-cpu`: * fixes importing in CPU-only environment (broken in 25.04, see rapidsai#6403) * enables import tests during conda builds, to reduce the risk of such issues going undetected in the future ## Notes for Reviewers ### Why all these changes in Python code? See some of the challenges I faced documented in rapidsai#6400 (comment). In short, `import cuml` when it was installed via `cuml-cpu` will break at import time whenever modules imported with `cuml.internals.safe_imports.gpu_only_import()` are used in any of the following ways: * type hints * decorators * any other module-level direct use Like this: ```text cuml.internals.safe_imports.UnavailableError: cudf is not installed in non GPU-enabled installations ``` ### How long has this been broken? What's the root cause? It seems like something changed within 25.04... earlier versions of cuML are not affected by these issues: rapidsai#6403 (comment) I don't know what the root cause is. Maybe some changes to `cuml`'s top-level imports in 25.04 is now pulling in the modules with these problems at runtime, when previously it wasn't? I'm really not sure. ### Benefits of these Changes This adds a bit of test coverage in CI, minimally verifying that `cuml-cpu` is installable and that `import cuml` works in an environment without a GPU. Inspired by: * similar changes in `cuvs`: rapidsai/cuvs#750 * this conversation I recently had with @betatim : rapidsai/cuvs#743 (comment) ### How I tested this Saw stuff like this in `conda-python-build` jobs, confirming that the import tests were running and passing: ```text BUILD START: ['cuml-cpu-25.04.00a137-py310_250312_g153b21870_137.conda'] ... import: 'cuml' ... Resource usage statistics from testing cuml-cpu: ... Time elapsed: 0:00:10.0 ... TEST END: /tmp/conda-bld-output/linux-64/cuml-cpu-25.04.00a137-py310_250312_g153b21870_137.conda ``` Authors: - James Lamb (https://github.com/jameslamb) Approvers: - Gil Forsyth (https://github.com/gforsyth) - Simon Adorf (https://github.com/csadorf) - Tim Head (https://github.com/betatim) URL: rapidsai#6400
…ai#6453) Updated the args_to_cpu method to allow NoneType as a valid argument alongside numbers and strings. Fixes skl tests: - test_inplace_data_preprocessing[42-False-csr_matrix] - test_linear_regression_sample_weight_consistency[42-False-csr_matrix] Authors: - Simon Adorf (https://github.com/csadorf) Approvers: - Jim Crist-Harif (https://github.com/jcrist) - Victor Lafargue (https://github.com/viclafargue) URL: rapidsai#6453
…" (rapidsai#6455) This PR adds the nightly CI check back and removes the temporarily increase length as nightlies were blocked due to failing changes in RAFT that are now resolved as of https://github.com/rapidsai/raft/actions/runs/13913530930 Authors: - Divye Gala (https://github.com/divyegala) - Simon Adorf (https://github.com/csadorf) Approvers: - Bradley Dice (https://github.com/bdice) URL: rapidsai#6455
…into branch-25.04-merge-25.02-2
For unknown reasons conda is unintentionally preferring an old build of `rapids-dask-dependency` that relies on `dask` nightlies rather than the current pin of `2025.2.0`. Since the current plan is to no longer install dask nightlies in project CI, removing the dask nightlies channel should prevent this problem going forward. Authors: - Jim Crist-Harif (https://github.com/jcrist) Approvers: - Gil Forsyth (https://github.com/gforsyth) URL: rapidsai#6485
A recent refactor (rapidsai#6089) made `sklearn` accidentally required to import `cuml`. This fixes that. I've tested that `cuml` can be imported now without `sklearn` installed. I'll push up a follow-up PR adding a minimal build import check to CI, but for now I believe this fixup should be sufficient to resolve the issue before release. Authors: - Jim Crist-Harif (https://github.com/jcrist) Approvers: - Tim Head (https://github.com/betatim) - Victor Lafargue (https://github.com/viclafargue) URL: rapidsai#6483
Previously this notebook used a couple internal `cuml` APIs. This PR switches them for public APIs instead. Authors: - Jim Crist-Harif (https://github.com/jcrist) Approvers: - Tim Head (https://github.com/betatim) URL: rapidsai#6488
This PR adds support for handling sparse input arrays in the KMeans algorithm by dispatching to CPU implementation when sparse arrays are detected during fitting. It also updates the sparse array detection utilities to be more robust and consistent across the codebase. Fixes scikit-learn test `test_kmeans_results[float64-lloyd-sparse_array]` in combination with rapidsai#6442 . ## Changes - Added `_should_dispatch_cpu` method to KMeans to handle sparse input arrays - Updated `is_sparse` utility function to use `issparse` instead of `isspmatrix` for better compatibility - Updated sparse array detection in `input_utils.py` to use the new `issparse` method ## Testing - Verified that KMeans correctly dispatches to CPU implementation when sparse arrays are detected Authors: - Simon Adorf (https://github.com/csadorf) - Jim Crist-Harif (https://github.com/jcrist) Approvers: - Victor Lafargue (https://github.com/viclafargue) - Jim Crist-Harif (https://github.com/jcrist) URL: rapidsai#6448
This fixes a failure in `test_to_sparse_dask_array` with dask main. It seems that the workarounds previously implemented are fixed in cupy / dask and can now be removed from cuml. xref rapidsai/dask-upstream-testing#37, specifically the failure [here](https://github.com/rapidsai/dask-upstream-testing/actions/runs/14053066285/job/39346850200#step:10:933). Not sure if anyone has the context to say for sure, but I'm curious how well we think the existing test suite would catch any regressions here. I haven't done any kind of performance / memory profiling to make sure there aren't any more subtle regressions. Authors: - Tom Augspurger (https://github.com/TomAugspurger) Approvers: - Jim Crist-Harif (https://github.com/jcrist) URL: rapidsai#6489
This PR promotes experimental FIL to the new stable FIL. This is purely a Python-level change. `cuml.fil.fil.ForestInference` now resolves to a thin wrapper around `cuml.experimental.fil.fil.ForestInference` with warnings about upcoming changes to the output shape of FIL predictions. Random forest estimators continue to use legacy FIL because of their usage of `TreeliteModel`, an obsolete implementation detail of legacy FIL. A future change should switch this to Treelite's native `treelite.Model` wrapper. The legacy FIL implementation has been moved to `cuml.legacy.fil.fil.ForestInference`. This can be removed in 25.06. The thin wrapper around `cuml.experimental.fil.fil.ForestInference` can also be removed in 25.06 once users have a deprecation cycle to adapt to new output shapes. This is marked as a breaking change because it removes the `shape_str` attribute from `ForestInference` objects. This attribute is not used anywhere in cuML and appears to have existed primarily for debugging. Resolve rapidsai#6460. Authors: - William Hicks (https://github.com/wphicks) - Jim Crist-Harif (https://github.com/jcrist) Approvers: - Jim Crist-Harif (https://github.com/jcrist) - Simon Adorf (https://github.com/csadorf) - Dante Gama Dessavre (https://github.com/dantegd) URL: rapidsai#6464
…into branch-25.04-merge-25.02-2
Co-authored-by: Simon Adorf <sadorf@nvidia.com> Co-authored-by: Jake Awe <jawe@nvidia.com> Co-authored-by: William Hicks <whicks@nvidia.com>
This PR adds the `conda-python-scikit-learn-accel-tests` job to the nightly test workflow. This ensures that scikit-learn acceleration tests are run as part of the nightly test suite, matching the behavior in the PR workflow. Authors: - Simon Adorf (https://github.com/csadorf) - Jim Crist-Harif (https://github.com/jcrist) Approvers: - Tim Head (https://github.com/betatim) - James Lamb (https://github.com/jameslamb) URL: rapidsai#6457
Adds the `.solver_` estimated attribute in addition to the `.solver` hyperparameter. Switches the default cuml `solver` hyperparameter from "eig" to "auto" (backwards-compatible). Authors: - Simon Adorf (https://github.com/csadorf) Approvers: - Tim Head (https://github.com/betatim) URL: rapidsai#6415
|
Check out this pull request on See visual diffs & provide feedback on Jupyter Notebooks. Powered by ReviewNB |
github-actions
Bot
added
conda
csadorf
added
improvement
csadorf
changed the title
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
14 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
No description provided.