Skip to content

Unblock CI for 25.04#6519

Merged
raydouglass merged 2 commits intorapidsai:branch-25.04from
csadorf:fix/unblock-ci-for-25.04
Apr 8, 2025
Merged

Unblock CI for 25.04#6519
raydouglass merged 2 commits intorapidsai:branch-25.04from
csadorf:fix/unblock-ci-for-25.04

Conversation

@csadorf
Copy link
Copy Markdown
Contributor

@csadorf csadorf commented Apr 8, 2025

This PR addresses two issues that are currently blocking the cuML CI on the 25.04 release branch:

  1. Out-of-memory (OOM) errors occurring in SVM tests on CUDA 11.8:

    • Several SVM-related tests, particularly test_svc_methods, are failing with OOM errors and segmentation faults
    • This only surfaces with CUDA 11.8 and is likely due to memory allocation patterns
    • As a temporary workaround, we skip these tests on CUDA 11.8 while the root cause is investigated
  2. XGBoost test dependency compatibility:

We expect to remove the constraint on the xgboost version once the issue is resolved in a future xgboost release.

We expect to be able to address the SVM test issue by reducing its memory footprint (see #6514), however here we are taking a more conservative approach to ensure that the CI pipeline is stable.

The remaining failing CI job is optional, the issue is going to be addressed on branch-25.06.

@csadorf csadorf requested a review from a team as a code owner April 8, 2025 14:39
@csadorf csadorf requested review from bdice and cjnolet April 8, 2025 14:39
@csadorf csadorf self-assigned this Apr 8, 2025
@github-actions github-actions Bot added the Cython / Python Cython or Python issue label Apr 8, 2025
@csadorf csadorf force-pushed the fix/unblock-ci-for-25.04 branch from 445cd53 to 2a8d33d Compare April 8, 2025 14:48
@csadorf csadorf requested a review from a team as a code owner April 8, 2025 14:48
@github-actions github-actions Bot added the conda conda issue label Apr 8, 2025
@csadorf csadorf added bug Something isn't working non-breaking Non-breaking change ci labels Apr 8, 2025
Copy link
Copy Markdown
Member

@jakirkham jakirkham left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks Simon! 🙏

Please update the Conda recipes as well

@csadorf
Copy link
Copy Markdown
Contributor Author

csadorf commented Apr 8, 2025

Thanks Simon! 🙏

Please update the Conda recipes as well

@jakirkham Xgboost is not a hard cuml dependency. It is currently not included in the conda build recipes. I am updating our test dependencies in the PR.

Copy link
Copy Markdown
Member

@jakirkham jakirkham left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for clarifying Simon! 🙏

Ok looks good

@raydouglass raydouglass merged commit fba79e4 into rapidsai:branch-25.04 Apr 8, 2025
81 of 83 checks passed
@csadorf csadorf deleted the fix/unblock-ci-for-25.04 branch April 8, 2025 19:41
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

bug Something isn't working ci conda conda issue Cython / Python Cython or Python issue non-breaking Non-breaking change

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants