Skip to content

Use sccache-dist build cluster for conda and wheel builds#1495

Merged
trxcllnt merged 60 commits intorapidsai:release/25.12from
trxcllnt:fea/use-sccache-build-cluster
Nov 24, 2025
Merged

Use sccache-dist build cluster for conda and wheel builds#1495
trxcllnt merged 60 commits intorapidsai:release/25.12from
trxcllnt:fea/use-sccache-build-cluster

Conversation

@trxcllnt
Copy link
Copy Markdown
Contributor

@trxcllnt trxcllnt commented Nov 4, 2025

Description

RAPIDS has deployed an autoscaling cloud build cluster that can be used to accelerate building large RAPIDS projects.

This PR updates the conda and wheel builds to use the build cluster.

This contributes to rapidsai/build-planning#228.

@trxcllnt trxcllnt requested review from a team as code owners November 4, 2025 18:35
@trxcllnt trxcllnt added the improvement Improves an existing functionality label Nov 4, 2025
@trxcllnt trxcllnt requested a review from a team as a code owner November 4, 2025 18:35
@trxcllnt trxcllnt added the non-breaking Introduces a non-breaking change label Nov 4, 2025
@trxcllnt trxcllnt force-pushed the fea/use-sccache-build-cluster branch from a452f46 to 3bc8c22 Compare November 10, 2025 23:04
@trxcllnt trxcllnt force-pushed the fea/use-sccache-build-cluster branch from e7f882d to 94ec4e7 Compare November 18, 2025 10:21
@trxcllnt trxcllnt force-pushed the fea/use-sccache-build-cluster branch from 94ec4e7 to 91792de Compare November 18, 2025 16:21
@robertmaynard
Copy link
Copy Markdown
Contributor

@trxcllnt Can we also extend the rocky8-clib-standalone-build (12.9/13.X) binary builds ( added in #1524 ) to use the build cluster as well?

@trxcllnt
Copy link
Copy Markdown
Contributor Author

@robertmaynard Yep, this PR updates that job in both pr.yaml and build.yaml (confirmed in CI here)

@robertmaynard
Copy link
Copy Markdown
Contributor

@robertmaynard Yep, this PR updates that job in both pr.yaml and build.yaml (confirmed in CI here)

Excellent, thanks

Copy link
Copy Markdown
Member

@cjnolet cjnolet left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM from library engineering side! Thanks for enabling this @trxcllnt!

@cjnolet cjnolet moved this from Todo to In Progress in Unstructured Data Processing Nov 19, 2025
@trxcllnt trxcllnt force-pushed the fea/use-sccache-build-cluster branch from 6bc9f08 to ab5a180 Compare November 19, 2025 20:34
Copy link
Copy Markdown
Contributor

@rockhowse rockhowse left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@trxcllnt one small DRY request (Feel free to batch commit if it looks good), one question about a differing pattern regarding removal of sccache --show-adv-stats call, and a potentially missing set of expanded SCCACHE_ related env var config in one of the conda recipes.

Comment thread ci/build_standalone_c.sh
Comment thread ci/build_standalone_c.sh Outdated
Comment thread ci/build_standalone_c.sh Outdated
Comment thread ci/build_standalone_c.sh
Comment thread conda/recipes/cuvs-bench-cpu/recipe.yaml
Copy link
Copy Markdown
Contributor

@rockhowse rockhowse left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Excllent let's get this merged in... the spice must flow!

@trxcllnt trxcllnt force-pushed the fea/use-sccache-build-cluster branch from 1f2334d to 483a38a Compare November 22, 2025 01:37
@trxcllnt trxcllnt merged commit 94ea498 into rapidsai:release/25.12 Nov 24, 2025
89 checks passed
@github-project-automation github-project-automation Bot moved this from In Progress to Done in Unstructured Data Processing Nov 24, 2025
@trxcllnt trxcllnt deleted the fea/use-sccache-build-cluster branch November 24, 2025 20:24
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

improvement Improves an existing functionality non-breaking Introduces a non-breaking change

Projects

Status: Done

Development

Successfully merging this pull request may close these issues.

5 participants