handle more RAPIDS version formats in update-version.sh, refactor dependencies.yaml

[jameslamb]

Contributes to rapidsai/build-planning#13.

Updates update-version.sh to correctly handle RAPIDS dependencies like cudf-cu12==24.2.*.

This also pulls in some dependency refactoring originally added in #161, which allows greater use of dependencies.yaml globs (and therefore less maintenance effort to support new CUDA versions).

How I tested this

The portability of this updated sed command was tested here: rapidsai/cudf#14825 (comment).

In this repo, I ran the following:

./ci/release/update-version.sh '0.36.00'
git diff

./ci/release/update-version.sh '0.37.00
git diff

Confirmed that that first git diff changed all the things I expected, and that second one showed 0 changes.

[bdice]

I included some changes that we know are needed for #167 / #171. The changes in b0142c2 will reduce merge conflicts as we continue with multiple work streams (adding CUDA 12.2, fixing branch updates, adding wheels).

[jameslamb]

I included some changes that we know are needed for #167 / #171

These all look good to me, thanks @bdice

[jakirkham]

It looks like some conflicts crept in here. Maybe these files need to be regenerated with rapids-dependency-file-generator

[jakirkham]

Have resolved conflicts with PR ( #164 )

[jakirkham]

As we are moving to rapids-dask-dependency in this PR, think we should update the Conda recipes as well. In particular am noticing these lines here

ucxx/conda/recipes/ucxx/meta.yaml

Lines 274 to 275 in 2249bed

	- dask >=2023.9.2
	- distributed >=2023.9.2

[jameslamb]

As we are moving to rapids-dask-dependency in this PR, think we should update the Conda recipes as well. In particular am noticing these lines here

@jakirkham I just did that in 20a86a4

I don't see any other uses of dask or distributed that need to be replaced.

[jameslamb]

whoops that needs to be a RAPIDS version not UCX's version

The reported errors are:
- Encountered problems while solving:
-   - nothing provides rapids-dask-dependency 0.37.* needed by distributed-ucxx-0.37.00a-py3.10_240125_g20a86a4_18

(build link)

Will fix that momentarily.

Resolved

[jakirkham]

Thanks James! 🙏

[bdice]

I noticed a small issue, rapids-dask-dependency was pinned as =24.4 while update-version.sh would pin it as =24.04. Then I re-ran ci/release/update-version.sh 0.37.00 and realized that there were a bunch of problems from the docs PR #164 not aligning with update-version.sh, and PRs targeting branch-0.36 that were not updated for branch-0.37 (they were merged after branch-0.37 was created). I pushed e06b20c to fix these issues. Sorry for the late change - I know we want to get this merged soon, but this felt sufficiently related to fix in this PR.

[jameslamb]

Sorry for the late change - I know we want to get this merged soon, but this felt sufficiently related to fix in this PR.

Totally fine, thanks for that! I hadn't noticed it.

[jakirkham]

Thanks Bradley! 🙏

Had missed that

[jakirkham]

/merge

[jakirkham]

Thanks all! 🙏