23.08 Release Selector and Install Page Updates #414

jarmak-nv · 2023-07-21T17:54:34Z

closes #413

This PR contains the updates to the release selector required for the conda-forge CUDA 12.0 work as well as the docker overhaul.

To do:

CUDA 12 conda-forge updates
Docker Overhaul updates to the selector
Docker overhaul updates to the install page
23.08 release updates

This PR updates:

Selector
- Enables CUDA 12.0 for conda packages
- Disables CUDA 12.0 for arm conda packages
- Disables cuSignal from conda CUDA 12.0
- Removes the note on cuSignal for conda CUDA 12.0
- Removes PyCaret as a conda additional package, it's now only available on pip for the newest versions
  - https://pypi.org/project/pycaret/#history
  - https://anaconda.org/conda-forge/pycaret/files
- Updated to new docker setup
  - Much simpler, only base and notebooks
  - No OS selection
  - Same CUDA 12 restrictions as Conda
Install page
- Adds 2 troubleshooting steps about conda CUDA 12
  - No ARM
  - PyTorch CUDA version compatibilities
- Expands the CUDA version compatibility section and breaks it up into docker/conda and pip

I've also merged in #412 to simplify merging at release.

jarmak-nv · 2023-07-24T21:26:50Z

I think this is good to review. I was initially holding back to consider adding the docker changes to the install page, ~~but I'll do that in a separate PR to open this to review now.~~

Selector changes are updated for Docker since CUDA 12 update impacts that as well.

Because arm isn't yet supported in CUDA 12, I've left 11.8 as the default. Happy to discuss/change.

jarmak-nv · 2023-07-24T21:31:40Z

@raydouglass would also appreciate your eyes on the selector when you find a moment to make sure I set up the Docker changes correctly - thanks! (I can't add you as a reviewer otherwise I'd just do that.)

bdice

Comments attached.

_includes/selector.html

install/install.md

Co-authored-by: Bradley Dice <[email protected]>

jarmak-nv · 2023-07-25T02:18:12Z

Oh one other question I had for the group: with libmamba being a default installed solver now, thoughts on adding

--solver=libmamba to all conda commands?

bdice · 2023-07-25T03:07:40Z

Oh one other question I had for the group: with libmamba being a default installed solver now, thoughts on adding

--solver=libmamba to all conda commands?

We've wanted to stay in line with conda defaults as much as possible. I am not sure that updated versions of conda are widespread enough to recommend this to all users. However, I'm very eager for it to become the default and recommend that all users update their conda versions...

jarmak-nv · 2023-07-25T14:34:59Z

We've wanted to stay in line with conda defaults as much as possible. I am not sure that updated versions of conda are widespread enough to recommend this to all users. However, I'm very eager for it to become the default and recommend that all users update their conda versions...

Yeah this is a good point, conda installs do often feel like a "set and forget" thing. It's not beautiful, but we could do:

conda create -n rapids-23.06 -c rapidsai -c conda-forge -c nvidia rapids=23.06 python=3.10 cuda-version=11.8 --solver=libmamba || \
conda create -n rapids-23.06 -c rapidsai -c conda-forge -c nvidia rapids=23.06 python=3.10 cuda-version=11.8

Not strongly attached to this, but think it's worth bringing up just because mamba is so much faster and if people can use it, they should.

mmccarty · 2023-07-25T14:39:23Z

We discussed this a while back and agreed to recommend libmamba where possible.

cc @jakirkham @raydouglass for viz

jarmak-nv · 2023-07-25T14:47:22Z

💯 to recommending it, the challenge is when to assume it. If we include --solver=libmamba in the base command we provide from the selector, we're assuming users already have that solver available which is only true if you're on the newest conda install, or are on a late 2022 install + performed the steps to install the libmamba solver.

However, the error you get from conda is very clear if you try to use --solver=libmamba and you don't have it installed. I don't think it's too much of a blocker, but it isn't a perfectly clean "we should do this" imo.

(My weak pref is we do include it and add a troubleshooting step in the docs for when libmamba isn't installed.)

mmccarty · 2023-07-25T14:49:39Z

Well yes, we also need to add / link to instructions on how to install it. I don't suppose we want yet another options in the selector.

bdice · 2023-07-25T14:53:25Z

(My weak pref is we do include it and add a troubleshooting step in the docs for when libmamba isn't installed.)

I like this proposal! Let’s add it “always” in our commands and then give clear troubleshooting instructions on how to fix it if their conda version is too old. Include a copy of the error message so it’s easy to find. Maybe add a note at the bottom of the conda selector like “We add --solver=libmamba for speeding up the environment solving process. This requires conda>=X.Y.Z. Alternatively, use mamba to install.”

ajschmidt8

almost there! there are a lot of changes in this release 🙂

_includes/selector.html

ajschmidt8 · 2023-08-01T17:50:08Z

@jarmak-nv et all, this is looking good. Great tool we have here.

Very minor thing but in the css for the "command" text id make it match so the command label stays to the top:
.cmd-label {
    color: white;
    width: 6em;
    text-transform: uppercase;
    font-weight: 600;
    margin-top: 1rem;
    margin-right: 0.6em;
    vertical-align: top;
}

I think this should also include text-align: right.

It looks like the other labels have that property, but not the command label.

jakirkham

Thanks Ben! 🙏

Looks good from my perspective. Will defer to others on JS, etc.

ajschmidt8

this is awesome.

the release selector is sooo much simpler as a result of the Docker image changes.

bdice

A few remaining comments, but otherwise LGTM!

_includes/selector.html

bdice · 2023-08-01T20:47:45Z

_includes/selector.html

                var py_cuda_pkgs = [this.highlightPkgOrImg("python") + "=" + python_version, this.highlightPkgOrImg("cuda-version") + "=" + cuda_version].join(" ");
                var conda_channels = [rapids_channel, "conda-forge", "nvidia"]
-                    .map(ch => this.highlightFlag("-c") + " " + ch + " ")
+                    .map(ch => "-" + this.highlightFlag("c") + " " + ch + " ")


I think the dashes are usually highlighted the same way as the flag name?

Suggested change

.map(ch => "-" + this.highlightFlag("c") + " " + ch + " ")

.map(ch => this.highlightFlag("-c") + " " + ch + " ")

My vim highlighting looks like:

interesting. okay. I will update in a bit. I just made them match the existing highlights.

bdice · 2023-08-01T20:48:13Z

_includes/selector.html

                var all_pkgs = pkgs_vers + py_cuda_pkgs;

-                var conda_create_ln = this.highlightCmd('conda') + " create " + this.highlightFlag("-n") + " rapids-" + rapids_version + " " + conda_channels;
+                var conda_create_ln = this.highlightCmd('conda') + " create --" + this.highlightFlag("solver") + "=libmamba " + this.highlightFlag("-n") + " rapids-" + rapids_version + " " + conda_channels;


Suggested change

var conda_create_ln = this.highlightCmd('conda') + " create --" + this.highlightFlag("solver") + "=libmamba " + this.highlightFlag("-n") + " rapids-" + rapids_version + " " + conda_channels;

var conda_create_ln = this.highlightCmd('conda') + " create " + this.highlightFlag("--solver") + "=libmamba " + this.highlightFlag("-n") + " rapids-" + rapids_version + " " + conda_channels;

bdice · 2023-08-01T20:48:34Z

_includes/selector.html

-                var pip_install = "pip install ";
-                var index_url = " --extra-index-url=https://pypi.nvidia.com";
+                var pip_install = `${this.highlightCmd("pip")} install`;
+                var index_url = `--${this.highlightFlag("extra-index-url")}=https://pypi.nvidia.com`;


Suggested change

var index_url = `--${this.highlightFlag("extra-index-url")}=https://pypi.nvidia.com`;

var index_url = `${this.highlightFlag("--extra-index-url")}=https://pypi.nvidia.com`;

bdice · 2023-08-01T20:49:47Z

_includes/selector.html


                var indent = "    ";
-                var cmd = this.highlightCmd("docker") + " run --" + this.highlightFlag("gpus") + " all --pull always --" + this.highlightFlag("rm") + " -" + this.highlightFlag("it") + " \\\n" +
+                var cmd = this.highlightCmd("docker") + " run --" + this.highlightFlag("gpus") + " all --" + this.highlightFlag("pull") + " always --" + this.highlightFlag("rm") + " -" + this.highlightFlag("it") + " \\\n" +


Suggested change

var cmd = this.highlightCmd("docker") + " run --" + this.highlightFlag("gpus") + " all --" + this.highlightFlag("pull") + " always --" + this.highlightFlag("rm") + " -" + this.highlightFlag("it") + " \\\n" +

var cmd = this.highlightCmd("docker") + " run " + this.highlightFlag("--gpus") + " all " + this.highlightFlag("--pull") + " always " + this.highlightFlag("--rm") + " " + this.highlightFlag("-it") + " \\\n" +

bdice · 2023-08-01T20:50:16Z

_includes/selector.html

                var indent = "    ";
-                var cmd = this.highlightCmd("docker") + " run --" + this.highlightFlag("gpus") + " all --pull always --" + this.highlightFlag("rm") + " -" + this.highlightFlag("it") + " \\\n" +
+                var cmd = this.highlightCmd("docker") + " run --" + this.highlightFlag("gpus") + " all --" + this.highlightFlag("pull") + " always --" + this.highlightFlag("rm") + " -" + this.highlightFlag("it") + " \\\n" +
                    indent + "--" + this.highlightFlag("shm-size") + "=1g --" + this.highlightFlag("ulimit") + " memlock=-1 --" + this.highlightFlag("ulimit") + " stack=67108864" + " \\\n" +


Suggested change

indent + "--" + this.highlightFlag("shm-size") + "=1g --" + this.highlightFlag("ulimit") + " memlock=-1 --" + this.highlightFlag("ulimit") + " stack=67108864" + " \\\n" +

indent + this.highlightFlag("--shm-size") + "=1g " + this.highlightFlag("--ulimit") + " memlock=-1 " + this.highlightFlag("--ulimit") + " stack=67108864" + " \\\n" +

_includes/selector.html

Co-authored-by: Bradley Dice <[email protected]>

jarmak-nv · 2023-08-02T16:48:20Z

Somewhat perplexed why the deploy preview isn't showing the cuProj updates. My local preview shows fine, but it's like they're not there at all in netlify.

ajschmidt8 · 2023-08-02T17:14:21Z

Somewhat perplexed why the deploy preview isn't showing the cuProj updates. My local preview shows fine, but it's like they're not there at all in netlify.

Must've been a hiccup on Netlify's end. I reran the build from our admin dashboard and it looks like it's showing now.

jakirkham · 2023-08-04T19:46:12Z

Is there anything else needed here?

jarmak-nv · 2023-08-04T19:49:23Z

@jakirkham It's all good - @ajschmidt8 just wants a moment to verify/modify highlighting per #414 (comment) but otherwise no concerns.

Edit to add: it's not super high prio on his radar since the link above is the last thing before merge and lots else going on so movement might not happen until closer to release day.

jakirkham · 2023-08-08T19:02:23Z

Maybe let's do final reviews. Think we can punt on smaller stylistic changes

ajschmidt8 · 2023-08-09T17:48:29Z

Maybe let's do final reviews. Think we can punt on smaller stylistic changes

+1.

Bradley, I don't want to merge your suggestions without reviewing them first, but I don't have time to check them all for consistency at the moment. We can address them in a future PR.

jakirkham · 2023-08-14T21:19:54Z

Thanks all! 🙏

jarmak-nv added 3 commits July 21, 2023 12:47

Initial conda cuda 12.0 doc updates

1ab6560

Correcting unsupported pytorch installation steps

dc20940

Small formatting cleanup

03a928a

jarmak-nv changed the title ~~Conda CUDA 12.0 doc updates~~ 23.08 Release Selector and Install Page Updates Jul 21, 2023

jarmak-nv added 10 commits July 21, 2023 16:32

Starting to incorporate docker updates

f28ec8c

Replace missing end div

24b2bd4

- unneeded docker options, + docker cuda 12 logic

4813509

Add 'a' to nightly version number in Docker

e56d827

Install page CUDA 12 update, layout preview test

0c2abfd

Header formatting update

c0a4de0

Removing leading spaces

1c5d26e

Change to info circles from check circles

767661a

Wording and minor heading format update

ddd284b

Remove 'base' from docker repo name

c560250

jarmak-nv marked this pull request as ready for review July 24, 2023 21:27

jarmak-nv requested a review from a team as a code owner July 24, 2023 21:27

bdice reviewed Jul 24, 2023

View reviewed changes

jarmak-nv and others added 4 commits July 24, 2023 20:35

Apply suggestions from code review

689bc35

Co-authored-by: Bradley Dice <[email protected]>

- old docker logic + dask-sql name fix

cb8e582

Remove more old Docker logic

717027d

Testing {{ site.data.releases.stable.version }}

82f531b

cmd-label css tweak from @exactlyallan

48f5316

ajschmidt8 reviewed Aug 1, 2023

View reviewed changes

_includes/selector.html Show resolved Hide resolved

jakirkham approved these changes Aug 1, 2023

View reviewed changes

jarmak-nv and others added 4 commits August 1, 2023 13:10

Truly disable cuSignal, add text-align: right

6f2e794

rm extraneous comment

b4cd396

run prettier to format

4215f19

unselect cuSignal when selecting CUDA 12

0c4ba90

ajschmidt8 approved these changes Aug 1, 2023

View reviewed changes

ajschmidt8 added 2 commits August 1, 2023 14:56

rm Source method CSS

1b8e178

make cmd highlighting & line breaks consistent

950e909

bdice approved these changes Aug 1, 2023

View reviewed changes

jarmak-nv mentioned this pull request Aug 2, 2023

cuProj Python and C++ Documentation rapidsai/cuspatial#1237

Merged

3 tasks

jarmak-nv and others added 3 commits August 2, 2023 11:03

Remove comma in pip note per review

bf6f63c

Co-authored-by: Bradley Dice <[email protected]>

Add cuProj 🎉

6323860

Add cuProj to meta package for notes

7c1e669

jarmak-nv added 2 commits August 2, 2023 14:34

Combine cuSpatial and cuProj

ad75c34

Remove spaces to avoid extra vertical spacing

0350454

Fix typo breaking port commands in docker install

9b88a51

Add WSL2 libcuda.so error + GitHub Issue

e030cd1

bdice approved these changes Aug 10, 2023

View reviewed changes

jakirkham approved these changes Aug 10, 2023

View reviewed changes

raydouglass merged commit 509f416 into rapidsai:main Aug 14, 2023

	.map(ch => "-" + this.highlightFlag("c") + " " + ch + " ")
	.map(ch => this.highlightFlag("-c") + " " + ch + " ")

	var conda_create_ln = this.highlightCmd('conda') + " create --" + this.highlightFlag("solver") + "=libmamba " + this.highlightFlag("-n") + " rapids-" + rapids_version + " " + conda_channels;
	var conda_create_ln = this.highlightCmd('conda') + " create " + this.highlightFlag("--solver") + "=libmamba " + this.highlightFlag("-n") + " rapids-" + rapids_version + " " + conda_channels;

	var index_url = `--${this.highlightFlag("extra-index-url")}=https://pypi.nvidia.com`;
	var index_url = `${this.highlightFlag("--extra-index-url")}=https://pypi.nvidia.com`;

	var cmd = this.highlightCmd("docker") + " run --" + this.highlightFlag("gpus") + " all --" + this.highlightFlag("pull") + " always --" + this.highlightFlag("rm") + " -" + this.highlightFlag("it") + " \\\n" +
	var cmd = this.highlightCmd("docker") + " run " + this.highlightFlag("--gpus") + " all " + this.highlightFlag("--pull") + " always " + this.highlightFlag("--rm") + " " + this.highlightFlag("-it") + " \\\n" +

	indent + "--" + this.highlightFlag("shm-size") + "=1g --" + this.highlightFlag("ulimit") + " memlock=-1 --" + this.highlightFlag("ulimit") + " stack=67108864" + " \\\n" +
	indent + this.highlightFlag("--shm-size") + "=1g " + this.highlightFlag("--ulimit") + " memlock=-1 " + this.highlightFlag("--ulimit") + " stack=67108864" + " \\\n" +

23.08 Release Selector and Install Page Updates #414

23.08 Release Selector and Install Page Updates #414

Uh oh!

Conversation

jarmak-nv commented Jul 21, 2023 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

jarmak-nv commented Jul 24, 2023 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

jarmak-nv commented Jul 24, 2023

Uh oh!

bdice left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

jarmak-nv commented Jul 25, 2023

Uh oh!

bdice commented Jul 25, 2023 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

jarmak-nv commented Jul 25, 2023

Uh oh!

mmccarty commented Jul 25, 2023

Uh oh!

jarmak-nv commented Jul 25, 2023

Uh oh!

mmccarty commented Jul 25, 2023

Uh oh!

bdice commented Jul 25, 2023

Uh oh!

ajschmidt8 left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

ajschmidt8 commented Aug 1, 2023

Uh oh!

jakirkham left a comment

Choose a reason for hiding this comment

Uh oh!

ajschmidt8 left a comment

Choose a reason for hiding this comment

Uh oh!

bdice left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

bdice Aug 1, 2023

Choose a reason for hiding this comment

Uh oh!

ajschmidt8 Aug 1, 2023

Choose a reason for hiding this comment

Uh oh!

bdice Aug 1, 2023

Choose a reason for hiding this comment

Uh oh!

bdice Aug 1, 2023

Choose a reason for hiding this comment

Uh oh!

bdice Aug 1, 2023

Choose a reason for hiding this comment

Uh oh!

bdice Aug 1, 2023

Choose a reason for hiding this comment

Uh oh!

Uh oh!

jarmak-nv commented Aug 2, 2023

Uh oh!

ajschmidt8 commented Aug 2, 2023

Uh oh!

jakirkham commented Aug 4, 2023

jarmak-nv commented Jul 21, 2023 •

edited

Loading

jarmak-nv commented Jul 24, 2023 •

edited

Loading

bdice commented Jul 25, 2023 •

edited

Loading

jarmak-nv commented Aug 4, 2023 •

edited

Loading