[NVIDIA] Blackwell Family

[johnnynunez]

https://developer.nvidia.com/blog/nvidia-blackwell-and-nvidia-cuda-12-9-introduce-family-specific-architecture-features/

cc @simon-mo

[gemini-code-assist]

Code Review

This pull request updates the CMake configuration to support the new NVIDIA Blackwell architecture family, aligning with CUDA 12.9+ features. The changes introduce new architecture codes and update the minimum required CUDA version for Blackwell-specific kernels. While this is a necessary update, I've identified a critical issue with how the new architecture suffixes are handled, which will likely cause build failures. Additionally, there's a potential regression for users on CUDA 12.8 that should be addressed.

[pytorch-bot]

No ciflow labels are configured for this repo.
For information on how to enable CIFlow bot see this wiki

[pytorch-bot]

No ciflow labels are configured for this repo.
For information on how to enable CIFlow bot see this wiki

[pytorch-bot]

No ciflow labels are configured for this repo.
For information on how to enable CIFlow bot see this wiki

[hmellor]

Thanks for the PR! Could you please add some more information to the PR description about what this enables, for example:

• Which new hardware does this enable?
• What effect does it have on the wheel size?

[johnnynunez]

Thanks for the PR! Could you please add some more information to the PR description about what this enables, for example:

• Which new hardware does this enable?
• What effect does it have on the wheel size?

This enable correctly:
Spark
Thor
Gb300
Without increase binaries size.

[mgoin]

@johnnynunez do you want to update cutlass separately? 4.2 tag hasn't been made yet

[2025-09-13T21:11:55Z] #25 13.59 CMake Error at cutlass-subbuild/cutlass-populate-prefix/tmp/cutlass-populate-gitclone.cmake:61 (message):
[2025-09-13T21:11:55Z] #25 13.59   Failed to checkout tag: 'v4.2.0'

[johnnynunez]

@johnnynunez do you want to update cutlass separately? 4.2 tag hasn't been made yet

[2025-09-13T21:11:55Z] #25 13.59 CMake Error at cutlass-subbuild/cutlass-populate-prefix/tmp/cutlass-populate-gitclone.cmake:61 (message):
[2025-09-13T21:11:55Z] #25 13.59   Failed to checkout tag: 'v4.2.0'

hello, cutlass v4.2.0 is out today!

NVIDIA/cutlass@6a35b4d

[johnnynunez]

@mgoin it is out now: https://github.com/NVIDIA/cutlass/releases/tag/v4.2.0

[johnnynunez]

cc @Aidyn-A could you review this PR?

[Aidyn-A]

I have left some of the comments. I would like you to test the build, make sure it is 100% successful and double test all the kernels on all machines. I doubt that those CUTLASS kernels need extra arch flags, as CUTLASS kernels tend to be arch specific (e.g. cutlass kernel written for sm_100 can be unusable on sm_120 or sm_110). Please keep in mind that not all arch conditional instructions can be replaced with family conditional.

Additionally, I have not dig into it, but I am pretty sure, the function cuda_archs_loose_intersection needs to be modified for family conditional flags like it does for arch conditional:

vllm/cmake/utils.cmake

Lines 315 to 325 in f4cd80f

	set(_CUDA_ARCHS)
	foreach(_arch ${_SRC_CUDA_ARCHS})
	if(_arch MATCHES "\\a$")
	list(REMOVE_ITEM _SRC_CUDA_ARCHS "${_arch}")
	string(REPLACE "a" "" _base "${_arch}")
	if ("${_base}" IN_LIST TGT_CUDA_ARCHS)
	list(REMOVE_ITEM _TGT_CUDA_ARCHS "${_base}")
	list(APPEND _CUDA_ARCHS "${_arch}")
	endif()
	endif()
	endforeach()

Lastly. The regular arch flags and their family conditional counterparts are conflicting with each other. For example:

nvcc code.cu -gencode arch=compute_120,code=sm_120 -gencode arch=compute_120f,code=sm_120f

will end up failing:

nvcc fatal   : The same GPU code (`sm_120`) generated for non family-specific and family-specific GPU arch

That being said, here it is important to be very precise on what flags to apply. I would modify cuda_archs_loose_intersection to exclude the basic arch flag if family conditional is being passed.

[jasl]

https://github.com/vllm-project/vllm/pull/24673/files#diff-c1cdf2ae7a3604efb26f96752f519b9d86fe38fd05dacca78301c522fb20d819R256-R262

Does cutlass_moe_mm_sm100 work on SM120?

[DrStone1971]

https://github.com/vllm-project/vllm/pull/24673/files#diff-c1cdf2ae7a3604efb26f96752f519b9d86fe38fd05dacca78301c522fb20d819R256-R262

Does cutlass_moe_mm_sm100 work on SM120?

is strange this code:

#if defined CUDA_VERSION
if (cuda_device_capability >= 100) {
return CUDA_VERSION >= 12080;
}
if (cuda_device_capability >= 90) {
return CUDA_VERSION >= 12030;
}
#endif

i have need to analyze and test.

@jasl have a simple code for testing if capavility is active and run ?

[jasl]

https://github.com/vllm-project/vllm/pull/24673/files#diff-c1cdf2ae7a3604efb26f96752f519b9d86fe38fd05dacca78301c522fb20d819R256-R262

@DrStone71

My testing machine is an x86 with RTX Pro 6000 (SM120)

I'm not sure this kernel labeled _sm100 will work on SM120.
Looking at the CMakeList https://github.com/vllm-project/vllm/blob/main/CMakeLists.txt#L678-L696 it only works for SM100.
So I guess we need an upper bound check here

And cutlass_moe_mm_sm100 causes trouble for me when running vllm, I got importError: /home/jasl/.venv/lib/python3.12/site-packages/vllm/_C.abi3.so: undefined symbol: _Z20cutlass_moe_mm_sm100RN2at6TensorERKS0_S3_S3_S3_S3_S3_S3_S3_S3_bb

I have a patch that works for me jasl@b52d720
(I'm not sure this is correct)

Because SM120 doesn't contain SM100 features, it's difficult to disable the compilation for non-SM100 platforms

[jasl]

With my additional patch, vllm@CUDA 13 is working on my RTX Pro 6000

I'm testing on my Thor now.

[DrStone1971]

is need a check with Cuda 13 and Sm_120 ?

i use a physical machine, there is a need of particular environment for testing (sw version or pip sw ?)

DrStone71

[johnnynunez]

is need a check with Cuda 13 and Sm_120 ?

i use a physical machine, there is a need of particular environment for testing (sw version or pip sw ?)

DrStone71

Test MoE, gptoss etc

[youkaichao]

why is it 13.0 rather than 12.9?

[youkaichao]

@johnnynunez