[Bug] cuda 12.4 encounter undefined symbol: cuGreenCtxStreamCreate

[yuan-luo]

Checklist

• 1. I have searched related issues but cannot get the expected help.
• 2. The bug has not been fixed in the latest version.
• 3. Please note that if the bug-related issue you submitted lacks corresponding environment info and a minimal reproducible demo, it will be challenging for us to reproduce and resolve the issue, reducing the likelihood of receiving feedback.
• 4. If the issue you raised is not a bug but a question, please raise a discussion at https://github.com/sgl-project/sglang/discussions/new/choose Otherwise, it will be closed.
• 5. Please use English, otherwise it will be closed.

Describe the bug

After rebase the latest sglang, it encountered import error: common_ops.abi3.so: undefined symbol: cuGreenCtxStreamCreate.

Reproduction

sglang commit id: e1020dc

$python ./sgl-kernel/tests/test_per_token_quant_fp8.py
Traceback (most recent call last):
  File "/root/luoyuan.luo/workspace/SGLang/./sgl-kernel/tests/test_per_token_quant_fp8.py", line 6, in <module>
    from sgl_kernel import sgl_per_token_quant_fp8
  File "/opt/conda/lib/python3.10/site-packages/sgl_kernel/__init__.py", line 13, in <module>
    from sgl_kernel import common_ops
ImportError: /opt/conda/lib/python3.10/site-packages/sgl_kernel/common_ops.abi3.so: undefined symbol: cuGreenCtxStreamCreate

Environment

sglang: 0.4.9.post2
cuda: 12.4
torch: 2.7.1

H20

[yuan-luo]

It's probably related with #8090.

But it's weird, because it already has filter for CUDA>=12.5:

#if CUDA_VERSION >= 12050
std::vector<int64_t> create_greenctx_stream_direct(CUgreenCtx gctx[2]) {
  CUstream streamA;
  CUstream streamB;

  CUDA_DRV(cuGreenCtxStreamCreate(&streamA, gctx[0], CU_STREAM_NON_BLOCKING, 0));
  CUDA_DRV(cuGreenCtxStreamCreate(&streamB, gctx[1], CU_STREAM_NON_BLOCKING, 0));

  std::vector<int64_t> vec = {(int64_t)streamA, (int64_t)streamB};
  return vec;
}
#endif

......

#if CUDA_VERSION >= 12050
  stream_handles = create_greenctx_stream_direct(gctx);
#else
  stream_handles = create_greenctx_stream_fallback(gctx);
#endif

[yuan-luo]

I found the following FAQ, not sure it's related with ccache or not. Anyway, does not work for me.

- When encountering this error while compiling using ccache: `ImportError: /usr/local/lib/python3.10/dist-packages/sgl_kernel/common_ops.abi3.so: undefined symbol: _ZN3c108ListType3getERKNSt7__cxx1112basic_stringIcSt11char_traitsIcESaIcEEENS_4Type24SingletonOrSharedTypePtrIS9_EE`, please modify the last command as follows to resolve it: `python3 -m uv build --wheel -Cbuild-dir=build . --color=always --no-build-isolation` .

[yuan-luo]

Per discussed with @AniZpZ offline, might be related with #7649.

[AniZpZ]

It's probably related with #8090.

But it's weird, because it already has filter for CUDA>=12.5:

#if CUDA_VERSION >= 12050
std::vector<int64_t> create_greenctx_stream_direct(CUgreenCtx gctx[2]) {
  CUstream streamA;
  CUstream streamB;

  CUDA_DRV(cuGreenCtxStreamCreate(&streamA, gctx[0], CU_STREAM_NON_BLOCKING, 0));
  CUDA_DRV(cuGreenCtxStreamCreate(&streamB, gctx[1], CU_STREAM_NON_BLOCKING, 0));

  std::vector<int64_t> vec = {(int64_t)streamA, (int64_t)streamB};
  return vec;
}
#endif

......

#if CUDA_VERSION >= 12050
  stream_handles = create_greenctx_stream_direct(gctx);
#else
  stream_handles = create_greenctx_stream_fallback(gctx);
#endif

could you please make sure you use the same CUDA version in both compiling and running?
compiling in CUDA >= 12.5 and running in CUDA <= 12.4 may cause the problem

i will make a permanent fix in the code today

[yuan-luo]

I just tried CUDA 12.4 compile and CUDA 12.4 running has no problem.
I encountered this issue is CUDA12.8 compile and CUDA 12.4 running.
Hence, close this issue.