[Hardware] [Feat] Setup platform dependent package installation

[tjtanaa]

PLEASE FILL IN THE PR DESCRIPTION HERE ENSURING ALL CHECKLIST ITEMS (AT THE BOTTOM) HAVE BEEN CONSIDERED.

Purpose

This PR implements the RFC #997 .
It fixes #909 as well.

The dockerfiles and documentation has been updated to reflect this.

Current behaviour

Default behaviour (CUDA)

pip install and uv pip install default will install CUDA configuration.

Other platforms

To install dependencies for other platform (CPU/ROCm/NPU/XPU), has to be installed using the following approach.

VLLM_OMNI_TARGET_DEVICE=<cpu,rocm,npu,xpu> pip install -e .

VLLM_OMNI_TARGET_DEVICE=<cpu,rocm,npu,xpu> uv pip install -e .

# using platform auto-detection
pip install -e . --no-build-isolation

uv pip install -e . --no-build-isolation

Test Plan

Validated locally on ROCm platform.

Test Result

Validated locally on ROCm platform that it works and we need --no-build-isolation flag.

CC @gcanlin @PopSoda2002 @david6666666

Simple bugfix

Fix the license deprecation warning

  /usr/local/lib/python3.12/dist-packages/setuptools/config/_apply_pyprojecttoml.py:82: SetuptoolsDeprecationWarning: `project.li
cense` as a TOML table is deprecated
  !!
          ********************************************************************************
          Please use a simple string containing a SPDX expression for `project.license`. You can also use `project.license-files`
. (Both options available on setuptools>=77.0.0).
          By 2026-Feb-18, you need to update your project and remove deprecated calls
          or your builds will no longer be supported.
          See https://packaging.python.org/en/latest/guides/writing-pyproject-toml/#license for details.
          ********************************************************************************
  !!
    corresp(dist, value, root_dir)
  /usr/local/lib/python3.12/dist-packages/setuptools/config/_apply_pyprojecttoml.py:61: SetuptoolsDeprecationWarning: License cla
ssifiers are deprecated.

---

Essential Elements of an Effective PR Description Checklist

• The purpose of the PR, such as "Fix some issue (link existing issues this PR will resolve)".
• The test plan, such as providing test command.
• The test results, such as pasting the results comparison before and after, or e2e results
• (Optional) The necessary documentation update, such as updating supported_models.md and examples for a new model.
• (Optional) Release notes update. If your change is user facing, please update the release notes draft.

BEFORE SUBMITTING, PLEASE READ https://github.com/vllm-project/vllm-omni/blob/main/CONTRIBUTING.md (anything written below this line will be removed by GitHub Actions)

[chatgpt-codex-connector]

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 7eb9c59675

ℹ️ About Codex in GitHub

Codex has been enabled to automatically review pull requests in this repo. Reviews are triggered when you

• Open a pull request for review
• Mark a draft as ready
• Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

When you sign up for Codex through ChatGPT, Codex can also answer questions or update the PR, like "@codex address that feedback".

[chatgpt-codex-connector]

Ensure requirements/*.txt are packaged in sdist/wheel

This setup logic reads requirements/{device}.txt at build time, but the new requirements/ directory isn’t included in package data (only vllm_omni.model_executor.stage_configs is listed) and there’s no MANIFEST.in, so source distributions will omit those files. In that case load_requirements returns an empty list and the built wheel installs with no runtime dependencies, which breaks pip install from sdist (e.g., when no wheel is available for a platform/Python). Consider adding the requirements files to the sdist/wheel inputs (MANIFEST.in or tool.setuptools package-data) or moving the lists back into pyproject.toml so they’re always available.

Useful? React with 👍 / 👎.

[tjtanaa]

To reviewer: please help me to ensure all three authors are there. Thanks

Signed-off-by: tjtanaa <[email protected]>
Co-authored-by: gcanlin <[email protected]>
Co-authored-by: PopSoda2002 <[email protected]>

[tjtanaa]

@david6666666 I have updated the PR, so that the fa-fwd is pinned.
@gcanlin and @faaany find there is no issue for npu and xpu.

[gcanlin]

CI failed. It looks like fa3-fwd didn't install successfully.

Exception: No module named 'fa3_fwd_interface'

Do we need --no-build-isolation on CUDA?

[yma11]

This note seems should be put above Installation of vLLM.

[tjtanaa]

Done. I was following the instruction from CUDA. Since we can move this before the installation of vLLM, I have put it in the gpu.md.

[tjtanaa]

CI failed. It looks like fa3-fwd didn't install successfully.

Exception: No module named 'fa3_fwd_interface'

Do we need --no-build-isolation on CUDA?

I have added that to the ci dockerfile, let's see.

[gcanlin]

CI failed. It looks like fa3-fwd didn't install successfully.

Exception: No module named 'fa3_fwd_interface'

Do we need --no-build-isolation on CUDA?

I have added that to the ci dockerfile, let's see.

My concern is that it will change the user behavior when installing. Is it acceptable?

My original point is we avoid to introduce the extra flag like [cuda], [npu]. But now, even if we unify installation way in different hardware, we have to add the longer flag. If we couldn't avoid it totally, maybe [cuda] style installation way should be considered.

[tjtanaa]

CI failed. It looks like fa3-fwd didn't install successfully.

Exception: No module named 'fa3_fwd_interface'

Do we need --no-build-isolation on CUDA?

I have added that to the ci dockerfile, let's see.

My concern is that it will change the user behavior when installing. Is it acceptable?

My original point is we avoid to introduce the extra flag like [cuda], [npu]. But now, even if we unify installation way in different hardware, we have to add the longer flag. If we couldn't avoid it totally, maybe [cuda] style installation way should be considered.

@gcanlin I think let's go for vllm-omni[cuda], vllm-omni[rocm]

I am currently also working on helping to setup release pipeline, both docker image and wheel.

I noticed that if we use setup.py approach, then when we are distributing on pypi, we cannot distribute wheel, we can only distribute Source tar.gz . The reason is that the dependency METADATA is determined at BUILD time, not INSTALL time.

If we go for vllm-omni[cuda], vllm-omni[rocm] , then this will work at INSTALL time.

Then for test-time dependencies, we can use the tag cuda-tests, rocm-tests, npu-tests etc.

[hsliuustc0106]

it's not optimal to require cuda users to use "pip install vllm-omni[cuda]"

[david6666666]

This PR will not be included in v0.14.0 for now; this will be explained in the documentation first, and a better method will be used later.

[Isotr0py]

Do we still need to include torch in building dependency? I think vLLM-Omni doesn't need to build pytorch kernels?

[tjtanaa]

We need to include torch if we want to make pip install . to install cuda dependencies as we are using if torch.version.cuda is not None: in the setup.py. Without torch in the pyproject.toml, if torch.version.cuda is not None: will be None and it will not install cuda dependencies.
I am thinking of proposing not to support CPU for now and always assume the fallback to be CUDA platform. Then we don't need to include torch into the pyproject.toml.

[tjtanaa]

I have updated the PR description to reflect the current state of the code. Please take alook.

[Isotr0py]

LGTM now

[PopSoda2002]

Good work!