[Feature] support multimodal inputs with multiple requests

[Gaohan123]

Purpose

This PR supports multimodal inputs for qwen2.5-omni. The modalities include text, image, audio, video and their combinations. The implementation partly refers to PR #57 in vllm-omni and PR#26634 in vllm main branch.

Test Plan

Here I both tests offline and online inference. First please follow README.md in root repo to finish installation.

Online inference

vllm serve Qwen/Qwen2.5-Omni-7B --omni --port 8091

Follow README.md in examples/online_serving

Offline inference

Follow README.md in examples/offline_inference/qwen2_5_omni

Test Result

---

Essential Elements of an Effective PR Description Checklist

• The purpose of the PR, such as "Fix some issue (link existing issues this PR will resolve)".
• The test plan, such as providing test command.
• The test results, such as pasting the results comparison before and after, or e2e results
• (Optional) The necessary documentation update, such as updating supported_models.md and examples for a new model.
• (Optional) Release notes update. If your change is user facing, please update the release notes draft.

BEFORE SUBMITTING, PLEASE READ https://github.com/vllm-project/vllm-omni/blob/main/CONTRIBUTING.md (anything written below this line will be removed by GitHub Actions)

[hsliuustc0106]

please refactor the models folder for each model, all qwen2_5_omni related files should be placed in an independent folder

[Gaohan123]

fixed

[hsliuustc0106]

could this vision.py be merged into utils.py?

[Gaohan123]

The file aligns with the path in vllm main

[hsliuustc0106]

def split_list_into_ranges_fast(lst: torch.Tensor, interval: int) -> list[list[int]]:
    if lst.numel() == 0:
        return []
    
    # Move to CPU and convert to list once (High Speedup)
    # using .item() inside a loop is very slow.
    data_list = lst.detach().cpu().tolist()
    
    # Calculate max on the list or tensor (Tensor max is fast enough)
    max_val = int(torch.max(lst).item())
    
    # Pre-allocate buckets
    ranges: list[list[int]] = [[] for _ in range((max_val // interval) + 1)]
    
    for num in data_list:
        index = int(num // interval)
        ranges[index].append(num)
        
    return ranges

[Gaohan123]

fixed. Thanks for advice.

[hsliuustc0106]

comments from gemini:

here is a critical performance issue: your _extract_audio_from_video_async method is defined as async, but it performs blocking synchronous I/O (file downloads, librosa.load, file writes).

In an async framework like vLLM, this will freeze the entire inference engine (heartbeats, other requests, token generation) while librosa processes the video.

[Gaohan123]

fixed.

[tzhouam]

lgtm, merging