[OpenVINO] Support Qwen3VL model

[popovaan]

What does this PR do?

Conversion cmd-line for Qwen/Qwen3-VL-2B-Instruct:

optimum-cli export openvino -m Qwen/Qwen3-VL-2B-Instruct ./Qwen3-VL

Inference of Qwen/Qwen3-VL-2B-Instruct using OpenVINO backend:

from transformers import AutoTokenizer, AutoProcessor
from transformers.video_utils import load_video
from huggingface_hub import hf_hub_download
from optimum.intel.openvino import OVModelForVisualCausalLM

model_dir = "./Qwen3-VL/"

tokenizer = AutoTokenizer.from_pretrained(model_dir)
processor = AutoProcessor.from_pretrained(model_dir)
model = OVModelForVisualCausalLM.from_pretrained(model_dir)

# Prepare video input
video_path = hf_hub_download(
                repo_id="raushan-testing-hf/videos-test",
                filename="sample_demo_1.mp4",
                repo_type="dataset",
            )
input_video, _ = load_video(video_path, num_frames=10, backend="opencv")
question = "Why is this video funny?"
inputs = model.preprocess_inputs(processor=processor, text=question, video=input_video)

# Run inference
output_ids = model.generate(**inputs, max_new_tokens=10)
output_text = tokenizer.decode(output_ids[0])

print(output_text)

Continuation of #1452

Before submitting

• [N/A] This PR fixes a typo or improves the docs (you can dismiss the other checks if that's the case).
• Did you make sure to update the documentation with your changes?
• Did you write any new necessary tests?

[popovaan]

@IlyasMoutawwakil @echarlaix could you please review the PR again?

[echarlaix]

Thanks a lot for the PR @popovaan !!

[echarlaix]

could you extend on why we need this config (instead of adding any modifications to the qwen3_vl config, depending on _behavior) ?

[popovaan]

If we want to avoid a separate text config, the qwen3_vl config should support handling of past key values. Simple inheritance from TextDecoderWithPositionIdsOnnxConfig does not solve this issue, as the config still behaves like a visual-language config and the past key values functionality is not triggered for some reason.

I tried several approaches to work around this, but none of them work properly. If avoiding a separate config is crucial, I suggest making this in a separate PR, as this model is awaited by the customer and it seems to be not trivial.

[echarlaix]

not directly related to this PR but what if the models forward is called multiple times (not through generate) ?

[popovaan]

I suppose accuracy will be low, as rope_deltas need to be cleared before each forward.

[popovaan]

@echarlaix thanks for the review! I will apply remaining comments today.
@IlyasMoutawwakil could you please review this PR again too?

[popovaan]

@echarlaix @IlyasMoutawwakil please review this PR.

[echarlaix]

Thanks a lot @popovaan for the great addition !

[rkazants]

@popovaan, @echarlaix, I recommend to run slow tests after precommit tests are complete. Otherwise, multiple model download requests from both scopes (precommit and slow tests) can lead to network errors.