[OpenVINO] Add support for Mistral3 by kyoui-dev · Pull Request #1627 · huggingface/optimum-intel

kyoui-dev · 2026-03-02T10:28:46Z

What does this PR do?

Conversion cmd-line for mistralai/Mistral-Small-3.1-24B-Instruct-2503:

optimum-cli export openvino -m mistralai/Mistral-Small-3.1-24B-Instruct-2503 ./Mistral-Small-3.1-24B --task image-text-to-text

Inference of mistralai/Mistral-Small-3.1-24B-Instruct-2503 using OpenVINO backend:

from transformers import AutoTokenizer, AutoProcessor
from transformers.image_utils import load_image
from huggingface_hub import hf_hub_download
from optimum.intel.openvino import OVModelForVisualCausalLM


model_dir = "./Mistral-Small-3.1-24B"

tokenizer = AutoTokenizer.from_pretrained(model_dir)
processor = AutoProcessor.from_pretrained(model_dir)
model = OVModelForVisualCausalLM.from_pretrained(model_dir)

# Prepare image input
image_path = hf_hub_download(
                repo_id="raushan-testing-hf/images_test",
                filename="australia.jpg",
                repo_type="dataset",
        )
image_input = load_image(image_path)
question = "Describe this image."
inputs = model.preprocess_inputs(processor=processor, text=question, image=image_input)

# Run inference
output_ids = model.generate(**inputs, max_new_tokens=10)
output_text = tokenizer.decode(output_ids[0])

print(output_text)

Fixes #1338

Before submitting

N/A This PR fixes a typo or improves the docs (you can dismiss the other checks if that's the case).
Did you make sure to update the documentation with your changes?
Did you write any new necessary tests?

kyoui-dev · 2026-03-02T10:31:03Z

Hi @popovaan,

Could you please take a look at my PR?

Thank you!

popovaan · 2026-03-02T14:57:27Z

Please add tests to the PR and use a local path for now, until we have a published tiny model.

rkazants

please add tests and create tiny model for it

kyoui-dev · 2026-03-03T16:34:20Z

Hi @popovaan and @rkazants,

I've added the tests and created a tiny model locally for now.

Here is the script I used to create the tiny model:

import os
from transformers import (
    AutoConfig,
    AutoModelForImageTextToText,
    AutoProcessor,
    AutoTokenizer,
)

model_id = "mistralai/Mistral-Small-3.1-24B-Instruct-2503"
config = AutoConfig.from_pretrained(model_id)

config.text_config.num_hidden_layers = 2
config.text_config.hidden_size = 8
config.text_config.intermediate_size = 64
config.text_config.num_attention_heads = 8
config.text_config.num_key_value_heads = 4
config.text_config.head_dim = 32

config.vision_config.num_hidden_layers = 2
config.vision_config.hidden_size = 128
config.vision_config.intermediate_size = 64
config.vision_config.num_attention_heads = 4
config.vision_config.head_dim = 32

model = AutoModelForImageTextToText.from_config(config)
tokenizer = AutoTokenizer.from_pretrained(model_id)
processor = AutoProcessor.from_pretrained(model_id)

output_dir = "./tiny-random-mistral3"
os.makedirs(output_dir, exist_ok=True)
model.save_pretrained(output_dir)
tokenizer.save_pretrained(output_dir)
processor.save_pretrained(output_dir)

I’d appreciate it if you could review the updates.

Copilot

Pull request overview

Adds OpenVINO export + inference support for the new mistral3 (Mistral-Small-3.1) visual language model family, integrating it into the OpenVINO VLM export pipeline and test matrices.

Changes:

Introduces a Mistral3-specific OVModelForVisualCausalLM implementation and export-time patchers to handle non-traceable vision components.
Registers new OpenVINO export configs/behaviors for Mistral3, including a dedicated multi_modal_projector submodel.
Extends OpenVINO test coverage + documentation to include the mistral3 architecture (gated by transformers>=4.50.0).

Reviewed changes

Copilot reviewed 11 out of 11 changed files in this pull request and generated 2 comments.

Show a summary per file

File	Description
`optimum/intel/openvino/modeling_visual_language.py`	Adds `_OVMistral3ForCausalLM` runtime logic (vision embeddings + vision/text merge + preprocessing) and registers it in the architecture mapping.
`optimum/exporters/openvino/model_patcher.py`	Adds Mistral3-specific forward patchers to make vision embedding + projector exportable to OV IR.
`optimum/exporters/openvino/model_configs.py`	Registers Mistral3 in TasksManager custom loading, adds Mistral3 OpenVINO config + `multi_modal_projector` export config and dummy input generator.
`optimum/exporters/openvino/utils.py`	Marks `mistral3` as a multi-submodel VLM for OpenVINO export.
`tests/openvino/utils_tests.py`	Adds `mistral3` model fixture + expected INT8 node counts for its exported submodels.
`tests/openvino/test_seq2seq.py`	Adds `mistral3` to supported visual-causal-lm integration tests for `transformers>=4.50.0`.
`tests/openvino/test_export.py`	Adds `mistral3` to supported export architectures for `transformers>=4.50.0`.
`tests/openvino/test_exporters_cli.py`	Adds CLI exporter coverage for (`image-text-to-text`, `mistral3`) for `transformers>=4.50.0`.
`tests/openvino/test_quantization.py`	Adds `mistral3` to auto-compression architecture list for `transformers>=4.50.0`.
`tests/openvino/test_genai.py`	Allows `mistral3` to be routed through `AutoModelForImageTextToText` in the GenAI pipeline helper.
`docs/source/openvino/models.mdx`	Documents `Mistral3` as a supported architecture.

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

tests/openvino/utils_tests.py

popovaan · 2026-03-04T09:14:50Z

Hi @popovaan and @rkazants,

I've added the tests and created a tiny model locally for now.

Here is the script I used to create the tiny model:

import os
from transformers import (
    AutoConfig,
    AutoModelForImageTextToText,
    AutoProcessor,
    AutoTokenizer,
)

model_id = "mistralai/Mistral-Small-3.1-24B-Instruct-2503"
config = AutoConfig.from_pretrained(model_id)

config.text_config.num_hidden_layers = 2
config.text_config.hidden_size = 8
config.text_config.intermediate_size = 64
config.text_config.num_attention_heads = 8
config.text_config.num_key_value_heads = 4
config.text_config.head_dim = 32

config.vision_config.num_hidden_layers = 2
config.vision_config.hidden_size = 128
config.vision_config.intermediate_size = 64
config.vision_config.num_attention_heads = 4
config.vision_config.head_dim = 32

model = AutoModelForImageTextToText.from_config(config)
tokenizer = AutoTokenizer.from_pretrained(model_id)
processor = AutoProcessor.from_pretrained(model_id)

output_dir = "./tiny-random-mistral3"
os.makedirs(output_dir, exist_ok=True)
model.save_pretrained(output_dir)
tokenizer.save_pretrained(output_dir)
processor.save_pretrained(output_dir)

I’d appreciate it if you could review the updates.

Thanks for sharing this script. I’ve published this model on Hugging Face, please use it in the tests in your PR.

https://huggingface.co/optimum-intel-internal-testing/tiny-random-mistral3

kyoui-dev · 2026-03-04T09:45:28Z

Thanks! I've updated it. Please let me know if anything else is needed.

echarlaix

thanks for the addition @kyoui-dev !

optimum/exporters/openvino/model_configs.py

docs/source/openvino/models.mdx

optimum/exporters/openvino/model_configs.py

optimum/exporters/openvino/model_patcher.py

optimum/exporters/openvino/model_configs.py

rkazants · 2026-03-05T06:17:20Z

@kyoui-dev, please double-check that all newly added tests are passing locally on your machine:

kyoui-dev · 2026-03-07T05:38:33Z

Hi @echarlaix, thanks for the review!

I've addressed all the comments you left. Could you take another look when you get a chance?

kyoui-dev · 2026-03-07T06:55:06Z

Hi @rkazants,

All newly added tests are passing on my local machine. Could you take a look?

(optimum-intel) kyoui-dev@kyoui-MacBookPro optimum-intel % pytest \
  tests/openvino/test_export.py \
  tests/openvino/test_exporters_cli.py \
  tests/openvino/test_quantization.py \
  tests/openvino/test_seq2seq.py \
  -k "mistral3 or test_exporters_cli_25_image_text_to_text or test_ovmodel_load_with_compressed_weights_17 or test_ovmodel_load_with_uncompressed_weights_17" \
  -v
============================================================= test session starts ==============================================================
platform darwin -- Python 3.13.7, pytest-7.4.4, pluggy-1.6.0 -- /Users/kyoui-dev/Desktop/GitHub/optimum-intel/.venv/bin/python3
cachedir: .pytest_cache
rootdir: /Users/kyoui-dev/Desktop/GitHub/optimum-intel
configfile: pyproject.toml
plugins: anyio-4.12.1
collected 633 items / 626 deselected / 7 selected                                                                                              

tests/openvino/test_export.py::ExportModelTest::test_export_27_mistral3 PASSED                                                           [ 14%]
tests/openvino/test_exporters_cli.py::OVCLIExportTestCase::test_exporters_cli_25_image_text_to_text PASSED                               [ 28%]
tests/openvino/test_quantization.py::OVWeightCompressionTest::test_ovmodel_load_with_compressed_weights_17 PASSED                        [ 42%]
tests/openvino/test_quantization.py::OVWeightCompressionTest::test_ovmodel_load_with_uncompressed_weights_17 PASSED                      [ 57%]
tests/openvino/test_seq2seq.py::OVModelForVisualCausalLMIntegrationTest::test_compare_to_transformers_03_mistral3 PASSED                 [ 71%]
tests/openvino/test_seq2seq.py::OVModelForVisualCausalLMIntegrationTest::test_generate_utils_03_mistral3 PASSED                          [ 85%]
tests/openvino/test_seq2seq.py::OVModelForVisualCausalLMIntegrationTest::test_model_can_be_loaded_after_saving_03_mistral3 PASSED        [100%]

=============================================================== warnings summary ===============================================================
<frozen importlib._bootstrap>:488
  <frozen importlib._bootstrap>:488: DeprecationWarning: builtin type SwigPyPacked has no __module__ attribute

<frozen importlib._bootstrap>:488
  <frozen importlib._bootstrap>:488: DeprecationWarning: builtin type SwigPyObject has no __module__ attribute

.venv/lib/python3.13/site-packages/torch/jit/_script.py:1480
  /Users/kyoui-dev/Desktop/GitHub/optimum-intel/.venv/lib/python3.13/site-packages/torch/jit/_script.py:1480: DeprecationWarning: `torch.jit.script` is deprecated. Please switch to `torch.compile` or `torch.export`.
    warnings.warn(

tests/openvino/test_export.py::ExportModelTest::test_export_27_mistral3
tests/openvino/test_quantization.py::OVWeightCompressionTest::test_ovmodel_load_with_compressed_weights_17
tests/openvino/test_quantization.py::OVWeightCompressionTest::test_ovmodel_load_with_uncompressed_weights_17
tests/openvino/test_seq2seq.py::OVModelForVisualCausalLMIntegrationTest::test_compare_to_transformers_03_mistral3
tests/openvino/test_seq2seq.py::OVModelForVisualCausalLMIntegrationTest::test_generate_utils_03_mistral3
tests/openvino/test_seq2seq.py::OVModelForVisualCausalLMIntegrationTest::test_model_can_be_loaded_after_saving_03_mistral3
  /Users/kyoui-dev/Desktop/GitHub/optimum-intel/.venv/lib/python3.13/site-packages/optimum/exporters/base.py:151: FutureWarning: functools.partial will be a method descriptor in future Python versions; wrap it in staticmethod() if you want to preserve the old behavior
    self._normalized_config = self.NORMALIZED_CONFIG_CLASS(self._config)

tests/openvino/test_export.py::ExportModelTest::test_export_27_mistral3
tests/openvino/test_quantization.py::OVWeightCompressionTest::test_ovmodel_load_with_compressed_weights_17
tests/openvino/test_quantization.py::OVWeightCompressionTest::test_ovmodel_load_with_uncompressed_weights_17
tests/openvino/test_seq2seq.py::OVModelForVisualCausalLMIntegrationTest::test_compare_to_transformers_03_mistral3
tests/openvino/test_seq2seq.py::OVModelForVisualCausalLMIntegrationTest::test_generate_utils_03_mistral3
tests/openvino/test_seq2seq.py::OVModelForVisualCausalLMIntegrationTest::test_model_can_be_loaded_after_saving_03_mistral3
  /Users/kyoui-dev/Desktop/GitHub/optimum-intel/optimum/exporters/openvino/model_configs.py:1772: FutureWarning: functools.partial will be a method descriptor in future Python versions; wrap it in staticmethod() if you want to preserve the old behavior
    InputEmbedOpenvVINOConfig.NORMALIZED_CONFIG_CLASS = internal_export_config.NORMALIZED_CONFIG_CLASS

tests/openvino/test_export.py: 4 warnings
tests/openvino/test_quantization.py: 8 warnings
tests/openvino/test_seq2seq.py: 12 warnings
  /Users/kyoui-dev/Desktop/GitHub/optimum-intel/.venv/lib/python3.13/site-packages/torch/jit/_trace.py:1000: DeprecationWarning: `torch.jit.trace` is deprecated. Please switch to `torch.compile` or `torch.export`.
    warnings.warn(

tests/openvino/test_export.py: 8 warnings
tests/openvino/test_quantization.py: 16 warnings
tests/openvino/test_seq2seq.py: 24 warnings
  /Users/kyoui-dev/Desktop/GitHub/optimum-intel/.venv/lib/python3.13/site-packages/torch/jit/_trace.py:1139: DeprecationWarning: `torch.jit.trace_method` is deprecated. Please switch to `torch.compile` or `torch.export`.
    warnings.warn(

tests/openvino/test_export.py::ExportModelTest::test_export_27_mistral3
tests/openvino/test_quantization.py::OVWeightCompressionTest::test_ovmodel_load_with_compressed_weights_17
tests/openvino/test_quantization.py::OVWeightCompressionTest::test_ovmodel_load_with_uncompressed_weights_17
tests/openvino/test_seq2seq.py::OVModelForVisualCausalLMIntegrationTest::test_compare_to_transformers_03_mistral3
tests/openvino/test_seq2seq.py::OVModelForVisualCausalLMIntegrationTest::test_generate_utils_03_mistral3
tests/openvino/test_seq2seq.py::OVModelForVisualCausalLMIntegrationTest::test_model_can_be_loaded_after_saving_03_mistral3
  /Users/kyoui-dev/Desktop/GitHub/optimum-intel/.venv/lib/python3.13/site-packages/transformers/cache_utils.py:132: TracerWarning: Converting a tensor to a Python boolean might cause the trace to be incorrect. We can't record the data flow of Python values, so this value will be treated as a constant in the future. This means that the trace might not generalize to other inputs!
    if not self.is_initialized or self.keys.numel() == 0:

tests/openvino/test_export.py::ExportModelTest::test_export_27_mistral3
tests/openvino/test_quantization.py::OVWeightCompressionTest::test_ovmodel_load_with_compressed_weights_17
tests/openvino/test_quantization.py::OVWeightCompressionTest::test_ovmodel_load_with_uncompressed_weights_17
tests/openvino/test_seq2seq.py::OVModelForVisualCausalLMIntegrationTest::test_compare_to_transformers_03_mistral3
tests/openvino/test_seq2seq.py::OVModelForVisualCausalLMIntegrationTest::test_generate_utils_03_mistral3
tests/openvino/test_seq2seq.py::OVModelForVisualCausalLMIntegrationTest::test_model_can_be_loaded_after_saving_03_mistral3
  /Users/kyoui-dev/Desktop/GitHub/optimum-intel/.venv/lib/python3.13/site-packages/transformers/masking_utils.py:207: TracerWarning: Converting a tensor to a Python boolean might cause the trace to be incorrect. We can't record the data flow of Python values, so this value will be treated as a constant in the future. This means that the trace might not generalize to other inputs!
    if (padding_length := kv_length + kv_offset - attention_mask.shape[-1]) > 0:

tests/openvino/test_export.py::ExportModelTest::test_export_27_mistral3
tests/openvino/test_quantization.py::OVWeightCompressionTest::test_ovmodel_load_with_compressed_weights_17
tests/openvino/test_quantization.py::OVWeightCompressionTest::test_ovmodel_load_with_uncompressed_weights_17
tests/openvino/test_seq2seq.py::OVModelForVisualCausalLMIntegrationTest::test_compare_to_transformers_03_mistral3
tests/openvino/test_seq2seq.py::OVModelForVisualCausalLMIntegrationTest::test_generate_utils_03_mistral3
tests/openvino/test_seq2seq.py::OVModelForVisualCausalLMIntegrationTest::test_model_can_be_loaded_after_saving_03_mistral3
  /Users/kyoui-dev/Desktop/GitHub/optimum-intel/optimum/exporters/openvino/model_patcher.py:233: TracerWarning: torch.tensor results are registered as constants in the trace. You can safely ignore this warning if you use this function to create tensors out of constant variables that would be the same every time you call this function. In any other case, this might cause the trace to be incorrect.
    torch.tensor(0.0, device=mask.device, dtype=dtype),

tests/openvino/test_export.py::ExportModelTest::test_export_27_mistral3
tests/openvino/test_quantization.py::OVWeightCompressionTest::test_ovmodel_load_with_compressed_weights_17
tests/openvino/test_quantization.py::OVWeightCompressionTest::test_ovmodel_load_with_uncompressed_weights_17
tests/openvino/test_seq2seq.py::OVModelForVisualCausalLMIntegrationTest::test_compare_to_transformers_03_mistral3
tests/openvino/test_seq2seq.py::OVModelForVisualCausalLMIntegrationTest::test_generate_utils_03_mistral3
tests/openvino/test_seq2seq.py::OVModelForVisualCausalLMIntegrationTest::test_model_can_be_loaded_after_saving_03_mistral3
  /Users/kyoui-dev/Desktop/GitHub/optimum-intel/optimum/exporters/openvino/model_patcher.py:234: TracerWarning: torch.tensor results are registered as constants in the trace. You can safely ignore this warning if you use this function to create tensors out of constant variables that would be the same every time you call this function. In any other case, this might cause the trace to be incorrect.
    torch.tensor(torch.finfo(torch.float16).min, device=mask.device, dtype=dtype),

tests/openvino/test_export.py::ExportModelTest::test_export_27_mistral3
tests/openvino/test_quantization.py::OVWeightCompressionTest::test_ovmodel_load_with_compressed_weights_17
tests/openvino/test_quantization.py::OVWeightCompressionTest::test_ovmodel_load_with_uncompressed_weights_17
tests/openvino/test_seq2seq.py::OVModelForVisualCausalLMIntegrationTest::test_compare_to_transformers_03_mistral3
tests/openvino/test_seq2seq.py::OVModelForVisualCausalLMIntegrationTest::test_generate_utils_03_mistral3
tests/openvino/test_seq2seq.py::OVModelForVisualCausalLMIntegrationTest::test_model_can_be_loaded_after_saving_03_mistral3
  /Users/kyoui-dev/Desktop/GitHub/optimum-intel/.venv/lib/python3.13/site-packages/transformers/integrations/sdpa_attention.py:81: TracerWarning: Converting a tensor to a Python boolean might cause the trace to be incorrect. We can't record the data flow of Python values, so this value will be treated as a constant in the future. This means that the trace might not generalize to other inputs!
    is_causal = query.shape[2] > 1 and attention_mask is None and getattr(module, "is_causal", True)

tests/openvino/test_export.py::ExportModelTest::test_export_27_mistral3
tests/openvino/test_quantization.py::OVWeightCompressionTest::test_ovmodel_load_with_compressed_weights_17
tests/openvino/test_quantization.py::OVWeightCompressionTest::test_ovmodel_load_with_uncompressed_weights_17
tests/openvino/test_seq2seq.py::OVModelForVisualCausalLMIntegrationTest::test_compare_to_transformers_03_mistral3
tests/openvino/test_seq2seq.py::OVModelForVisualCausalLMIntegrationTest::test_generate_utils_03_mistral3
tests/openvino/test_seq2seq.py::OVModelForVisualCausalLMIntegrationTest::test_model_can_be_loaded_after_saving_03_mistral3
  /Users/kyoui-dev/Desktop/GitHub/optimum-intel/.venv/lib/python3.13/site-packages/transformers/models/pixtral/modeling_pixtral.py:482: TracerWarning: Iterating over a tensor might cause the trace to be incorrect. Passing a tensor of different shape won't change the number of iterations executed (and might lead to errors or silently give incorrect results).
    for embed, size in zip(patch_embeds, image_sizes)

tests/openvino/test_export.py::ExportModelTest::test_export_27_mistral3
tests/openvino/test_quantization.py::OVWeightCompressionTest::test_ovmodel_load_with_compressed_weights_17
tests/openvino/test_quantization.py::OVWeightCompressionTest::test_ovmodel_load_with_uncompressed_weights_17
tests/openvino/test_seq2seq.py::OVModelForVisualCausalLMIntegrationTest::test_compare_to_transformers_03_mistral3
tests/openvino/test_seq2seq.py::OVModelForVisualCausalLMIntegrationTest::test_generate_utils_03_mistral3
tests/openvino/test_seq2seq.py::OVModelForVisualCausalLMIntegrationTest::test_model_can_be_loaded_after_saving_03_mistral3
  /Users/kyoui-dev/Desktop/GitHub/optimum-intel/.venv/lib/python3.13/site-packages/transformers/models/pixtral/modeling_pixtral.py:429: TracerWarning: torch.tensor results are registered as constants in the trace. You can safely ignore this warning if you use this function to create tensors out of constant variables that would be the same every time you call this function. In any other case, this might cause the trace to be incorrect.
    block_end_idx = torch.tensor(patch_embeds_list).cumsum(-1)

tests/openvino/test_export.py::ExportModelTest::test_export_27_mistral3
tests/openvino/test_quantization.py::OVWeightCompressionTest::test_ovmodel_load_with_compressed_weights_17
tests/openvino/test_quantization.py::OVWeightCompressionTest::test_ovmodel_load_with_uncompressed_weights_17
tests/openvino/test_seq2seq.py::OVModelForVisualCausalLMIntegrationTest::test_compare_to_transformers_03_mistral3
tests/openvino/test_seq2seq.py::OVModelForVisualCausalLMIntegrationTest::test_generate_utils_03_mistral3
tests/openvino/test_seq2seq.py::OVModelForVisualCausalLMIntegrationTest::test_model_can_be_loaded_after_saving_03_mistral3
  /Users/kyoui-dev/Desktop/GitHub/optimum-intel/.venv/lib/python3.13/site-packages/transformers/models/pixtral/modeling_pixtral.py:430: TracerWarning: torch.tensor results are registered as constants in the trace. You can safely ignore this warning if you use this function to create tensors out of constant variables that would be the same every time you call this function. In any other case, this might cause the trace to be incorrect.
    block_start_idx = torch.tensor([0] + patch_embeds_list[:-1]).cumsum(-1)

tests/openvino/test_export.py::ExportModelTest::test_export_27_mistral3
tests/openvino/test_quantization.py::OVWeightCompressionTest::test_ovmodel_load_with_compressed_weights_17
tests/openvino/test_quantization.py::OVWeightCompressionTest::test_ovmodel_load_with_uncompressed_weights_17
tests/openvino/test_seq2seq.py::OVModelForVisualCausalLMIntegrationTest::test_compare_to_transformers_03_mistral3
tests/openvino/test_seq2seq.py::OVModelForVisualCausalLMIntegrationTest::test_generate_utils_03_mistral3
tests/openvino/test_seq2seq.py::OVModelForVisualCausalLMIntegrationTest::test_model_can_be_loaded_after_saving_03_mistral3
  /Users/kyoui-dev/Desktop/GitHub/optimum-intel/.venv/lib/python3.13/site-packages/transformers/models/pixtral/modeling_pixtral.py:431: TracerWarning: Iterating over a tensor might cause the trace to be incorrect. Passing a tensor of different shape won't change the number of iterations executed (and might lead to errors or silently give incorrect results).
    for start, end in zip(block_start_idx, block_end_idx):

-- Docs: https://docs.pytest.org/en/stable/how-to/capture-warnings.html
========================================= 7 passed, 626 deselected, 141 warnings in 154.94s (0:02:34) ==========================================

HuggingFaceDocBuilderDev · 2026-03-09T10:04:54Z

The docs for this PR live here. All of your documentation changes will be reflected on that endpoint. The docs are available until 30 days after the last update.

popovaan · 2026-03-11T14:29:37Z

This error seem to be related with this PR, please take a look:
https://github.com/huggingface/optimum-intel/actions/runs/22792915683/job/66592942836?pr=1627

Also please fix the code style with following set of commands:

ruff check --config pyproject.toml --fix .
ruff format --config pyproject.toml .
ruff check --config pyproject.toml .
ruff format --check --config pyproject.toml .
 
black .
black --check .

kyoui-dev · 2026-03-13T08:23:01Z

Hi @popovaan,

Thank you for letting me know. I've fixed the test and the code style with ruff and black. Could you check?

rkazants · 2026-03-13T12:20:34Z

@kyoui-dev, please check tests locally before running our CI.

kyoui-dev · 2026-03-14T03:36:10Z

@rkazants, I just ran the related tests locally and they passed on my end. Please let me know if anything else is needed.

tests/openvino/test_quantization.py

popovaan · 2026-03-17T15:06:35Z

Could you please locally run OpenVINO GenAI WhoWhatBenchmark tool to check the accuracy of the full model (not the tiny one) and share the results?
https://github.com/openvinotoolkit/openvino.genai/tree/master/tools/who_what_benchmark

Here is the instruction: https://github.com/openvinotoolkit/openvino.genai/blob/master/tools/who_what_benchmark/README.md#compare-visual-language-models-with-image-inputs-vlms

popovaan · 2026-03-19T08:21:13Z

Could you please locally run OpenVINO GenAI WhoWhatBenchmark tool to check the accuracy of the full model (not the tiny one) and share the results? https://github.com/openvinotoolkit/openvino.genai/tree/master/tools/who_what_benchmark

Here is the instruction: https://github.com/openvinotoolkit/openvino.genai/blob/master/tools/who_what_benchmark/README.md#compare-visual-language-models-with-image-inputs-vlms

Please use --weight-format fp16 during model conversion to avoid quantization.

kyoui-dev · 2026-03-19T12:08:39Z

Hi @popovaan,

I’ll run it locally and share the results once it’s done. Thanks for the guidance!

kyoui-dev · 2026-03-22T23:36:52Z

@popovaan,

I'm running the evaluation pipeline, but the smallest model for Mistral3 is 24B, so it's taking quite a long time on my local machine. Is there another way I could run this? If not, I can still run it locally — it’ll just take a while.

Evaluate pipeline:   0%|                                                                                                                                              | 0/24 [00:00<?, ?it/s]Setting `pad_token_id` to `eos_token_id`:2 for open-end generation.
Evaluate pipeline:   4%|█████▎                                                                                                                        | 1/24 [2:47:16<64:07:11, 10036.15s/it]Setting `pad_token_id` to `eos_token_id`:2 for open-end generation.
Evaluate pipeline:   8%|██████████▎                                                                                                                 | 2/24 [22:30:16<280:24:41, 45885.54s/it]Setting `pad_token_id` to `eos_token_id`:2 for open-end generation.
Evaluate pipeline:  12%|███████████████▌                                                                                                            | 3/24 [37:40:36<290:55:56, 49874.13s/it]Setting `pad_token_id` to `eos_token_id`:2 for open-end generation.
Evaluate pipeline:  17%|████████████████████▋                                                                                                       | 4/24 [55:12:04<306:03:35, 55090.79s/it]Setting `pad_token_id` to `eos_token_id`:2 for open-end generation.

tests/openvino/test_seq2seq.py

kyoui-dev · 2026-03-24T11:26:20Z

@popovaan,

I tried running the evaluation locally, but it eventually got killed due to resource limits on my machine. Would you recommend a different way to run it?

popovaan · 2026-03-24T11:34:57Z

@popovaan,

I tried running the evaluation locally, but it eventually got killed due to resource limits on my machine. Would you recommend a different way to run it?

Please try reducing the number of samples, for example:
--num-samples 4

Add support for Mistral3

f3e3433

rkazants reviewed Mar 2, 2026

View reviewed changes

rkazants requested a review from popovaan March 2, 2026 19:09

kyoui-dev force-pushed the mistral3 branch from 57bc52b to f3e3433 Compare March 3, 2026 15:59

Add tests

4808de0

rkazants requested review from IlyasMoutawwakil, Copilot and echarlaix March 4, 2026 06:39

Copilot started reviewing on behalf of rkazants March 4, 2026 06:39 View session

Copilot AI reviewed Mar 4, 2026

View reviewed changes

tests/openvino/utils_tests.py Outdated Show resolved Hide resolved

tests/openvino/utils_tests.py Outdated Show resolved Hide resolved

Fix typo

3100050

Update test model path

976843b

echarlaix reviewed Mar 4, 2026

View reviewed changes

kyoui-dev added 4 commits March 7, 2026 12:46

Remove redundant custom mapping

52c4b61

Fix model naming in docs

422709a

Refactor projector input generator to inherit from LLava

a6bb58e

Update comments and links

56eb853

kyoui-dev added 3 commits March 13, 2026 16:57

Fix test

2f288b0

Fix code style

a46ff83

Merge branch 'main' into mistral3

7dcc1e4

kyoui-dev added 2 commits March 14, 2026 12:05

Fix test

1d4c1e9

Fix code style

316ffa4

kyoui-dev force-pushed the mistral3 branch from 242d705 to 316ffa4 Compare March 16, 2026 10:02

Fix code style

aa397b2

kyoui-dev requested review from echarlaix and rkazants March 17, 2026 03:39

popovaan reviewed Mar 17, 2026

View reviewed changes

tests/openvino/test_quantization.py Outdated Show resolved Hide resolved

Revert unintended code style change

46cf664

echarlaix reviewed Mar 23, 2026

View reviewed changes

tests/openvino/test_seq2seq.py Outdated Show resolved Hide resolved

Revert test change

bbc77ef

Conversation

kyoui-dev commented Mar 2, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

What does this PR do?

Before submitting

Uh oh!

kyoui-dev commented Mar 2, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

popovaan commented Mar 2, 2026

Uh oh!

rkazants left a comment

Choose a reason for hiding this comment

Uh oh!

kyoui-dev commented Mar 3, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Pull request overview

Reviewed changes

Uh oh!

Uh oh!

Uh oh!

popovaan commented Mar 4, 2026

Uh oh!

kyoui-dev commented Mar 4, 2026

Uh oh!

echarlaix left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

rkazants commented Mar 5, 2026

Uh oh!

kyoui-dev commented Mar 7, 2026

Uh oh!

kyoui-dev commented Mar 7, 2026

Uh oh!

HuggingFaceDocBuilderDev commented Mar 9, 2026

Uh oh!

popovaan commented Mar 11, 2026

Uh oh!

kyoui-dev commented Mar 13, 2026

Uh oh!

rkazants commented Mar 13, 2026

Uh oh!

kyoui-dev commented Mar 14, 2026

Uh oh!

Uh oh!

popovaan commented Mar 17, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

popovaan commented Mar 19, 2026

Uh oh!

kyoui-dev commented Mar 19, 2026

Uh oh!

kyoui-dev commented Mar 22, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

Uh oh!

kyoui-dev commented Mar 24, 2026

Uh oh!

popovaan commented Mar 24, 2026

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

6 participants

kyoui-dev commented Mar 2, 2026 •

edited

Loading

kyoui-dev commented Mar 2, 2026 •

edited

Loading

kyoui-dev commented Mar 3, 2026 •

edited

Loading

popovaan commented Mar 17, 2026 •

edited

Loading

kyoui-dev commented Mar 22, 2026 •

edited

Loading