Upgrade transformers to v4.50.3

[hmellor]

Supersedes #13602

Depends on:

• Replace LogitsWarper with LogitsProcessor noamgat/lm-format-enforcer#159
• Lazy import libraries in src/transformers/image_utils.py huggingface/transformers#36435
• Fix processor kwargs qwen2 vl huggingface/transformers#36890
• [chameleon] fix num image token check huggingface/transformers#36918
• [blip-2] Fix dtype mismatch when keep in fp32  huggingface/transformers#37068
• [generate] beam search -- fix output cropping huggingface/transformers#37080

Models which are not compatible with the latest version of Transformers:

• https://huggingface.co/openbmb/MiniCPM3-4B/discussions/40
• https://huggingface.co/openbmb/MiniCPM-o-2_6/discussions/40 (test skipped for now)
• https://huggingface.co/allenai/Molmo-7B-D-0924/discussions/41 (test skipped for now)

[github-actions]

👋 Hi! Thank you for contributing to the vLLM project.

💬 Join our developer Slack at https://slack.vllm.ai to discuss your PR in #pr-reviews, coordinate on features in #feat- channels, or join special interest groups in #sig- channels.

Just a reminder: PRs would not trigger full CI run by default. Instead, it would only run fastcheck CI which starts running only a small and essential subset of CI tests to quickly catch errors. You can run other CI tests on top of those by going to your fastcheck build on Buildkite UI (linked in the PR checks section) and unblock them. If you do not have permission to unblock, ping simon-mo or khluu to add you in our Buildkite org.

Once the PR is approved and ready to go, your PR reviewer(s) can run CI to test the changes comprehensively before merging.

To run CI, PR reviewers can either: Add ready label to the PR or enable auto-merge.

🚀

[mergify]

This pull request has merge conflicts that must be resolved before it can be
merged. Please rebase the PR, @hmellor.

https://docs.github.com/en/pull-requests/collaborating-with-pull-requests/working-with-forks/syncing-a-fork

[DarkLight1337]

Has this been resolved? We want to upgrade to v4.50 for Gemma3 release.

[hmellor]

Not yet, we're expecting a release sometime next week

[aarnphm]

Can we also bump lm-format-enforcer version for v0?

[hmellor]

V1 and V0 do not have different requirements. lm-format-enforcer was last bumped 3 weeks ago, which version do you need?

[aarnphm]

Oh, I thought it wasn't bumped. please discard that message my bad 😃

[DarkLight1337]

As mentioned before, let's just skip those tests where the HF repo can't keep up with the latest transformers version

[hmellor]

What is the preferred method of skipping?

• Delete model from list of models?
• Comment model and add reason?
• Add pytest.mark.skip?

[DarkLight1337]

Let's add a new field max_transformers_version to _HfExamplesInfo, and update check_transformers_version accordingly

(The tests for Isotr0py/deepseek-vl2-tiny and TIGER-Lab/Mantis-8B-siglip-llama3 are already being skipped in test_models.py, we can also update that code to use max_transformers_version)

[hmellor]

Let's add a new field max_transformers_version to _HfExamplesInfo, and update check_transformers_version accordingly

Ok I can make this change and use it for models which are not currently compatible with vLLM because they're outdated.

(The tests for Isotr0py/deepseek-vl2-tiny and TIGER-Lab/Mantis-8B-siglip-llama3 are already being skipped in test_models.py, we can also update that code to use max_transformers_version)

For these ones it's only the HfRunner that has the maximum Transformers version requirement. These models do actually work with vLLM.

[DarkLight1337]

Let's add a new field max_transformers_version to _HfExamplesInfo, and update check_transformers_version accordingly

Ok I can make this change and use it for models which are not currently compatible with vLLM because they're outdated.

(The tests for Isotr0py/deepseek-vl2-tiny and TIGER-Lab/Mantis-8B-siglip-llama3 are already being skipped in test_models.py, we can also update that code to use max_transformers_version)

For these ones it's only the HfRunner that has the maximum Transformers version requirement. These models do actually work with vLLM.

Yeah they do work with vLLM. I think the other models would also work with vLLM if not for the outdated imports...

[hmellor]

Yeah the outdated imports are the only blocker

[hmellor]

The two further issues I found have been added to the description. Fix PRs have already been merged in transformers, we'd just need another patch release.

[mergify]

This pull request has merge conflicts that must be resolved before it can be
merged. Please rebase the PR, @hmellor.

https://docs.github.com/en/pull-requests/collaborating-with-pull-requests/working-with-forks/syncing-a-fork