[RFC]: replace deprecated torch_dtype by dtype

[yannicks1]

Motivation.

Huggingface recently updated the argument from torch_dtype to a simple dtype to be coherent with PyTorch (see PR here).

Although their implementation is backward compatible, we could consider also making the change torch_dtype -> dtype to not trigger the warning:

`torch_dtype` is deprecated! Use `dtype` instead!

The warning is triggered for every call that involves huggingface (even if it is just for loading the model).

For example:

vllm bench latency --model "any_model_from_hf_hub" . . .

triggers the above warning.

Proposed Change.

Following changes are suggested:

• Replace torch_dtype with dtype to silence above warning and be prepared to when torch_dtype is fully removed in hf.
• Ensure backwards compatibility from vLLM user perspective.
• (Optional) add a test for backwards compatibility.

Feedback Period.

No response

CC List.

No response

Any Other Things.

No response

Before submitting a new issue...

• Make sure you already searched for relevant issues, and asked the chatbot living at the bottom right corner of the documentation page, which can answer lots of frequently asked questions.

[robertgshaw2-redhat]

sounds reasonable to me

[DarkLight1337]

Same. cc @hmellor @Isotr0py

[hmellor]

Yeah this is a resonable change to make.

The only thing to be aware of is that torch_dtype was first added in v4.56.1, which would become a hard lower bound for vLLM compatibility.

[yannicks1]

agree @hmellor, I see transformers >= 4.55.2 in requirements/common.txt that is where the hard lower bound is specified, right?

[hmellor]

that is where the hard lower bound is specified, right?

I believe so, yes. So when this lower bound is >= 4.56.1 we can update the variable name

[Moorer624]

Has this been released? I'm on most current Comfyui version and still get these errors.

[hmellor]

This was just released in v0.11.1 (this release was stuck for a while unfortunately)