[Misc] Use model_redirect to redirect the model name to a local folder.

[noooop]

TLDR

Use model_redirect to redirect the model name to a local folder.
Use model name in the code without hard-coding the model path.

Usage

One redirect rule per line, model_name and redirect_name separated by \t.

For example, given a redirection facebook/opt-125m -> /data/LLM-model/opt-125m

echo -e "facebook/opt-125m\t/data/LLM-model/opt-125m\n" > .model.redirect

VLLM_MODEL_REDIRECT_PATH=".model.redirect" vllm serve facebook/opt-125m

should be equivalent to

vllm serve /data/LLM-model/opt-125m --served-model-name facebook/opt-125m

Use Case

1. Use model name instead of model path to serve a local model, (e.g. your own trained model).
2. Allow models from different sources, such as ModelScope and huggingface, using non-standard paths, local models to coexist harmoniously
3. Offline mode. If you use model name, it need to request huggingface several times to confirm the model path and whether the model is up-to-date. Redirecting model names to a local folder can be done completely without using the network.
4. Pin model version. Using the model name will automatically request huggingface and download the latest model version. Sometimes you need a specific model version. Redirecting model names to a local folder allows you to avoid hardcoding revisions.

[github-actions]

👋 Hi! Thank you for contributing to the vLLM project.

💬 Join our developer Slack at https://slack.vllm.ai to discuss your PR in #pr-reviews, coordinate on features in #feat- channels, or join special interest groups in #sig- channels.

Just a reminder: PRs would not trigger full CI run by default. Instead, it would only run fastcheck CI which starts running only a small and essential subset of CI tests to quickly catch errors. You can run other CI tests on top of those by going to your fastcheck build on Buildkite UI (linked in the PR checks section) and unblock them. If you do not have permission to unblock, ping simon-mo or khluu to add you in our Buildkite org.

Once the PR is approved and ready to go, your PR reviewer(s) can run CI to test the changes comprehensively before merging.

To run CI, PR reviewers can either: Add ready label to the PR or enable auto-merge.

🚀

[DarkLight1337]

I think this is not necessary anymore since we have migrated the CI/CD to using our own file storage which has pre-downloaded models.

[noooop]

@DarkLight1337

This feature is very convenient for everyone who needs to load models locally.

No need to use the model path every time

Not just for vllm CI/CD

[mergify]

This pull request has merge conflicts that must be resolved before it can be
merged. Please rebase the PR, @noooop.

https://docs.github.com/en/pull-requests/collaborating-with-pull-requests/working-with-forks/syncing-a-fork

[DarkLight1337]

Don't we already have HF_HOME to change the directory containing locally downloaded HF repos?

[noooop]

Some of my models use modelscope, some use hf, and some, e.g. deepseek v3, need to be downloaded to another partition.

I found that using model_overwrite is very convenient

[DarkLight1337]

Personally I don't use ModelScope, so I'll have @Isotr0py review this instead.

[noooop]

@Isotr0py
ready to review

examples/offline_inference/basic/use_model_overwrite.py will be deleted.

I think the config is too complicated, and it will also request hf multiple times.

[Isotr0py]

Hmmm, I'm fine to have a function to redirect the model_repo to a downloaded local directory manually, but I don't really like the name of "overwrite" and the introduction of .model.overwrite (seems it's not a formal file used by neither HF nor modelscope)...

[Isotr0py]

Suggested change

	def model_overwrite(model: str):
	def maybe_model_redirect(model: str):

I prefer to use maybe_model_redirect here.

[noooop]

maybe_model_redirect fine

[Isotr0py]

We should move the helper function to vllm.transformers_utils.utils if it's used by both config and tokenizer.

[Isotr0py]

I think we just need to call the redirect function when initialize ModelConfig, so that we don't need to add it here and there.

[noooop]

I don't want to override any model name in ModelConfig.

Yes, reading config is too complicated, and I don't want to write it everywhere, too.

[Isotr0py]

Is this a formal file in modelscope? I can't find it in modelscope's API documentations.

[noooop]

I named it

[Isotr0py]

I think the config is too complicated, and it will also request hf multiple times.

I think we just need to redirect to local location when initializing ModelConfig, so that we don't need to request hf multiple times.

[noooop]

I think we just need to redirect to local location when initializing ModelConfig, so that we don't need to request hf multiple times.

1. Redirection to local location when initializing ModelConfig. Log output and server names will be very weird.
   It may even trigger strange bugs

2. Redirection in each submodule. Redirection logic is written in many places.

3. Add a model_path parameter to each module？It's a huge project.

It's all very hacky anyway.

@Isotr0py

Maybe I should stop this stupid idea.

[Isotr0py]

Otherwise, log output and server name will be very weird.

IMO, if we redirect the model_repo to local directory manually, we should also make sure the model's name updated accordingly, otherwise it's a little bit hacky and make bugs caused by outdated custom code difficult to find. (For HF, model with custom code won't be updated automatically when loading an outdated loacl checkpoint)

About the server name, we can use --served-model-name to use model repo name.

[noooop]

I personally prefer 2

2. Redirection in each submodule. Redirection logic is written in many places.

@Isotr0py

Looking forward to hearing your opinions

[Isotr0py]

LGTM now!

[noooop]

Thanks for reviewing the code

[noooop]

@Isotr0py

Please restart the test

[noooop]

I'm not sure if this PR is the cause of the problem. It seems that these tests also reported errors yesterday.

QVQ

[Isotr0py]

The entrypoint test failure should be unrelated, I can confirm it's passed locally. The V1 test is flaky currently. 😅

[noooop]

The entrypoint test failure should be unrelated, I can confirm it's passed locally. The V1 test is flaky currently. 😅

QVQ