Simplify weight loading in Transformers backend

[hmellor]

• TransformersModel no longer inherits from nn.Module so self.model = AutoModel.from_config(...) is no longer instantly registered as a child module of TransformersModel
• In TransformersForCausalLM and TransformersForMultimodalLM we set self.model to the inner model from TransformersModel
• Now, hf_to_vllm_mapper is no longer necessary in TransformersForCausalLM and can be converted to a class variable in TransformersForMultimodalLM
• Now that hf_to_vllm_mapper is no longer a @property in any Transformers backend classes, we can update SupportsQuant to access the class attribute directly as instructed in the comments

[gemini-code-assist]

Code Review

This pull request simplifies the Transformers backend by removing nn.Module inheritance from TransformersModel, streamlining weight mappers. However, the weight mapper in TransformersForMultimodalLM has a critical issue that will cause weight loading to fail for some models. The review includes a fix for this issue, as well as suggestions for code simplification and readability.

[gemini-code-assist]

The updated hf_to_vllm_mapper for TransformersForMultimodalLM appears to be incorrect. The AutoWeightsLoader is initialized with self (the TransformersForMultimodalLM instance), which has self.model as an attribute containing the PreTrainedModel. Therefore, parameter names within the model are expected to be prefixed with model. (e.g., model.language_model...).

The current mappings, such as "language_model.model": "language_model", will cause the loader to look for a top-level language_model attribute on TransformersForMultimodalLM, which doesn't exist. The target prefixes should include model. to correctly map to the nested structure.

For example, language_model.model from the checkpoint should map to model.language_model in the vLLM model.

hf_to_vllm_mapper = WeightsMapper(
        orig_to_new_prefix={
            "language_model.model": "model.language_model",
            "text_model.model": "model.text_model",
            "text_model.lm_head": "model.lm_head",
            "language_model.lm_head": "model.lm_head",
            # deal with Qwen2-VL mapping
            "model.layers": "model.language_model.layers",
        })

[github-actions]

👋 Hi! Thank you for contributing to the vLLM project.

💬 Join our developer Slack at https://slack.vllm.ai to discuss your PR in #pr-reviews, coordinate on features in #feat- channels, or join special interest groups in #sig- channels.

Just a reminder: PRs would not trigger full CI run by default. Instead, it would only run fastcheck CI which starts running only a small and essential subset of CI tests to quickly catch errors. You can run other CI tests on top of those by going to your fastcheck build on Buildkite UI (linked in the PR checks section) and unblock them. If you do not have permission to unblock, ping simon-mo or khluu to add you in our Buildkite org.

Once the PR is approved and ready to go, your PR reviewer(s) can run CI to test the changes comprehensively before merging.

To run CI, PR reviewers can either: Add ready label to the PR or enable auto-merge.

🚀

[Isotr0py]

LGTM as long as tests can pass.