support load qwen2-72b-instruct lora

[NiuBlibing]

Like #4007, to support qwen2-72b-instruct's lora adapter with 1,2,4,8 tp-size.

Ref #3793

[NiuBlibing]

Currntly punica kernel cannot support Qwen2-72B-Instruct because of 3696 could not be divided by 64. Hope #5036 or #5356 will work.

[jeejeelee]

Could you provide your running script?

I can test Qwen2-72B-Instruct+LoRA on my local device using #5036.

[NiuBlibing]

Could you provide your running script?

I can test Qwen2-72B-Instruct+LoRA on my local device using #5356.

I just start it with vllm cli.

python -m vllm.entrypoints.openai.api_server --served-model-name Qwen2-72B-Chat-test --model ./Qwen/Qwen2-72B-Instruct/ --gpu-memory-utilization 0.9 --tensor-parallel-size 8 --enable-lora --lora-dtype bfloat16 --lora-modules test=/path/to/lora/

[jeejeelee]

Could you provide your running script?
I can test Qwen2-72B-Instruct+LoRA on my local device using #5356.

I just start it with vllm cli.

python -m vllm.entrypoints.openai.api_server --served-model-name Qwen2-72B-Chat-test --model ./Qwen/Qwen2-72B-Instruct/ --gpu-memory-utilization 0.9 --tensor-parallel-size 8 --enable-lora --lora-dtype bfloat16 --lora-modules test=/path/to/lora/

Sorry, Actually, #5036 was used for the testing.

I have completed the test, #5036 can resolve this issue.

However, there are still some other issues that need to be resolved with #5036, I will process ASAP