[V1] LoRA - Enable Serving Usecase#12883

Merged

DarkLight1337 merged 19 commits intovllm-project:mainfrom

neuralmagic:varun/v1-lora-serving-fix

Feb 14, 2025

Commits on Feb 12, 2025

plumb add_lora to enable serving usecase
Varun Sundar Rabindranath
committed
Add add_lora_decoder
Varun Sundar Rabindranath
committed
fix return types
Varun Sundar Rabindranath
committed
add add_lora tests
Varun Sundar Rabindranath
committed
fixes
Varun Sundar Rabindranath
committed
increase max_loras for bigger margin
Varun Sundar Rabindranath
committed
fix lint
Varun Sundar Rabindranath
committed

Commits on Feb 13, 2025

[fix] wait longer
Varun Sundar Rabindranath
committed
[fix] Run tests in eagermode
Varun Sundar Rabindranath
committed
drop max loras to avoid ci oom
Varun Sundar Rabindranath
committed
drop gpu util to avoid oom
Varun Sundar Rabindranath
committed
drop max_model_len to satisfy gpu mem util
Varun Sundar Rabindranath
committed
use local lora without tokenizer
Varun Sundar Rabindranath
committed
fix tokenizer stuff
Varun Sundar Rabindranath
committed
reuse llm instance
Varun Sundar Rabindranath
committed
reduce default max loras
Varun Sundar Rabindranath
committed
reduce max-loras
Varun Sundar Rabindranath
committed
update default max loras
Varun Sundar Rabindranath
committed

Commits on Feb 14, 2025

fix comment
Varun Sundar Rabindranath
committed