Skip to content

Commit 7268d69

Browse files
authored
[Fix] Benchmark "EngineClient" has no attribute "model_config" (vllm-project#17976)
Signed-off-by: Brayden Zhong <[email protected]>
1 parent 82da8b2 commit 7268d69

File tree

1 file changed

+3
-2
lines changed

1 file changed

+3
-2
lines changed

throughput.py

Lines changed: 3 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -148,9 +148,10 @@ async def run_vllm_async(
148148

149149
async with build_async_engine_client_from_engine_args(
150150
engine_args, disable_frontend_multiprocessing) as llm:
151+
model_config = await llm.get_model_config()
151152
assert all(
152-
llm.model_config.max_model_len >= (request.prompt_len +
153-
request.expected_output_len)
153+
model_config.max_model_len >= (request.prompt_len +
154+
request.expected_output_len)
154155
for request in requests), (
155156
"Please ensure that max_model_len is greater than the sum of"
156157
" prompt_len and expected_output_len for all requests.")

0 commit comments

Comments
 (0)