Skip to content

Commit 8ca8954

Browse files
authored
[Bugfix][Misc]: fix graph capture for decoder (#9549)
1 parent f6b9729 commit 8ca8954

File tree

1 file changed

+1
-1
lines changed

1 file changed

+1
-1
lines changed

vllm/worker/model_runner.py

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -828,7 +828,7 @@ def build(self) -> ModelInputForGPU:
828828

829829
cuda_graph_pad_size = self._get_cuda_graph_pad_size(
830830
num_seqs=len(seq_lens),
831-
max_decode_seq_len=max_encoder_seq_len,
831+
max_decode_seq_len=max_decode_seq_len,
832832
max_encoder_seq_len=max_encoder_seq_len)
833833

834834
batch_size = len(input_tokens)

0 commit comments

Comments
 (0)