Skip to content

Commit 51342f5

Browse files
author
chang-wenbin
committed
support MLA overlap-schedule
1 parent cb2d7c0 commit 51342f5

1 file changed

Lines changed: 2 additions & 1 deletion

File tree

fastdeploy/model_executor/models/deepseek_v3.py

Lines changed: 2 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -1105,7 +1105,8 @@ def forward(
11051105
residual: paddle.Tensor,
11061106
):
11071107
""" """
1108-
if hidden_states.shape[0] > 0:
1108+
need_do_attention = forward_meta.max_len_tensor_cpu[1] > 0 or forward_meta.max_len_tensor_cpu[2] > 0
1109+
if hidden_states.shape[0] > 0 and need_do_attention:
11091110
hidden_states, residual = self.input_layernorm(
11101111
hidden_states, residual_input=residual, forward_meta=forward_meta
11111112
)

0 commit comments

Comments
 (0)