Skip to content

Commit ef54499

Browse files
ugolowicastachowiczhabana
authored andcommitted
[SW-209062] Disable default sdpa in Albert (#22)
Transformers v4.45 introduced sdpa as the default implementation in Albet. This caused performance drop. Adding Albert to the list of models which don't yet have sdpa implementation in Gaudi and use thus eager attention.
1 parent 9a49200 commit ef54499

1 file changed

Lines changed: 1 addition & 1 deletion

File tree

optimum/habana/transformers/models/modeling_all_models.py

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -115,7 +115,7 @@ def gaudi_conv1d_forward(self, x):
115115
@classmethod
116116
def gaudi_check_and_enable_sdpa(cls, config, hard_check_only: bool = False) -> PretrainedConfig:
117117
# This model doesn't support SDPA in Gaudi yet, fallback to original code.
118-
MODELS_ATTN_IMPLEMENTATION_EAGER = ["bart", "gpt_bigcode", "mistral", "mixtral", "wav2vec2", "roberta"]
118+
MODELS_ATTN_IMPLEMENTATION_EAGER = ["albert", "bart", "gpt_bigcode", "mistral", "mixtral", "wav2vec2", "roberta"]
119119

120120
if config.model_type in MODELS_ATTN_IMPLEMENTATION_EAGER:
121121
config._attn_implementation = "eager"

0 commit comments

Comments
 (0)