Fix for: 'Gemma3ModelOutputWithPast' object has no attribute 'loss'

[LimLims]

Issue when training gemma3:

2025-05-29 23:57:34,110 - auto_antislop_main - ERROR - An error occurred during finetuning: 'Gemma3ModelOutputWithPast' ob
ject has no attribute 'loss'
Traceback (most recent call last):
  File "/home/ubuntu/auto-antislop/main.py", line 312, in main
    run_dpo_finetune(config, experiment_run_dir)
  File "/home/ubuntu/auto-antislop/core/finetuning.py", line 643, in run_dpo_finetune
    pre_train_rows, pre_train_stats = _gap_stats(model, train_ds.select(range(min(N_VAL_ITEMS, len(train_ds)))), collate,
"train_pre")
                                      ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
^^^^^^^^^^^^
  File "/home/ubuntu/auto-antislop/core/finetuning.py", line 566, in _gap_stats
    logits_next = model(ids, attention_mask=attn).logits[:, -1, :]
                  ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/ubuntu/.local/lib/python3.11/site-packages/torch/nn/modules/module.py", line 1739, in _wrapped_call_impl
    return self._call_impl(*args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/ubuntu/.local/lib/python3.11/site-packages/torch/nn/modules/module.py", line 1750, in _call_impl
    return forward_call(*args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/ubuntu/.local/lib/python3.11/site-packages/peft/peft_model.py", line 1757, in forward
    return self.base_model(
           ^^^^^^^^^^^^^^^^
  File "/home/ubuntu/.local/lib/python3.11/site-packages/torch/nn/modules/module.py", line 1739, in _wrapped_call_impl
    return self._call_impl(*args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/ubuntu/.local/lib/python3.11/site-packages/torch/nn/modules/module.py", line 1750, in _call_impl
    return forward_call(*args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/ubuntu/.local/lib/python3.11/site-packages/peft/tuners/tuners_utils.py", line 193, in forward
    return self.model.forward(*args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/ubuntu/auto-antislop/utils/model_helpers.py", line 103, in safe_fwd
    out = old_fwd(self, *a, **kw)
          ^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/ubuntu/.local/lib/python3.11/site-packages/unsloth_zoo/temporary_patches/gemma.py", line 384, in forward
    loss = outputs.loss
           ^^^^^^^^^^^^
AttributeError: 'Gemma3ModelOutputWithPast' object has no attribute 'loss'

The cause is in this file:

https://github.com/unslothai/unsloth-zoo/blob/main/unsloth_zoo/temporary_patches/gemma.py

specifically both instances of:

loss = outputs.loss

when the model does its forward pass, if there are no labels specified, we don't expect the "loss" attribute to be there. the fix is:

loss = getattr(outputs, "loss", loss)

in both places.

Also-- the labels = None assignment might be a mistake, since it makes the later if block dead code.

[eu90h]

Thanks, I had the same problem and adjusting the loss fixed it.