[Bug] Empty Logits passed even though os.environ['UNSLOTH_RETURN_LOGITS'] = '1' unable to use compute metrics in evaluation

[devlup]

1. Did you update? pip install --upgrade unsloth unsloth_zoo Yes
2. Colab or Kaggle or local / cloud local
3. Number GPUs used, use nvidia-smi A100
4. Which notebook? Please link!
5. Which Unsloth version, TRL version, transformers version, PyTorch version? latest main
6. Which trainer? SFTTrainer, GRPOTrainer etc SFT trainer

TypeError: Unsupported types (<class 'unsloth_compiled_module_gpt_oss.EmptyLogits'>) passed to _pad_across_processes. Only nested list/tuple/dicts of objects that are valid for is_torch_tensor should be passed.
in evaluation_loop
logits = self.accelerator.pad_across_processes(logits, dim=1, pad_index=-100)

Put Minimal code to reproduce error here ###Remove Hugging Face token###

🦥 You can also ask via our Reddit page: https://www.reddit.com/r/unsloth/

[devlup]

@rolandtannous i don't think it is the duplicate issue, For me the flag is set as os.environ["UNSLOTH_RETURN_LOGITS"] = "1" but in unsloth_compiled_cache/unsloth_compiled_module_gpt_oss.py line : 725 logits were not generated and set to empty logits elif self.loss_function.name.endswith("ForCausalLMLoss") and labels is not None:
lm_head_weight = self.lm_head.weight
lm_head_bias = getattr(self.lm_head, "bias", None)

    # ========= NEW fused =========
    _hidden_states = hidden_states[:, slice_indices, :]
    torch._dynamo.mark_dynamic(_hidden_states, 1)
    torch._dynamo.mark_dynamic(labels, 1)
    loss = unsloth_compiled_fused_ce_loss_function(
        hidden_states        = _hidden_states,
        lm_head_weight       = lm_head_weight,
        lm_head_bias         = lm_head_bias,
        output_labels        = labels,
        logit_scale_multiply = () if () != () else 0,
        logit_scale_divide   = () if () != () else 0,
        logit_softcapping    = () if () not in (None, (),) else 0,
        vocab_size           = (self.vocab_size),
        n_items              = n_items,
        requires_grad_       = requires_grad_,
    )

    # ========= OLD non fused =========
    # logits = self.lm_head(hidden_states[:, slice_indices, :].to(lm_head_weight.device))
    # torch._dynamo.mark_dynamic(logits, 1)
    # torch._dynamo.mark_dynamic(labels, 1)
    # loss = unsloth_compiled_ce_loss_function(
    #     output_logits        = logits,
    #     output_labels        = labels,
    #     logit_scale_multiply = () if () != () else 0,
    #     logit_scale_divide   = () if () != () else 0,
    #     logit_softcapping    = () if () not in (None, (),) else 0,
    #     vocab_size           = (self.vocab_size),
    #     n_items              = n_items,
    #     requires_grad_       = requires_grad_,
    # )

[danielhanchen]

Will investigate - sorry on the issue!

[thusinh1969]

Is this fixed ? We need logits for custom loss.

Thanks,
Steve

[mpras32]

I am also facing a similar issue when using a custom evaluator

[AlbertMarashi]

In my case, I wanted to obtain per-token losses, but unfortunately, something must've been changed, and is now returning a scalar number instead of a loss per token.

Previously, it appeared that setting os.environ["UNSLOTH_RETURN_LOGITS"] = "1" would return me the individual token losses (e.g. like reduction="none", but this appears to no longer be the case.

        outputs = model(
            input_ids=inputs["input_ids"],
            labels=inputs["labels"],
            position_ids=inputs["position_ids"],
        )
        per_token_loss = outputs.loss  # Shape: (total_tokens - 1,), assuming reduction='none'

[pluesclues]

Hello, custom evaluation and UNSLOTH_RETURN_LOGITS="1" should work now, please update unsloth again via pip install --upgrade --force-reinstall --no-deps unsloth_zoo unsloth