Eval bug: starting from b8287 every response for Qwen 3.5 35b A3B in Instruct mode starts with "</think>"

[kelheor]

Name and Version

b8287

Operating systems

Windows

GGML backends

CUDA

Hardware

Ryzen 5950x + 4090 + 128Gb DDR4

Models

Qwen3.5-35B-A3B-MXFP4_MOE.gguf from Unsloth

Problem description & steps to reproduce

Starting from b8287 every response for Qwen 3.5 35b A3B in Instruct mode starts with "</think>"
Version b8286 and below works just fine.

First Bad Commit

Starting from b8287

Relevant log output

No issues in logs

[drHuangMHT]

Hi! I'm not seeing </think> on cuda-server container b8334 with model Qwen3.5-122B-A10B, with additional argument --chat-template-kwargs '{"enable_thinking": false}' to disable thinking(which should be --reasoning off on recent builds). OS is Linux(debian). Might as well try the argument, and I'll try 35B-A3B when I get the weights!

[drHuangMHT]

I am unable to reproduce the bug. Running on Linux host, container ghcr.io/ggml-org/llama.cpp:server-cuda13-b8334, model Qwen3.5-35B-A3B-Q8_0 from unsloth. The raw response does not show </think> tag at the beginning, with the same prompt. Reasoning is disabled by --reasoning off, and my frontend handles thinking normally.

[kelheor]

I am unable to reproduce the bug. Running on Linux host, container ghcr.io/ggml-org/llama.cpp:server-cuda13-b8334, model Qwen3.5-35B-A3B-Q8_0 from unsloth. The raw response does not show </think> tag at the beginning, with the same prompt. Reasoning is disabled by --reasoning off, and my frontend handles thinking normally.

I'm using --chat-template-kwargs '{"enable_thinking":false}' and --reasoning-budget 0
Can you try Qwen3.5-35B-A3B-MXFP4_MOE.gguf ?

Also, it doesn't matter if I add additional instructions (like personas) or use default state - still same issue.
I'm also using CUDA 12, not 13.

[drHuangMHT]

Confirmed with --reasoning off --reasoning-budget 0 on Qwen3.5-35B-A3B-Q8_0 that the output start with </think>.
Related PR: #20297, see comment #20297 (comment)

If your template doesn't support disabling thinking, you can use --reasoning-budget 0 as the sampler "forcing" solution, but beware that models might not like it (this is basically the equivalent of hacking the template to insert at the start of the response).

According to the author this seems to be expected behavior. The frontend does not handle a single close tag without an opening tag. Since --reasoning off works for Qwen3.5 models, you may drop --reasoning-budget 0

[kelheor]

According to the author this seems to be expected behavior. The frontend does not handle a single close tag without an opening tag. Since --reasoning off works for Qwen3.5 models, you may drop --reasoning-budget 0

Ok, I removed "--reasoning-budget 0" and now it works as it should.