Eval bug: gpt-oss reasoning_effort does nothing

[createthis]

Name and Version

ggml_cuda_init: GGML_CUDA_FORCE_MMQ: no
ggml_cuda_init: GGML_CUDA_FORCE_CUBLAS: no
ggml_cuda_init: found 1 CUDA devices:
Device 0: NVIDIA RTX PRO 6000 Blackwell Workstation Edition, compute capability 12.0, VMM: yes
version: 6097 (9515c61)
built with cc (Ubuntu 13.3.0-6ubuntu2~24.04) 13.3.0 for x86_64-linux-gnu

Operating systems

Linux

GGML backends

CUDA

Hardware

dual EPYC 9355, blackwell 6000 pro

Models

gpt-oss-120b-F16.gguf

Problem description & steps to reproduce

reasoning_effort seems to have no effect.

Startup command:

./build/bin/llama-server \
    --model /data/gpt-oss-120b-GGUF/gpt-oss-120b-F16.gguf \
    --alias gpt-oss-120b-F16 \
    --no-webui \
    --numa numactl \
    --threads 32 \
    --ctx-size 131072 \
    --n-gpu-layers 37 \
    -ot "exps.*\.blk.*\.ffn_.*=CUDA0" \
    --no-op-offload \
    -ub 4096 -b 4096 \
    --seed 3407 \
    --temp 0.6 \
    --top-p 1.0 \
    --log-colors \
    --flash-attn \
    --host 0.0.0.0 \
    --jinja \
    --chat-template-kwargs '{"reasoning_effort": "high"}' \
    --port 11434

First Bad Commit

No response

Relevant log output

output says:


Knowledge cutoff: 2024-06
Current date: 2025-08-06

Reasoning: medium

[nai-kon]

Thank you for the issue. I have the same problem.
I'm hosting gpt-oss-20b on llama-server and calling it from the OpenAI API library chat.completions.create().
I'm changing reasoning_effort of create(..., reasoning_effort="low"), but the actual system prompt remains "reasoning: medium" and seems not working.

Here is quick sample.
llama-server -m gpt-oss-20b-mxfp4.gguf -c 0 -fa --jinja --reasoning-format none -ngl 1000 (I'm using b6096 build)

from openai import OpenAI

model = OpenAI(api_key="dummy", base_url="http://127.0.0.1:8080")
completion = model.chat.completions.create(
    model="dummy",
    messages=[{"role": "user", "content": "Write fizzbuzz in Python"}],
    reasoning_effort="low",  # not effective
)

print(completion)

[KebobZ]

It seems to be working for me:

build/bin/llama-server  \
  -m gpt-oss-120b-mxfp4-00001-of-00003.gguf \
  --host 0.0.0.0 \
  -c 131072 \
  --jinja \
  --chat-template-kwargs '{"reasoning_effort": "high"}' \
  --temp 1.0 \
  --top-p 1.0 \
  --top-k 0 \
  -v

and the verbose response shows:

$ curl -s http://localhost:8080/v1/chat/completions -H "Content-Type: application/json" -d '{
    "messages": [{"role": "user", "content": "Hello, world!"}]
  }' | jq -r .__verbose.prompt

<|start|>system<|message|>You are ChatGPT, a large language model trained by OpenAI.
Knowledge cutoff: 2024-06
Current date: 2025-08-06

reasoning: high

# Valid channels: analysis, commentary, final. Channel must be included for every message.
Calls to these tools must go to the commentary channel: 'functions'.<|end|><|start|>user<|message|>Hello, world!<|end|><|start|>assistant

[nai-kon]

@KebobZ

Thank you for the comment. Now I found it worked with OpenAI library by setting this.

extra_body={"chat_template_kwargs":{"reasoning_effort": "low"}}

model = OpenAI(api_key="dummy", base_url="http://127.0.0.1:8080")
completion = model.chat.completions.create(
    model="dummy",
    messages=[{"role": "user", "content": "Write fizzbuzz in Python"}],
    # reasoning_effort="low",  This is not work
    extra_body={"chat_template_kwargs":{"reasoning_effort": "low"}},  # This worked
)

[createthis]

@KebobZ did you make that quant yourself from safetensors? It doesn’t look like an unsloth filename to me.

[emzaedu]

--chat-template-kwargs "{\"reasoning_effort\": \"high\"}"
working for me

[createthis]

@emzaedu I'm running commit 9515c6 from Aug 5th. I built a bf16 from scratch to rule out any potential issues in the unsloth quant. Same issue.

Can I ask which quant you're using and where you got it from? Also which commit of llama.cpp you're running? Also, can you share your startup command?

[victorb]

I'm running commit 9515c6 from Aug 5th

Including the chat_template_kwargs in the HTTP request so it ends up something like:

{
    model: "gpt-oss-20b-MXFP4.gguf",
    max_tokens: 4096,
    chat_template_kwargs: {
        reasoning_effort: "high", // or "low"
    },
    // ...
}

works correctly for me as well, given barely any reasoning_content on low but a ton with high, with llama-server and serving lmstudio-community/gpt-oss-120b-GGUF together with --jinja. Built from source from 1d72c84. Full command I use is:

$ CUDA_VISIBLE_DEVICES="0" ./build/bin/llama-server -fa --gpu-layers 99 --threads $(nproc) --threads-batch $(nproc) --batch-size 128 --ctx-size 131072 --jinja -v --verbose-prompt --model /mnt/nas/models/lmstudio-community/gpt-oss-120b-GGUF/gpt-oss-120b-MXFP4-00001-of-00002.gguf

[createthis]

@victorb does the CLI argument work for you?

[grigio]

@victorb thanks I tried via cli and it works

[createthis]

Still an issue with the latest unsloth F16 quant.

[ggerganov]

The parameter takes effect, even though the log always shows "medium". You can verify this by posting a simple logical task at temp 0.0 with "low" and "high" reasoning.

[abc-nix]

Still an issue with the latest unsloth F16 quant.

The --chat-template-kwargs '{"reasoning_effort": "high"}' command option works for me. Are you using the --jinja option also?

If you are still having issues with it, then save the unsloth's chat template into a file, edit the line that sets reasoning effort from medium to high ( {%- set reasoning_effort = "high" %}), save and load the chat template with the model using --chat-template-file /path/to/chat_template.jinja --jinja. It will always be using high then.

But, as reported by others, the command option to overwrite the template argument should be working. This workaround is not necessary, (I use it because I edit the system prompt to make the model be less restrictive with its policy, but that is a different topic).

[createthis]

Are you using the --jinja option also?

yes.

The parameter takes effect, even though the log always shows "medium".

I think I can confirm this. When using the --verbose option I see this in the transcripts:

"__verbose": { "prompt": "<|start|>system<|message|>You are ChatGPT, a large language model trained by OpenAI.\nKnowledge cutoff: 2024-06\nCurrent date: 2025-08-11\n\nReasoning: high\n\n# Valid channels: analysis, commentary, final. Channel must be included for every message.<|end|><|start|>developer<|message|>