Skip to content

chat : detect reasoning markers when enable_thinking changes system prompt#20859

Merged
pwilkin merged 1 commit intoggml-org:masterfrom
jhen0409:jhen/patch-1
Mar 23, 2026
Merged

chat : detect reasoning markers when enable_thinking changes system prompt#20859
pwilkin merged 1 commit intoggml-org:masterfrom
jhen0409:jhen/patch-1

Conversation

@jhen0409
Copy link
Copy Markdown
Collaborator

@jhen0409 jhen0409 commented Mar 22, 2026

I found models like SmolLM3-3B chat template changes the system prompt depends on enable_thinking (!left_trimmed.empty() && !right_trimmed.empty()), so it will skip parse the reasoning markers.

Codex (gpt-5.4, xhigh) also helped check this bug fix.

@jhen0409 jhen0409 requested review from a team and ggerganov as code owners March 22, 2026 10:01
@github-actions github-actions bot added the testing Everything test related label Mar 22, 2026
@jhen0409 jhen0409 requested a review from pwilkin March 22, 2026 10:02
@pwilkin
Copy link
Copy Markdown
Member

pwilkin commented Mar 22, 2026

#20424 should have fixed it.

@jhen0409
Copy link
Copy Markdown
Collaborator Author

#20424 should have fixed it.

@pwilkin There are the results I got for left_trimmed and right_trimmed using the template:

compare_thinking_enabled: left_trimmed: no_think

## Custom Instructions

You are a helpful AI assistant named SmolLM, trained by Hugging Face.

<|im_start|>user
U_USER_MSG Hello END_U<|im_end|>
<|im_start|>assistant
<think>

</think>
compare_thinking_enabled: right_trimmed: think

## Custom Instructions

You are a helpful AI assistant named SmolLM, trained by Hugging Face. Your role as an assistant involves thoroughly exploring questions through a systematic thinking process before providing the final precise and accurate solutions. This requires engaging in a comprehensive cycle of analysis, summarizing, exploration, reassessment, reflection, backtracking, and iteration to develop well-considered thinking process. Please structure your response into two main sections: Thought and Solution using the specified format: <think> Thought section </think> Solution section. In the Thought section, detail your reasoning process in steps. Each step should include detailed considerations such as analysing questions, summarizing relevant findings, brainstorming new ideas, verifying the accuracy of the current steps, refining any errors, and revisiting previous steps. In the Solution section, based on various attempts, explorations, and reflections from the Thought section, systematically present the final solution that you deem correct. The Solution section should be logical, accurate, and concise and detail necessary steps needed to reach the conclusion.

<|im_start|>user
U_USER_MSG Hello END_U<|im_end|>
<|im_start|>assistant

So compare_thinking_enabled will be always false for this template before this PR.

Copy link
Copy Markdown
Member

@pwilkin pwilkin left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Ah, I'm sorry, I was tired and I look at the description and thought it was for something else before looking at the code.

Yes, this is valid, thanks.

@pwilkin pwilkin merged commit 7a0b6a6 into ggml-org:master Mar 23, 2026
48 checks passed
@jhen0409 jhen0409 deleted the jhen/patch-1 branch March 23, 2026 08:24
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

testing Everything test related

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants