Kimi-K2-Thinking native tool calling format#17251
Closed
KiruyaMomochi wants to merge 5 commits intoggml-org:masterfrom
Closed
Kimi-K2-Thinking native tool calling format#17251KiruyaMomochi wants to merge 5 commits intoggml-org:masterfrom
KiruyaMomochi wants to merge 5 commits intoggml-org:masterfrom
Conversation
Author
|
Thanks! Will try #16932. |
Author
|
Finally get some to test and it worked well! |
Author
|
Close in favour of #16932. |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
The implementation might support Kimi-K2-Instruct too, but I don't have enough disk space to test now :(
Almost silly copy-paste from DeepSeek V3.1 #15533, modified according to https://github.com/MoonshotAI/Kimi-K2/blob/main/docs/tool_call_guidance.md: matching function id instead of plain function name.
Considerations:
<think>tag at the end, sothinking_forced_openis false. Should we test it by modify the template manually?tojson(separators=(',', ':')). Although the value ofseparatorsis the same as default value, but we must remove it to make the template work for minja.<|tool▁calls▁begin|>tool...and ignoring<|tool▁call▁begin|>, but I have not observed such behavior in Kimi-K2-Thinking and always get<|tool_calls_section_begin|><|tool_call_begin|>, therefore I'm removing the?in the function regex.llama.cpp/common/chat.cpp
Line 1751 in c4abcb2
Actually, I always get an extra
<|tool_calls_section_end|>when keeping?, but I have not been able to fix it, so finally removed the?.For maintainers: I may have a busy weekend so fell free to edit directly if I'm not able to reply in time.
Closes #17155.