Conversation
|
@kirnat Can you test this? |
|
Thanks a lot, unfortunately this PR didn't seem to solve this particular issue with tools calls being picked up properly. Hopefully someone else can confirm so it's not local to my specific setup. When you tried originally with DS 3.1, do you remember which quantized version you used? I could test with the same if it would be meaningful to rule out any issues with chat template? |
|
Just had a report come in about the new Kimi-K2 tool-calling as moonshot just released a tool-calling test harness verifier tool and someone was having an issue: https://huggingface.co/ubergarm/Kimi-K2-Instruct-0905-GGUF/discussions/1#68d5df5f268bd3e07916baba I asked them to try again with this PR and report back if possible. |
I don't have the model to test as the previous PR is a cherrypick from the mainline. |
Some users still report underwhelming results with tool callings after Deepseek V3.1 tool call support is merged (#771). This PR is the initial attempt to fix it. It simply updates minja.hpp and json.hpp from mainline. This is needed even if it does not fix Deepseek v3.1 tool calling issue.