What happened?
I am trying to make some grammars for my workflows, some of them more complex but I noticed that even simple ones still fail. For example, this is very simple grammar that blocks <think>The user sequence made of three tokens, it adds more variety to the thought patterns of Kimi K2 Thinking since otherwise it almost always starts with "The user", which if allowed decreases creativity. It also just a basic example of a failing grammar (as opposed to a more complex grammars that force certain data format):
root ::= ( !<[163606]> root | think-block )*
think-block ::= <[163606]> after-think
after-think ::= !<[1008]> root | the-block
the-block ::= <[1008]> !<[2742]> root
I think this simple example is sufficient because regardless of grammar complexity I get about the same errors. Unfortunately, no matter what I have tried, like formatting it differently or changing the approach, it seems to be failing in about the same way.
If I am doing something wrong and ik_llama.cpp maybe have different syntax than llama.cpp for token-based grammar rules, then I would appreciate any suggestion what else I could try.
Below I have attached the relevant log (in actual log these errors end up repeated few times, then it generated the output without applying the grammar). The same grammar works as expected in the mainline llama.cpp.
Name and Version
The latest git version
What operating system are you seeing the problem on?
Linux
Relevant log output
parse: error parsing grammar: expecting ')' at !<[163606]> root | think-block )*
think-block ::= <[163606]> after-think
after-think ::= !<[1008]> root | the-block
the-block ::= <[1008]> !<[2742]> root
root ::= ( !<[163606]> root | think-block )*
think-block ::= <[163606]> after-think
after-think ::= !<[1008]> root | the-block
the-block ::= <[1008]> !<[2742]> root
llama_grammar_init_impl: failed to parse grammar
What happened?
I am trying to make some grammars for my workflows, some of them more complex but I noticed that even simple ones still fail. For example, this is very simple grammar that blocks
<think>The usersequence made of three tokens, it adds more variety to the thought patterns of Kimi K2 Thinking since otherwise it almost always starts with "The user", which if allowed decreases creativity. It also just a basic example of a failing grammar (as opposed to a more complex grammars that force certain data format):I think this simple example is sufficient because regardless of grammar complexity I get about the same errors. Unfortunately, no matter what I have tried, like formatting it differently or changing the approach, it seems to be failing in about the same way.
If I am doing something wrong and ik_llama.cpp maybe have different syntax than llama.cpp for token-based grammar rules, then I would appreciate any suggestion what else I could try.
Below I have attached the relevant log (in actual log these errors end up repeated few times, then it generated the output without applying the grammar). The same grammar works as expected in the mainline llama.cpp.
Name and Version
The latest git version
What operating system are you seeing the problem on?
Linux
Relevant log output