Skip to content

Conversation

@Seven-Streams
Copy link
Collaborator

This PR enables the xgrammar to compile the grammar just in time. To determine whether a grammar should be compiled just in time, you can use the code as follows:

grammar = xgr.Grammar.builtin_json_grammar() 
tokenizer = AutoTokenizer.from_pretrained("meta-llama/Llama-2-7b-chat-hf")
compiler = xgr.GrammarCompiler(xgr.TokenizerInfo.from_huggingface(tokenizer), is_jit=True)
compiled_grammar = compiler.compile_grammar(grammar)
... # the same codes with the pre-computed compiled_grammar.

Signed-off-by: Yuchuan [email protected]

@Seven-Streams Seven-Streams force-pushed the main-dev/2025-08-21/jit branch 2 times, most recently from c7bd4ce to 0d72de2 Compare September 9, 2025 08:28
@Seven-Streams Seven-Streams force-pushed the main-dev/2025-08-21/jit branch from 0d72de2 to 93f3696 Compare September 18, 2025 00:26
@Seven-Streams Seven-Streams force-pushed the main-dev/2025-08-21/jit branch from 93f3696 to bf20557 Compare September 18, 2025 00:27
@Seven-Streams Seven-Streams marked this pull request as draft November 8, 2025 01:21
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant