Skip to content

Commit 1140991

Browse files
authored
[V1] Fix vocab size calculation for structured output (#14826)
Signed-off-by: Russell Bryant <[email protected]>
1 parent 613c5bb commit 1140991

File tree

1 file changed

+1
-1
lines changed

1 file changed

+1
-1
lines changed

vllm/v1/structured_output/__init__.py

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -40,7 +40,7 @@ def _delayed_init(self):
4040
tokenizer_group.ping()
4141

4242
tokenizer = tokenizer_group.get_lora_tokenizer(None)
43-
self.vocab_size = tokenizer.max_token_id
43+
self.vocab_size = tokenizer.max_token_id + 1
4444
if isinstance(tokenizer, MistralTokenizer):
4545
# NOTE: ideally, xgrammar should handle this accordingly.
4646
# refer to https://github.com/mlc-ai/xgrammar/blob/d77c0a0173ef14779c918e3be7966ba852f7910f/python/xgrammar/tokenizer_info.py#L98

0 commit comments

Comments
 (0)