Skip to content

convert : register Qwen 3.5 ForCausalLM for text only#20119

Merged
CISC merged 1 commit intomasterfrom
cisc/convert-qwen35-forcausal
Mar 5, 2026
Merged

convert : register Qwen 3.5 ForCausalLM for text only#20119
CISC merged 1 commit intomasterfrom
cisc/convert-qwen35-forcausal

Conversation

@CISC
Copy link
Member

@CISC CISC commented Mar 4, 2026

Support text only versions of Qwen 3.5.

Fixes #20116
Fixes #20102

@CISC CISC requested review from danbev and ggerganov March 4, 2026 23:28
@github-actions github-actions bot added the python python script changes label Mar 5, 2026
Copy link
Member

@danbev danbev left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This does fix the reported issue but the conversion fails with the following error for me:

Traceback (most recent call last):
  File "/home/danbev/work/ai/llama.cpp/examples/model-conversion/../../convert_hf_to_gguf.py", line 12130, in <module>
    main()
  File "/home/danbev/work/ai/llama.cpp/examples/model-conversion/../../convert_hf_to_gguf.py", line 12124, in main
    model_instance.write()
  File "/home/danbev/work/ai/llama.cpp/examples/model-conversion/../../convert_hf_to_gguf.py", line 716, in write
    self.prepare_metadata(vocab_only=False)
  File "/home/danbev/work/ai/llama.cpp/examples/model-conversion/../../convert_hf_to_gguf.py", line 857, in prepare_metadata
    self.set_vocab()
  File "/home/danbev/work/ai/llama.cpp/examples/model-conversion/../../convert_hf_to_gguf.py", line 829, in set_vocab
    self._set_vocab_gpt2()
  File "/home/danbev/work/ai/llama.cpp/examples/model-conversion/../../convert_hf_to_gguf.py", line 1336, in _set_vocab_gpt2
    tokens, toktypes, tokpre = self.get_vocab_base()
                               ^^^^^^^^^^^^^^^^^^^^^
  File "/home/danbev/work/ai/llama.cpp/examples/model-conversion/../../convert_hf_to_gguf.py", line 1005, in get_vocab_base
    tokenizer = AutoTokenizer.from_pretrained(self.dir_model)
                ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/danbev/work/ai/llama.cpp/venv/lib/python3.11/site-packages/transformers/models/auto/tokenization_auto.py", line 1153, in from_pretrained
    raise ValueError(
ValueError: Tokenizer class TokenizersBackend does not exist or is not currently imported.
make: *** [Makefile:41: causal-convert-model] Error 1

But this looks like a custom class and with the following the conversion worked:

diff --git a/tokenizer_config.json b/tokenizer_config.json
index 6be6ce1..a5adeb7 100644
--- a/tokenizer_config.json
+++ b/tokenizer_config.json
@@ -23,7 +23,7 @@
   "pad_token": "<|endoftext|>",
   "pretokenize_regex": "(?i:'s|'t|'re|'ve|'m|'ll|'d)|[^\\r\\n\\p{L}\\p{N}]?[\\p{L}\\p{M}]+|\\p{N}| ?[^\\s\\p{L}\\p{M}\\p{N}]+[\\r\\n]*|\\s*[\\r\\n]+|\\s+(?!\\S)|\\s+",
   "split_special_tokens": false,
-  "tokenizer_class": "TokenizersBackend",
+  "tokenizer_class": "Qwen2TokenizerFast",
   "unk_token": null,
   "video_token": "<|video_pad|>",
   "vision_bos_token": "<|vision_start|>",

@CISC
Copy link
Member Author

CISC commented Mar 5, 2026

This does fix the reported issue but the conversion fails with the following error for me:

Thanks for testing! :)

I think this is a new class in transformers 5...

Edit: Not sure about the validity of using it like that, but seen it before, also a bit strange with the pretokenize_regex in tokenizer_config.json, likely that this will give a different chkhsh, but don't think I'll bother with that in this PR.

@CISC CISC merged commit cf23251 into master Mar 5, 2026
9 checks passed
@CISC CISC deleted the cisc/convert-qwen35-forcausal branch March 5, 2026 09:30
bartowski1182 pushed a commit to bartowski1182/llama.cpp that referenced this pull request Mar 10, 2026
Ethan-a2 pushed a commit to Ethan-a2/llama.cpp that referenced this pull request Mar 20, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

python python script changes

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Misc. bug: convert_hf_to_gguf.py does not support the text version of Qwen3.5 Qwen3_5ForCausalLM is not supported

2 participants