Skip to content

Add Ernie 4.5 MOE and 0.3B Support#759

Merged
ikawrakow merged 2 commits intomainfrom
fcp/ernie4.5_moe
Sep 5, 2025
Merged

Add Ernie 4.5 MOE and 0.3B Support#759
ikawrakow merged 2 commits intomainfrom
fcp/ernie4.5_moe

Conversation

@firecoperana
Copy link
Copy Markdown
Collaborator

This is the port of Ernie 4.5 MoE model and Ernie 4.5 0.3B
In my test, response from the model looks ok.

Didn't test the python convert script. It's a straight copy/paste.

@firecoperana firecoperana self-assigned this Sep 4, 2025
@ikawrakow ikawrakow merged commit 33e0712 into main Sep 5, 2025
@Ph0rk0z
Copy link
Copy Markdown

Ph0rk0z commented Sep 5, 2025

This works for big ernie too?

@firecoperana
Copy link
Copy Markdown
Collaborator Author

Yes.

@AesSedai
Copy link
Copy Markdown
Contributor

AesSedai commented Sep 11, 2025

The python convert script is broken, because of the following error:

Traceback (most recent call last):
  File "/home/jarvis/development/ik_llama.cpp/convert_hf_to_gguf.py", line 2198, in <module>
    @ModelBase.register("Ernie4_5_ForCausalLM", "Ernie4_5ForCausalLM")
NameError: name 'ModelBase' is not defined

The mainline may use ModelBase, but in ik_llama it's just @Model.register

Edit: also, TextModel should just be Model in class Ernie4_5Model(Model):

@firecoperana
Copy link
Copy Markdown
Collaborator Author

Can you verify if #774 fix the error?

@AesSedai
Copy link
Copy Markdown
Contributor

@firecoperana confirmed that PR un-breaks the convert script. I haven't tried an ERNIE convert, I was doing a GLM-4.5 quant and just noticed that the script was totally busted, that PR fixes the issue I was seeing.

@firecoperana firecoperana deleted the fcp/ernie4.5_moe branch October 26, 2025 16:57
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants