UPSTREAM PR #19064: Fix conversion for models that needed undo_permute = False but never declared it#1017
Open
UPSTREAM PR #19064: Fix conversion for models that needed undo_permute = False but never declared it#1017
undo_permute = False but never declared it#1017Conversation
|
Based on the analysis, no functions were identified with meaningful performance changes between the base and target versions. The function_insights_topk tool returned empty results for both response time and throughput time metrics, indicating that the code changes between these versions did not produce measurable performance impacts in the analyzed binaries. This suggests that the modifications were either:
Conclusion: No significant performance regression or improvement was detected between the two versions. See the complete breakdown in Version Insights |
a50395f to
8587aee
Compare
5fea2ef to
8a7ef20
Compare
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Mirrored from ggml-org/llama.cpp#19064
After #18866 modify_tensors was altered to call the super function
Previously, Afmoe and CogVLM both returned the tensors directly, bypassing the
super().modify_tensors()functionLlamaModel causes
undo_permuteto be set toTrue, which breaks the model by applying the Q/K permutation when the super function is calledThis should fix up those cases. With this change, PPL for Trinity-Mini returns to expected levels.
This error was discovered with the help of Claude Code but tested by a human