Skip to content

Lower sorted QMM gather threshold#2609

Merged
awni merged 1 commit intomainfrom
lower_sorted_gather
Sep 20, 2025
Merged

Lower sorted QMM gather threshold#2609
awni merged 1 commit intomainfrom
lower_sorted_gather

Conversation

@awni
Copy link
Copy Markdown
Member

@awni awni commented Sep 19, 2025

With GPT OSS for short prompts (256) it's much faster:

Pre: prompt_tps=480.979
Post: prompt_tps=974.984

@awni awni requested a review from angeloskath September 19, 2025 22:39
Copy link
Copy Markdown
Member

@angeloskath angeloskath left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

👍

@awni awni merged commit ec2ab42 into main Sep 20, 2025
6 of 7 checks passed
@awni awni deleted the lower_sorted_gather branch September 20, 2025 01:22
faisalmemon pushed a commit to faisalmemon/mlx that referenced this pull request Oct 30, 2025
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants