Skip to content

Navigation Menu

Appearance settings

View all features
- BY COMPANY SIZE
  Enterprises
  Small and medium teams
  Startups
  Nonprofits
- BY USE CASE
  App Modernization
  DevSecOps
  DevOps
  CI/CD
  View all use cases
- BY INDUSTRY
  Healthcare
  Financial services
  Manufacturing
  Government
  View all industries
View all solutions
- EXPLORE BY TOPIC
  AI
  Software Development
  DevOps
  Security
  View all topics
- EXPLORE BY TYPE
  Customer stories
  Events & webinars
  Ebooks & reports
  Business insights
  GitHub Skills
- SUPPORT & SERVICES
  Documentation
  Customer support
  Community forum
  Trust center
  Partners
View all resources
- COMMUNITY
  GitHub SponsorsFund open source developers
- PROGRAMS
  Security Lab
  Maintainer Community
  Accelerator
  Archive Program
- REPOSITORIES
  Topics
  Trending
  Collections
- ENTERPRISE SOLUTIONS
  Enterprise platformAI-powered developer platform
- AVAILABLE ADD-ONS
  GitHub Advanced SecurityEnterprise-grade security features
  Copilot for BusinessEnterprise-grade AI features
  Premium SupportEnterprise-grade 24/7 support
Pricing

Search code, repositories, users, issues, pull requests...

Search

Clear

Search syntax tips

Provide feedback

We read every piece of feedback, and take your input very seriously.

Include my email address so I can be contacted

Saved searches

Use saved searches to filter your results more quickly

Name

Query

To see all available qualifiers, see our documentation.

Appearance settings

You signed in with another tab or window. Reload to refresh your session. You signed out in another tab or window. Reload to refresh your session. You switched accounts on another tab or window. Reload to refresh your session.

Dismiss alert

ggml-org / ggml Public

Notifications You must be signed in to change notification settings
Fork 1.5k
Star 14.2k

Code
Issues 280
Pull requests 45
Discussions
Actions
Projects
Security
Insights

Additional navigation options

Code
Issues
Pull requests
Discussions
Actions
Projects
Security
Insights

4-bit Integer quantisation#27

Merged

ggerganov merged 38 commits intomasterggml-org/ggml:masterfrom

gqggml-org/ggml:gqCopy head branch name to clipboard

Mar 29, 2023

Conversation Commits38 (38)Checks Files changed

Merged

4-bit Integer quantisation#27
ggerganov merged 38 commits intomasterfrom
gq

Commits

Commits on Mar 29, 2023

gq : attempt at n-bit quantization
ggerganov
committed
gq : add amax based method 3
ggerganov
committed
gq : progress on method 2
ggerganov
committed
gq : method 4 (AVX2)
ggerganov
committed
gq : method 4 (ARM)
ggerganov
committed
gq : method 4 (AVX2 attempt) + method 5 (no min)
ggerganov
committed
gq : method 5 (ARM)
ggerganov
committed
gpt-2 : model conversion for Q4_0 quantization
ggerganov
committed
ggml : Q4_0 quantization support (ggml_get_rows())
ggerganov
committed
gpt-2 : loading Q4_0 quantized model
ggerganov
committed
ggml : q4_0 quantization support
ggerganov
committed
ggml : q4_1 quantization support (seems to work for bigger models)
ggerganov
committed
gpt-2 : add gpt-2-quantize tool for quantizing f32 GPT-2 models
ggerganov
committed
ggml : 4-bit quantization works (only scalar for now)
ggerganov
committed
gq : add method 6 (ARM)
ggerganov
committed
ggml : vectorized mad q4_0 (ARM)
ggerganov
committed
ggml : vectorized quantize_row_q4_0 (ARM)
ggerganov
committed
ggml : simplify mad q4_0 (ARM)
ggerganov
committed
ggml : minor indentations
ggerganov
committed
gpt-j : support for 4-bit quantized model inference
ggerganov
committed
ggml : GGML_ASSERT() instead of assert() where appropriate
ggerganov
committed
gpt : avoid ggml_transpose on model tensors (new models!)
ggerganov
committed
gpt-2 : minor
ggerganov
committed
gpt-j : fix conversion for FP16 models (such as GPT-JT-6B)
ggerganov
committed
ggml : add ggml_compute_forward_rope_f16()
ggerganov
committed
gpt : fix memory usage computation
ggerganov
committed
ggml : fix ggml_is_contiguous() to take into account blck size
ggerganov
committed
whisper : add whisper-qunatize tool
ggerganov
committed
whisper : add support for quantized models
ggerganov
committed
whisper : mem usage based on model format type
ggerganov
committed
gpt : seems not worth to use FP16 for KV cache
ggerganov
committed
gpt : support quantisation of f16 models files
ggerganov
committed
ggml : fixes for rpi4
ggerganov
committed
whisper : add Q4_1 model sizes
ggerganov
committed
ggml : add WASM SIMD for Q4_0
ggerganov
committed
utils : print quantization histograms
ggerganov
committed
ggml : sync all changes from llama.cpp and whisper.cpp
ggerganov
committed
ggml : finalize the Q4_1 quantization for ARM_NEON
ggerganov
committed

Footer

© 2026 GitHub, Inc.

Footer navigation

Terms
Privacy
Security
Status
Community
Docs
Contact

You can’t perform that action at this time.