Get Rows & Dequantize Function Adapted to repacked weights of type q4_K#4

Open

swetha097 wants to merge 91 commits intomasterfrom

swe_pr/q4_K_get_rows

Commits on Jun 3, 2025

Get_Rows & Dequantize implementation adapted to work for repacked weights of type q4_0

swetha097
authored and
mcw
committed

Commits on Jun 6, 2025

Resolve PR comments

swetha097
authored and
mcw
committed

Commits on Jun 16, 2025

Commits on Jun 17, 2025

Commits on Jun 18, 2025

Add the Get_Rows & Dequantize implementation adapted to work for repacked weights of type q4_K
swetha097
committed

Commits on Jun 20, 2025

Commits on Jun 21, 2025

Add `ggml_roll` (ggml/1274)

Acly
authored and
ggerganov
committed
ggml: Add Apple support for GGML_CPU_ALL_VARIANTS (llama/14258)

chaxu01
authored and
ggerganov
committed
ggml-cpu: fix uncaught underscore terminators (llama/14023)

taronaeo
authored and
ggerganov
committed
ggml-cpu: reduce asm calls for hsum (llama/14037)

taronaeo
authored and
ggerganov
committed
metal : add mean kernel (llama/14267)
ggerganov
committed
Vulkan: Set device max size for host memory to avoid OOM warning and fallback to CPU buffer (llama/14249)

0cc4m
authored and
ggerganov
committed
llamafile : support s390x SIMD instruction set (llama/14273)

taronaeo
authored and
ggerganov
committed
sycl: Cleanup codepaths in Get Rows in sycl backend (llama/14215)

ShanoToni
authored and
ggerganov
committed
build : suppress gcc15 compile warnings (llama/14261)

fanyang89
authored and
ggerganov
committed
ggml-cpu : remove unnecesary arm feature detection (llama/14281)

slaren
authored and
ggerganov
committed
CUDA: add conv_2d_dw (llama/14265)

am17an
authored and
ggerganov
committed
ggml: Update KleidiAI to v1.9.0 (llama/14277)

chaxu01
authored and
ggerganov
committed
ggml : fix repack work size for mul_mat_id (llama/14292)
ggerganov
committed
cuda : synchronize graph capture and cublas handle destruction (llama/14288)

slaren
authored and
ggerganov
committed
Implement GGML_CPU_ALL_VARIANTS for PowerPC (llama/14286)

authored and
ggerganov
committed
sycl: add usage of enqueue_functions extension (llama/14244)

s-Nick
authored and
ggerganov
committed
CUDA: add conv_2d_transpose (llama/14287)

am17an
authored and
ggerganov
committed
sync : ggml
ggerganov
committed
talk-llama : sync llama.cpp
ggerganov
committed

Commits on Jun 23, 2025

Commits on Jun 27, 2025

Commits on Jul 1, 2025

ggml-quants : rename best_mad to best_error (ggml/1283)

danbev
authored and
ggerganov
committed
ggml-cpu : "align corners" for bilinear upscale/downscale (ggml/1285)

Acly
authored and
ggerganov
committed
metal : fix thread-safety (llama/14300)
ggerganov
committed
Add support for VK_EXT_debug_utils to add labels to Vulkan objects. (llama/13792)

mtavenrath
authored and
ggerganov
committed
CUDA: add mean operation (llama/14313)

am17an
authored and
ggerganov
committed
HIP: enable vec fattn on RDNA4 (llama/14323)

IMbackK
authored and
ggerganov
committed
CUDA: mul_mat_v support for batch sizes > 1 (llama/14262)

JohannesGaessler
authored and
ggerganov
committed
CUDA/HIP: optimize mmv paths taken for HIP devices (llama/14324)

authored and
ggerganov
committed
opencl: ref count `ggml_backend_opencl_context` and refactor profiling (llama/14254)

lhez
authored and
ggerganov
committed
sycl: GGML_SYCL_DISABLE_OPT on by default for all Intel Devices (llama/13973)

ShanoToni
authored and
ggerganov
committed
ggml : do not output unprintable characters on GGUF load failure (llama/14381)

CISC
authored and
ggerganov
committed
ggml-cpu: enable IBM NNPA Vector Intrinsics (llama/14317)

authored and
ggerganov
committed
musa: enable fp16 mma (all) and cublas on qy2 (llama/13842)

authored and
ggerganov
committed
metal : batch rows copy in a single threadgroup (llama/14384)
ggerganov
committed
metal : add special-case mat-vec mul for ne00 == 4 (llama/14385)
ggerganov
committed
cmake: regen vulkan shaders when shaders-gen sources change (llama/14398)

mtmcp
authored and
ggerganov
committed
ggml : add ggml_set_rows (llama/14274)

rgerganov
and
ggerganov
committed
vulkan: Fix GGML_VULKAN_SHADER_DEBUG_INFO (llama/14427)

jeffbolznv
authored and
ggerganov
committed
fix async_mode bug (llama/14432)

bachelor-dou
authored and
ggerganov
committed
vulkan: lock accesses of pinned_memory vector (llama/14333)

jeffbolznv
authored and
ggerganov
committed
vulkan: handle noncontig in the final case of ggml_vk_get_cpy_pipeline (llama/14378)

jeffbolznv
authored and
ggerganov
committed
CUDA: add bf16 and f32 support to cublas_mul_mat_batched (llama/14361)

am17an
authored and
ggerganov
committed
vulkan: Add fusion support for RMS_NORM+MUL (llama/14366)

authored and
ggerganov
committed
ggml : implement REGLU/GEGLU/SWIGLU ops (llama/14158)

committed
ggml : fix unmerged GGML_FPxx_TO_FPxx refactoring (llama/14443)

CISC
authored and
ggerganov
committed
SYCL: disable faulty fp16 exp kernel (llama/14395)

qnixsynapse
authored and
ggerganov
committed
scripts : make the shell scripts cross-platform (llama/14341)

vedranmiletic
authored and
ggerganov
committed
cmake : Remove redundant include path in CMakeLists.txt (llama/14452)

xiaobing318
authored and
ggerganov
committed
ggml-cpu: sycl: Re-enable exp f16 (llama/14462)

Rbiessy
authored and
ggerganov
committed
metal : disable fast-math for some cpy kernels (llama/14460)
ggerganov
committed
Add Conv2d for CPU (llama/14388)

am17an
authored and
ggerganov
committed
opencl : add GEGLU, REGLU, SWIGLU (llama/14456)

lhez
authored and
ggerganov
committed
ggml : remove trailing whitespace (llama/0)
ggerganov
committed
talk-llama : sync llama.cpp
ggerganov
committed
sync : ggml
ggerganov
committed

Commits on Jul 2, 2025

feat: support vad for addon.node (#3301)

buxuku
and
linxiaodong
authored

Commits on Jul 3, 2025

Merge remote-tracking branch 'origin/master' into swe_pr/q4_K_get_rows
swetha097
committed