Throughput improvement for small batch sizes#17342

uttampc1 wants to merge 1 commit intoggml-org:masterggml-org/llama.cpp:masterfrom

uttampc1:core-scaling-optuttampc1/llama.cpp:core-scaling-optCopy head branch name to clipboard