Commit 3b4fca1
authored
ggml-cpu: Enable tiled matmul on AIX (ggml-org#25199)
The matmul_tiled path uses large local stack buffers for A_pack and B_pack. On AIX this can trigger a segmentation fault, so reduce the buffer footprint there to keep the tiled path usable.
Performance Impact:
~ 2x gains in PP_Speed for FP32, Q4_0 and Q8_0 models tested with llama-bench, llama-batched-bench and llama-cli.
Models used: Llama3.2 3b Instruct F32, qwen 2.5 3b Q4_0 and Q8_01 parent 86961ef commit 3b4fca1
1 file changed
Lines changed: 20 additions & 14 deletions
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
2321 | 2321 | | |
2322 | 2322 | | |
2323 | 2323 | | |
2324 | | - | |
2325 | | - | |
2326 | | - | |
2327 | | - | |
2328 | | - | |
| 2324 | + | |
2329 | 2325 | | |
| 2326 | + | |
| 2327 | + | |
| 2328 | + | |
| 2329 | + | |
| 2330 | + | |
| 2331 | + | |
| 2332 | + | |
| 2333 | + | |
2330 | 2334 | | |
2331 | | - | |
| 2335 | + | |
2332 | 2336 | | |
2333 | 2337 | | |
2334 | 2338 | | |
2335 | | - | |
| 2339 | + | |
2336 | 2340 | | |
2337 | 2341 | | |
2338 | | - | |
| 2342 | + | |
2339 | 2343 | | |
2340 | 2344 | | |
2341 | | - | |
| 2345 | + | |
2342 | 2346 | | |
2343 | 2347 | | |
2344 | 2348 | | |
| |||
2354 | 2358 | | |
2355 | 2359 | | |
2356 | 2360 | | |
2357 | | - | |
2358 | 2361 | | |
2359 | 2362 | | |
2360 | 2363 | | |
| |||
3195 | 3198 | | |
3196 | 3199 | | |
3197 | 3200 | | |
| 3201 | + | |
| 3202 | + | |
| 3203 | + | |
3198 | 3204 | | |
3199 | | - | |
3200 | | - | |
3201 | | - | |
| 3205 | + | |
| 3206 | + | |
| 3207 | + | |
| 3208 | + | |
3202 | 3209 | | |
3203 | 3210 | | |
3204 | 3211 | | |
3205 | 3212 | | |
3206 | 3213 | | |
3207 | | - | |
3208 | 3214 | | |
3209 | 3215 | | |
3210 | 3216 | | |
| |||
0 commit comments