Split by rows instead of layers for llama.cpp multi-gpu

[Ph0rk0z]

On some cards, the new splitting by layer causes performance. Even on 3090s, the utilization goes from over 50 to 43. P40s actually have demonstrable losses. This parameter lets you split by rows like theoriginal behavior and should fix those speed issues. Default behavior should still be splitting by layers.

[oobabooga]

Is there a reason to not have split by rows by default if it leads to better performance?

[Ph0rk0z]

I kept the default behavior of l.cpp and also have no way to test 4090 or all different combinations. I can say P40 gains its 2 or 3 t/s back and 3090 goes from 40% utilization per GPU to over 5X%.

Nothing makes it like pre ggml-org/llama.cpp#4606 unfortunately.

[oobabooga]

Fair enough