bug: performance issues with accumulators for wide matrices

[Nashtare]

@adr1anh I recall you mentioned it in our previous call, would you mind sharing some additional details so that myself or someone else can investigate? Was it actually related to the implementation of this repo or also affecting upstream P3?

[adr1anh]

Sure! Basically our use of columnwise_dot_product_batched works quite well for us because of our narrow trace regime, allowing us to evaluate the next row for all matrices with little overhead.

Unfortunately the savings seem to diminish with wider matrices, to the point where it might be more efficient to run the unbatched version twice. My hunch is that this is due to the accumulator being too large and adding memory pressure.