Improve performance

[DilumAluthge]

This is the continuation of MasonProtter/Gaius.jl#11

cc: @MasonProtter @chriselrod

[DilumAluthge]

BLIS multithreading docs: https://github.com/flame/blis/blob/master/docs/Multithreading.md

[DilumAluthge]

BLIS graphic:

[chriselrod]

By M, K, and N loops I mean the 3rd, 4th, and 5th loops, respectively.

Like the BLIS docs say, threading the K loop isn't particularly practical. Maybe that's something we could explore eventually for the case where K is much larger than M and N. We'd basically just do a bunch of separate multiplications and sum them together afterwards.

So I'd follow the plan of spawning NUM_CORES tasks, and then having them synchronize as necessary through the multiplication.
Depending on the sizes of M and N, we'd choose how to partition them to make use of the threads, and how packing fits into the picture.

1. Small M and small N - we probably need to divide up both M and N to get good thread utilization. We probably won't bother packing B here, and may or may not pack A depending on its properties.
2. Large M and small N - partition M.
3. Small M and large N - I haven't experimented here, so I don't really know what works well. Maybe try packing neither A or B?
4. Large M and large N - partition M, and repeatedly synchronize every time we repack B.

[DilumAluthge]

4. Large M and large N - partition M, and repeatedly synchronize every time we repack B.

Is this equivalent to saying that for large M and large N, we parallelize loop 3 only (and do not parallelize loop 5)?

[chriselrod]

Yes, where "large" is defined as whatever's big enough for that to make sense.

Some testing will have to show whether that is M > (mr * W * NUM_CORES) or M > Mc * NUM_CORES.

[DilumAluthge]

Obviously there is more work to do, and more improvements that we are going to add, but I'm going to close this broad issue as fixed by #41

And then we can open more focused issues for specific perf improvements.