BitwiseHamming distance for NN Descent

[jinsolp]

Adding bitwise hamming distance for NN Descent

[jinsolp]

The only diff here is wrapping the wmma operation part with if (metric != cuvs::distance::DistanceType::BitwiseHamming) so that we don't perform unnecessary computations for BitwiseHamming. (the diff seems to detect all additional indents as changes which makes it confusing)

[divyegala]

We should be able to allocate just half the fp16 matrix for bitwise hamming, right?

[jinsolp]

Yep that's what we do here!

[tarang-jain]

wherever you are doing this, have you ensured that the dim that you pass along is correctly scaled? If you were to convert a half to two uint8s, the dim would have to be doubled.
Furthermore, I haven't looked into the nn descent logic entirely, but in case you are operating in the half space, I dont think you'd have to reinterpret_cast everywhere to uint8. Its more efficient to stay in the half space and do something like:
__popc(static_cast<uint32_t>(data_n1[d] ^ data_n2[d]) & 0xffffu)

[jinsolp]

wherever you are doing this, have you ensured that the dim that you pass along is correctly scaled? If you were to convert a half to two uint8s, the dim would have to be doubled.

I am allocating dim/2 dimensions for the half type array. To make things straightforward, the dim is always used as the dimension of the "given dataset" (fp32, int8 etc). And dimensions for allocating the fp16 type device array is configured based on the data type (dim/2 for int8 and uint8, as-is for other types)

I dont think you'd have to reinterpret_cast everywhere to uint8.

the issue with keeping the pointer fp16 and doing this for data_dim/2 dimensions results in having to check if the last byte is a valid value or not inside the kernel (because the original int8 data could have an odd number of dimensions). I thought it would be easier to cast to int8 to loop over the original data_dim instead : )

[tarang-jain]

I see. Yes there can be an odd number of dims, for which we can fall back to int8, but if its not an odd number of dims, we can do it in the half space. I'd argue that we can do even more -- if it is divisible by 4, reinterpret_cast to uint32_t so you'd have to popcount only over one fourth the dim (I'm doing the same thing with BitwiseHamming in ivf-flat). However, considering the deadlines, we can look into these optimizations later. Can we create a github issue for this and write it as a TODO here?
Regarding the dims, I just wanted to verify if the dims being used everywhere need to be scaled, but it looks like you have already checked those things, so apart from the creation of that github issue this PR looks okay to me.

[jinsolp]

Makes sense, added an issue here #1127

[divyegala]

/merge