Support for signed int8 and float16 for HNSW and IVF using Flat and PQ encoders

[naveentatikonda]

Summary

Using float32 datatype to ingest vectors using IndexHNSW or IndexIVF (using Flat or PQ encoders) is getting expensive in terms of storage and memory especially for large scale use cases. Adding support for signed int8 and fp16 vector datatype support helps to reduce these memory footprints.

As of now, it supports uint8 for indexing Binary vectors and Scalar Quantizer has the fp16 support. Do you have any plans to support them for other methods and encoders mentioned above ?

[mdouze]

HNSW and IVF both support FP16, SQ8 and PQ compression, see
https://github.com/facebookresearch/faiss/wiki/The-index-factory

[navneet1v]

@naveentatikonda while creating the index you need to send SQfp16 instead Flat(which is fp64 or fp32) to take use of floats as 16bit.

@mdouze please correct if that is not the understanding.

[naveentatikonda]

@mdouze Thank you for your reply. It is actually hard to understand the encoder and decoder logic in this fp16 quantizer. Can you please confirm if we provide a quantized vector as input where each dimension is of type fp16 and try to ingest them using HNSW and SQfp16 encoder, will faiss support it ?

If it is supported, will it quantize this vector further and does the vector value change ?

[naveentatikonda]

@mdouze Thank you for your reply. It is actually hard to understand the encoder and decoder logic in this fp16 quantizer. Can you please confirm if we provide a quantized vector as input where each dimension is of type fp16 and try to ingest them using HNSW and SQfp16 encoder, will faiss support it ?

If it is supported, will it quantize this vector further and does the vector value change ?

@mdouze Did you get a chance to look into my question?

[mdouze]

Sorry I did not understand the question.
If you use SQfp16, the internal representation will be FP16.
You cannot provide a quantized vector, the index will do it at add time.
Is that clear?

[naveentatikonda]

@mdouze As of now, we have AVX2 SIMD optimization support for x86 architecture in ScalarQuantizer for QT_fp16. But, we don't have similar optimization for ARM based architecture. So, I'm working on adding NEON SIMD support for QT_fp16. What are your thoughts on adding NEON optimization support to QT_fp16 ?

[mdouze]

That would be great! There a areas of Faiss that are optimized with Neon but not the scalar quantizer. This is mainly because SQ was implemented before impl/simdlib.h existed.

[mdouze]

Yes sure, NEON is supported for some operations but not for scalar quantizer. This is mainly because impl/simdlib.h was not implemented when scalar quantization was added to Faiss.