[BUG] FAISS+cuVS not building with the instructions

[cjnolet]

@alexanderguzhva was not able to build FAISS+cuVS after following the instructions. This is not good- we should expect that users will be able to check out the cuVS codebase or install the conda package and be able to build it with FAISS.

[tarang-jain]

Can I get the build command used and the error logs?

[cjnolet]

@alexanderguzhva any chance you could provide a link to the instructions you were using?

[alexanderguzhva]

@cjnolet, @tarang-jain as far as I remember, I compiled faiss on ubuntu 22.04 with the default gcc-11 (or maybe clang-18, not sure) by running git clone https://github.com/facebookresearch/faiss and later applying the standard CUVS-enabled compilation cmake -DFAISS_ENABLE_CUVS=ON -DFAISS_ENABLE_GPU=ON ... command. There was a compilation error, related to a C++ compiler (basically, some method call mismatches?)

I'll be able to provide more accurate instructions a bit later.

[alexanderguzhva]

@cjnolet @tarang-jain were you able to reproduce the problem?

[tarang-jain]

I have been using gcc_linux-64=12 and it works for me. Can you share the error log? It is hard to guess what you are seeing without looking at the logs.

[alexanderguzhva]

@tarang-jain it seems that it was a lack of installation instructions on my end

I've seen that FAISS got upgraded to cuvs 24.12 (faiss commit 5637bb88964c25c5fd916d9df7550e7e48b383fc).
So, I took a fresh empty docker container, downloaded faiss via git clone https://github.com/facebookresearch/faiss and run

root@903efab45b2d:~/faiss/faiss# cmake -DFAISS_ENABLE_PYTHON=OFF -B build -DFAISS_ENABLE_GPU=ON -DFAISS_ENABLE_CUVS=ON .

CMake Warning at CMakeLists.txt:85 (find_package):
  By not providing "Findcuvs.cmake" in CMAKE_MODULE_PATH this project has
  asked CMake to find a package configuration file provided by "cuvs", but
  CMake did not find one.

  Could not find a package configuration file provided by "cuvs" with any of
  the following names:

    cuvsConfig.cmake
    cuvs-config.cmake

  Add the installation prefix of "cuvs" to CMAKE_PREFIX_PATH or set
  "cuvs_DIR" to a directory containing one of the above files.  If "cuvs"
  provides a separate development package or SDK, be sure it has been
  installed.

-- Configuring done (2.1s)
CMake Error at faiss/gpu/CMakeLists.txt:338 (target_link_libraries):
  Target "faiss_gpu" links to:

    cuvs::cuvs

  but the target was not found.  Possible reasons include:

    * There is a typo in the target name.
    * A find_package call is missing for an IMPORTED target.
    * An ALIAS target is missing.



CMake Error at faiss/gpu/test/CMakeLists.txt:28 (target_link_libraries):
  Target "faiss_gpu_test_helper" links to:

    cuvs::cuvs

  but the target was not found.  Possible reasons include:

    * There is a typo in the target name.
    * A find_package call is missing for an IMPORTED target.
    * An ALIAS target is missing.



-- Generating done (0.2s)
CMake Generate step failed.  Build files cannot be regenerated correctly.

So, Faiss does not auto-install cuvs via something like https://github.com/rapidsai/cuvs/blob/branch-24.12/examples/cmake/thirdparty/get_cuvs.cmake .

So, it seems that the right way to go ahead is to install cuvs as a prereq using

git clone https://github.com/rapidsai/cuvs
git checkout branch-24.12
INSTALL_PREFIX=/usr/local ./build.sh libcuvs

, then install and build rmm as well

git clone https://github.com/rapidsai/rmm
git checkout branch-24.12
INSTALL_PREFIX=/usr/local ./build.sh libcuvs

, then build faiss using cmake -DFAISS_ENABLE_PYTHON=OFF -B build -DFAISS_ENABLE_GPU=ON -DFAISS_ENABLE_CUVS=ON .

The problem is that the necessity of INSTALL_PREFIX=/usr/local is not super evident, and also Faiss does not mention installation instructions for cuvs in its INSTALL.md file.

Faiss got compiled. It seems that appropriate building instructions need to be added to INSTALL.md of Faiss. Also, it is not clear how to completely disable compiling baseline Faiss GPU implementation and keep cuvs-based only (setting -DFAISS_ENABLE_GPU=OFF -DFAISS_ENABLE_CUVS=ON for cmake disables GPU)

[tarang-jain]

That's right. libcuvs would have to be installed in the build environment. I have an open PR to update the docs to reflect this. FAISS' github CI tests building FAISS from source with a conda install of libcuvs, which is also what I would recommend you to do:

A conda install of libcuvs ensures that the upstream dependencies such as -- librmm and libraft-headers are automatically installed. Note that FAISS' CMake only fetches and links against the already installed cuvs targets, but does not build cuvs from source. To install libcuvs:

For CUDA 12.0 to 12.5:

conda install -c rapidsai -c conda-forge -c nvidia  \
    libcuvs=24.12 'cuda-version>=12.0,<=12.5'

For CUDA 11.4 to 11.8:

conda install -c rapidsai -c conda-forge -c nvidia  \
    libcuvs=24.12 'cuda-version>=11.4,<=11.8'

With a conda install, you wouldn't need the INSTALL_PREFIX=/usr/local.

With -DFAISS_ENABLE_CUVS=ON, -DFAISS_ENABLE_GPU will also have to be set to ON. So the FAISS_ENABLE_GPU flag can be though of more like a flag to enable/disable compiling CUDA code, and would therefore have to be set to ON in order to use the cuvs GPU implementations. There are certain scenarios in which irrespective of whether use_cuvs is set to True, FAISS' classical GPU implementation will be used. This is true for algorithms that do not have cuvs support (for example GpuIndexIVFScalarQuantizer) and in situations where the Pascal GPU architecture (such as P100) is used, for which RAPIDS support has been phased out.

Thanks for pointing this out. The documentation is still WIP and we should have clearer instructions in their next stable release.