-
Notifications
You must be signed in to change notification settings - Fork 4.3k
Closed
Description
I am using Faiss-GPU, and my project occasionally encounters the error Faiss assertion 'allocs_.count(device) == 0' failed in virtual void faiss::gpu::StandardGpuResourcesImpl::initializeForDevice(int) at /xx/xx/faiss/gpu/StandardGpuResources.cpp:347 after running normally for some time. This causes the service to stop.
Below is the code in my project for using GPU search:
def __init__(self,*args):
...
self.gpu_res = faiss.StandardGpuResources() # use a single GPU
...
def search_gpu(self, embeddings):
num = embeddings.shape[0]
self.rwlock.acquire_read()
try:
if self.index.ntotal == 0:
return np.full((num, self.k), -1, dtype=np.float32), np.full((num, self.k), -1, dtype=np.int64)
gpu_index = faiss.index_cpu_to_gpu(self.gpu_res, 0, self.index)
D, I = gpu_index.search(embeddings, self.k)
finally:
self.rwlock.release_read()
return D, IThis service is a web service. Could the error be caused by concurrency issues leading to a failure in StandardGpuResources()? Please give me some possible reasons so I can investigate, such as CUDA version, Faiss version, or other factors?
faiss-gpu version: v1.7.4
Reactions are currently unavailable
Metadata
Metadata
Assignees
Labels
No labels