Adding example for tuning build and search params using Optuna

[dpadmanabhan03]

No description provided.

[copy-pr-bot]

This pull request requires additional validation before any workflows can run on NVIDIA's runners.

Pull request vetters can view their responsibilities here.

Contributors can view more details about this message here.

[cjnolet]

/ok to test

[review-notebook-app]

Check out this pull request on 

See visual diffs & provide feedback on Jupyter Notebooks.

---

Powered by ReviewNB

[cjnolet]

The challenge with vector search is that there's a dual objective here- it's not just about the final recall achieved but rather the tradeoff between a "good enough" recall with "good enough" search latency / throughput. The problem with only using recall is that you will usually find the search latency is at its highest. I think we have to consider both in here up front, otherwise this example isn't particularly useful in the practice. Maybe we can accept a target recall try to minimize search latency that reaches at least to the target recall.

Reply via ReviewNB

[cjnolet]

Line #12.        n_lists = trial.suggest_int("n_lists", math.sqrt(dataset_size)*0.5, math.sqrt(dataset_size)*1.5)

This looks great, but I think we should provide a wider range of values to try so that we can see the objective in action and hit our target recall/latency.

Reply via ReviewNB

[cjnolet]

@dpadmanabhan03 I think this example looks great. I think we should expand it to cover ivf-flat, ivf-pq, and cagra and add the dual objectives in there to accept the target recall. We should also try to prefer the lowest build times by using the inverse of the build time in the score for the objective.

[cjnolet]

Hi @dpadmanabhan03. The notebook is looking great. Can you also copy/add a notebook that has a couple cpu comparisons in there? (You can use FAISS as the CPU library since it has graphbased HNSW and the ivf methods)

[cjnolet]

/ok to test

[cjnolet]

/ok to test

[cjnolet]

/merge