ScaNN: Overlapped gather for AVQ

[rmaschal]

Adds a class cluster_loader for AVQ that enables overlapping the gather operation and HtoD copy with GPU computation.

There are two scenarios:

1. dataset on device: This is identical to the previous code, using raft::matrix::gather to perform the gather on device.
2. dataset on host: cluster_loader allocates to pinned buffers in host for fast (and possibly async) copies of cluster vectors DtoH. The actual gather operation is performed on cpu, into the pinned buffer. Copies can be overlapped with GPU work (namely AVQ update of the previous cluster) if scheduled on a separate stream.

[tfeher]

Thanks @rmaschal for the PR, looks good overall, I just have one suggestion for a cosmetic change.

In general, had to think about why do we need a specific version of gather here. Although we have h2d gather in RAFT (admittedly not properly exposed), here an explicit prefetch method enables the user to schedule additional GPU operations in parallel. That looks good to me.

[tfeher]

Thanks for the update, LGTM!

[bkarsin]

PR looks good to me, thanks for the clarification on my questions.

[bkarsin]

PR looks good to me, thanks (and very sorry for the mistaken close/open)

[tfeher]

/merge

[stic]

Hi @bkarsin,
The assumption is that there is one cluster or clusters will be of same / similar size? Asking as if I've read it correctly max for buffers will be per biggest cluster?