Fix overflow issue in interpolation search

[mariosasko]

Fixes #2335

More info about this error can be found here.

[mariosasko]

Seems like the CI failure is unrelated to this PR (fixed with the merge).

@lhoestq Can you please verify that everything is OK in terms of speed? Another solution is to change the offsets array dtype to np.int64 (but this doesn't scale in theory compared to Python integer which is unbound). I'm not sure why on my 64-bit machine the default numpy dtype is np.int32 tho.

[mtala3t]

The fix is OK.

[lhoestq]

Hi ! Thanks for the fix.
Unfortunately in terms of speed this is not acceptable :/
The get_batch_of_1024_random_rows metric or the benchmark_getitem_100B benchmark is almost at 1sec instead of a few milliseconds.

Would it be possible to avoid the overflow by simply passing dtype=np.int64 to np.cumsum ?
On windows machines the default is int32 unfortunately so we have to force the dtype to be int64

[mariosasko]

Yes, casting the array to np.int64 should work as well. Another option would be to cast the array elements (arr[i], arr[j]) in interpolation search to Python integers (bound only with memory) before multiplication (the error stems from this part: (j - i) * (x - arr[i])) when working with big values. But for now, the first option is OK for the sake of simplicity.

[lhoestq]

Thanks :)