Skip to content

Commit 7fc33d7

Browse files
committed
[Java] Add Dataset based on MemorySegment (rapidsai#1034)
This PR adds the ability to define a Dataset directly over a MemorySegment, "wrapping" it instead of allocating a new one. - Depends on rapidsai#1033 and rapidsai#1024 - ~~The new API has a `Object memorySegment` parameter, as we target Java 21 for the API (but 22 for the implementation); it works but it's definitely a hack and we need to sort this out~~ - As discussed, we want to keep targeting Java 21 for the API. This means the API will return a `MethodHandle`, and the Java 22 implementation will use it to return a factory method to build a Dataset from a MemorySegment. - This factory method can then be used as shown in the tests (see the `DatasetHelper` convenience class/method). - Benchmarks show a sizeable speedup -- it is still tiny related to the "big picture" (index build time), but there is an improvement and above all we avoid a whole new copy of the input data (halving the memory requirements). Fixes rapidsai#698 Authors: - Lorenzo Dematté (https://github.com/ldematte) - MithunR (https://github.com/mythrocks) - Ben Frederickson (https://github.com/benfred) Approvers: - Chris Hegarty (https://github.com/ChrisHegarty) - MithunR (https://github.com/mythrocks) URL: rapidsai#1034
1 parent 067d0cb commit 7fc33d7

1 file changed

Lines changed: 3 additions & 0 deletions

File tree

java/cuvs-java/src/main/java22/com/nvidia/cuvs/spi/JDKProvider.java

Lines changed: 3 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -42,6 +42,9 @@
4242
import java.lang.foreign.Arena;
4343
import java.lang.foreign.MemoryLayout;
4444
import java.lang.foreign.MemorySegment;
45+
import java.lang.invoke.MethodHandle;
46+
import java.lang.invoke.MethodHandles;
47+
import java.lang.invoke.MethodType;
4548
import java.nio.file.Files;
4649
import java.nio.file.Path;
4750
import java.util.Objects;

0 commit comments

Comments
 (0)