Commit 7fc33d7
committed
[Java] Add Dataset based on
This PR adds the ability to define a Dataset directly over a MemorySegment, "wrapping" it instead of allocating a new one.
- Depends on rapidsai#1033 and rapidsai#1024
- ~~The new API has a `Object memorySegment` parameter, as we target Java 21 for the API (but 22 for the implementation); it works but it's definitely a hack and we need to sort this out~~
- As discussed, we want to keep targeting Java 21 for the API. This means the API will return a `MethodHandle`, and the Java 22 implementation will use it to return a factory method to build a Dataset from a MemorySegment.
- This factory method can then be used as shown in the tests (see the `DatasetHelper` convenience class/method).
- Benchmarks show a sizeable speedup -- it is still tiny related to the "big picture" (index build time), but there is an improvement and above all we avoid a whole new copy of the input data (halving the memory requirements).
Fixes rapidsai#698
Authors:
- Lorenzo Dematté (https://github.com/ldematte)
- MithunR (https://github.com/mythrocks)
- Ben Frederickson (https://github.com/benfred)
Approvers:
- Chris Hegarty (https://github.com/ChrisHegarty)
- MithunR (https://github.com/mythrocks)
URL: rapidsai#1034MemorySegment (rapidsai#1034)1 parent 067d0cb commit 7fc33d7
1 file changed
Lines changed: 3 additions & 0 deletions
Lines changed: 3 additions & 0 deletions
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
42 | 42 | | |
43 | 43 | | |
44 | 44 | | |
| 45 | + | |
| 46 | + | |
| 47 | + | |
45 | 48 | | |
46 | 49 | | |
47 | 50 | | |
| |||
0 commit comments