This PR significantly improves POSIX I/O write performance as well as cold-page-cache read by opportunistically using Direct I/O. The speedup for sequential write is approximately 3~4x. The opportunistic POSIX Direct I/O feature can be controlled in two ways: - Environment variables: - `"KVIKIO_AUTO_DIRECT_IO_READ"`: defaults to `false`. - `"KVIKIO_AUTO_DIRECT_IO_WRITE"`: defaults to `true`. - C++/Python API - `defaults::set_auto_direct_io_read(bool flag)`/`kvikio.defaults.set("posix_auto_direct_io_read", flag)` - `defaults::set_auto_direct_io_write(bool flag)`/`kvikio.defaults.set("posix_auto_direct_io_write", flag)` In addition, this PR refactors the bounce buffer class. To improve clarity, relevant classes and variables have been renamed and a lot of comments added. The bounce buffer class is now templated by allocator to accommodate different use cases: - `PageAlignedBounceBufferPool`: used for Direct I/O to/from unaligned host buffer. Does not require CUDA context. - `CudaPinnedBounceBufferPool`: used for buffered I/O to/from device buffer. Requires CUDA context. This is the original implementation on the main branch. - `CudaPageAlignedPinnedBounceBufferPool`: used for Direct I/O to/from device buffer. Requires CUDA context. ## Performance results See #863 (comment) ## Goal - Addresses most part of #761 - Addresses the reported write performance issue in cudf ## Non-goal - This PR does not add opportunistic Direct I/O as file handle's function parameters. This will be revisited in a future PR. - This PR does not address one of the objectives in #520, which is to unify the implementation of the bounce buffer in POSIX IO and in Remote IO. This will be revisited in a future PR. Authors: - Tianyu Liu (https://github.com/kingcrimsontianyu) Approvers: - Mads R. B. Kristensen (https://github.com/madsbk) - Vukasin Milovanovic (https://github.com/vuule) URL: #863

Double buffered bounce buffer #520

Description

Metadata

Metadata

Assignees

Labels

Type

Projects

Milestone

Relationships

Development

Issue actions