Skip to content

Commit f6806e9

Browse files
committed
[native] Expose index join perf related session properties
Expose following Velox index join performance optimizations related query configs in Prestissimo as session properties: - native_index_lookup_join_max_prefetch_batches: Specifies the max number of input batches to prefetch to do index lookup ahead. If it is zero, then process one input batch at a time. - native_index_lookup_join_split_output: If this is true, then the index join operator might split output for each input batch based on the output batch size control. Otherwise, it tries to produce a single output for each input batch. - native_unnest_split_output: If this is true, then the unnest operator might split output for each input batch based on the output batch size control. Otherwise, it produces a single output for each input batch.
1 parent e53f403 commit f6806e9

4 files changed

Lines changed: 84 additions & 4 deletions

File tree

presto-docs/src/main/sphinx/presto_cpp/properties-session.rst

Lines changed: 29 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -487,3 +487,32 @@ This is used in global arbitration victim selection.
487487

488488
Maximum number of splits to listen to by the SplitListener per table scan node per
489489
native worker.
490+
491+
``native_index_lookup_join_max_prefetch_batches``
492+
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
493+
494+
* **Type:** ``integer``
495+
* **Default value:** ``0``
496+
497+
Specifies the max number of input batches to prefetch to do index lookup ahead.
498+
If it is zero, then process one input batch at a time.
499+
500+
``native_index_lookup_join_split_output``
501+
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
502+
503+
* **Type:** ``boolean``
504+
* **Default value:** ``true``
505+
506+
If this is true, then the index join operator might split output for each input
507+
batch based on the output batch size control. Otherwise, it tries to produce a
508+
single output for each input batch.
509+
510+
``native_unnest_split_output``
511+
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
512+
513+
* **Type:** ``boolean``
514+
* **Default value:** ``true``
515+
516+
If this is true, then the unnest operator might split output for each input
517+
batch based on the output batch size control. Otherwise, it produces a single
518+
output for each input batch.

presto-native-execution/presto_cpp/main/SessionProperties.cpp

Lines changed: 29 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -509,6 +509,35 @@ SessionProperties::SessionProperties() {
509509
0,
510510
QueryConfig::kMaxNumSplitsListenedTo,
511511
std::to_string(c.maxNumSplitsListenedTo()));
512+
513+
addSessionProperty(
514+
kIndexLookupJoinMaxPrefetchBatches,
515+
"Specifies the max number of input batches to prefetch to do index"
516+
"lookup ahead. If it is zero, then process one input batch at a time.",
517+
INTEGER(),
518+
false,
519+
QueryConfig::kIndexLookupJoinMaxPrefetchBatches,
520+
std::to_string(c.indexLookupJoinMaxPrefetchBatches()));
521+
522+
addSessionProperty(
523+
kIndexLookupJoinSplitOutput,
524+
"If this is true, then the index join operator might split output for"
525+
"each input batch based on the output batch size control. Otherwise, it tries to"
526+
"produce a single output for each input batch.",
527+
BOOLEAN(),
528+
false,
529+
QueryConfig::kIndexLookupJoinSplitOutput,
530+
std::to_string(c.indexLookupJoinSplitOutput()));
531+
532+
addSessionProperty(
533+
kUnnestSplitOutput,
534+
"In streaming aggregation, wait until we have enough number of output"
535+
"rows to produce a batch of size specified by this. If set to 0, then"
536+
"Operator::outputBatchRows will be used as the min output batch rows.",
537+
BOOLEAN(),
538+
false,
539+
QueryConfig::kUnnestSplitOutput,
540+
std::to_string(c.unnestSplitOutput()));
512541
}
513542

514543
const std::unordered_map<std::string, std::shared_ptr<SessionProperty>>&

presto-native-execution/presto_cpp/main/SessionProperties.h

Lines changed: 19 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -231,8 +231,7 @@ class SessionProperties {
231231
static constexpr const char* kQueryTraceDir = "native_query_trace_dir";
232232

233233
/// The plan node id whose input data will be traced.
234-
static constexpr const char* kQueryTraceNodeId =
235-
"native_query_trace_node_id";
234+
static constexpr const char* kQueryTraceNodeId = "native_query_trace_node_id";
236235

237236
/// The max trace bytes limit. Tracing is disabled if zero.
238237
static constexpr const char* kQueryTraceMaxBytes =
@@ -322,10 +321,27 @@ class SessionProperties {
322321
static constexpr const char* kMaxNumSplitsListenedTo =
323322
"native_max_num_splits_listened_to";
324323

324+
/// Specifies the max number of input batches to prefetch to do index lookup
325+
/// ahead. If it is zero, then process one input batch at a time.
326+
static constexpr const char* kIndexLookupJoinMaxPrefetchBatches =
327+
"native_index_lookup_join_max_prefetch_batches";
328+
329+
/// If this is true, then the index join operator might split output for each
330+
/// input batch based on the output batch size control. Otherwise, it tries to
331+
/// produce a single output for each input batch.
332+
static constexpr const char* kIndexLookupJoinSplitOutput =
333+
"native_index_lookup_join_split_output";
334+
335+
/// If this is true, then the unnest operator might split output for each
336+
/// input batch based on the output batch size control. Otherwise, it produces
337+
/// a single output for each input batch.
338+
static constexpr const char* kUnnestSplitOutput =
339+
"native_unnest_split_output";
340+
325341
static SessionProperties* instance();
326342

327343
SessionProperties();
328-
344+
329345
/// Utility function to translate a config name in Presto to its equivalent in
330346
/// Velox. Returns 'name' as is if there is no mapping.
331347
const std::string toVeloxConfig(const std::string& name) const;

presto-native-execution/presto_cpp/main/tests/SessionPropertiesTest.cpp

Lines changed: 7 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -117,7 +117,13 @@ TEST_F(SessionPropertiesTest, validateMapping) {
117117
{SessionProperties::kNativeQueryMemoryReclaimerPriority,
118118
core::QueryConfig::kQueryMemoryReclaimerPriority},
119119
{SessionProperties::kMaxNumSplitsListenedTo,
120-
core::QueryConfig::kMaxNumSplitsListenedTo}};
120+
core::QueryConfig::kMaxNumSplitsListenedTo},
121+
{SessionProperties::kIndexLookupJoinMaxPrefetchBatches,
122+
core::QueryConfig::kIndexLookupJoinMaxPrefetchBatches},
123+
{SessionProperties::kIndexLookupJoinSplitOutput,
124+
core::QueryConfig::kIndexLookupJoinSplitOutput},
125+
{SessionProperties::kUnnestSplitOutput,
126+
core::QueryConfig::kUnnestSplitOutput}};
121127

122128
const auto& sessionProperties =
123129
SessionProperties::instance()->testingSessionProperties();

0 commit comments

Comments
 (0)