Commit 7c6c076
Support multiple catalogs for Presto spark native execution (prestodb#25943)
## Description
This PR implements native catalog properties support for Presto Spark,
enabling proper configuration and management of catalog properties for
native execution processes.
**Key Changes:**
* **Added `NativeExecutionCatalogProperties` class**: A new class that
holds catalog properties for native execution processes, where each
catalog generates a separate `<catalog-name>.properties` file.
* **Enhanced `WorkerProperty` and `PrestoSparkWorkerProperty`**:
Extended the worker property classes to support catalog properties
configuration and proper property file generation.
* **Updated Native Execution Module**: Modified `NativeExecutionModule`
and `NativeExecutionProcess` to integrate catalog properties into the
native execution workflow.
* **Improved Configuration Integration**: Updated `PrestoSparkModule`,
`PrestoSparkServiceFactory` to properly wire the new catalog properties
functionality.
* **Enhanced Test Coverage**: Added and updated tests in
`TestNativeExecutionSystemConfig`, `TestNativeExecutionProcess`, and
other test classes to ensure proper catalog properties handling.
## Motivation and Context
This change is required to support proper catalog configuration in
Presto Spark's native execution mode. Previously, catalog properties
were not properly managed for native execution processes, which limited
the ability to configure connectors effectively in native mode.
## Test Plan
* **Unit Tests**: Added test `NativeExecutionCatalogProperties`
* **Integration Tests**: Updated `TestNativeExecutionProcess` and
`TestPrestoSparkHttpClient` to verify catalog properties integration
## Contributor checklist
- [x] Please make sure your submission complies with our [contributing
guide](https://github.com/prestodb/presto/blob/master/CONTRIBUTING.md),
in particular [code
style](https://github.com/prestodb/presto/blob/master/CONTRIBUTING.md#code-style)
and [commit
standards](https://github.com/prestodb/presto/blob/master/CONTRIBUTING.md#commit-standards).
- [x] PR description addresses the issue accurately and concisely. If
the change is non-trivial, a GitHub Issue is referenced.
- [x] Adequate tests were added if applicable.
- [x] CI passed.
## Release Notes
```
== NO RELEASE NOTE ==
```1 parent ec69909 commit 7c6c076
12 files changed
Lines changed: 104 additions & 145 deletions
File tree
- presto-native-execution/src/test/java/com/facebook/presto/spark
- presto-spark-base/src
- main/java/com/facebook/presto/spark
- execution
- nativeprocess
- property
- test/java/com/facebook/presto/spark/execution
- http
- property
Lines changed: 2 additions & 2 deletions
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
18 | 18 | | |
19 | 19 | | |
20 | 20 | | |
21 | | - | |
22 | 21 | | |
23 | 22 | | |
24 | 23 | | |
| |||
125 | 124 | | |
126 | 125 | | |
127 | 126 | | |
128 | | - | |
| 127 | + | |
| 128 | + | |
129 | 129 | | |
130 | 130 | | |
131 | 131 | | |
| |||
Lines changed: 0 additions & 2 deletions
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
120 | 120 | | |
121 | 121 | | |
122 | 122 | | |
123 | | - | |
124 | 123 | | |
125 | 124 | | |
126 | 125 | | |
| |||
284 | 283 | | |
285 | 284 | | |
286 | 285 | | |
287 | | - | |
288 | 286 | | |
289 | 287 | | |
290 | 288 | | |
| |||
Lines changed: 2 additions & 1 deletion
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
27 | 27 | | |
28 | 28 | | |
29 | 29 | | |
| 30 | + | |
30 | 31 | | |
31 | 32 | | |
32 | 33 | | |
| |||
70 | 71 | | |
71 | 72 | | |
72 | 73 | | |
73 | | - | |
| 74 | + | |
74 | 75 | | |
75 | 76 | | |
76 | 77 | | |
| |||
Lines changed: 12 additions & 16 deletions
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
13 | 13 | | |
14 | 14 | | |
15 | 15 | | |
16 | | - | |
17 | | - | |
18 | | - | |
| 16 | + | |
19 | 17 | | |
20 | 18 | | |
21 | 19 | | |
| |||
32 | 30 | | |
33 | 31 | | |
34 | 32 | | |
| 33 | + | |
35 | 34 | | |
36 | 35 | | |
37 | 36 | | |
| |||
40 | 39 | | |
41 | 40 | | |
42 | 41 | | |
43 | | - | |
| 42 | + | |
44 | 43 | | |
45 | 44 | | |
46 | 45 | | |
47 | 46 | | |
48 | | - | |
| 47 | + | |
49 | 48 | | |
50 | 49 | | |
51 | 50 | | |
52 | 51 | | |
53 | 52 | | |
54 | | - | |
| 53 | + | |
55 | 54 | | |
56 | | - | |
| 55 | + | |
57 | 56 | | |
58 | 57 | | |
59 | 58 | | |
| |||
74 | 73 | | |
75 | 74 | | |
76 | 75 | | |
| 76 | + | |
| 77 | + | |
| 78 | + | |
| 79 | + | |
| 80 | + | |
77 | 81 | | |
78 | 82 | | |
79 | | - | |
80 | | - | |
81 | | - | |
82 | | - | |
83 | | - | |
84 | | - | |
85 | | - | |
86 | | - | |
87 | | - | |
| 83 | + | |
88 | 84 | | |
89 | 85 | | |
90 | 86 | | |
| |||
Lines changed: 1 addition & 2 deletions
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
343 | 343 | | |
344 | 344 | | |
345 | 345 | | |
346 | | - | |
347 | | - | |
| 346 | + | |
348 | 347 | | |
349 | 348 | | |
350 | 349 | | |
| |||
Lines changed: 37 additions & 0 deletions
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
| 1 | + | |
| 2 | + | |
| 3 | + | |
| 4 | + | |
| 5 | + | |
| 6 | + | |
| 7 | + | |
| 8 | + | |
| 9 | + | |
| 10 | + | |
| 11 | + | |
| 12 | + | |
| 13 | + | |
| 14 | + | |
| 15 | + | |
| 16 | + | |
| 17 | + | |
| 18 | + | |
| 19 | + | |
| 20 | + | |
| 21 | + | |
| 22 | + | |
| 23 | + | |
| 24 | + | |
| 25 | + | |
| 26 | + | |
| 27 | + | |
| 28 | + | |
| 29 | + | |
| 30 | + | |
| 31 | + | |
| 32 | + | |
| 33 | + | |
| 34 | + | |
| 35 | + | |
| 36 | + | |
| 37 | + | |
Lines changed: 0 additions & 86 deletions
This file was deleted.
Lines changed: 3 additions & 3 deletions
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
19 | 19 | | |
20 | 20 | | |
21 | 21 | | |
22 | | - | |
| 22 | + | |
23 | 23 | | |
24 | 24 | | |
25 | 25 | | |
26 | | - | |
| 26 | + | |
27 | 27 | | |
28 | 28 | | |
29 | 29 | | |
30 | | - | |
| 30 | + | |
31 | 31 | | |
32 | 32 | | |
Lines changed: 24 additions & 8 deletions
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
64 | 64 | | |
65 | 65 | | |
66 | 66 | | |
67 | | - | |
| 67 | + | |
68 | 68 | | |
69 | | - | |
| 69 | + | |
70 | 70 | | |
71 | 71 | | |
72 | 72 | | |
73 | 73 | | |
74 | | - | |
| 74 | + | |
75 | 75 | | |
76 | 76 | | |
77 | 77 | | |
78 | 78 | | |
79 | 79 | | |
80 | | - | |
| 80 | + | |
81 | 81 | | |
82 | 82 | | |
83 | | - | |
| 83 | + | |
84 | 84 | | |
85 | | - | |
| 85 | + | |
86 | 86 | | |
87 | 87 | | |
88 | 88 | | |
| |||
95 | 95 | | |
96 | 96 | | |
97 | 97 | | |
98 | | - | |
| 98 | + | |
99 | 99 | | |
100 | 100 | | |
101 | 101 | | |
102 | 102 | | |
103 | | - | |
| 103 | + | |
104 | 104 | | |
105 | 105 | | |
106 | 106 | | |
| |||
122 | 122 | | |
123 | 123 | | |
124 | 124 | | |
| 125 | + | |
| 126 | + | |
| 127 | + | |
| 128 | + | |
| 129 | + | |
| 130 | + | |
| 131 | + | |
| 132 | + | |
| 133 | + | |
| 134 | + | |
| 135 | + | |
| 136 | + | |
| 137 | + | |
| 138 | + | |
| 139 | + | |
| 140 | + | |
125 | 141 | | |
Lines changed: 2 additions & 2 deletions
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
22 | 22 | | |
23 | 23 | | |
24 | 24 | | |
25 | | - | |
| 25 | + | |
26 | 26 | | |
27 | 27 | | |
28 | 28 | | |
| |||
88 | 88 | | |
89 | 89 | | |
90 | 90 | | |
91 | | - | |
| 91 | + | |
92 | 92 | | |
93 | 93 | | |
94 | 94 | | |
| |||
0 commit comments