Commit c0c52dd
[SPARK-32268][SQL][FOLLOWUP] Add RewritePredicateSubquery below the InjectRuntimeFilter
### What changes were proposed in this pull request?
Add `RewritePredicateSubquery` below the `InjectRuntimeFilter` in `SparkOptimizer`.
### Why are the changes needed?
It seems if the runtime use in-subquery to do the filter, it won't be converted to semi-join as the design said.
This pr fixes the issue.
### Does this PR introduce _any_ user-facing change?
No, not released
### How was this patch tested?
Improve the test by adding: ensure the semi-join exists if the runtime filter use in-subquery code path.
Closes #35998 from ulysses-you/SPARK-32268-FOllOWUP.
Authored-by: ulysses-you <ulyssesyou18@gmail.com>
Signed-off-by: Wenchen Fan <wenchen@databricks.com>1 parent 8fab597 commit c0c52dd
2 files changed
Lines changed: 14 additions & 2 deletions
File tree
- sql/core/src
- main/scala/org/apache/spark/sql/execution
- test/scala/org/apache/spark/sql
Lines changed: 2 additions & 1 deletion
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
44 | 44 | | |
45 | 45 | | |
46 | 46 | | |
47 | | - | |
| 47 | + | |
| 48 | + | |
48 | 49 | | |
49 | 50 | | |
50 | 51 | | |
| |||
Lines changed: 12 additions & 1 deletion
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
19 | 19 | | |
20 | 20 | | |
21 | 21 | | |
22 | | - | |
| 22 | + | |
| 23 | + | |
23 | 24 | | |
24 | 25 | | |
25 | 26 | | |
| |||
213 | 214 | | |
214 | 215 | | |
215 | 216 | | |
| 217 | + | |
| 218 | + | |
| 219 | + | |
| 220 | + | |
| 221 | + | |
| 222 | + | |
| 223 | + | |
| 224 | + | |
| 225 | + | |
216 | 226 | | |
217 | 227 | | |
218 | 228 | | |
| |||
234 | 244 | | |
235 | 245 | | |
236 | 246 | | |
| 247 | + | |
237 | 248 | | |
238 | 249 | | |
239 | 250 | | |
| |||
0 commit comments