Commit 2f3997f
[SPARK-28306][SQL][FOLLOWUP] Fix NormalizeFloatingNumbers rule idempotence for equi-join with
## What changes were proposed in this pull request?
Idempotence of the `NormalizeFloatingNumbers` rule was broken due to the implementation of `ExtractEquiJoinKeys`. There is no reason that we don't remove `EqualNullSafe` join keys from an equi-join's `otherPredicates`.
## How was this patch tested?
A new UT.
Closes #25126 from yeshengm/spark-28306.
Authored-by: Yesheng Ma <kimi.ysma@gmail.com>
Signed-off-by: gatorsmile <gatorsmile@gmail.com><=> predicates1 parent 8d1e87a commit 2f3997f
2 files changed
Lines changed: 24 additions & 7 deletions
File tree
- sql/catalyst/src
- main/scala/org/apache/spark/sql/catalyst/planning
- test/scala/org/apache/spark/sql/catalyst/optimizer
Lines changed: 10 additions & 6 deletions
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
118 | 118 | | |
119 | 119 | | |
120 | 120 | | |
121 | | - | |
122 | | - | |
| 121 | + | |
| 122 | + | |
| 123 | + | |
| 124 | + | |
123 | 125 | | |
124 | | - | |
125 | | - | |
| 126 | + | |
| 127 | + | |
| 128 | + | |
| 129 | + | |
126 | 130 | | |
127 | 131 | | |
128 | 132 | | |
129 | 133 | | |
130 | | - | |
| 134 | + | |
131 | 135 | | |
132 | 136 | | |
133 | | - | |
| 137 | + | |
134 | 138 | | |
135 | 139 | | |
136 | 140 | | |
| |||
Lines changed: 14 additions & 1 deletion
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
19 | 19 | | |
20 | 20 | | |
21 | 21 | | |
22 | | - | |
| 22 | + | |
23 | 23 | | |
24 | 24 | | |
25 | 25 | | |
| |||
78 | 78 | | |
79 | 79 | | |
80 | 80 | | |
| 81 | + | |
| 82 | + | |
| 83 | + | |
| 84 | + | |
| 85 | + | |
| 86 | + | |
| 87 | + | |
| 88 | + | |
| 89 | + | |
| 90 | + | |
| 91 | + | |
| 92 | + | |
| 93 | + | |
81 | 94 | | |
82 | 95 | | |
0 commit comments