Commit 37d6b3c
[SPARK-32761][SQL][3.0] Allow aggregating multiple foldable distinct expressions
### What changes were proposed in this pull request?
For queries with multiple foldable distinct columns, since they will be eliminated during
execution, it's not mandatory to let `RewriteDistinctAggregates` handle this case. And
in the current code, `RewriteDistinctAggregates` *dose* miss some "aggregating with
multiple foldable distinct expressions" cases.
For example: `select count(distinct 2), count(distinct 2, 3)` will be missed.
But in the planner, this will trigger an error that "multiple distinct expressions" are not allowed.
As the foldable distinct columns can be eliminated finally, we can allow this in the aggregation
planner check.
### Why are the changes needed?
bug fix
### Does this PR introduce _any_ user-facing change?
No
### How was this patch tested?
added test case
Authored-by: Linhong Liu <linhong.liudatabricks.com>
Signed-off-by: Wenchen Fan <wenchendatabricks.com>
(cherry picked from commit a410658)
Closes #30052 from linhongliu-db/SPARK-32761-3.0.
Authored-by: Linhong Liu <linhong.liu@databricks.com>
Signed-off-by: Wenchen Fan <wenchen@databricks.com>1 parent 160f458 commit 37d6b3c
2 files changed
Lines changed: 8 additions & 2 deletions
File tree
- sql/core/src
- main/scala/org/apache/spark/sql/execution
- test/scala/org/apache/spark/sql
Lines changed: 4 additions & 2 deletions
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
517 | 517 | | |
518 | 518 | | |
519 | 519 | | |
520 | | - | |
| 520 | + | |
| 521 | + | |
521 | 522 | | |
522 | 523 | | |
523 | 524 | | |
| |||
548 | 549 | | |
549 | 550 | | |
550 | 551 | | |
551 | | - | |
| 552 | + | |
| 553 | + | |
552 | 554 | | |
553 | 555 | | |
554 | 556 | | |
| |||
Lines changed: 4 additions & 0 deletions
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
2467 | 2467 | | |
2468 | 2468 | | |
2469 | 2469 | | |
| 2470 | + | |
| 2471 | + | |
| 2472 | + | |
| 2473 | + | |
2470 | 2474 | | |
2471 | 2475 | | |
2472 | 2476 | | |
0 commit comments