Commit 66fa6bd
[SPARK-19451][SQL] rangeBetween method should accept Long value as boundary
## What changes were proposed in this pull request?
Long values can be passed to `rangeBetween` as range frame boundaries, but we silently convert it to Int values, this can cause wrong results and we should fix this.
Further more, we should accept any legal literal values as range frame boundaries. In this PR, we make it possible for Long values, and make accepting other DataTypes really easy to add.
This PR is mostly based on Herman's previous amazing work: hvanhovell@596f53c
After this been merged, we can close #16818 .
## How was this patch tested?
Add new tests in `DataFrameWindowFunctionsSuite` and `TypeCoercionSuite`.
Author: Xingbo Jiang <[email protected]>
Closes #18540 from jiangxb1987/rangeFrame.
(cherry picked from commit 92d8563)
Signed-off-by: gatorsmile <[email protected]>1 parent 24a9bac commit 66fa6bd
17 files changed
Lines changed: 533 additions & 304 deletions
File tree
- sql
- catalyst/src
- main/scala/org/apache/spark/sql/catalyst
- analysis
- expressions
- parser
- trees
- test/scala/org/apache/spark/sql/catalyst
- analysis
- parser
- trees
- core/src
- main/scala/org/apache/spark/sql
- execution/window
- expressions
- test
- resources/sql-tests
- inputs
- results
- scala/org/apache/spark/sql
- hive/src/test/scala/org/apache/spark/sql/catalyst
Lines changed: 4 additions & 11 deletions
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
106 | 106 | | |
107 | 107 | | |
108 | 108 | | |
109 | | - | |
110 | | - | |
111 | | - | |
112 | | - | |
113 | | - | |
| 109 | + | |
| 110 | + | |
| 111 | + | |
114 | 112 | | |
115 | 113 | | |
116 | 114 | | |
| |||
119 | 117 | | |
120 | 118 | | |
121 | 119 | | |
| 120 | + | |
122 | 121 | | |
123 | 122 | | |
124 | 123 | | |
125 | | - | |
126 | | - | |
127 | | - | |
128 | | - | |
129 | | - | |
130 | | - | |
131 | 124 | | |
132 | 125 | | |
133 | 126 | | |
| |||
Lines changed: 23 additions & 0 deletions
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
58 | 58 | | |
59 | 59 | | |
60 | 60 | | |
| 61 | + | |
61 | 62 | | |
62 | 63 | | |
63 | 64 | | |
| |||
785 | 786 | | |
786 | 787 | | |
787 | 788 | | |
| 789 | + | |
| 790 | + | |
| 791 | + | |
| 792 | + | |
| 793 | + | |
| 794 | + | |
| 795 | + | |
| 796 | + | |
| 797 | + | |
| 798 | + | |
| 799 | + | |
| 800 | + | |
| 801 | + | |
| 802 | + | |
| 803 | + | |
| 804 | + | |
| 805 | + | |
| 806 | + | |
| 807 | + | |
| 808 | + | |
| 809 | + | |
| 810 | + | |
788 | 811 | | |
Lines changed: 7 additions & 0 deletions
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
74 | 74 | | |
75 | 75 | | |
76 | 76 | | |
| 77 | + | |
| 78 | + | |
| 79 | + | |
| 80 | + | |
| 81 | + | |
| 82 | + | |
| 83 | + | |
77 | 84 | | |
78 | 85 | | |
79 | 86 | | |
| |||
0 commit comments