-
Notifications
You must be signed in to change notification settings - Fork 29k
[SPARK-28645][SQL] ParseException is thrown when the window is redefined #30512
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Conversation
|
Test build #131824 has finished for PR 30512 at commit
|
|
Test build #131832 has finished for PR 30512 at commit
|
|
cc @cloud-fan |
|
The fix looks fine. In the PR description, could you add an example query output with/without this fix? |
OK |
| FROM | ||
| basic_pays | ||
| WINDOW | ||
| w AS (ORDER BY salary DESC ROWS BETWEEN UNBOUNDED PRECEDING AND 1 FOLLOWING), |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
is the window spec name case insensitive?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I checked the PgSQL and find the alias of window frame is case insensitive.
|
thanks, merging to master! |
|
@cloud-fan @maropu Thanks for your work! |
What changes were proposed in this pull request?
Currently in Spark one could redefine a window. For instance:
select count(*) OVER w FROM tenk1 WINDOW w AS (ORDER BY unique1), w AS (ORDER BY unique1);The window
wis defined two times. In PgSQL, on the other hand, a thrown will happen:ERROR: window "w" is already definedWhy are the changes needed?
The current implement gives the following window definitions a higher priority. But it wasn't Spark's intention and users can't know from any document of Spark.
This PR fixes the bug.
Does this PR introduce any user-facing change?
Yes.
There is an example query output with/without this fix.
The output before this fix:
The output after this fix:
How was this patch tested?
Jenkins test.