-
Notifications
You must be signed in to change notification settings - Fork 29k
[SPARK-11594][SQL][REPL] Cannot create UDAF in REPL #9568
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Conversation
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I am not catching a java.lang.InternalError here, because I think these shouldn't be caught at all (the error could have different cause). We enable the use to redefine the name to work arround this problem.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I am wondering if we can catch the error and re-throw it with a more informative error message (we put InternalError as the cause of our new error)? In our error message, we can let the user know he/she potentially hit that Scala issue and the workaround is to override def name.
|
Test build #45364 has finished for PR 9568 at commit
|
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
So, this is the cause of that InternalError?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
The getSimpleName call on the defined class causes the internal error.
|
This is actually a scala problem: |
|
How about we add the link to the scala issue in our description and in our code comment? |
|
Sounds like a good idea. I'll add this in the morning. |
|
Move to scala 2.10.5 fixed this. Closing PR. |
This PR enables users to create a UDAF in the REPL without getting a
java.lang.InternalError.