[SPARK-11595] [SQL] Fixes ADD JAR when the input path contains URL scheme #9569
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
liancheng
changed the title
Contributor
Author
|
PR #9570 backports this one to branch-1.5. |
|
Test build #45365 has finished for PR 9569 at commit
|
|
Test build #45370 has finished for PR 9569 at commit
|
Contributor
|
LGTM, merging this into master and 1.6 branch. |
asfgit
pushed a commit
that referenced
this pull request
Nov 9, 2015
…heme Author: Cheng Lian <[email protected]> Closes #9569 from liancheng/spark-11595.fix-add-jar. (cherry picked from commit 150f6a8) Signed-off-by: Davies Liu <[email protected]>
asfgit
pushed a commit
that referenced
this pull request
Nov 12, 2015
…ains URL scheme This PR backports #9569 to branch-1.5. Author: Cheng Lian <[email protected]> Closes #9570 from liancheng/spark-11595.for-branch-1.5.
|
|
||
| def addJar(path: String): Unit = { | ||
| clientLoader.addJar(path) | ||
| val uri = new Path(path).toUri |
There was a problem hiding this comment.
我在1.6.1 版本上测试了一下这次的修改，发现add jar 的方式的确是可以通过的。
但是 如果按照 CREATE FUNCTION xxx AS 'org.hue.udf.MyUpper' USING JAR 'hdfs://xxx/myudfs.jar';
的方式， 看log是现实了加载了 ， jar。 但是实际运行时会报， class not found 。
请问这个bug 有思路么？
@liancheng
There was a problem hiding this comment.
https://issues.apache.org/jira/browse/SPARK-16833, this isseue shows that sometimes it work.
looks like sometimes private val jars = new ConcurrentHashMap[String, File]() in class NettyStreamManager doesn't successful add jar.
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
5 participants
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
No description provided.