Skip to content

Commit 590b9a0

Browse files
Ngone51cloud-fan
authored andcommitted
[SPARK-31010][SQL][FOLLOW-UP] Add Java UDF suggestion in error message of untyped Scala UDF
### What changes were proposed in this pull request? Added Java UDF suggestion in the in error message of untyped Scala UDF. ### Why are the changes needed? To help user migrate their use case from deprecate untyped Scala UDF to other supported UDF. ### Does this PR introduce any user-facing change? No. It haven't been released. ### How was this patch tested? Pass Jenkins. Closes #28070 from Ngone51/spark_31010. Authored-by: yi.wu <yi.wu@databricks.com> Signed-off-by: Wenchen Fan <wenchen@databricks.com>
1 parent 2a6aa8e commit 590b9a0

1 file changed

Lines changed: 7 additions & 3 deletions

File tree

sql/core/src/main/scala/org/apache/spark/sql/functions.scala

Lines changed: 7 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -4864,9 +4864,13 @@ object functions {
48644864
"information. Spark may blindly pass null to the Scala closure with primitive-type " +
48654865
"argument, and the closure will see the default value of the Java type for the null " +
48664866
"argument, e.g. `udf((x: Int) => x, IntegerType)`, the result is 0 for null input. " +
4867-
"You could use typed Scala UDF APIs (e.g. `udf((x: Int) => x)`) to avoid this problem, " +
4868-
s"or set ${SQLConf.LEGACY_ALLOW_UNTYPED_SCALA_UDF.key} to true and use this API with " +
4869-
s"caution."
4867+
"To get rid of this error, you could:\n" +
4868+
"1. use typed Scala UDF APIs, e.g. `udf((x: Int) => x)`\n" +
4869+
"2. use Java UDF APIs, e.g. `udf(new UDF1[String, Integer] { " +
4870+
"override def call(s: String): Integer = s.length() }, IntegerType)`, " +
4871+
"if input types are all non primitive\n" +
4872+
s"3. set ${SQLConf.LEGACY_ALLOW_UNTYPED_SCALA_UDF.key} to true and " +
4873+
s"use this API with caution"
48704874
throw new AnalysisException(errorMsg)
48714875
}
48724876
SparkUserDefinedFunction(f, dataType, inputEncoders = Nil)

0 commit comments

Comments
 (0)