Skip to content

Commit db65be5

Browse files
author
Davies Liu
committed
fix the Python UDF in Scala 2.10
1 parent a153e41 commit db65be5

1 file changed

Lines changed: 1 addition & 1 deletion

File tree

sql/core/src/main/scala/org/apache/spark/sql/execution/python/ExtractPythonUDFs.scala

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -132,7 +132,7 @@ private[spark] object ExtractPythonUDFs extends Rule[SparkPlan] {
132132
val validUdfs = udfs.filter { case udf =>
133133
// Check to make sure that the UDF can be evaluated with only the input of this child.
134134
udf.references.subsetOf(child.outputSet)
135-
}
135+
}.toArray // force the iterator to an array, that can't be serialized in Scala 2.10
136136
if (validUdfs.nonEmpty) {
137137
val resultAttrs = udfs.zipWithIndex.map { case (u, i) =>
138138
AttributeReference(s"pythonUDF$i", u.dataType)()

0 commit comments

Comments
 (0)