Skip to content

Commit e923849

Browse files
peter-tothbeliefer
authored andcommitted
[SPARK-41468][SQL][FOLLOWUP] Handle NamedLambdaVariables in EquivalentExpressions
### What changes were proposed in this pull request? This is a follow-up PR to apache#39010 to handle `NamedLambdaVariable`s too. ### Why are the changes needed? To avoid possible issues with higer-order functions. ### Does this PR introduce _any_ user-facing change? No. ### How was this patch tested? Existing UTs. Closes apache#39046 from peter-toth/SPARK-41468-fix-planexpressions-in-equivalentexpressions-follow-up. Authored-by: Peter Toth <[email protected]> Signed-off-by: Wenchen Fan <[email protected]>
1 parent 06634d4 commit e923849

File tree

1 file changed

+3
-2
lines changed

1 file changed

+3
-2
lines changed

sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/EquivalentExpressions.scala

Lines changed: 3 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -144,9 +144,10 @@ class EquivalentExpressions {
144144

145145
private def supportedExpression(e: Expression) = {
146146
!e.exists {
147-
// `LambdaVariable` is usually used as a loop variable, which can't be evaluated ahead of the
148-
// loop. So we can't evaluate sub-expressions containing `LambdaVariable` at the beginning.
147+
// `LambdaVariable` is usually used as a loop variable and `NamedLambdaVariable` is used in
148+
// higher-order functions, which can't be evaluated ahead of the execution.
149149
case _: LambdaVariable => true
150+
case _: NamedLambdaVariable => true
150151

151152
// `PlanExpression` wraps query plan. To compare query plans of `PlanExpression` on executor,
152153
// can cause error like NPE.

0 commit comments

Comments
 (0)