Commit d80cc90
[SPARK-12526][SPARKR] ifelse
`ifelse`, `when`, `otherwise` is unable to take `Column` typed S4 object as values.
For example:
```r
ifelse(lit(1) == lit(1), lit(2), lit(3))
ifelse(df$mpg > 0, df$mpg, 0)
```
will both fail with
```r
attempt to replicate an object of type 'environment'
```
The PR replaces `ifelse` calls with `if ... else ...` inside the function implementations to avoid attempt to vectorize(i.e. `rep()`). It remains to be discussed whether we should instead support vectorization in these functions for consistency because `ifelse` in base R is vectorized but I cannot foresee any scenarios these functions will want to be vectorized in SparkR.
For reference, added test cases which trigger failures:
```r
. Error: when(), otherwise() and ifelse() with column on a DataFrame ----------
error in evaluating the argument 'x' in selecting a method for function 'collect':
error in evaluating the argument 'col' in selecting a method for function 'select':
attempt to replicate an object of type 'environment'
Calls: when -> when -> ifelse -> ifelse
1: withCallingHandlers(eval(code, new_test_environment), error = capture_calls, message = function(c) invokeRestart("muffleMessage"))
2: eval(code, new_test_environment)
3: eval(expr, envir, enclos)
4: expect_equal(collect(select(df, when(df$a > 1 & df$b > 2, lit(1))))[, 1], c(NA, 1)) at test_sparkSQL.R:1126
5: expect_that(object, equals(expected, label = expected.label, ...), info = info, label = label)
6: condition(object)
7: compare(actual, expected, ...)
8: collect(select(df, when(df$a > 1 & df$b > 2, lit(1))))
Error: Test failures
Execution halted
```
Author: Forest Fang <[email protected]>
Closes apache#10481 from saurfang/spark-12526., when, otherwise` unable to take Column as value1 parent 73862a1 commit d80cc90
3 files changed
Lines changed: 18 additions & 7 deletions
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
215 | 215 | | |
216 | 216 | | |
217 | 217 | | |
218 | | - | |
| 218 | + | |
219 | 219 | | |
220 | 220 | | |
221 | 221 | | |
| |||
225 | 225 | | |
226 | 226 | | |
227 | 227 | | |
228 | | - | |
| 228 | + | |
229 | 229 | | |
230 | 230 | | |
231 | 231 | | |
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
37 | 37 | | |
38 | 38 | | |
39 | 39 | | |
40 | | - | |
| 40 | + | |
41 | 41 | | |
42 | 42 | | |
43 | 43 | | |
| |||
2262 | 2262 | | |
2263 | 2263 | | |
2264 | 2264 | | |
2265 | | - | |
| 2265 | + | |
2266 | 2266 | | |
2267 | 2267 | | |
2268 | 2268 | | |
| |||
2277 | 2277 | | |
2278 | 2278 | | |
2279 | 2279 | | |
2280 | | - | |
| 2280 | + | |
| 2281 | + | |
| 2282 | + | |
| 2283 | + | |
2281 | 2284 | | |
2282 | 2285 | | |
2283 | 2286 | | |
2284 | 2287 | | |
2285 | | - | |
2286 | | - | |
| 2288 | + | |
| 2289 | + | |
2287 | 2290 | | |
2288 | 2291 | | |
2289 | 2292 | | |
| |||
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
1120 | 1120 | | |
1121 | 1121 | | |
1122 | 1122 | | |
| 1123 | + | |
| 1124 | + | |
| 1125 | + | |
| 1126 | + | |
| 1127 | + | |
| 1128 | + | |
| 1129 | + | |
| 1130 | + | |
1123 | 1131 | | |
1124 | 1132 | | |
1125 | 1133 | | |
| |||
0 commit comments