Skip to content

Commit 31aedcf

Browse files
author
cafreeman
committed
Refactor test to call an external R script
`test_includeJAR.R` now executes an external sparkR script which, in turn, initializes a new spark context with the test JAR, runs the test functions, and returns the output to the original test. As a result, we can now run this test inside the SparkR test suite and get around the fact that we can't add the "sparkJars" argument to `sparkR.init()` once the JVM has already started.
1 parent 2c22073 commit 31aedcf

2 files changed

Lines changed: 31 additions & 12 deletions

File tree

R/pkg/inst/tests/jarTest.R

Lines changed: 17 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,17 @@
1+
2+
library(SparkR)
3+
4+
sc <- sparkR.init()
5+
6+
helloTest <- SparkR:::callJStatic("sparkR.test.hello",
7+
"helloWorld",
8+
"Dave")
9+
10+
basicFunction <- SparkR:::callJStatic("sparkR.test.basicFunction",
11+
"addStuff",
12+
2L,
13+
2L)
14+
15+
sparkR.stop()
16+
output <- c(helloTest, basicFunction)
17+
writeLines(output)

R/pkg/inst/tests/test_includeJAR.R

Lines changed: 14 additions & 12 deletions
Original file line numberDiff line numberDiff line change
@@ -16,20 +16,22 @@
1616
#
1717
context("include an external JAR in SparkContext")
1818

19-
filePath <- file.path(Sys.getenv("SPARK_HOME"),
20-
"R/lib/SparkR/test_support/sparktestjar_2.10-1.0.jar")
21-
sparkR.stop()
22-
sc <- sparkR.init(master = "local", sparkJars = filePath)
19+
runScript <- function() {
20+
sparkHome <- Sys.getenv("SPARK_HOME")
21+
jarPath <- paste("--jars",
22+
shQuote(file.path(sparkHome, "R/lib/SparkR/test_support/sparktestjar_2.10-1.0.jar")))
23+
scriptPath <- file.path(sparkHome, "R/lib/SparkR/tests/jarTest.R")
24+
submitPath <- file.path(sparkHome, "bin/spark-submit")
25+
res <- system2(command = submitPath,
26+
args = c(jarPath, scriptPath),
27+
stdout = TRUE)
28+
tail(res, 2)
29+
}
2330

2431
test_that("sparkJars tag in SparkContext", {
25-
helloTest <- SparkR:::callJStatic("sparkR.test.hello",
26-
"helloWorld",
27-
"Dave")
32+
testOutput <- runScript()
33+
helloTest <- testOutput[1]
2834
expect_true(helloTest == "Hello, Dave")
29-
30-
basicFunction <- SparkR:::callJStatic("sparkR.test.basicFunction",
31-
"addStuff",
32-
2L,
33-
2L)
35+
basicFunction <- testOutput[2]
3436
expect_true(basicFunction == 4L)
3537
})

0 commit comments

Comments
 (0)