Skip to content

[SPARK-15359] [Mesos] Mesos dispatcher should handle DRIVER_ABORTED status from mesosDriver.run() #13143

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Closed
devaraj-kavali wants to merge 4 commits into from
Closed

[SPARK-15359] [Mesos] Mesos dispatcher should handle DRIVER_ABORTED status from mesosDriver.run() #13143

Changes from 1 commit
Commits
Show all changes
4 commits
Select commit Hold shift + click to select a range
c16fb5f
[SPARK-15359] [Mesos] Mesos dispatcher should handle DRIVER_ABORTED s…
May 17, 2016
946ad7d
Merge branch 'master' into SPARK-15359
Dec 28, 2016
131edc3
Merge branch 'master' into SPARK-15359
Apr 18, 2017
b9893a3
Changed from newDriver.run() to newDriver.start()
Apr 18, 2017
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
19 changes: 15 additions & 4 deletions core/src/main/scala/org/apache/spark/scheduler/cluster/mesos/MesosSchedulerUtils.scala
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -120,14 +120,25 @@ private[mesos] trait MesosSchedulerUtils extends Logging {
val ret = mesosDriver.run()
logInfo("driver.run() returned with code " + ret)
if (ret != null && ret.equals(Status.DRIVER_ABORTED)) {
error = Some(new SparkException("Error starting driver, DRIVER_ABORTED"))
markErr()
val ex = new SparkException("Error starting driver, DRIVER_ABORTED")
// if the driver gets aborted after the successful registration
Copy link
Contributor

@tnachen tnachen Jun 22, 2016

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

s/after the successful registration/after registration/g

Copy link
Contributor

@tnachen tnachen Jun 23, 2016

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Also to simplify the code, can we just throw SparkExecption here? Then the catch will then handle all cases

if (registerLatch.getCount == 0) {
throw ex
} else {
error = Some(ex)
markErr()
}
}
} catch {
case e: Exception =>
logError("driver.run() failed", e)
error = Some(e)
markErr()
// if any exception occurs after the successful registration
if (registerLatch.getCount == 0) {
throw e
} else {
error = Some(e)
markErr()
}
}
}
}.start()
Expand Down