Skip to content

Commit cf37cd5

Browse files
yanxiaoledongjoon-hyun
authored andcommitted
[SPARK-32557][CORE] Logging and swallowing the exception per entry in History server
### What changes were proposed in this pull request? This PR adds a try catch wrapping the History server scan logic to log and swallow the exception per entry. ### Why are the changes needed? As discussed in #29350 , one entry failure shouldn't affect others. ### Does this PR introduce _any_ user-facing change? No. ### How was this patch tested? Manually tested. Closes #29374 from yanxiaole/SPARK-32557. Authored-by: Yan Xiaole <[email protected]> Signed-off-by: Dongjoon Hyun <[email protected]>
1 parent 34d9f1c commit cf37cd5

1 file changed

Lines changed: 3 additions & 1 deletion

File tree

core/src/main/scala/org/apache/spark/deploy/history/FsHistoryProvider.scala

Lines changed: 3 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -27,6 +27,7 @@ import java.util.zip.ZipOutputStream
2727
import scala.collection.JavaConverters._
2828
import scala.collection.mutable
2929
import scala.io.Source
30+
import scala.util.control.NonFatal
3031
import scala.xml.Node
3132

3233
import com.fasterxml.jackson.annotation.JsonIgnore
@@ -539,7 +540,8 @@ private[history] class FsHistoryProvider(conf: SparkConf, clock: Clock)
539540
case _: FileNotFoundException => false
540541
}
541542

542-
case _: FileNotFoundException =>
543+
case NonFatal(e) =>
544+
logWarning(s"Error while filtering log ${reader.rootPath}", e)
543545
false
544546
}
545547
}

0 commit comments

Comments
 (0)