Skip to content

Commit b0bebcc

Browse files
committed
add new configurations to the doc
1 parent e51667c commit b0bebcc

File tree

2 files changed

+17
-0
lines changed

2 files changed

+17
-0
lines changed

core/src/main/scala/org/apache/spark/scheduler/EventLoggingListener.scala

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -80,6 +80,7 @@ private[spark] class EventLoggingListener(
8080
}
8181
private val compressionCodecName = compressionCodec.map { c =>
8282
CompressionCodec.getShortName(c.getClass.getName)
83+
8384
}
8485

8586
// Only defined if the file system scheme is not local

docs/configuration.md

Lines changed: 16 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -659,6 +659,22 @@ Apart from these, the following properties are also available, and may be useful
659659
finished.
660660
</td>
661661
</tr>
662+
<tr>
663+
<td><code>spark.eventLog.omitInternalAccumulables</code></td>
664+
<td>false</td>
665+
<td>
666+
Whether to log internal accumulables for Spark task metrics, useful for reducing the size
667+
of event logs. Spark task metrics will still be recorded under "Task Metrics" field.
668+
</td>
669+
</tr>
670+
<tr>
671+
<td><code>spark.eventLog.omitUpdatedBlockStatuses</code></td>
672+
<td>false</td>
673+
<td>
674+
Whether to include Updated Block Statuses metrics in Spark event log, useful for reducing
675+
the size of event logs.
676+
</td>
677+
</tr>
662678
<tr>
663679
<td><code>spark.ui.enabled</code></td>
664680
<td>true</td>

0 commit comments

Comments
 (0)