[SPARK-45257][CORE] Enable spark.eventLog.compress by default

### What changes were proposed in this pull request?

This PR aims to enable `spark.eventLog.compress` by default for Apache Spark 4.0.0.

### Why are the changes needed?

- To save the event log storage cost by compressing the logs with ZStandard codec by default

### Does this PR introduce _any_ user-facing change?

Although we added a migration guide, the old Spark history servers are able to read the compressed logs.

### How was this patch tested?

 Pass the CIs.

### Was this patch authored or co-authored using generative AI tooling?

No.

Closes #43036 from dongjoon-hyun/SPARK-45257.

Lead-authored-by: Dongjoon Hyun <dhyun@apple.com>
Co-authored-by: Dongjoon Hyun <dongjoon@apache.org>
Signed-off-by: Dongjoon Hyun <dhyun@apple.com>

Author: dongjoon-hyun

core/src/main/scala/org/apache/spark/internal/config/package.scala

@@ -165,7 +165,7 @@ package object config {
     ConfigBuilder("spark.eventLog.compress")
       .version("1.0.0")
       .booleanConf
-      .createWithDefault(false)
+      .createWithDefault(true)
 
   private[spark] val EVENT_LOG_BLOCK_UPDATES =
     ConfigBuilder("spark.eventLog.logBlockUpdates.enabled")

docs/configuration.md

@@ -1311,7 +1311,7 @@ Apart from these, the following properties are also available, and may be useful
 </tr>
 <tr>
   <td><code>spark.eventLog.compress</code></td>
-  <td>false</td>
+  <td>true</td>
   <td>
     Whether to compress logged events, if <code>spark.eventLog.enabled</code> is true.
   </td>

docs/core-migration-guide.md

@@ -22,6 +22,10 @@ license: |
 * Table of contents
 {:toc}
 
+## Upgrading from Core 3.4 to 4.0
+
+- Since Spark 4.0, Spark will compress event logs. To restore the behavior before Spark 4.0, you can set `spark.eventLog.compress` to `false`.
+
 ## Upgrading from Core 3.3 to 3.4
 
 - Since Spark 3.4, Spark driver will own `PersistentVolumnClaim`s and try to reuse if they are not assigned to live executors. To restore the behavior before Spark 3.4, you can set `spark.kubernetes.driver.ownPersistentVolumeClaim` to `false` and `spark.kubernetes.driver.reusePersistentVolumeClaim` to `false`.