Skip to content

[SPARK-46546][DOCS] Fix the formatting of tables in running-on-yarn pages #44540

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Closed
panbingkun wants to merge 7 commits into from
Closed

[SPARK-46546][DOCS] Fix the formatting of tables in running-on-yarn pages #44540

Changes from 5 commits
Commits
Show all changes
7 commits
Select commit Hold shift + click to select a range
674496d
[SPARK-46546][DOCS] Fix the formatting of tables in `running-on-kuber…
panbingkun Dec 30, 2023
d8c5460
Make the table fixed width
panbingkun Jan 3, 2024
3635512
Revert "Make the table fixed width"
panbingkun Jan 3, 2024
cc11057
Revert "[SPARK-46546][DOCS] Fix the formatting of tables in `running-…
panbingkun Jan 3, 2024
39a112d
[SPARK-46546][DOCS] Fix the formatting of tables in running-on-yarn p…
panbingkun Jan 3, 2024
ca304b6
update
panbingkun Jan 3, 2024
f87a260
[SPARK-46546][DOCS] Fix the formatting of tables in running-on-yarn p…
panbingkun Jan 3, 2024
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
5 changes: 4 additions & 1 deletion docs/running-on-yarn.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -866,7 +866,7 @@ to avoid garbage collection issues during shuffle.
The following extra configuration options are available when the shuffle service is running on YARN:

<table>
<thead><tr><th>Property Name</th><th>Default</th><th>Meaning</th></tr></thead>
<thead><tr><th>Property Name</th><th>Default</th><th>Meaning</th><th>Since Version</th></tr></thead>
<tr>
<td><code>spark.yarn.shuffle.stopOnFailure</code></td>
Copy link
Contributor Author

@panbingkun panbingkun Jan 3, 2024 •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

PR: https://github.com/apache/spark/pull/14162/files#diff-71e304edb6adff7be2edd8855cd040b965240627aa6ebe5b5e941b2fc41e090dR75
Starting from the following version of the document: https://spark.apache.org/docs/2.1.0/running-on-yarn.html
branch 2.0(non-exist): https://github.com/apache/spark/blob/branch-2.0/docs/running-on-yarn.md?plain=1
branch 2.1(exist): https://github.com/apache/spark/blob/branch-2.1/docs/running-on-yarn.md?plain=1

=> 2.1.0

<td><code>false</code></td>
Expand All @@ -875,6 +875,7 @@ The following extra configuration options are available when the shuffle service
initialization. This prevents application failures caused by running containers on
NodeManagers where the Spark Shuffle Service is not running.
</td>
<td></td>
Copy link
Member

@yaooqinn yaooqinn Jan 3, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Please ensure that the value is fulfilled.

Copy link
Contributor

@LuciferYang LuciferYang Jan 3, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

+1

Copy link
Contributor Author

@panbingkun panbingkun Jan 3, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Okay, let me find the versions of each option.

Copy link
Contributor Author

@panbingkun panbingkun Jan 3, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Done

</tr>
<tr>
<td><code>spark.yarn.shuffle.service.metrics.namespace</code></td>
Copy link
Contributor Author

@panbingkun panbingkun Jan 3, 2024 •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

PR: https://github.com/apache/spark/pull/31936/files#diff-71e304edb6adff7be2edd8855cd040b965240627aa6ebe5b5e941b2fc41e090dR105
Starting from the following version of the document: https://spark.apache.org/docs/3.2.0/running-on-yarn.html
branch-3.1(non-exist): https://github.com/apache/spark/blob/branch-3.1/docs/running-on-yarn.md?plain=1
branch-3.2(exist): https://github.com/apache/spark/blob/branch-3.2/docs/running-on-yarn.md?plain=1

=> 3.2.0

Expand All @@ -883,6 +884,7 @@ The following extra configuration options are available when the shuffle service
The namespace to use when emitting shuffle service metrics into Hadoop metrics2 system of the
NodeManager.
</td>
<td></td>
</tr>
<tr>
<td><code>spark.yarn.shuffle.service.logs.namespace</code></td>
Copy link
Contributor Author

@panbingkun panbingkun Jan 3, 2024 •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

PR: https://github.com/apache/spark/pull/34079/files#diff-661e1c80840458affe09c28d71a7901d9bf15d6e87edd0c26dec1c8fd78b128aR810
Starting from the following version of the document: https://spark.apache.org/docs/3.3.0/running-on-yarn.html
branch-3.2(non-exist): https://github.com/apache/spark/blob/branch-3.2/docs/running-on-yarn.md?plain=1
branch-3.3(exist): https://github.com/apache/spark/blob/branch-3.3/docs/running-on-yarn.md?plain=1

=> 3.3.0

Expand All @@ -894,6 +896,7 @@ The following extra configuration options are available when the shuffle service
may expect the logger name to look like a class name, it's generally recommended to provide a value which
would be a valid Java package or class name and not include spaces.
</td>
<td></td>
</tr>
<tr>
<td><code>spark.shuffle.service.db.backend</code></td>
Expand Down