YARN-11391 Add yarn RM DNS support #5213

lnbest0707 · 2022-12-13T02:56:41Z

Description of PR

The patch reuses the resolver introduced from the HDFS side and apply similar logic to YARN RM service addresses resolving.
To utilize the YARN DNS support and use DNS as endpoint, simply upgrade the hadoop binary and revise configs from, for example:

 <property>
   <name>yarn.resourcemanager.address.rm1</name>
   <value>rm1_address:8032</value>
 </property>
 <property>
   <name>yarn.resourcemanager.scheduler.address.rm1</name>
   <value>rm1_address:8030</value>
 </property>
 <property>
   <name>yarn.resourcemanager.resource-tracker.address.rm1</name>
   <value>rm1_address:8031</value>
 </property>
 <property>
   <name>yarn.resourcemanager.admin.address.rm1</name>
   <value>rm1_address:8033</value>
 </property>
 <property>
   <name>yarn.resourcemanager.webapp.address.rm1</name>
   <value>rm1_address:8088</value>
 </property>
 <property>
   <name>yarn.resourcemanager.webapp.https.address.rm1</name>
   <value>rm1_address:8090</value>
 </property>
 <property>
   <name>yarn.resourcemanager.address.rm2</name>
   <value>rm2_address:8032</value>
 </property>
 <property>
   <name>yarn.resourcemanager.scheduler.address.rm2</name>
   <value>rm2_address:8030</value>
 </property>
 <property>
   <name>yarn.resourcemanager.resource-tracker.address.rm2</name>
   <value>rm2_address:8031</value>
 </property>
 <property>
   <name>yarn.resourcemanager.admin.address.rm2</name>
   <value>rm2_address:8033</value>
 </property>
 <property>
   <name>yarn.resourcemanager.webapp.address.rm2</name>
   <value>rm2_address:8088</value>
 </property>
 <property>
   <name>yarn.resourcemanager.webapp.https.address.rm2</name>
   <value>rm2_address:8090</value>
 </property>
 <property>
   <name>yarn.resourcemanager.ha.rm-ids</name>
   <value>rm1,rm2</value>
 </property>

to:

  <property>
    <name>yarn.resourcemanager.address.rm1</name>
    <value>rm_multi_a_dns:8032</value>
  </property>
  <property>
    <name>yarn.resourcemanager.scheduler.address.rm1</name>
    <value>rm_multi_a_dns:8030</value>
  </property>
  <property>
    <name>yarn.resourcemanager.resource-tracker.address.rm1</name>
    <value>rm_multi_a_dns:8031</value>
  </property>
  <property>
    <name>yarn.resourcemanager.admin.address.rm1</name>
    <value>rm_multi_a_dns:8033</value>
  </property>
  <property>
    <name>yarn.resourcemanager.webapp.address.rm1</name>
    <value>rm_multi_a_dns:8088</value>
  </property>
  <property>
    <name>yarn.resourcemanager.webapp.https.address.rm1</name>
    <value>rm_multi_a_dns:8090</value>
  </property>
  <property>
    <name>yarn.resourcemanager.ha.rm-ids</name>
    <value>rm1</value>
  </property>
  <property>
    <name>yarn.resourcemanager.ha.resolve-needed</name>
    <value>true</value>
  </property>
  <property>
    <name>yarn.resourcemanager.ha.resolver.useFQDN</name>
    <value>true</value> # required in secure mode
  </property>
  <property>
    <name>yarn.resourcemanager.ha.refresh-period-ms</name>
    <value>180000</value> # 3 min
  </property>

where rm_multi_a_dns is a multi-A DNS record for rm1_address and rm2_address. This means the following output on the terminal.

$ dig +short <rm_multi_a_dns> | xargs -n +1 dig +short -x | sort
<rm1_address>
<rm2_address>

For the newly introduced flags, please refer to yarn-default.xml.

How was this patch tested?

For code changes:

Does the title or this PR starts with the corresponding JIRA issue id (e.g. 'HADOOP-17799. Your PR title ...')?
Object storage: have the integration tests been executed and the endpoint declared according to the connector-specific documentation?
If adding new dependencies to the code, are these dependencies licensed in a way that is compatible for inclusion under ASF 2.0?
If applicable, have you updated the LICENSE, LICENSE-binary, NOTICE-binary files?

slfan1989 · 2022-12-13T06:20:23Z

...rn-project/hadoop-yarn/hadoop-yarn-api/src/main/java/org/apache/hadoop/yarn/conf/HAUtil.java

+        }
+        // If the address needs to be resolved, get all of the IP addresses
+        // from this address and pass them into the map
+        LOG.info("Multi-A domain name " + addr +


The log format uses {}, do not splicing, we should be the way of slf4j

slfan1989 · 2022-12-13T06:20:40Z

...rn-project/hadoop-yarn/hadoop-yarn-api/src/main/java/org/apache/hadoop/yarn/conf/HAUtil.java

+    try {
+      address = NetUtils.createSocketAddr(addr);
+    } catch (Exception e) {
+      LOG.warn("Exception in creating socket address " + addr, e);


Forgot to pick the patch for new pr. Thanks for the comments, revised.

hadoop-yetus · 2022-12-13T07:15:11Z

💔 -1 overall

Vote	Subsystem	Runtime	Logfile	Comment
+0 🆗	reexec	0m 54s		Docker mode activated.
			_ Prechecks _
+1 💚	dupname	0m 0s		No case conflicting files found.
+0 🆗	codespell	0m 1s		codespell was not available.
+0 🆗	detsecrets	0m 2s		detect-secrets was not available.
+0 🆗	xmllint	0m 2s		xmllint was not available.
+1 💚	@author	0m 0s		The patch does not contain any @author tags.
+1 💚	test4tests	0m 0s		The patch appears to include 2 new or modified test files.
			_ trunk Compile Tests _
+0 🆗	mvndep	16m 2s		Maven dependency ordering for branch
+1 💚	mvninstall	28m 52s		trunk passed
+1 💚	compile	25m 13s		trunk passed with JDK Ubuntu-11.0.17+8-post-Ubuntu-1ubuntu220.04
+1 💚	compile	21m 48s		trunk passed with JDK Private Build-1.8.0_352-8u352-ga-1~20.04-b08
+1 💚	checkstyle	4m 7s		trunk passed
+1 💚	mvnsite	3m 25s		trunk passed
-1 ❌	javadoc	1m 7s	/branch-javadoc-hadoop-common-project_hadoop-common-jdkUbuntu-11.0.17+8-post-Ubuntu-1ubuntu220.04.txt	hadoop-common in trunk failed with JDK Ubuntu-11.0.17+8-post-Ubuntu-1ubuntu220.04.
+1 💚	javadoc	2m 16s		trunk passed with JDK Private Build-1.8.0_352-8u352-ga-1~20.04-b08
+1 💚	spotbugs	6m 39s		trunk passed
+1 💚	shadedclient	24m 17s		branch has no errors when building and testing our client artifacts.
			_ Patch Compile Tests _
+0 🆗	mvndep	0m 23s		Maven dependency ordering for patch
+1 💚	mvninstall	2m 19s		the patch passed
+1 💚	compile	24m 33s		the patch passed with JDK Ubuntu-11.0.17+8-post-Ubuntu-1ubuntu220.04
-1 ❌	javac	24m 33s	/results-compile-javac-root-jdkUbuntu-11.0.17+8-post-Ubuntu-1ubuntu220.04.txt	root-jdkUbuntu-11.0.17+8-post-Ubuntu-1ubuntu220.04 with JDK Ubuntu-11.0.17+8-post-Ubuntu-1ubuntu220.04 generated 18 new + 2814 unchanged - 0 fixed = 2832 total (was 2814)
+1 💚	compile	21m 39s		the patch passed with JDK Private Build-1.8.0_352-8u352-ga-1~20.04-b08
-1 ❌	javac	21m 39s	/results-compile-javac-root-jdkPrivateBuild-1.8.0_352-8u352-ga-1~20.04-b08.txt	root-jdkPrivateBuild-1.8.0_352-8u352-ga-1~~20.04-b08 with JDK Private Build-1.8.0_352-8u352-ga-1~~20.04-b08 generated 18 new + 2611 unchanged - 0 fixed = 2629 total (was 2611)
+1 💚	blanks	0m 0s		The patch has no blanks issues.
-0 ⚠️	checkstyle	3m 55s	/results-checkstyle-root.txt	root: The patch generated 21 new + 185 unchanged - 0 fixed = 206 total (was 185)
+1 💚	mvnsite	3m 24s		the patch passed
-1 ❌	javadoc	0m 59s	/patch-javadoc-hadoop-common-project_hadoop-common-jdkUbuntu-11.0.17+8-post-Ubuntu-1ubuntu220.04.txt	hadoop-common in the patch failed with JDK Ubuntu-11.0.17+8-post-Ubuntu-1ubuntu220.04.
+1 💚	javadoc	2m 14s		the patch passed with JDK Private Build-1.8.0_352-8u352-ga-1~20.04-b08
-1 ❌	spotbugs	2m 4s	/new-spotbugs-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-common.html	hadoop-yarn-project/hadoop-yarn/hadoop-yarn-common generated 1 new + 0 unchanged - 0 fixed = 1 total (was 0)
+1 💚	shadedclient	24m 34s		patch has no errors when building and testing our client artifacts.
			_ Other Tests _
+1 💚	unit	18m 34s		hadoop-common in the patch passed.
-1 ❌	unit	1m 6s	/patch-unit-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-api.txt	hadoop-yarn-api in the patch passed.
+1 💚	unit	5m 26s		hadoop-yarn-common in the patch passed.
+1 💚	asflicense	0m 55s		The patch does not generate ASF License warnings.
		256m 53s

Reason	Tests
SpotBugs	module:hadoop-yarn-project/hadoop-yarn/hadoop-yarn-common
	Inconsistent synchronization of org.apache.hadoop.yarn.client.ConfiguredRMFailoverProxyProvider.rmServiceIds; locked 90% of time Unsynchronized access at RequestHedgingRMFailoverProxyProvider.java:90% of time Unsynchronized access at RequestHedgingRMFailoverProxyProvider.java:[line 147]
Failed junit tests	hadoop.yarn.conf.TestYarnConfigurationFields

Subsystem	Report/Notes
Docker	ClientAPI=1.41 ServerAPI=1.41 base: https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-5213/1/artifact/out/Dockerfile
GITHUB PR	#5213
Optional Tests	dupname asflicense compile javac javadoc mvninstall mvnsite unit shadedclient spotbugs checkstyle codespell detsecrets xmllint
uname	Linux 3b111a613b6d 4.15.0-200-generic #211-Ubuntu SMP Thu Nov 24 18:16:04 UTC 2022 x86_64 x86_64 x86_64 GNU/Linux
Build tool	maven
Personality	dev-support/bin/hadoop.sh
git revision	trunk / `e897e65`
Default Java	Private Build-1.8.0_352-8u352-ga-1~20.04-b08
Multi-JDK versions	/usr/lib/jvm/java-11-openjdk-amd64:Ubuntu-11.0.17+8-post-Ubuntu-1ubuntu220.04 /usr/lib/jvm/java-8-openjdk-amd64:Private Build-1.8.0_352-8u352-ga-1~20.04-b08
Test Results	https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-5213/1/testReport/
Max. process+thread count	1244 (vs. ulimit of 5500)
modules	C: hadoop-common-project/hadoop-common hadoop-yarn-project/hadoop-yarn/hadoop-yarn-api hadoop-yarn-project/hadoop-yarn/hadoop-yarn-common U: .
Console output	https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-5213/1/console
versions	git=2.25.1 maven=3.6.3 spotbugs=4.2.2
Powered by	Apache Yetus 0.14.0 https://yetus.apache.org

This message was automatically generated.

hadoop-yetus · 2022-12-13T11:24:24Z

💔 -1 overall

Vote	Subsystem	Runtime	Logfile	Comment
+0 🆗	reexec	0m 59s		Docker mode activated.
			_ Prechecks _
+1 💚	dupname	0m 0s		No case conflicting files found.
+0 🆗	codespell	0m 1s		codespell was not available.
+0 🆗	detsecrets	0m 1s		detect-secrets was not available.
+0 🆗	xmllint	0m 1s		xmllint was not available.
+1 💚	@author	0m 0s		The patch does not contain any @author tags.
+1 💚	test4tests	0m 0s		The patch appears to include 2 new or modified test files.
			_ trunk Compile Tests _
+0 🆗	mvndep	15m 30s		Maven dependency ordering for branch
+1 💚	mvninstall	28m 54s		trunk passed
+1 💚	compile	25m 9s		trunk passed with JDK Ubuntu-11.0.17+8-post-Ubuntu-1ubuntu220.04
+1 💚	compile	25m 23s		trunk passed with JDK Private Build-1.8.0_352-8u352-ga-1~20.04-b08
+1 💚	checkstyle	3m 59s		trunk passed
+1 💚	mvnsite	3m 30s		trunk passed
-1 ❌	javadoc	1m 10s	/branch-javadoc-hadoop-common-project_hadoop-common-jdkUbuntu-11.0.17+8-post-Ubuntu-1ubuntu220.04.txt	hadoop-common in trunk failed with JDK Ubuntu-11.0.17+8-post-Ubuntu-1ubuntu220.04.
+1 💚	javadoc	2m 15s		trunk passed with JDK Private Build-1.8.0_352-8u352-ga-1~20.04-b08
+1 💚	spotbugs	6m 48s		trunk passed
+1 💚	shadedclient	24m 34s		branch has no errors when building and testing our client artifacts.
			_ Patch Compile Tests _
+0 🆗	mvndep	0m 28s		Maven dependency ordering for patch
+1 💚	mvninstall	2m 20s		the patch passed
+1 💚	compile	24m 38s		the patch passed with JDK Ubuntu-11.0.17+8-post-Ubuntu-1ubuntu220.04
+1 💚	javac	24m 38s		the patch passed
+1 💚	compile	22m 4s		the patch passed with JDK Private Build-1.8.0_352-8u352-ga-1~20.04-b08
+1 💚	javac	22m 4s		the patch passed
+1 💚	blanks	0m 0s		The patch has no blanks issues.
-0 ⚠️	checkstyle	4m 0s	/results-checkstyle-root.txt	root: The patch generated 2 new + 185 unchanged - 0 fixed = 187 total (was 185)
+1 💚	mvnsite	3m 27s		the patch passed
-1 ❌	javadoc	1m 0s	/patch-javadoc-hadoop-common-project_hadoop-common-jdkUbuntu-11.0.17+8-post-Ubuntu-1ubuntu220.04.txt	hadoop-common in the patch failed with JDK Ubuntu-11.0.17+8-post-Ubuntu-1ubuntu220.04.
+1 💚	javadoc	2m 16s		the patch passed with JDK Private Build-1.8.0_352-8u352-ga-1~20.04-b08
-1 ❌	spotbugs	2m 1s	/new-spotbugs-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-common.html	hadoop-yarn-project/hadoop-yarn/hadoop-yarn-common generated 1 new + 0 unchanged - 0 fixed = 1 total (was 0)
+1 💚	shadedclient	24m 22s		patch has no errors when building and testing our client artifacts.
			_ Other Tests _
+1 💚	unit	18m 17s		hadoop-common in the patch passed.
+1 💚	unit	1m 9s		hadoop-yarn-api in the patch passed.
+1 💚	unit	5m 25s		hadoop-yarn-common in the patch passed.
+1 💚	asflicense	0m 54s		The patch does not generate ASF License warnings.
		261m 1s

Reason	Tests
SpotBugs	module:hadoop-yarn-project/hadoop-yarn/hadoop-yarn-common
	Inconsistent synchronization of org.apache.hadoop.yarn.client.ConfiguredRMFailoverProxyProvider.rmServiceIds; locked 90% of time Unsynchronized access at RequestHedgingRMFailoverProxyProvider.java:90% of time Unsynchronized access at RequestHedgingRMFailoverProxyProvider.java:[line 147]

Subsystem	Report/Notes
Docker	ClientAPI=1.41 ServerAPI=1.41 base: https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-5213/2/artifact/out/Dockerfile
GITHUB PR	#5213
Optional Tests	dupname asflicense compile javac javadoc mvninstall mvnsite unit shadedclient spotbugs checkstyle codespell detsecrets xmllint
uname	Linux e39533b024e5 4.15.0-200-generic #211-Ubuntu SMP Thu Nov 24 18:16:04 UTC 2022 x86_64 x86_64 x86_64 GNU/Linux
Build tool	maven
Personality	dev-support/bin/hadoop.sh
git revision	trunk / `66b8fe5`
Default Java	Private Build-1.8.0_352-8u352-ga-1~20.04-b08
Multi-JDK versions	/usr/lib/jvm/java-11-openjdk-amd64:Ubuntu-11.0.17+8-post-Ubuntu-1ubuntu220.04 /usr/lib/jvm/java-8-openjdk-amd64:Private Build-1.8.0_352-8u352-ga-1~20.04-b08
Test Results	https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-5213/2/testReport/
Max. process+thread count	3134 (vs. ulimit of 5500)
modules	C: hadoop-common-project/hadoop-common hadoop-yarn-project/hadoop-yarn/hadoop-yarn-api hadoop-yarn-project/hadoop-yarn/hadoop-yarn-common U: .
Console output	https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-5213/2/console
versions	git=2.25.1 maven=3.6.3 spotbugs=4.2.2
Powered by	Apache Yetus 0.14.0 https://yetus.apache.org

This message was automatically generated.

slfan1989 · 2022-12-20T11:32:08Z

@lnbest0707 Can we rebase again?

hadoop-yetus · 2022-12-21T05:56:00Z

💔 -1 overall

Vote	Subsystem	Runtime	Logfile	Comment
+0 🆗	reexec	0m 57s		Docker mode activated.
			_ Prechecks _
+1 💚	dupname	0m 0s		No case conflicting files found.
+0 🆗	codespell	0m 1s		codespell was not available.
+0 🆗	detsecrets	0m 1s		detect-secrets was not available.
+0 🆗	xmllint	0m 1s		xmllint was not available.
+1 💚	@author	0m 0s		The patch does not contain any @author tags.
+1 💚	test4tests	0m 0s		The patch appears to include 2 new or modified test files.
			_ trunk Compile Tests _
+0 🆗	mvndep	15m 5s		Maven dependency ordering for branch
+1 💚	mvninstall	28m 58s		trunk passed
+1 💚	compile	25m 28s		trunk passed with JDK Ubuntu-11.0.17+8-post-Ubuntu-1ubuntu220.04
+1 💚	compile	21m 55s		trunk passed with JDK Private Build-1.8.0_352-8u352-ga-1~20.04-b08
+1 💚	checkstyle	4m 1s		trunk passed
+1 💚	mvnsite	3m 28s		trunk passed
-1 ❌	javadoc	1m 9s	/branch-javadoc-hadoop-common-project_hadoop-common-jdkUbuntu-11.0.17+8-post-Ubuntu-1ubuntu220.04.txt	hadoop-common in trunk failed with JDK Ubuntu-11.0.17+8-post-Ubuntu-1ubuntu220.04.
+1 💚	javadoc	2m 15s		trunk passed with JDK Private Build-1.8.0_352-8u352-ga-1~20.04-b08
+1 💚	spotbugs	6m 34s		trunk passed
+1 💚	shadedclient	24m 8s		branch has no errors when building and testing our client artifacts.
			_ Patch Compile Tests _
+0 🆗	mvndep	0m 25s		Maven dependency ordering for patch
+1 💚	mvninstall	2m 15s		the patch passed
+1 💚	compile	24m 21s		the patch passed with JDK Ubuntu-11.0.17+8-post-Ubuntu-1ubuntu220.04
+1 💚	javac	24m 21s		the patch passed
+1 💚	compile	21m 49s		the patch passed with JDK Private Build-1.8.0_352-8u352-ga-1~20.04-b08
+1 💚	javac	21m 49s		the patch passed
+1 💚	blanks	0m 0s		The patch has no blanks issues.
-0 ⚠️	checkstyle	3m 53s	/results-checkstyle-root.txt	root: The patch generated 2 new + 185 unchanged - 0 fixed = 187 total (was 185)
+1 💚	mvnsite	3m 18s		the patch passed
-1 ❌	javadoc	1m 0s	/patch-javadoc-hadoop-common-project_hadoop-common-jdkUbuntu-11.0.17+8-post-Ubuntu-1ubuntu220.04.txt	hadoop-common in the patch failed with JDK Ubuntu-11.0.17+8-post-Ubuntu-1ubuntu220.04.
+1 💚	javadoc	2m 10s		the patch passed with JDK Private Build-1.8.0_352-8u352-ga-1~20.04-b08
-1 ❌	spotbugs	2m 2s	/new-spotbugs-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-common.html	hadoop-yarn-project/hadoop-yarn/hadoop-yarn-common generated 1 new + 0 unchanged - 0 fixed = 1 total (was 0)
+1 💚	shadedclient	24m 22s		patch has no errors when building and testing our client artifacts.
			_ Other Tests _
+1 💚	unit	18m 11s		hadoop-common in the patch passed.
+1 💚	unit	1m 8s		hadoop-yarn-api in the patch passed.
+1 💚	unit	5m 23s		hadoop-yarn-common in the patch passed.
+1 💚	asflicense	0m 52s		The patch does not generate ASF License warnings.
		255m 22s

Reason	Tests
SpotBugs	module:hadoop-yarn-project/hadoop-yarn/hadoop-yarn-common
	Inconsistent synchronization of org.apache.hadoop.yarn.client.ConfiguredRMFailoverProxyProvider.rmServiceIds; locked 90% of time Unsynchronized access at RequestHedgingRMFailoverProxyProvider.java:90% of time Unsynchronized access at RequestHedgingRMFailoverProxyProvider.java:[line 147]

Subsystem	Report/Notes
Docker	ClientAPI=1.41 ServerAPI=1.41 base: https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-5213/3/artifact/out/Dockerfile
GITHUB PR	#5213
Optional Tests	dupname asflicense compile javac javadoc mvninstall mvnsite unit shadedclient spotbugs checkstyle codespell detsecrets xmllint
uname	Linux bcc82f68a73a 4.15.0-200-generic #211-Ubuntu SMP Thu Nov 24 18:16:04 UTC 2022 x86_64 x86_64 x86_64 GNU/Linux
Build tool	maven
Personality	dev-support/bin/hadoop.sh
git revision	trunk / `c90de11`
Default Java	Private Build-1.8.0_352-8u352-ga-1~20.04-b08
Multi-JDK versions	/usr/lib/jvm/java-11-openjdk-amd64:Ubuntu-11.0.17+8-post-Ubuntu-1ubuntu220.04 /usr/lib/jvm/java-8-openjdk-amd64:Private Build-1.8.0_352-8u352-ga-1~20.04-b08
Test Results	https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-5213/3/testReport/
Max. process+thread count	1244 (vs. ulimit of 5500)
modules	C: hadoop-common-project/hadoop-common hadoop-yarn-project/hadoop-yarn/hadoop-yarn-api hadoop-yarn-project/hadoop-yarn/hadoop-yarn-common U: .
Console output	https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-5213/3/console
versions	git=2.25.1 maven=3.6.3 spotbugs=4.2.2
Powered by	Apache Yetus 0.14.0 https://yetus.apache.org

This message was automatically generated.

hadoop-yetus · 2022-12-21T10:44:15Z

💔 -1 overall

Vote	Subsystem	Runtime	Logfile	Comment
+0 🆗	reexec	0m 56s		Docker mode activated.
			_ Prechecks _
+1 💚	dupname	0m 0s		No case conflicting files found.
+0 🆗	codespell	0m 1s		codespell was not available.
+0 🆗	detsecrets	0m 1s		detect-secrets was not available.
+0 🆗	xmllint	0m 1s		xmllint was not available.
+1 💚	@author	0m 0s		The patch does not contain any @author tags.
+1 💚	test4tests	0m 0s		The patch appears to include 2 new or modified test files.
			_ trunk Compile Tests _
+0 🆗	mvndep	14m 47s		Maven dependency ordering for branch
+1 💚	mvninstall	28m 57s		trunk passed
+1 💚	compile	25m 14s		trunk passed with JDK Ubuntu-11.0.17+8-post-Ubuntu-1ubuntu220.04
+1 💚	compile	21m 47s		trunk passed with JDK Private Build-1.8.0_352-8u352-ga-1~20.04-b08
+1 💚	checkstyle	4m 6s		trunk passed
+1 💚	mvnsite	3m 27s		trunk passed
-1 ❌	javadoc	1m 8s	/branch-javadoc-hadoop-common-project_hadoop-common-jdkUbuntu-11.0.17+8-post-Ubuntu-1ubuntu220.04.txt	hadoop-common in trunk failed with JDK Ubuntu-11.0.17+8-post-Ubuntu-1ubuntu220.04.
+1 💚	javadoc	2m 15s		trunk passed with JDK Private Build-1.8.0_352-8u352-ga-1~20.04-b08
+1 💚	spotbugs	6m 35s		trunk passed
+1 💚	shadedclient	24m 19s		branch has no errors when building and testing our client artifacts.
			_ Patch Compile Tests _
+0 🆗	mvndep	0m 24s		Maven dependency ordering for patch
+1 💚	mvninstall	3m 9s		the patch passed
+1 💚	compile	25m 51s		the patch passed with JDK Ubuntu-11.0.17+8-post-Ubuntu-1ubuntu220.04
+1 💚	javac	25m 51s		the patch passed
+1 💚	compile	22m 11s		the patch passed with JDK Private Build-1.8.0_352-8u352-ga-1~20.04-b08
+1 💚	javac	22m 11s		the patch passed
+1 💚	blanks	0m 0s		The patch has no blanks issues.
-0 ⚠️	checkstyle	4m 6s	/results-checkstyle-root.txt	root: The patch generated 2 new + 185 unchanged - 0 fixed = 187 total (was 185)
+1 💚	mvnsite	3m 25s		the patch passed
-1 ❌	javadoc	1m 0s	/patch-javadoc-hadoop-common-project_hadoop-common-jdkUbuntu-11.0.17+8-post-Ubuntu-1ubuntu220.04.txt	hadoop-common in the patch failed with JDK Ubuntu-11.0.17+8-post-Ubuntu-1ubuntu220.04.
+1 💚	javadoc	2m 17s		the patch passed with JDK Private Build-1.8.0_352-8u352-ga-1~20.04-b08
-1 ❌	spotbugs	2m 2s	/new-spotbugs-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-common.html	hadoop-yarn-project/hadoop-yarn/hadoop-yarn-common generated 1 new + 0 unchanged - 0 fixed = 1 total (was 0)
+1 💚	shadedclient	24m 22s		patch has no errors when building and testing our client artifacts.
			_ Other Tests _
+1 💚	unit	18m 56s		hadoop-common in the patch passed.
+1 💚	unit	1m 16s		hadoop-yarn-api in the patch passed.
+1 💚	unit	5m 58s		hadoop-yarn-common in the patch passed.
+1 💚	asflicense	1m 2s		The patch does not generate ASF License warnings.
		260m 13s

Reason	Tests
SpotBugs	module:hadoop-yarn-project/hadoop-yarn/hadoop-yarn-common
	Inconsistent synchronization of org.apache.hadoop.yarn.client.ConfiguredRMFailoverProxyProvider.rmServiceIds; locked 90% of time Unsynchronized access at RequestHedgingRMFailoverProxyProvider.java:90% of time Unsynchronized access at RequestHedgingRMFailoverProxyProvider.java:[line 147]

Subsystem	Report/Notes
Docker	ClientAPI=1.41 ServerAPI=1.41 base: https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-5213/4/artifact/out/Dockerfile
GITHUB PR	#5213
Optional Tests	dupname asflicense compile javac javadoc mvninstall mvnsite unit shadedclient spotbugs checkstyle codespell detsecrets xmllint
uname	Linux a7a8b209ab6a 4.15.0-200-generic #211-Ubuntu SMP Thu Nov 24 18:16:04 UTC 2022 x86_64 x86_64 x86_64 GNU/Linux
Build tool	maven
Personality	dev-support/bin/hadoop.sh
git revision	trunk / `4d7fe49`
Default Java	Private Build-1.8.0_352-8u352-ga-1~20.04-b08
Multi-JDK versions	/usr/lib/jvm/java-11-openjdk-amd64:Ubuntu-11.0.17+8-post-Ubuntu-1ubuntu220.04 /usr/lib/jvm/java-8-openjdk-amd64:Private Build-1.8.0_352-8u352-ga-1~20.04-b08
Test Results	https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-5213/4/testReport/
Max. process+thread count	3028 (vs. ulimit of 5500)
modules	C: hadoop-common-project/hadoop-common hadoop-yarn-project/hadoop-yarn/hadoop-yarn-api hadoop-yarn-project/hadoop-yarn/hadoop-yarn-common U: .
Console output	https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-5213/4/console
versions	git=2.25.1 maven=3.6.3 spotbugs=4.2.2
Powered by	Apache Yetus 0.14.0 https://yetus.apache.org

This message was automatically generated.

lnbest0707 · 2022-12-21T17:11:47Z

@slfan1989 Tried to rebase multiple times, still same error. Am I doing it in a correct way? Sry for naive questions.

slfan1989 · 2022-12-22T09:17:35Z

@slfan1989 Tried to rebase multiple times, still same error. Am I doing it in a correct way? Sry for naive questions.

If the error cannot be resolved, we can recreate a new pr based on the trunk.

I think this pr is very valuable, thanks for your contribution.

lnbest0707 · 2022-12-22T23:42:58Z

Close and recreate to bypass some irrelevant errors.

hadoop-yetus · 2022-12-23T04:00:16Z

💔 -1 overall

Vote	Subsystem	Runtime	Logfile	Comment
+0 🆗	reexec	1m 2s		Docker mode activated.
			_ Prechecks _
+1 💚	dupname	0m 0s		No case conflicting files found.
+0 🆗	codespell	0m 1s		codespell was not available.
+0 🆗	detsecrets	0m 1s		detect-secrets was not available.
+0 🆗	xmllint	0m 1s		xmllint was not available.
+1 💚	@author	0m 0s		The patch does not contain any @author tags.
+1 💚	test4tests	0m 0s		The patch appears to include 2 new or modified test files.
			_ trunk Compile Tests _
+0 🆗	mvndep	15m 5s		Maven dependency ordering for branch
+1 💚	mvninstall	28m 36s		trunk passed
+1 💚	compile	25m 51s		trunk passed with JDK Ubuntu-11.0.17+8-post-Ubuntu-1ubuntu220.04
+1 💚	compile	22m 8s		trunk passed with JDK Private Build-1.8.0_352-8u352-ga-1~20.04-b08
+1 💚	checkstyle	4m 15s		trunk passed
+1 💚	mvnsite	3m 33s		trunk passed
-1 ❌	javadoc	1m 7s	/branch-javadoc-hadoop-common-project_hadoop-common-jdkUbuntu-11.0.17+8-post-Ubuntu-1ubuntu220.04.txt	hadoop-common in trunk failed with JDK Ubuntu-11.0.17+8-post-Ubuntu-1ubuntu220.04.
+1 💚	javadoc	2m 16s		trunk passed with JDK Private Build-1.8.0_352-8u352-ga-1~20.04-b08
+1 💚	spotbugs	6m 39s		trunk passed
+1 💚	shadedclient	24m 19s		branch has no errors when building and testing our client artifacts.
			_ Patch Compile Tests _
+0 🆗	mvndep	0m 23s		Maven dependency ordering for patch
+1 💚	mvninstall	2m 17s		the patch passed
+1 💚	compile	24m 42s		the patch passed with JDK Ubuntu-11.0.17+8-post-Ubuntu-1ubuntu220.04
+1 💚	javac	24m 42s		the patch passed
+1 💚	compile	21m 47s		the patch passed with JDK Private Build-1.8.0_352-8u352-ga-1~20.04-b08
+1 💚	javac	21m 47s		the patch passed
+1 💚	blanks	0m 0s		The patch has no blanks issues.
-0 ⚠️	checkstyle	4m 2s	/results-checkstyle-root.txt	root: The patch generated 2 new + 185 unchanged - 0 fixed = 187 total (was 185)
+1 💚	mvnsite	3m 34s		the patch passed
-1 ❌	javadoc	1m 1s	/patch-javadoc-hadoop-common-project_hadoop-common-jdkUbuntu-11.0.17+8-post-Ubuntu-1ubuntu220.04.txt	hadoop-common in the patch failed with JDK Ubuntu-11.0.17+8-post-Ubuntu-1ubuntu220.04.
+1 💚	javadoc	2m 14s		the patch passed with JDK Private Build-1.8.0_352-8u352-ga-1~20.04-b08
-1 ❌	spotbugs	2m 1s	/new-spotbugs-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-common.html	hadoop-yarn-project/hadoop-yarn/hadoop-yarn-common generated 1 new + 0 unchanged - 0 fixed = 1 total (was 0)
+1 💚	shadedclient	24m 30s		patch has no errors when building and testing our client artifacts.
			_ Other Tests _
+1 💚	unit	18m 8s		hadoop-common in the patch passed.
+1 💚	unit	1m 7s		hadoop-yarn-api in the patch passed.
+1 💚	unit	5m 25s		hadoop-yarn-common in the patch passed.
+1 💚	asflicense	0m 58s		The patch does not generate ASF License warnings.
		257m 27s

Reason	Tests
SpotBugs	module:hadoop-yarn-project/hadoop-yarn/hadoop-yarn-common
	Inconsistent synchronization of org.apache.hadoop.yarn.client.ConfiguredRMFailoverProxyProvider.rmServiceIds; locked 90% of time Unsynchronized access at RequestHedgingRMFailoverProxyProvider.java:90% of time Unsynchronized access at RequestHedgingRMFailoverProxyProvider.java:[line 147]

Subsystem	Report/Notes
Docker	ClientAPI=1.41 ServerAPI=1.41 base: https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-5213/5/artifact/out/Dockerfile
GITHUB PR	#5213
Optional Tests	dupname asflicense compile javac javadoc mvninstall mvnsite unit shadedclient spotbugs checkstyle codespell detsecrets xmllint
uname	Linux c51a8269c61a 4.15.0-200-generic #211-Ubuntu SMP Thu Nov 24 18:16:04 UTC 2022 x86_64 x86_64 x86_64 GNU/Linux
Build tool	maven
Personality	dev-support/bin/hadoop.sh
git revision	trunk / `e1d2a74`
Default Java	Private Build-1.8.0_352-8u352-ga-1~20.04-b08
Multi-JDK versions	/usr/lib/jvm/java-11-openjdk-amd64:Ubuntu-11.0.17+8-post-Ubuntu-1ubuntu220.04 /usr/lib/jvm/java-8-openjdk-amd64:Private Build-1.8.0_352-8u352-ga-1~20.04-b08
Test Results	https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-5213/5/testReport/
Max. process+thread count	2418 (vs. ulimit of 5500)
modules	C: hadoop-common-project/hadoop-common hadoop-yarn-project/hadoop-yarn/hadoop-yarn-api hadoop-yarn-project/hadoop-yarn/hadoop-yarn-common U: .
Console output	https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-5213/5/console
versions	git=2.25.1 maven=3.6.3 spotbugs=4.2.2
Powered by	Apache Yetus 0.14.0 https://yetus.apache.org

This message was automatically generated.

YARN-11391 Add yarn RM DNS support

e897e65

lnbest0707 mentioned this pull request Dec 13, 2022

YARN-11391 Add yarn RM DNS support #5196

Closed

4 tasks

slfan1989 reviewed Dec 13, 2022

View reviewed changes

Xin Gao added 2 commits December 12, 2022 23:01

YARN-11391 Fix Jenkins reported issue

ae187ea

YARN-11391 Improve logging format

66b8fe5

Merge branch 'apache:trunk' into YARN-11391

c90de11

Merge branch 'apache:trunk' into YARN-11391

4d7fe49

Merge branch 'apache:trunk' into YARN-11391

e1d2a74

lnbest0707 closed this Dec 22, 2022

YARN-11391 Add yarn RM DNS support #5213

YARN-11391 Add yarn RM DNS support #5213

Uh oh!

Conversation

lnbest0707 commented Dec 13, 2022

Description of PR

How was this patch tested?

For code changes:

Uh oh!

slfan1989 Dec 13, 2022

Choose a reason for hiding this comment

Uh oh!

slfan1989 Dec 13, 2022

Choose a reason for hiding this comment

Uh oh!

lnbest0707 Dec 13, 2022

Choose a reason for hiding this comment

Uh oh!

hadoop-yetus commented Dec 13, 2022

Uh oh!

hadoop-yetus commented Dec 13, 2022

Uh oh!

slfan1989 commented Dec 20, 2022

Uh oh!

hadoop-yetus commented Dec 21, 2022

Uh oh!

hadoop-yetus commented Dec 21, 2022

Uh oh!

lnbest0707 commented Dec 21, 2022

Uh oh!

slfan1989 commented Dec 22, 2022 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

lnbest0707 commented Dec 22, 2022

Uh oh!

hadoop-yetus commented Dec 23, 2022

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

slfan1989 commented Dec 22, 2022 •

edited

Loading