HBASE-27250 MasterRpcService#setRegionStateInMeta does not support re… #4664

huaxiangsun · 2022-07-28T02:16:38Z

…plica region encodedNames or region names

huaxiangsun · 2022-08-04T19:12:51Z

force pushed update to address spotless and spotbugs issues.

hbase-server/src/main/java/org/apache/hadoop/hbase/master/MasterRpcServices.java

huaxiangsun · 2022-08-05T17:23:43Z

unit | 282m 8s | hbase-server in the patch failed.
I run the failed two test cases, TestSyncReplicationRemoveRemoteWAL,TestFuzzyRowFilterEndToEnd locally, it passed for me.

huaxiangsun · 2022-09-07T22:48:27Z

Run TestSyncReplicationRemoveRemoteWAL and TestFuzzyRowFilterEndToEnd locally, they passed.

Apache9

And checked the code again, when calculating encodedRegionName, we will include replicaId so I think the old implementation can support replica region, as the encodedRegionName for a secondary replica region is different from the primary replica region.

hbase-server/src/main/java/org/apache/hadoop/hbase/master/MasterRpcServices.java

huaxiangsun · 2022-09-14T16:32:19Z

Thanks for comments, checking!

Apache9 · 2022-09-20T09:21:01Z

Any updates here? Thanks.

ndimiduk · 2023-02-08T12:52:29Z

I'm seeing some production issues and I think this feature would be useful. Let me try to pick it up.

…plica region encodedNames or region names

…ame is valid

ndimiduk · 2023-02-14T07:54:06Z

Oh hey, it let me push to @huaxiangsun 's branch.

I have rebased onto master and picked up the plumbing required through AM to handle read replica regions in more places. As a nice bonus, I think HBCKServerCrashProcedure will now clean up replicas too. devtests are passing locally. I'd like to add an HBCKSCP test that explicitly covers this use-case. LEt's see what builedbot has to say.

hbase-server/src/main/java/org/apache/hadoop/hbase/MetaTableAccessor.java

- make several MetaTableAccessor methods more precise in their handling of replica regions - hbck2 setRegionStateInMeta and HBCKServerCrashProcedure handle read replicas - separate AM helper methods -- loading RegionInfo from cache vs. refreshing cache from meta - AM helper method support loading RegionInfo from cache via either region name and encoded region name (both caches are maintained, and under lock)

Apache-HBase · 2023-02-14T20:15:46Z

🎊 +1 overall

Vote	Subsystem	Runtime	Comment
+0 🆗	reexec	0m 24s	Docker mode activated.
		_ Prechecks _
+1 💚	dupname	0m 0s	No case conflicting files found.
+1 💚	hbaseanti	0m 0s	Patch does not have any anti-patterns.
+1 💚	@author	0m 0s	The patch does not contain any @author tags.
		_ master Compile Tests _
+1 💚	mvninstall	3m 14s	master passed
+1 💚	compile	2m 26s	master passed
+1 💚	checkstyle	0m 32s	master passed
+1 💚	spotless	0m 38s	branch has no errors when running spotless:check.
+1 💚	spotbugs	1m 20s	master passed
		_ Patch Compile Tests _
+1 💚	mvninstall	3m 13s	the patch passed
+1 💚	compile	2m 24s	the patch passed
+1 💚	javac	2m 24s	the patch passed
+1 💚	checkstyle	0m 33s	the patch passed
+1 💚	whitespace	0m 0s	The patch has no whitespace issues.
+1 💚	hadoopcheck	12m 44s	Patch does not cause any errors with Hadoop 3.2.4 3.3.4.
+1 💚	spotless	0m 38s	patch has no errors when running spotless:check.
+1 💚	spotbugs	1m 28s	the patch passed
		_ Other Tests _
+1 💚	asflicense	0m 8s	The patch does not generate ASF License warnings.
		37m 22s

Subsystem	Report/Notes
Docker	ClientAPI=1.42 ServerAPI=1.42 base: https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-4664/6/artifact/yetus-general-check/output/Dockerfile
GITHUB PR	#4664
Optional Tests	dupname asflicense javac spotbugs hadoopcheck hbaseanti spotless checkstyle compile
uname	Linux 6ca2a45117c6 5.4.0-1094-aws #102~18.04.1-Ubuntu SMP Tue Jan 10 21:07:03 UTC 2023 x86_64 x86_64 x86_64 GNU/Linux
Build tool	maven
Personality	dev-support/hbase-personality.sh
git revision	master / `8ba56cc`
Default Java	Eclipse Adoptium-11.0.17+8
Max. process+thread count	85 (vs. ulimit of 30000)
modules	C: hbase-server U: hbase-server
Console output	https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-4664/6/console
versions	git=2.34.1 maven=3.8.6 spotbugs=4.7.3
Powered by	Apache Yetus 0.12.0 https://yetus.apache.org

This message was automatically generated.

Apache-HBase · 2023-02-14T23:19:09Z

🎊 +1 overall

Vote	Subsystem	Runtime	Comment
+0 🆗	reexec	0m 25s	Docker mode activated.
-0 ⚠️	yetus	0m 3s	Unprocessed flag(s): --brief-report-file --spotbugs-strict-precheck --whitespace-eol-ignore-list --whitespace-tabs-ignore-list --quick-hadoopcheck
		_ Prechecks _
		_ master Compile Tests _
+1 💚	mvninstall	4m 0s	master passed
+1 💚	compile	0m 46s	master passed
+1 💚	shadedjars	4m 43s	branch has no errors when building our shaded downstream artifacts.
+1 💚	javadoc	0m 26s	master passed
		_ Patch Compile Tests _
+1 💚	mvninstall	3m 34s	the patch passed
+1 💚	compile	0m 45s	the patch passed
+1 💚	javac	0m 45s	the patch passed
+1 💚	shadedjars	4m 44s	patch has no errors when building our shaded downstream artifacts.
+1 💚	javadoc	0m 23s	the patch passed
		_ Other Tests _
+1 💚	unit	197m 5s	hbase-server in the patch passed.
		220m 44s

Subsystem	Report/Notes
Docker	ClientAPI=1.42 ServerAPI=1.42 base: https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-4664/6/artifact/yetus-jdk11-hadoop3-check/output/Dockerfile
GITHUB PR	#4664
Optional Tests	javac javadoc unit shadedjars compile
uname	Linux 0ef22ee2d5ad 5.4.0-1093-aws #102~18.04.2-Ubuntu SMP Wed Dec 7 00:31:59 UTC 2022 x86_64 x86_64 x86_64 GNU/Linux
Build tool	maven
Personality	dev-support/hbase-personality.sh
git revision	master / `8ba56cc`
Default Java	Eclipse Adoptium-11.0.17+8
Test Results	https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-4664/6/testReport/
Max. process+thread count	2579 (vs. ulimit of 30000)
modules	C: hbase-server U: hbase-server
Console output	https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-4664/6/console
versions	git=2.34.1 maven=3.8.6
Powered by	Apache Yetus 0.12.0 https://yetus.apache.org

This message was automatically generated.

Apache-HBase · 2023-02-14T23:24:00Z

🎊 +1 overall

Vote	Subsystem	Runtime	Comment
+0 🆗	reexec	0m 23s	Docker mode activated.
-0 ⚠️	yetus	0m 3s	Unprocessed flag(s): --brief-report-file --spotbugs-strict-precheck --whitespace-eol-ignore-list --whitespace-tabs-ignore-list --quick-hadoopcheck
		_ Prechecks _
		_ master Compile Tests _
+1 💚	mvninstall	3m 3s	master passed
+1 💚	compile	0m 36s	master passed
+1 💚	shadedjars	4m 35s	branch has no errors when building our shaded downstream artifacts.
+1 💚	javadoc	0m 21s	master passed
		_ Patch Compile Tests _
+1 💚	mvninstall	2m 45s	the patch passed
+1 💚	compile	0m 34s	the patch passed
+1 💚	javac	0m 34s	the patch passed
+1 💚	shadedjars	4m 35s	patch has no errors when building our shaded downstream artifacts.
+1 💚	javadoc	0m 21s	the patch passed
		_ Other Tests _
+1 💚	unit	204m 27s	hbase-server in the patch passed.
		225m 33s

Subsystem	Report/Notes
Docker	ClientAPI=1.42 ServerAPI=1.42 base: https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-4664/6/artifact/yetus-jdk8-hadoop3-check/output/Dockerfile
GITHUB PR	#4664
Optional Tests	javac javadoc unit shadedjars compile
uname	Linux 32e2f2736bc0 5.4.0-1093-aws #102~18.04.2-Ubuntu SMP Wed Dec 7 00:31:59 UTC 2022 x86_64 x86_64 x86_64 GNU/Linux
Build tool	maven
Personality	dev-support/hbase-personality.sh
git revision	master / `8ba56cc`
Default Java	Temurin-1.8.0_352-b08
Test Results	https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-4664/6/testReport/
Max. process+thread count	2550 (vs. ulimit of 30000)
modules	C: hbase-server U: hbase-server
Console output	https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-4664/6/console
versions	git=2.34.1 maven=3.8.6
Powered by	Apache Yetus 0.12.0 https://yetus.apache.org

This message was automatically generated.

petersomogyi

This change looks good to me. Thanks @ndimiduk for moving this patch forward!

ndimiduk · 2023-02-21T12:48:27Z

Merged to master. Thanks all for the effort!

This comment was marked as outdated.

Sign in to view

huaxiangsun force-pushed the master.HBASE-27250 branch from f626df5 to c283ea7 Compare August 4, 2022 19:11

This comment was marked as outdated.

Sign in to view

Apache9 reviewed Aug 5, 2022

View reviewed changes

hbase-server/src/main/java/org/apache/hadoop/hbase/master/MasterRpcServices.java Outdated Show resolved Hide resolved

hbase-server/src/main/java/org/apache/hadoop/hbase/master/MasterRpcServices.java Outdated Show resolved Hide resolved

This comment was marked as outdated.

Sign in to view

Apache9 reviewed Sep 14, 2022

View reviewed changes

hbase-server/src/main/java/org/apache/hadoop/hbase/master/MasterRpcServices.java Outdated Show resolved Hide resolved

Huaxiang Sun added 2 commits February 14, 2023 08:43

HBASE-27250 MasterRpcService#setRegionStateInMeta does not support re…

956ee05

…plica region encodedNames or region names

Added sanity check to make sure input region encoded name or region n…

9ebdea1

…ame is valid

ndimiduk force-pushed the master.HBASE-27250 branch from ab8030a to a3585b1 Compare February 14, 2023 07:51

This comment was marked as outdated.

Sign in to view

ndimiduk reviewed Feb 14, 2023

View reviewed changes

hbase-server/src/main/java/org/apache/hadoop/hbase/MetaTableAccessor.java Outdated Show resolved Hide resolved

ndimiduk added 2 commits February 14, 2023 18:27

consolidate, extend tests to cover read replica regions

47df4f5

ndimiduk force-pushed the master.HBASE-27250 branch from a3585b1 to 47df4f5 Compare February 14, 2023 18:18

ndimiduk requested review from Apache9, petersomogyi and wchevreuil February 15, 2023 09:09

petersomogyi approved these changes Feb 16, 2023

View reviewed changes

ndimiduk merged commit 22dbb7a into apache:master Feb 21, 2023

HBASE-27250 MasterRpcService#setRegionStateInMeta does not support re… #4664

HBASE-27250 MasterRpcService#setRegionStateInMeta does not support re… #4664

Uh oh!

Conversation

huaxiangsun commented Jul 28, 2022

Uh oh!

This comment was marked as outdated.

This comment was marked as outdated.

This comment was marked as outdated.

huaxiangsun commented Aug 4, 2022

Uh oh!

This comment was marked as outdated.

This comment was marked as outdated.

This comment was marked as outdated.

Uh oh!

Uh oh!

huaxiangsun commented Aug 5, 2022

Uh oh!

This comment was marked as outdated.

huaxiangsun commented Sep 7, 2022

Uh oh!

Apache9 left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

huaxiangsun commented Sep 14, 2022

Uh oh!

Apache9 commented Sep 20, 2022

Uh oh!

ndimiduk commented Feb 8, 2023

Uh oh!

ndimiduk commented Feb 14, 2023

Uh oh!

This comment was marked as outdated.

This comment was marked as outdated.

This comment was marked as outdated.

This comment was marked as outdated.

This comment was marked as outdated.

This comment was marked as outdated.

Uh oh!

Apache-HBase commented Feb 14, 2023

Uh oh!

Apache-HBase commented Feb 14, 2023

Uh oh!

Apache-HBase commented Feb 14, 2023

Uh oh!

petersomogyi left a comment

Choose a reason for hiding this comment

Uh oh!

ndimiduk commented Feb 21, 2023

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

5 participants