HBASE-27715 Refactoring the long tryAdvanceEntry method in WALEntrySt… #5105

Apache9 · 2023-03-14T15:47:07Z

…ream

Apache9 · 2023-03-14T15:48:04Z

hbase-server/src/main/java/org/apache/hadoop/hbase/replication/regionserver/WALEntryStream.java

-            return HasNext.RETRY;
-          }
-          Pair<WALTailingReader.State, Boolean> p = readNextEntryAndRecordReaderPosition();
-          state = pair.getFirst();


There is a typo, this maybe the problem why some replication related tests are unstable...

Apache-HBase · 2023-03-14T16:29:14Z

🎊 +1 overall

Vote	Subsystem	Runtime	Comment
+0 🆗	reexec	0m 58s	Docker mode activated.
		_ Prechecks _
+1 💚	dupname	0m 0s	No case conflicting files found.
+1 💚	hbaseanti	0m 0s	Patch does not have any anti-patterns.
+1 💚	@author	0m 0s	The patch does not contain any @author tags.
		_ master Compile Tests _
+1 💚	mvninstall	3m 50s	master passed
+1 💚	compile	2m 22s	master passed
+1 💚	checkstyle	0m 35s	master passed
+1 💚	spotless	0m 43s	branch has no errors when running spotless:check.
+1 💚	spotbugs	1m 30s	master passed
		_ Patch Compile Tests _
+1 💚	mvninstall	3m 17s	the patch passed
+1 💚	compile	2m 22s	the patch passed
+1 💚	javac	2m 22s	the patch passed
+1 💚	checkstyle	0m 32s	the patch passed
+1 💚	whitespace	0m 0s	The patch has no whitespace issues.
+1 💚	hadoopcheck	12m 10s	Patch does not cause any errors with Hadoop 3.2.4 3.3.4.
+1 💚	spotless	0m 41s	patch has no errors when running spotless:check.
+1 💚	spotbugs	1m 36s	the patch passed
		_ Other Tests _
+1 💚	asflicense	0m 11s	The patch does not generate ASF License warnings.
		38m 34s

Subsystem	Report/Notes
Docker	ClientAPI=1.42 ServerAPI=1.42 base: https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-5105/1/artifact/yetus-general-check/output/Dockerfile
GITHUB PR	#5105
Optional Tests	dupname asflicense javac spotbugs hadoopcheck hbaseanti spotless checkstyle compile
uname	Linux 6ce07025de98 5.4.0-144-generic #161-Ubuntu SMP Fri Feb 3 14:49:04 UTC 2023 x86_64 x86_64 x86_64 GNU/Linux
Build tool	maven
Personality	dev-support/hbase-personality.sh
git revision	master / `c2b64e7`
Default Java	Eclipse Adoptium-11.0.17+8
Max. process+thread count	85 (vs. ulimit of 30000)
modules	C: hbase-server U: hbase-server
Console output	https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-5105/1/console
versions	git=2.34.1 maven=3.8.6 spotbugs=4.7.3
Powered by	Apache Yetus 0.12.0 https://yetus.apache.org

This message was automatically generated.

Apache-HBase · 2023-03-14T20:10:50Z

🎊 +1 overall

Vote	Subsystem	Runtime	Comment
+0 🆗	reexec	0m 46s	Docker mode activated.
-0 ⚠️	yetus	0m 3s	Unprocessed flag(s): --brief-report-file --spotbugs-strict-precheck --whitespace-eol-ignore-list --whitespace-tabs-ignore-list --quick-hadoopcheck
		_ Prechecks _
		_ master Compile Tests _
+1 💚	mvninstall	4m 40s	master passed
+1 💚	compile	1m 8s	master passed
+1 💚	shadedjars	7m 5s	branch has no errors when building our shaded downstream artifacts.
+1 💚	javadoc	0m 35s	master passed
		_ Patch Compile Tests _
+1 💚	mvninstall	4m 14s	the patch passed
+1 💚	compile	1m 0s	the patch passed
+1 💚	javac	1m 0s	the patch passed
+1 💚	shadedjars	6m 50s	patch has no errors when building our shaded downstream artifacts.
+1 💚	javadoc	0m 37s	the patch passed
		_ Other Tests _
+1 💚	unit	228m 53s	hbase-server in the patch passed.
		260m 8s

Subsystem	Report/Notes
Docker	ClientAPI=1.42 ServerAPI=1.42 base: https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-5105/1/artifact/yetus-jdk8-hadoop3-check/output/Dockerfile
GITHUB PR	#5105
Optional Tests	javac javadoc unit shadedjars compile
uname	Linux 857ecedc88dd 5.4.0-1093-aws #102~18.04.2-Ubuntu SMP Wed Dec 7 00:31:59 UTC 2022 x86_64 x86_64 x86_64 GNU/Linux
Build tool	maven
Personality	dev-support/hbase-personality.sh
git revision	master / `c2b64e7`
Default Java	Temurin-1.8.0_352-b08
Test Results	https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-5105/1/testReport/
Max. process+thread count	2409 (vs. ulimit of 30000)
modules	C: hbase-server U: hbase-server
Console output	https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-5105/1/console
versions	git=2.34.1 maven=3.8.6
Powered by	Apache Yetus 0.12.0 https://yetus.apache.org

This message was automatically generated.

infraio · 2023-03-15T01:54:03Z

hbase-server/src/main/java/org/apache/hadoop/hbase/replication/regionserver/WALEntryStream.java

+    assert !pair.getSecond();
+    if (!state.eof()) {
+      // we still have something to read after reopen
+      return HasNext.YES;


// just sleep a bit and retry to see if there are new entries coming since the file is // still being written return HasNext.RETRY;"
When state not eof, the return value changed from HasNext.RETRY to HasNext.YES?

This should also be a problem in the old code. We should return something here but in the old code, we just fall into the next switch section...
And after consideration, I think here we also need to consider whether there are errors, let me update the code.

Updated. Will return RETRY if there are errors, otherwise return YES. If still get EOF, we can dequeue the current file.

…ream

2005hithlj

LGTM

Apache-HBase · 2023-03-15T07:12:48Z

🎊 +1 overall

Vote	Subsystem	Runtime	Comment
+0 🆗	reexec	0m 55s	Docker mode activated.
		_ Prechecks _
+1 💚	dupname	0m 0s	No case conflicting files found.
+1 💚	hbaseanti	0m 0s	Patch does not have any anti-patterns.
+1 💚	@author	0m 0s	The patch does not contain any @author tags.
		_ master Compile Tests _
+1 💚	mvninstall	3m 18s	master passed
+1 💚	compile	2m 24s	master passed
+1 💚	checkstyle	0m 33s	master passed
+1 💚	spotless	0m 41s	branch has no errors when running spotless:check.
+1 💚	spotbugs	1m 27s	master passed
		_ Patch Compile Tests _
+1 💚	mvninstall	3m 16s	the patch passed
+1 💚	compile	2m 23s	the patch passed
+1 💚	javac	2m 23s	the patch passed
+1 💚	checkstyle	0m 35s	the patch passed
+1 💚	whitespace	0m 0s	The patch has no whitespace issues.
+1 💚	hadoopcheck	12m 13s	Patch does not cause any errors with Hadoop 3.2.4 3.3.4.
+1 💚	spotless	0m 41s	patch has no errors when running spotless:check.
+1 💚	spotbugs	1m 34s	the patch passed
		_ Other Tests _
+1 💚	asflicense	0m 9s	The patch does not generate ASF License warnings.
		37m 26s

Subsystem	Report/Notes
Docker	ClientAPI=1.42 ServerAPI=1.42 base: https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-5105/2/artifact/yetus-general-check/output/Dockerfile
GITHUB PR	#5105
Optional Tests	dupname asflicense javac spotbugs hadoopcheck hbaseanti spotless checkstyle compile
uname	Linux 0f918569bd0e 5.4.0-144-generic #161-Ubuntu SMP Fri Feb 3 14:49:04 UTC 2023 x86_64 x86_64 x86_64 GNU/Linux
Build tool	maven
Personality	dev-support/hbase-personality.sh
git revision	master / `c8bee23`
Default Java	Eclipse Adoptium-11.0.17+8
Max. process+thread count	80 (vs. ulimit of 30000)
modules	C: hbase-server U: hbase-server
Console output	https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-5105/2/console
versions	git=2.34.1 maven=3.8.6 spotbugs=4.7.3
Powered by	Apache Yetus 0.12.0 https://yetus.apache.org

This message was automatically generated.

Apache-HBase · 2023-03-15T10:44:30Z

🎊 +1 overall

Vote	Subsystem	Runtime	Comment
+0 🆗	reexec	1m 2s	Docker mode activated.
-0 ⚠️	yetus	0m 3s	Unprocessed flag(s): --brief-report-file --spotbugs-strict-precheck --whitespace-eol-ignore-list --whitespace-tabs-ignore-list --quick-hadoopcheck
		_ Prechecks _
		_ master Compile Tests _
+1 💚	mvninstall	3m 8s	master passed
+1 💚	compile	0m 39s	master passed
+1 💚	shadedjars	4m 17s	branch has no errors when building our shaded downstream artifacts.
+1 💚	javadoc	0m 26s	master passed
		_ Patch Compile Tests _
+1 💚	mvninstall	2m 52s	the patch passed
+1 💚	compile	0m 41s	the patch passed
+1 💚	javac	0m 41s	the patch passed
+1 💚	shadedjars	4m 16s	patch has no errors when building our shaded downstream artifacts.
+1 💚	javadoc	0m 23s	the patch passed
		_ Other Tests _
+1 💚	unit	227m 13s	hbase-server in the patch passed.
		249m 12s

Subsystem	Report/Notes
Docker	ClientAPI=1.42 ServerAPI=1.42 base: https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-5105/2/artifact/yetus-jdk8-hadoop3-check/output/Dockerfile
GITHUB PR	#5105
Optional Tests	javac javadoc unit shadedjars compile
uname	Linux d54f017568de 5.4.0-137-generic #154-Ubuntu SMP Thu Jan 5 17:03:22 UTC 2023 x86_64 x86_64 x86_64 GNU/Linux
Build tool	maven
Personality	dev-support/hbase-personality.sh
git revision	master / `c8bee23`
Default Java	Temurin-1.8.0_352-b08
Test Results	https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-5105/2/testReport/
Max. process+thread count	2677 (vs. ulimit of 30000)
modules	C: hbase-server U: hbase-server
Console output	https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-5105/2/console
versions	git=2.34.1 maven=3.8.6
Powered by	Apache Yetus 0.12.0 https://yetus.apache.org

This message was automatically generated.

Apache-HBase · 2023-03-15T11:00:25Z

🎊 +1 overall

Vote	Subsystem	Runtime	Comment
+0 🆗	reexec	1m 3s	Docker mode activated.
-0 ⚠️	yetus	0m 2s	Unprocessed flag(s): --brief-report-file --spotbugs-strict-precheck --whitespace-eol-ignore-list --whitespace-tabs-ignore-list --quick-hadoopcheck
		_ Prechecks _
		_ master Compile Tests _
+1 💚	mvninstall	3m 34s	master passed
+1 💚	compile	0m 48s	master passed
+1 💚	shadedjars	4m 23s	branch has no errors when building our shaded downstream artifacts.
+1 💚	javadoc	0m 27s	master passed
		_ Patch Compile Tests _
+1 💚	mvninstall	3m 20s	the patch passed
+1 💚	compile	0m 51s	the patch passed
+1 💚	javac	0m 51s	the patch passed
+1 💚	shadedjars	4m 22s	patch has no errors when building our shaded downstream artifacts.
+1 💚	javadoc	0m 25s	the patch passed
		_ Other Tests _
+1 💚	unit	240m 49s	hbase-server in the patch passed.
		265m 7s

Subsystem	Report/Notes
Docker	ClientAPI=1.42 ServerAPI=1.42 base: https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-5105/2/artifact/yetus-jdk11-hadoop3-check/output/Dockerfile
GITHUB PR	#5105
Optional Tests	javac javadoc unit shadedjars compile
uname	Linux ed9a14a76c32 5.4.0-144-generic #161-Ubuntu SMP Fri Feb 3 14:49:04 UTC 2023 x86_64 x86_64 x86_64 GNU/Linux
Build tool	maven
Personality	dev-support/hbase-personality.sh
git revision	master / `c8bee23`
Default Java	Eclipse Adoptium-11.0.17+8
Test Results	https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-5105/2/testReport/
Max. process+thread count	2461 (vs. ulimit of 30000)
modules	C: hbase-server U: hbase-server
Console output	https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-5105/2/console
versions	git=2.34.1 maven=3.8.6
Powered by	Apache Yetus 0.12.0 https://yetus.apache.org

This message was automatically generated.

Apache9 · 2023-03-15T13:46:44Z

OK, good, all green. Let me merge.

…ream (#5105) Signed-off-by: Liangjun He <[email protected]> (cherry picked from commit 1f2e1f5)

Apache9 commented Mar 14, 2023

View reviewed changes

Apache9 self-assigned this Mar 15, 2023

infraio reviewed Mar 15, 2023

View reviewed changes

2005hithlj self-requested a review March 15, 2023 02:53

HBASE-27715 Refactoring the long tryAdvanceEntry method in WALEntrySt…

33a0e00

…ream

2005hithlj approved these changes Mar 15, 2023

View reviewed changes

Apache9 force-pushed the HBASE-27715 branch from 8e9ac79 to 33a0e00 Compare March 15, 2023 06:29

Apache9 merged commit 1f2e1f5 into apache:master Mar 15, 2023

Apache9 added a commit that referenced this pull request Mar 15, 2023

HBASE-27715 Refactoring the long tryAdvanceEntry method in WALEntrySt…

ef8a981

…ream (#5105) Signed-off-by: Liangjun He <[email protected]> (cherry picked from commit 1f2e1f5)

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Uh oh!

HBASE-27715 Refactoring the long tryAdvanceEntry method in WALEntrySt… #5105

HBASE-27715 Refactoring the long tryAdvanceEntry method in WALEntrySt… #5105

Uh oh!

Apache9 commented Mar 14, 2023

Uh oh!

Apache9 Mar 14, 2023

Uh oh!

Apache-HBase commented Mar 14, 2023

Uh oh!

Apache-HBase commented Mar 14, 2023

Uh oh!

infraio Mar 15, 2023

Uh oh!

Apache9 Mar 15, 2023

Uh oh!

Apache9 Mar 15, 2023

Uh oh!

2005hithlj left a comment

Uh oh!

Apache-HBase commented Mar 15, 2023

Uh oh!

Apache-HBase commented Mar 15, 2023

Uh oh!

Apache-HBase commented Mar 15, 2023

Uh oh!

Apache9 commented Mar 15, 2023

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

4 participants

HBASE-27715 Refactoring the long tryAdvanceEntry method in WALEntrySt… #5105

HBASE-27715 Refactoring the long tryAdvanceEntry method in WALEntrySt… #5105

Uh oh!

Conversation

Apache9 commented Mar 14, 2023

Uh oh!

Apache9 Mar 14, 2023

Choose a reason for hiding this comment

Uh oh!

Apache-HBase commented Mar 14, 2023

Uh oh!

Apache-HBase commented Mar 14, 2023

Uh oh!

infraio Mar 15, 2023

Choose a reason for hiding this comment

Uh oh!

Apache9 Mar 15, 2023

Choose a reason for hiding this comment

Uh oh!

Apache9 Mar 15, 2023

Choose a reason for hiding this comment

Uh oh!

2005hithlj left a comment

Choose a reason for hiding this comment

Uh oh!

Apache-HBase commented Mar 15, 2023

Uh oh!

Apache-HBase commented Mar 15, 2023

Uh oh!

Apache-HBase commented Mar 15, 2023

Uh oh!

Apache9 commented Mar 15, 2023

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

4 participants