HBASE-24740 Enable journal logging for HBase snapshot operation #2104

sguggilam · 2020-07-20T22:09:20Z

No description provided.

Apache-HBase · 2020-07-20T22:57:00Z

🎊 +1 overall

Vote	Subsystem	Runtime	Comment
+0 🆗	reexec	7m 7s	Docker mode activated.
		_ Prechecks _
+1 💚	dupname	0m 0s	No case conflicting files found.
+1 💚	hbaseanti	0m 0s	Patch does not have any anti-patterns.
+1 💚	@author	0m 0s	The patch does not contain any @author tags.
		_ master Compile Tests _
+1 💚	mvninstall	4m 7s	master passed
+1 💚	checkstyle	1m 12s	master passed
+1 💚	spotbugs	2m 8s	master passed
		_ Patch Compile Tests _
+1 💚	mvninstall	3m 44s	the patch passed
-0 ⚠️	checkstyle	1m 10s	hbase-server: The patch generated 1 new + 22 unchanged - 3 fixed = 23 total (was 25)
+1 💚	whitespace	0m 0s	The patch has no whitespace issues.
+1 💚	hadoopcheck	12m 17s	Patch does not cause any errors with Hadoop 3.1.2 3.2.1.
+1 💚	spotbugs	2m 16s	the patch passed
		_ Other Tests _
+1 💚	asflicense	0m 13s	The patch does not generate ASF License warnings.
		41m 53s

Subsystem	Report/Notes
Docker	Client=19.03.12 Server=19.03.12 base: https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-2104/1/artifact/yetus-general-check/output/Dockerfile
GITHUB PR	#2104
Optional Tests	dupname asflicense spotbugs hadoopcheck hbaseanti checkstyle
uname	Linux 1045a40fa5e9 4.15.0-101-generic #102-Ubuntu SMP Mon May 11 10:07:26 UTC 2020 x86_64 x86_64 x86_64 GNU/Linux
Build tool	maven
Personality	dev-support/hbase-personality.sh
git revision	master / `a857c6d`
checkstyle	https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-2104/1/artifact/yetus-general-check/output/diff-checkstyle-hbase-server.txt
Max. process+thread count	84 (vs. ulimit of 12500)
modules	C: hbase-server U: hbase-server
Console output	https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-2104/1/console
versions	git=2.17.1 maven=(cecedd343002696d0abb50b32b541b8a6ba2883f) spotbugs=3.1.12
Powered by	Apache Yetus 0.11.1 https://yetus.apache.org

This message was automatically generated.

sguggilam · 2020-07-20T23:22:01Z

@bharathv Please review and merge

bharathv

A few nits, but lgtm, can commit once fixed. @sguggilam Can you please confirm that you did a quick sanity check on the journal logging? (given there are no tests).

bharathv · 2020-07-20T23:23:50Z

hbase-server/src/main/java/org/apache/hadoop/hbase/snapshot/SnapshotManifest.java

          false);
    } else {
-      LOG.debug("Convert to Single Snapshot Manifest");
+      LOG.debug("Convert to Single Snapshot Manifest for " + this.desc.getName());


nit: parameterized logging

bharathv · 2020-07-20T23:25:01Z

hbase-server/src/main/java/org/apache/hadoop/hbase/master/snapshot/SnapshotManager.java

    if (master.getTableStateManager().isTableState(snapshotTable,
        TableState.State.ENABLED)) {
-      LOG.debug("Table enabled, starting distributed snapshot.");
+      LOG.debug("Table enabled, starting distributed snapshots for {}",


nit: This still requires the isDebugEnabled() guard because ClientSnapshotDescriptionUtils.toString(..) is not a trivial call.

bharathv · 2020-07-20T23:25:12Z

hbase-server/src/main/java/org/apache/hadoop/hbase/master/snapshot/SnapshotManager.java

    else if (master.getTableStateManager().isTableState(snapshotTable,
        TableState.State.DISABLED)) {
-      LOG.debug("Table is disabled, running snapshot entirely on master.");
+        LOG.debug("Table is disabled, running snapshot entirely on master for {}",


same as above.

sguggilam · 2020-07-21T00:09:42Z

Yes @bharathv , tested out the journal log entry and below is the snippet

Table snapshot journal :
Running FLUSH table snapshot emp1 C_M_SNAPSHOT_TABLE on table emp at 1595282584365
Consolidate snapshot: emp1 at 1595282584593
Loading Region manifests for emp1 at 1595282584593
Writing data manifest for emp1 at 1595282584603
Verifying snapshot: emp1 at 1595282584616

bharathv

Perfect, thanks. Will merge it once the precommit is green.

Apache-HBase · 2020-07-21T01:06:08Z

💔 -1 overall

Vote	Subsystem	Runtime	Comment
+0 🆗	reexec	0m 29s	Docker mode activated.
-0 ⚠️	yetus	0m 3s	Unprocessed flag(s): --brief-report-file --spotbugs-strict-precheck --whitespace-eol-ignore-list --whitespace-tabs-ignore-list --quick-hadoopcheck
		_ Prechecks _
		_ master Compile Tests _
+1 💚	mvninstall	3m 44s	master passed
+1 💚	compile	0m 55s	master passed
+1 💚	shadedjars	5m 35s	branch has no errors when building our shaded downstream artifacts.
+1 💚	javadoc	0m 40s	master passed
		_ Patch Compile Tests _
+1 💚	mvninstall	3m 29s	the patch passed
+1 💚	compile	0m 55s	the patch passed
+1 💚	javac	0m 55s	the patch passed
+1 💚	shadedjars	5m 40s	patch has no errors when building our shaded downstream artifacts.
+1 💚	javadoc	0m 35s	the patch passed
		_ Other Tests _
-1 ❌	unit	147m 4s	hbase-server in the patch failed.
		171m 3s

Subsystem	Report/Notes
Docker	Client=19.03.12 Server=19.03.12 base: https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-2104/1/artifact/yetus-jdk8-hadoop3-check/output/Dockerfile
GITHUB PR	#2104
Optional Tests	javac javadoc unit shadedjars compile
uname	Linux 40458fdb34fc 4.15.0-58-generic #64-Ubuntu SMP Tue Aug 6 11:12:41 UTC 2019 x86_64 x86_64 x86_64 GNU/Linux
Build tool	maven
Personality	dev-support/hbase-personality.sh
git revision	master / `a857c6d`
Default Java	1.8.0_232
unit	https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-2104/1/artifact/yetus-jdk8-hadoop3-check/output/patch-unit-hbase-server.txt
Test Results	https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-2104/1/testReport/
Max. process+thread count	4124 (vs. ulimit of 12500)
modules	C: hbase-server U: hbase-server
Console output	https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-2104/1/console
versions	git=2.17.1 maven=(cecedd343002696d0abb50b32b541b8a6ba2883f)
Powered by	Apache Yetus 0.11.1 https://yetus.apache.org

This message was automatically generated.

Signed-off-by: Bharath Vissapragada <[email protected]> (cherry picked from commit 430602a)

Apache-HBase · 2020-07-21T02:06:13Z

💔 -1 overall

Vote	Subsystem	Runtime	Comment
+0 🆗	reexec	1m 55s	Docker mode activated.
-0 ⚠️	yetus	0m 3s	Unprocessed flag(s): --brief-report-file --spotbugs-strict-precheck --whitespace-eol-ignore-list --whitespace-tabs-ignore-list --quick-hadoopcheck
		_ Prechecks _
		_ master Compile Tests _
+1 💚	mvninstall	5m 1s	master passed
+1 💚	compile	1m 14s	master passed
+1 💚	shadedjars	6m 33s	branch has no errors when building our shaded downstream artifacts.
-0 ⚠️	javadoc	0m 45s	hbase-server in master failed.
		_ Patch Compile Tests _
+1 💚	mvninstall	4m 55s	the patch passed
+1 💚	compile	1m 21s	the patch passed
+1 💚	javac	1m 21s	the patch passed
+1 💚	shadedjars	7m 9s	patch has no errors when building our shaded downstream artifacts.
-0 ⚠️	javadoc	0m 51s	hbase-server in the patch failed.
		_ Other Tests _
-1 ❌	unit	199m 28s	hbase-server in the patch failed.
		231m 5s

Subsystem	Report/Notes
Docker	Client=19.03.12 Server=19.03.12 base: https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-2104/1/artifact/yetus-jdk11-hadoop3-check/output/Dockerfile
GITHUB PR	#2104
Optional Tests	javac javadoc unit shadedjars compile
uname	Linux 08a73f07b4ec 4.15.0-74-generic #84-Ubuntu SMP Thu Dec 19 08:06:28 UTC 2019 x86_64 x86_64 x86_64 GNU/Linux
Build tool	maven
Personality	dev-support/hbase-personality.sh
git revision	master / `a857c6d`
Default Java	2020-01-14
javadoc	https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-2104/1/artifact/yetus-jdk11-hadoop3-check/output/branch-javadoc-hbase-server.txt
javadoc	https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-2104/1/artifact/yetus-jdk11-hadoop3-check/output/patch-javadoc-hbase-server.txt
unit	https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-2104/1/artifact/yetus-jdk11-hadoop3-check/output/patch-unit-hbase-server.txt
Test Results	https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-2104/1/testReport/
Max. process+thread count	3719 (vs. ulimit of 12500)
modules	C: hbase-server U: hbase-server
Console output	https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-2104/1/console
versions	git=2.17.1 maven=(cecedd343002696d0abb50b32b541b8a6ba2883f)
Powered by	Apache Yetus 0.11.1 https://yetus.apache.org

This message was automatically generated.

…he#2104) Signed-off-by: Bharath Vissapragada <[email protected]> (cherry picked from commit 430602a)

…he#2104) Signed-off-by: Bharath Vissapragada <[email protected]> (cherry picked from commit 430602a) (cherry picked from commit 4e5ec22) Change-Id: Ibe837047e9f93c608ec13972a9f6b1ac3622c92f

HBASE-24740 Enable journal logging for HBase snapshot operation

840cf09

bharathv approved these changes Jul 20, 2020

View reviewed changes

Incorporatte review comments

c69be59

bharathv approved these changes Jul 21, 2020

View reviewed changes

bharathv merged commit 430602a into apache:master Jul 21, 2020

bharathv pushed a commit that referenced this pull request Jul 21, 2020

HBASE-24740 Enable journal logging for HBase snapshot operation (#2104)

8341b65

Signed-off-by: Bharath Vissapragada <[email protected]> (cherry picked from commit 430602a)

bharathv pushed a commit that referenced this pull request Jul 21, 2020

HBASE-24740 Enable journal logging for HBase snapshot operation (#2104)

4e5ec22

Signed-off-by: Bharath Vissapragada <[email protected]> (cherry picked from commit 430602a)

bharathv pushed a commit that referenced this pull request Jul 21, 2020

HBASE-24740 Enable journal logging for HBase snapshot operation (#2104)

19d0f36

Signed-off-by: Bharath Vissapragada <[email protected]> (cherry picked from commit 430602a)

clarax pushed a commit to clarax/hbase that referenced this pull request Nov 15, 2020

HBASE-24740 Enable journal logging for HBase snapshot operation (apac…

6e5df0c

…he#2104) Signed-off-by: Bharath Vissapragada <[email protected]> (cherry picked from commit 430602a)

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Uh oh!

HBASE-24740 Enable journal logging for HBase snapshot operation #2104

HBASE-24740 Enable journal logging for HBase snapshot operation #2104

Uh oh!

sguggilam commented Jul 20, 2020

Uh oh!

Apache-HBase commented Jul 20, 2020

Uh oh!

sguggilam commented Jul 20, 2020

Uh oh!

bharathv left a comment

Uh oh!

bharathv Jul 20, 2020

Uh oh!

bharathv Jul 20, 2020

Uh oh!

bharathv Jul 20, 2020

Uh oh!

sguggilam commented Jul 21, 2020

Uh oh!

bharathv left a comment

Uh oh!

Apache-HBase commented Jul 21, 2020

Uh oh!

Apache-HBase commented Jul 21, 2020

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

HBASE-24740 Enable journal logging for HBase snapshot operation #2104

HBASE-24740 Enable journal logging for HBase snapshot operation #2104

Uh oh!

Conversation

sguggilam commented Jul 20, 2020

Uh oh!

Apache-HBase commented Jul 20, 2020

Uh oh!

sguggilam commented Jul 20, 2020

Uh oh!

bharathv left a comment

Choose a reason for hiding this comment

Uh oh!

bharathv Jul 20, 2020

Choose a reason for hiding this comment

Uh oh!

bharathv Jul 20, 2020

Choose a reason for hiding this comment

Uh oh!

bharathv Jul 20, 2020

Choose a reason for hiding this comment

Uh oh!

sguggilam commented Jul 21, 2020

Uh oh!

bharathv left a comment

Choose a reason for hiding this comment

Uh oh!

Apache-HBase commented Jul 21, 2020

Uh oh!

Apache-HBase commented Jul 21, 2020

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants