HBASE-26349 Improve recent change to IntegrationTestLoadCommonCrawl #3744

apurtell · 2021-10-11T22:18:45Z

Use a hybrid logical clock for timestamping entries.

Using BufferedMutator without HLC was not good because we assign client timestamps, and the store loop is fast enough that on rare occasion two temporally adjacent URLs in the set of WARCs are equivalent and the timestamp does not advance, leading later to a rare false positive CORRUPT finding.

While making changes, support direct S3N paths as input paths on the command line.

Apache-HBase · 2021-10-11T22:50:15Z

🎊 +1 overall

Vote	Subsystem	Runtime	Comment
+0 🆗	reexec	1m 3s	Docker mode activated.
-0 ⚠️	yetus	0m 4s	Unprocessed flag(s): --brief-report-file --spotbugs-strict-precheck --whitespace-eol-ignore-list --whitespace-tabs-ignore-list --quick-hadoopcheck
		_ Prechecks _
		_ master Compile Tests _
+1 💚	mvninstall	4m 49s	master passed
+1 💚	compile	0m 30s	master passed
+1 💚	shadedjars	8m 15s	branch has no errors when building our shaded downstream artifacts.
+1 💚	javadoc	0m 22s	master passed
		_ Patch Compile Tests _
+1 💚	mvninstall	4m 25s	the patch passed
+1 💚	compile	0m 30s	the patch passed
+1 💚	javac	0m 30s	the patch passed
+1 💚	shadedjars	8m 10s	patch has no errors when building our shaded downstream artifacts.
+1 💚	javadoc	0m 18s	the patch passed
		_ Other Tests _
+1 💚	unit	0m 57s	hbase-it in the patch passed.
		30m 37s

Subsystem	Report/Notes
Docker	ClientAPI=1.41 ServerAPI=1.41 base: https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-3744/1/artifact/yetus-jdk11-hadoop3-check/output/Dockerfile
GITHUB PR	#3744
Optional Tests	javac javadoc unit shadedjars compile
uname	Linux 1a23e73d3dd9 4.15.0-112-generic #113-Ubuntu SMP Thu Jul 9 23:41:39 UTC 2020 x86_64 x86_64 x86_64 GNU/Linux
Build tool	maven
Personality	dev-support/hbase-personality.sh
git revision	master / `ce44e16`
Default Java	AdoptOpenJDK-11.0.10+9
Test Results	https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-3744/1/testReport/
Max. process+thread count	692 (vs. ulimit of 30000)
modules	C: hbase-it U: hbase-it
Console output	https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-3744/1/console
versions	git=2.17.1 maven=3.6.3
Powered by	Apache Yetus 0.12.0 https://yetus.apache.org

This message was automatically generated.

Apache-HBase · 2021-10-11T22:53:30Z

🎊 +1 overall

Vote	Subsystem	Runtime	Comment
+0 🆗	reexec	1m 29s	Docker mode activated.
-0 ⚠️	yetus	0m 4s	Unprocessed flag(s): --brief-report-file --spotbugs-strict-precheck --whitespace-eol-ignore-list --whitespace-tabs-ignore-list --quick-hadoopcheck
		_ Prechecks _
		_ master Compile Tests _
+1 💚	mvninstall	4m 59s	master passed
+1 💚	compile	0m 31s	master passed
+1 💚	shadedjars	9m 42s	branch has no errors when building our shaded downstream artifacts.
+1 💚	javadoc	0m 19s	master passed
		_ Patch Compile Tests _
+1 💚	mvninstall	4m 41s	the patch passed
+1 💚	compile	0m 28s	the patch passed
+1 💚	javac	0m 28s	the patch passed
+1 💚	shadedjars	9m 14s	patch has no errors when building our shaded downstream artifacts.
+1 💚	javadoc	0m 15s	the patch passed
		_ Other Tests _
+1 💚	unit	0m 58s	hbase-it in the patch passed.
		33m 49s

Subsystem	Report/Notes
Docker	ClientAPI=1.41 ServerAPI=1.41 base: https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-3744/1/artifact/yetus-jdk8-hadoop3-check/output/Dockerfile
GITHUB PR	#3744
Optional Tests	javac javadoc unit shadedjars compile
uname	Linux 697c2e930c14 4.15.0-143-generic #147-Ubuntu SMP Wed Apr 14 16:10:11 UTC 2021 x86_64 x86_64 x86_64 GNU/Linux
Build tool	maven
Personality	dev-support/hbase-personality.sh
git revision	master / `ce44e16`
Default Java	AdoptOpenJDK-1.8.0_282-b08
Test Results	https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-3744/1/testReport/
Max. process+thread count	626 (vs. ulimit of 30000)
modules	C: hbase-it U: hbase-it
Console output	https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-3744/1/console
versions	git=2.17.1 maven=3.6.3
Powered by	Apache Yetus 0.12.0 https://yetus.apache.org

This message was automatically generated.

Apache-HBase · 2021-10-11T23:10:44Z

🎊 +1 overall

Vote	Subsystem	Runtime	Comment
+0 🆗	reexec	1m 20s	Docker mode activated.
		_ Prechecks _
+1 💚	dupname	0m 0s	No case conflicting files found.
+1 💚	hbaseanti	0m 0s	Patch does not have any anti-patterns.
+1 💚	@author	0m 0s	The patch does not contain any @author tags.
		_ master Compile Tests _
+1 💚	mvninstall	5m 7s	master passed
+1 💚	compile	0m 52s	master passed
+1 💚	checkstyle	0m 21s	master passed
+1 💚	spotbugs	0m 43s	master passed
		_ Patch Compile Tests _
+1 💚	mvninstall	4m 52s	the patch passed
+1 💚	compile	0m 47s	the patch passed
-0 ⚠️	javac	0m 47s	hbase-it generated 4 new + 99 unchanged - 4 fixed = 103 total (was 103)
+1 💚	checkstyle	0m 19s	the patch passed
+1 💚	whitespace	0m 0s	The patch has no whitespace issues.
+1 💚	hadoopcheck	25m 42s	Patch does not cause any errors with Hadoop 3.1.2 3.2.1 3.3.0.
+1 💚	spotbugs	0m 54s	the patch passed
		_ Other Tests _
+1 💚	asflicense	0m 14s	The patch does not generate ASF License warnings.
		51m 0s

Subsystem	Report/Notes
Docker	ClientAPI=1.41 ServerAPI=1.41 base: https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-3744/1/artifact/yetus-general-check/output/Dockerfile
GITHUB PR	#3744
Optional Tests	dupname asflicense javac spotbugs hadoopcheck hbaseanti checkstyle compile
uname	Linux 56772de24ec3 4.15.0-147-generic #151-Ubuntu SMP Fri Jun 18 19:21:19 UTC 2021 x86_64 x86_64 x86_64 GNU/Linux
Build tool	maven
Personality	dev-support/hbase-personality.sh
git revision	master / `ce44e16`
Default Java	AdoptOpenJDK-1.8.0_282-b08
javac	https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-3744/1/artifact/yetus-general-check/output/diff-compile-javac-hbase-it.txt
Max. process+thread count	86 (vs. ulimit of 30000)
modules	C: hbase-it U: hbase-it
Console output	https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-3744/1/console
versions	git=2.17.1 maven=3.6.3 spotbugs=4.2.2
Powered by	Apache Yetus 0.12.0 https://yetus.apache.org

This message was automatically generated.

Apache-HBase · 2021-10-12T02:06:16Z

🎊 +1 overall

Vote	Subsystem	Runtime	Comment
+0 🆗	reexec	1m 9s	Docker mode activated.
-0 ⚠️	yetus	0m 3s	Unprocessed flag(s): --brief-report-file --spotbugs-strict-precheck --whitespace-eol-ignore-list --whitespace-tabs-ignore-list --quick-hadoopcheck
		_ Prechecks _
		_ master Compile Tests _
+1 💚	mvninstall	5m 2s	master passed
+1 💚	compile	0m 33s	master passed
+1 💚	shadedjars	8m 45s	branch has no errors when building our shaded downstream artifacts.
+1 💚	javadoc	0m 23s	master passed
		_ Patch Compile Tests _
+1 💚	mvninstall	4m 38s	the patch passed
+1 💚	compile	0m 30s	the patch passed
+1 💚	javac	0m 30s	the patch passed
+1 💚	shadedjars	8m 32s	patch has no errors when building our shaded downstream artifacts.
+1 💚	javadoc	0m 20s	the patch passed
		_ Other Tests _
+1 💚	unit	0m 58s	hbase-it in the patch passed.
		32m 7s

Subsystem	Report/Notes
Docker	ClientAPI=1.41 ServerAPI=1.41 base: https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-3744/2/artifact/yetus-jdk11-hadoop3-check/output/Dockerfile
GITHUB PR	#3744
Optional Tests	javac javadoc unit shadedjars compile
uname	Linux c2c7b3b2f44b 4.15.0-65-generic #74-Ubuntu SMP Tue Sep 17 17:06:04 UTC 2019 x86_64 x86_64 x86_64 GNU/Linux
Build tool	maven
Personality	dev-support/hbase-personality.sh
git revision	master / `ce44e16`
Default Java	AdoptOpenJDK-11.0.10+9
Test Results	https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-3744/2/testReport/
Max. process+thread count	683 (vs. ulimit of 30000)
modules	C: hbase-it U: hbase-it
Console output	https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-3744/2/console
versions	git=2.17.1 maven=3.6.3
Powered by	Apache Yetus 0.12.0 https://yetus.apache.org

This message was automatically generated.

Apache-HBase · 2021-10-12T02:07:25Z

🎊 +1 overall

Vote	Subsystem	Runtime	Comment
+0 🆗	reexec	1m 42s	Docker mode activated.
-0 ⚠️	yetus	0m 3s	Unprocessed flag(s): --brief-report-file --spotbugs-strict-precheck --whitespace-eol-ignore-list --whitespace-tabs-ignore-list --quick-hadoopcheck
		_ Prechecks _
		_ master Compile Tests _
+1 💚	mvninstall	4m 20s	master passed
+1 💚	compile	0m 28s	master passed
+1 💚	shadedjars	9m 23s	branch has no errors when building our shaded downstream artifacts.
+1 💚	javadoc	0m 17s	master passed
		_ Patch Compile Tests _
+1 💚	mvninstall	4m 36s	the patch passed
+1 💚	compile	0m 30s	the patch passed
+1 💚	javac	0m 30s	the patch passed
+1 💚	shadedjars	9m 27s	patch has no errors when building our shaded downstream artifacts.
+1 💚	javadoc	0m 16s	the patch passed
		_ Other Tests _
+1 💚	unit	1m 3s	hbase-it in the patch passed.
		33m 16s

Subsystem	Report/Notes
Docker	ClientAPI=1.41 ServerAPI=1.41 base: https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-3744/2/artifact/yetus-jdk8-hadoop3-check/output/Dockerfile
GITHUB PR	#3744
Optional Tests	javac javadoc unit shadedjars compile
uname	Linux 63462a2f3f99 4.15.0-143-generic #147-Ubuntu SMP Wed Apr 14 16:10:11 UTC 2021 x86_64 x86_64 x86_64 GNU/Linux
Build tool	maven
Personality	dev-support/hbase-personality.sh
git revision	master / `ce44e16`
Default Java	AdoptOpenJDK-1.8.0_282-b08
Test Results	https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-3744/2/testReport/
Max. process+thread count	638 (vs. ulimit of 30000)
modules	C: hbase-it U: hbase-it
Console output	https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-3744/2/console
versions	git=2.17.1 maven=3.6.3
Powered by	Apache Yetus 0.12.0 https://yetus.apache.org

This message was automatically generated.

Apache-HBase · 2021-10-12T02:17:54Z

🎊 +1 overall

Vote	Subsystem	Runtime	Comment
+0 🆗	reexec	1m 4s	Docker mode activated.
		_ Prechecks _
+1 💚	dupname	0m 0s	No case conflicting files found.
+1 💚	hbaseanti	0m 0s	Patch does not have any anti-patterns.
+1 💚	@author	0m 0s	The patch does not contain any @author tags.
		_ master Compile Tests _
+1 💚	mvninstall	4m 33s	master passed
+1 💚	compile	0m 41s	master passed
+1 💚	checkstyle	0m 18s	master passed
+1 💚	spotbugs	0m 36s	master passed
		_ Patch Compile Tests _
+1 💚	mvninstall	4m 14s	the patch passed
+1 💚	compile	0m 40s	the patch passed
+1 💚	javac	0m 40s	the patch passed
+1 💚	checkstyle	0m 16s	the patch passed
+1 💚	whitespace	0m 0s	The patch has no whitespace issues.
+1 💚	hadoopcheck	21m 34s	Patch does not cause any errors with Hadoop 3.1.2 3.2.1 3.3.0.
+1 💚	spotbugs	0m 44s	the patch passed
		_ Other Tests _
+1 💚	asflicense	0m 12s	The patch does not generate ASF License warnings.
		43m 45s

Subsystem	Report/Notes
Docker	ClientAPI=1.41 ServerAPI=1.41 base: https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-3744/2/artifact/yetus-general-check/output/Dockerfile
GITHUB PR	#3744
Optional Tests	dupname asflicense javac spotbugs hadoopcheck hbaseanti checkstyle compile
uname	Linux e6346db93ae2 4.15.0-153-generic #160-Ubuntu SMP Thu Jul 29 06:54:29 UTC 2021 x86_64 x86_64 x86_64 GNU/Linux
Build tool	maven
Personality	dev-support/hbase-personality.sh
git revision	master / `ce44e16`
Default Java	AdoptOpenJDK-1.8.0_282-b08
Max. process+thread count	86 (vs. ulimit of 30000)
modules	C: hbase-it U: hbase-it
Console output	https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-3744/2/console
versions	git=2.17.1 maven=3.6.3 spotbugs=4.2.2
Powered by	Apache Yetus 0.12.0 https://yetus.apache.org

This message was automatically generated.

Apache-HBase · 2021-10-15T02:02:22Z

🎊 +1 overall

Vote	Subsystem	Runtime	Comment
+0 🆗	reexec	1m 17s	Docker mode activated.
-0 ⚠️	yetus	0m 3s	Unprocessed flag(s): --brief-report-file --spotbugs-strict-precheck --whitespace-eol-ignore-list --whitespace-tabs-ignore-list --quick-hadoopcheck
		_ Prechecks _
		_ master Compile Tests _
+1 💚	mvninstall	4m 37s	master passed
+1 💚	compile	0m 27s	master passed
+1 💚	shadedjars	9m 14s	branch has no errors when building our shaded downstream artifacts.
+1 💚	javadoc	0m 18s	master passed
		_ Patch Compile Tests _
+1 💚	mvninstall	4m 16s	the patch passed
+1 💚	compile	0m 27s	the patch passed
+1 💚	javac	0m 27s	the patch passed
+1 💚	shadedjars	9m 4s	patch has no errors when building our shaded downstream artifacts.
+1 💚	javadoc	0m 15s	the patch passed
		_ Other Tests _
+1 💚	unit	0m 59s	hbase-it in the patch passed.
		32m 6s

Subsystem	Report/Notes
Docker	ClientAPI=1.41 ServerAPI=1.41 base: https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-3744/3/artifact/yetus-jdk8-hadoop3-check/output/Dockerfile
GITHUB PR	#3744
Optional Tests	javac javadoc unit shadedjars compile
uname	Linux a1cb4ea0d7fd 4.15.0-143-generic #147-Ubuntu SMP Wed Apr 14 16:10:11 UTC 2021 x86_64 x86_64 x86_64 GNU/Linux
Build tool	maven
Personality	dev-support/hbase-personality.sh
git revision	master / `ad7d698`
Default Java	AdoptOpenJDK-1.8.0_282-b08
Test Results	https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-3744/3/testReport/
Max. process+thread count	624 (vs. ulimit of 30000)
modules	C: hbase-it U: hbase-it
Console output	https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-3744/3/console
versions	git=2.17.1 maven=3.6.3
Powered by	Apache Yetus 0.12.0 https://yetus.apache.org

This message was automatically generated.

Apache-HBase · 2021-10-15T02:06:05Z

🎊 +1 overall

Vote	Subsystem	Runtime	Comment
+0 🆗	reexec	1m 30s	Docker mode activated.
-0 ⚠️	yetus	0m 3s	Unprocessed flag(s): --brief-report-file --spotbugs-strict-precheck --whitespace-eol-ignore-list --whitespace-tabs-ignore-list --quick-hadoopcheck
		_ Prechecks _
		_ master Compile Tests _
+1 💚	mvninstall	5m 52s	master passed
+1 💚	compile	0m 37s	master passed
+1 💚	shadedjars	9m 29s	branch has no errors when building our shaded downstream artifacts.
+1 💚	javadoc	0m 21s	master passed
		_ Patch Compile Tests _
+1 💚	mvninstall	5m 27s	the patch passed
+1 💚	compile	0m 35s	the patch passed
+1 💚	javac	0m 35s	the patch passed
+1 💚	shadedjars	9m 25s	patch has no errors when building our shaded downstream artifacts.
+1 💚	javadoc	0m 20s	the patch passed
		_ Other Tests _
+1 💚	unit	1m 1s	hbase-it in the patch passed.
		35m 51s

Subsystem	Report/Notes
Docker	ClientAPI=1.41 ServerAPI=1.41 base: https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-3744/3/artifact/yetus-jdk11-hadoop3-check/output/Dockerfile
GITHUB PR	#3744
Optional Tests	javac javadoc unit shadedjars compile
uname	Linux 796c6178405f 4.15.0-147-generic #151-Ubuntu SMP Fri Jun 18 19:21:19 UTC 2021 x86_64 x86_64 x86_64 GNU/Linux
Build tool	maven
Personality	dev-support/hbase-personality.sh
git revision	master / `ad7d698`
Default Java	AdoptOpenJDK-11.0.10+9
Test Results	https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-3744/3/testReport/
Max. process+thread count	636 (vs. ulimit of 30000)
modules	C: hbase-it U: hbase-it
Console output	https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-3744/3/console
versions	git=2.17.1 maven=3.6.3
Powered by	Apache Yetus 0.12.0 https://yetus.apache.org

This message was automatically generated.

Apache-HBase · 2021-10-15T02:20:47Z

🎊 +1 overall

Vote	Subsystem	Runtime	Comment
+0 🆗	reexec	1m 59s	Docker mode activated.
		_ Prechecks _
+1 💚	dupname	0m 0s	No case conflicting files found.
+1 💚	hbaseanti	0m 0s	Patch does not have any anti-patterns.
+1 💚	@author	0m 0s	The patch does not contain any @author tags.
		_ master Compile Tests _
+1 💚	mvninstall	4m 30s	master passed
+1 💚	compile	3m 29s	master passed
+1 💚	checkstyle	0m 19s	master passed
+1 💚	spotbugs	0m 42s	master passed
		_ Patch Compile Tests _
+1 💚	mvninstall	4m 16s	the patch passed
+1 💚	compile	0m 49s	the patch passed
-0 ⚠️	javac	0m 49s	hbase-it generated 4 new + 99 unchanged - 4 fixed = 103 total (was 103)
+1 💚	checkstyle	0m 17s	the patch passed
+1 💚	whitespace	0m 0s	The patch has no whitespace issues.
+1 💚	hadoopcheck	23m 34s	Patch does not cause any errors with Hadoop 3.1.2 3.2.1 3.3.0.
+1 💚	spotbugs	0m 49s	the patch passed
		_ Other Tests _
+1 💚	asflicense	0m 16s	The patch does not generate ASF License warnings.
		50m 29s

Subsystem	Report/Notes
Docker	ClientAPI=1.41 ServerAPI=1.41 base: https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-3744/3/artifact/yetus-general-check/output/Dockerfile
GITHUB PR	#3744
Optional Tests	dupname asflicense javac spotbugs hadoopcheck hbaseanti checkstyle compile
uname	Linux ef29cd49d7d7 4.15.0-112-generic #113-Ubuntu SMP Thu Jul 9 23:41:39 UTC 2020 x86_64 x86_64 x86_64 GNU/Linux
Build tool	maven
Personality	dev-support/hbase-personality.sh
git revision	master / `ad7d698`
Default Java	AdoptOpenJDK-1.8.0_282-b08
javac	https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-3744/3/artifact/yetus-general-check/output/diff-compile-javac-hbase-it.txt
Max. process+thread count	96 (vs. ulimit of 30000)
modules	C: hbase-it U: hbase-it
Console output	https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-3744/3/console
versions	git=2.17.1 maven=3.6.3 spotbugs=4.2.2
Powered by	Apache Yetus 0.12.0 https://yetus.apache.org

This message was automatically generated.

…mmonCrawl Use a hybrid logical clock for timestamping entries. Using BufferedMutator without HLC was not good because we assign client timestamps, and the store loop is fast enough that on rare occasion two temporally adjacent URLs in the set of WARCs are equivalent and the timestamp does not advance, leading later to a rare false positive CORRUPT finding. While making changes, support direct S3N paths as input paths on the command line.

apurtell · 2021-10-19T00:49:40Z

Rebase

Apache-HBase · 2021-10-19T01:22:22Z

🎊 +1 overall

Vote	Subsystem	Runtime	Comment
+0 🆗	reexec	1m 30s	Docker mode activated.
-0 ⚠️	yetus	0m 3s	Unprocessed flag(s): --brief-report-file --spotbugs-strict-precheck --whitespace-eol-ignore-list --whitespace-tabs-ignore-list --quick-hadoopcheck
		_ Prechecks _
		_ master Compile Tests _
+1 💚	mvninstall	4m 27s	master passed
+1 💚	compile	0m 27s	master passed
+1 💚	shadedjars	9m 17s	branch has no errors when building our shaded downstream artifacts.
+1 💚	javadoc	0m 16s	master passed
		_ Patch Compile Tests _
+1 💚	mvninstall	4m 17s	the patch passed
+1 💚	compile	0m 28s	the patch passed
+1 💚	javac	0m 28s	the patch passed
+1 💚	shadedjars	9m 8s	patch has no errors when building our shaded downstream artifacts.
+1 💚	javadoc	0m 16s	the patch passed
		_ Other Tests _
+1 💚	unit	0m 58s	hbase-it in the patch passed.
		32m 11s

Subsystem	Report/Notes
Docker	ClientAPI=1.41 ServerAPI=1.41 base: https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-3744/4/artifact/yetus-jdk8-hadoop3-check/output/Dockerfile
GITHUB PR	#3744
Optional Tests	javac javadoc unit shadedjars compile
uname	Linux 62a0ddd665ab 4.15.0-143-generic #147-Ubuntu SMP Wed Apr 14 16:10:11 UTC 2021 x86_64 x86_64 x86_64 GNU/Linux
Build tool	maven
Personality	dev-support/hbase-personality.sh
git revision	master / `26ab9d0`
Default Java	AdoptOpenJDK-1.8.0_282-b08
Test Results	https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-3744/4/testReport/
Max. process+thread count	627 (vs. ulimit of 30000)
modules	C: hbase-it U: hbase-it
Console output	https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-3744/4/console
versions	git=2.17.1 maven=3.6.3
Powered by	Apache Yetus 0.12.0 https://yetus.apache.org

This message was automatically generated.

Apache-HBase · 2021-10-19T01:23:02Z

🎊 +1 overall

Vote	Subsystem	Runtime	Comment
+0 🆗	reexec	1m 43s	Docker mode activated.
-0 ⚠️	yetus	0m 3s	Unprocessed flag(s): --brief-report-file --spotbugs-strict-precheck --whitespace-eol-ignore-list --whitespace-tabs-ignore-list --quick-hadoopcheck
		_ Prechecks _
		_ master Compile Tests _
+1 💚	mvninstall	4m 42s	master passed
+1 💚	compile	0m 33s	master passed
+1 💚	shadedjars	9m 2s	branch has no errors when building our shaded downstream artifacts.
+1 💚	javadoc	0m 20s	master passed
		_ Patch Compile Tests _
+1 💚	mvninstall	4m 43s	the patch passed
+1 💚	compile	0m 31s	the patch passed
+1 💚	javac	0m 31s	the patch passed
+1 💚	shadedjars	8m 34s	patch has no errors when building our shaded downstream artifacts.
+1 💚	javadoc	0m 20s	the patch passed
		_ Other Tests _
+1 💚	unit	1m 0s	hbase-it in the patch passed.
		32m 46s

Subsystem	Report/Notes
Docker	ClientAPI=1.41 ServerAPI=1.41 base: https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-3744/4/artifact/yetus-jdk11-hadoop3-check/output/Dockerfile
GITHUB PR	#3744
Optional Tests	javac javadoc unit shadedjars compile
uname	Linux 8fa34b7c7241 4.15.0-65-generic #74-Ubuntu SMP Tue Sep 17 17:06:04 UTC 2019 x86_64 x86_64 x86_64 GNU/Linux
Build tool	maven
Personality	dev-support/hbase-personality.sh
git revision	master / `26ab9d0`
Default Java	AdoptOpenJDK-11.0.10+9
Test Results	https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-3744/4/testReport/
Max. process+thread count	698 (vs. ulimit of 30000)
modules	C: hbase-it U: hbase-it
Console output	https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-3744/4/console
versions	git=2.17.1 maven=3.6.3
Powered by	Apache Yetus 0.12.0 https://yetus.apache.org

This message was automatically generated.

Apache-HBase · 2021-10-19T01:33:20Z

🎊 +1 overall

Vote	Subsystem	Runtime	Comment
+0 🆗	reexec	0m 59s	Docker mode activated.
		_ Prechecks _
+1 💚	dupname	0m 0s	No case conflicting files found.
+1 💚	hbaseanti	0m 0s	Patch does not have any anti-patterns.
+1 💚	@author	0m 0s	The patch does not contain any @author tags.
		_ master Compile Tests _
+1 💚	mvninstall	4m 33s	master passed
+1 💚	compile	0m 41s	master passed
+1 💚	checkstyle	0m 18s	master passed
+1 💚	spotbugs	0m 36s	master passed
		_ Patch Compile Tests _
+1 💚	mvninstall	4m 10s	the patch passed
+1 💚	compile	0m 39s	the patch passed
-0 ⚠️	javac	0m 39s	hbase-it generated 4 new + 99 unchanged - 4 fixed = 103 total (was 103)
+1 💚	checkstyle	0m 16s	the patch passed
+1 💚	whitespace	0m 0s	The patch has no whitespace issues.
+1 💚	hadoopcheck	21m 17s	Patch does not cause any errors with Hadoop 3.1.2 3.2.1 3.3.0.
+1 💚	spotbugs	0m 45s	the patch passed
		_ Other Tests _
+1 💚	asflicense	0m 13s	The patch does not generate ASF License warnings.
		43m 8s

Subsystem	Report/Notes
Docker	ClientAPI=1.41 ServerAPI=1.41 base: https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-3744/4/artifact/yetus-general-check/output/Dockerfile
GITHUB PR	#3744
Optional Tests	dupname asflicense javac spotbugs hadoopcheck hbaseanti checkstyle compile
uname	Linux be58d99ca1d0 4.15.0-147-generic #151-Ubuntu SMP Fri Jun 18 19:21:19 UTC 2021 x86_64 x86_64 x86_64 GNU/Linux
Build tool	maven
Personality	dev-support/hbase-personality.sh
git revision	master / `26ab9d0`
Default Java	AdoptOpenJDK-1.8.0_282-b08
javac	https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-3744/4/artifact/yetus-general-check/output/diff-compile-javac-hbase-it.txt
Max. process+thread count	86 (vs. ulimit of 30000)
modules	C: hbase-it U: hbase-it
Console output	https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-3744/4/console
versions	git=2.17.1 maven=3.6.3 spotbugs=4.2.2
Powered by	Apache Yetus 0.12.0 https://yetus.apache.org

This message was automatically generated.

…3744) Use a hybrid logical clock for timestamping entries. Using BufferedMutator without HLC was not good because we assign client timestamps, and the store loop is fast enough that on rare occasion two temporally adjacent URLs in the set of WARCs are equivalent and the timestamp does not advance, leading later to a rare false positive CORRUPT finding. While making changes, support direct S3N paths as input paths on the command line. Signed-off-by: Viraj Jasani <[email protected]>

apurtell requested review from saintstack and virajjasani October 11, 2021 22:18

apurtell force-pushed the HBASE-26349 branch from 1522d2e to bd10277 Compare October 12, 2021 01:33

apurtell changed the title ~~HBASE-26349 Revert and improve recent change to IntegrationTestLoadCommonCrawl~~ HBASE-26349 Improve recent change to IntegrationTestLoadCommonCrawl Oct 12, 2021

virajjasani approved these changes Oct 13, 2021

View reviewed changes

apurtell force-pushed the HBASE-26349 branch from bd10277 to 7040828 Compare October 15, 2021 01:26

apurtell force-pushed the HBASE-26349 branch from 7040828 to 860e807 Compare October 19, 2021 00:49

Java warning fixes

54a3c8a

apurtell merged commit 9e73ea8 into apache:master Oct 19, 2021

apurtell deleted the HBASE-26349 branch October 19, 2021 20:45

HBASE-26349 Improve recent change to IntegrationTestLoadCommonCrawl #3744

HBASE-26349 Improve recent change to IntegrationTestLoadCommonCrawl #3744

Uh oh!

Conversation

apurtell commented Oct 11, 2021 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

Apache-HBase commented Oct 11, 2021

Uh oh!

Apache-HBase commented Oct 11, 2021

Uh oh!

Apache-HBase commented Oct 11, 2021

Uh oh!

Apache-HBase commented Oct 12, 2021

Uh oh!

Apache-HBase commented Oct 12, 2021

Uh oh!

Apache-HBase commented Oct 12, 2021

Uh oh!

Apache-HBase commented Oct 15, 2021

Uh oh!

Apache-HBase commented Oct 15, 2021

Uh oh!

Apache-HBase commented Oct 15, 2021

Uh oh!

apurtell commented Oct 19, 2021

Uh oh!

Apache-HBase commented Oct 19, 2021

Uh oh!

Apache-HBase commented Oct 19, 2021

Uh oh!

Apache-HBase commented Oct 19, 2021

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

apurtell commented Oct 11, 2021 •

edited

Loading