HBASE-24297 release scripts should be able to use an existing project clone #1725

busbey · 2020-05-17T23:11:33Z

Adds the ability to specify the remote git repo for the release scripts, including a local clone.

adds a optional -r [repo] arg
if the passed repo is on the local filesystem, creates a container mount
git clone operations consolidated into a function
when cloning a local repo configure git to share objects with the local repo instead of copying
when cloning a local repo in a container configure the clone to have a remote that will work back on the host.

… checkout

busbey · 2020-05-17T23:12:13Z

@mattf-apache mind taking a look?

Apache-HBase · 2020-05-17T23:22:25Z

🎊 +1 overall

Vote	Subsystem	Runtime	Comment
+0 🆗	reexec	0m 29s	Docker mode activated.
		_ Prechecks _
+1 💚	dupname	0m 0s	No case conflicting files found.
+0 🆗	shelldocs	0m 1s	Shelldocs was not available.
+1 💚	@author	0m 0s	The patch does not contain any @author tags.
		_ master Compile Tests _
+0 🆗	mvndep	0m 21s	Maven dependency ordering for branch
		_ Patch Compile Tests _
+0 🆗	mvndep	0m 8s	Maven dependency ordering for patch
-0 ⚠️	shellcheck	0m 3s	The patch generated 1 new + 4 unchanged - 0 fixed = 5 total (was 4)
+1 💚	whitespace	0m 0s	The patch has no whitespace issues.
		_ Other Tests _
+0 🆗	asflicense	0m 0s	ASF License check generated no output?
		2m 6s

Subsystem	Report/Notes
Docker	Client=19.03.8 Server=19.03.8 base: https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1725/1/artifact/yetus-general-check/output/Dockerfile
GITHUB PR	#1725
Optional Tests	dupname asflicense shellcheck shelldocs
uname	Linux 72d599c81e71 4.15.0-58-generic #64-Ubuntu SMP Tue Aug 6 11:12:41 UTC 2019 x86_64 x86_64 x86_64 GNU/Linux
Build tool	maven
Personality	dev-support/hbase-personality.sh
git revision	master / `15627bb`
shellcheck	https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1725/1/artifact/yetus-general-check/output/diff-patch-shellcheck.txt
Max. process+thread count	48 (vs. ulimit of 12500)
modules	C: U:
Console output	https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1725/1/console
versions	git=2.17.1 maven=(cecedd343002696d0abb50b32b541b8a6ba2883f) shellcheck=0.4.6
Powered by	Apache Yetus 0.11.1 https://yetus.apache.org

This message was automatically generated.

Apache-HBase · 2020-05-17T23:23:08Z

🎊 +1 overall

Vote	Subsystem	Runtime	Comment
+0 🆗	reexec	1m 29s	Docker mode activated.
-0 ⚠️	yetus	0m 2s	Unprocessed flag(s): --brief-report-file --spotbugs-strict-precheck --whitespace-eol-ignore-list --whitespace-tabs-ignore-list --quick-hadoopcheck
		_ Prechecks _
		_ master Compile Tests _
+0 🆗	mvndep	0m 20s	Maven dependency ordering for branch
		_ Patch Compile Tests _
+0 🆗	mvndep	0m 7s	Maven dependency ordering for patch
		_ Other Tests _
		2m 48s

Subsystem	Report/Notes
Docker	Client=19.03.8 Server=19.03.8 base: https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1725/1/artifact/yetus-jdk11-hadoop3-check/output/Dockerfile
GITHUB PR	#1725
Optional Tests
uname	Linux 8ef8be1757b8 4.15.0-74-generic #84-Ubuntu SMP Thu Dec 19 08:06:28 UTC 2019 x86_64 x86_64 x86_64 GNU/Linux
Build tool	maven
Personality	dev-support/hbase-personality.sh
git revision	master / `15627bb`
Max. process+thread count	47 (vs. ulimit of 12500)
modules	C: U:
Console output	https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1725/1/console
versions	git=2.17.1 maven=(cecedd343002696d0abb50b32b541b8a6ba2883f)
Powered by	Apache Yetus 0.11.1 https://yetus.apache.org

This message was automatically generated.

Apache-HBase · 2020-05-17T23:23:11Z

🎊 +1 overall

Vote	Subsystem	Runtime	Comment
+0 🆗	reexec	1m 28s	Docker mode activated.
-0 ⚠️	yetus	0m 3s	Unprocessed flag(s): --brief-report-file --spotbugs-strict-precheck --whitespace-eol-ignore-list --whitespace-tabs-ignore-list --quick-hadoopcheck
		_ Prechecks _
		_ master Compile Tests _
+0 🆗	mvndep	0m 24s	Maven dependency ordering for branch
		_ Patch Compile Tests _
+0 🆗	mvndep	0m 7s	Maven dependency ordering for patch
		_ Other Tests _
		2m 52s

Subsystem	Report/Notes
Docker	Client=19.03.8 Server=19.03.8 base: https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1725/1/artifact/yetus-jdk8-hadoop3-check/output/Dockerfile
GITHUB PR	#1725
Optional Tests
uname	Linux af93b7c703d2 4.15.0-74-generic #84-Ubuntu SMP Thu Dec 19 08:06:28 UTC 2019 x86_64 x86_64 x86_64 GNU/Linux
Build tool	maven
Personality	dev-support/hbase-personality.sh
git revision	master / `15627bb`
Max. process+thread count	47 (vs. ulimit of 12500)
modules	C: U:
Console output	https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1725/1/console
versions	git=2.17.1 maven=(cecedd343002696d0abb50b32b541b8a6ba2883f)
Powered by	Apache Yetus 0.11.1 https://yetus.apache.org

This message was automatically generated.

saintstack

LGTM.

This is a new feature w/ nice cleanup of git reference. We want to be able to specify the git repo to speed stuff up?

Thanks Sean.

dev-support/create-release/do-release-docker.sh

ndimiduk

+1 from me. some comment about maybe simplifying url sniffing. nice addition :)

dev-support/create-release/do-release-docker.sh

dev-support/create-release/release-util.sh

busbey · 2020-05-22T19:46:25Z

I think this last push has the nits you mentioned cleaned up @ndimiduk.

Y'all let me know if you want to review things more. Otherwise I'll merge when I get some time this weekend.

busbey · 2020-05-22T19:48:19Z

dev-support/create-release/release-util.sh

+  if [ -z "${PROJECT}" ] || [ "${PROJECT}" != "${PROJECT#/}" ]; then
+    error "Project name must be defined and not start with a '/'. PROJECT='${PROJECT}'"
+  fi
+  rm -rf "${PROJECT}"


I am a bit nervous here about malformed project names like "../.."

Apache-HBase · 2020-05-22T20:12:27Z

🎊 +1 overall

Vote	Subsystem	Runtime	Comment
+0 🆗	reexec	0m 29s	Docker mode activated.
		_ Prechecks _
+1 💚	dupname	0m 0s	No case conflicting files found.
+0 🆗	shelldocs	0m 0s	Shelldocs was not available.
+1 💚	@author	0m 0s	The patch does not contain any @author tags.
		_ master Compile Tests _
+0 🆗	mvndep	0m 22s	Maven dependency ordering for branch
		_ Patch Compile Tests _
+0 🆗	mvndep	0m 8s	Maven dependency ordering for patch
-0 ⚠️	shellcheck	0m 3s	The patch generated 1 new + 4 unchanged - 0 fixed = 5 total (was 4)
+1 💚	whitespace	0m 0s	The patch has no whitespace issues.
		_ Other Tests _
+0 🆗	asflicense	0m 0s	ASF License check generated no output?
		1m 52s

Subsystem	Report/Notes
Docker	Client=19.03.9 Server=19.03.9 base: https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1725/2/artifact/yetus-general-check/output/Dockerfile
GITHUB PR	#1725
Optional Tests	dupname asflicense shellcheck shelldocs
uname	Linux 567374aacfb8 4.15.0-58-generic #64-Ubuntu SMP Tue Aug 6 11:12:41 UTC 2019 x86_64 x86_64 x86_64 GNU/Linux
Build tool	maven
Personality	dev-support/hbase-personality.sh
git revision	master / `e1d7dc8`
shellcheck	https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1725/2/artifact/yetus-general-check/output/diff-patch-shellcheck.txt
Max. process+thread count	50 (vs. ulimit of 12500)
modules	C: U:
Console output	https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1725/2/console
versions	git=2.17.1 maven=(cecedd343002696d0abb50b32b541b8a6ba2883f) shellcheck=0.4.6
Powered by	Apache Yetus 0.11.1 https://yetus.apache.org

This message was automatically generated.

Apache-HBase · 2020-05-22T20:15:03Z

🎊 +1 overall

Vote	Subsystem	Runtime	Comment
+0 🆗	reexec	3m 26s	Docker mode activated.
-0 ⚠️	yetus	0m 3s	Unprocessed flag(s): --brief-report-file --spotbugs-strict-precheck --whitespace-eol-ignore-list --whitespace-tabs-ignore-list --quick-hadoopcheck
		_ Prechecks _
		_ master Compile Tests _
+0 🆗	mvndep	0m 22s	Maven dependency ordering for branch
		_ Patch Compile Tests _
+0 🆗	mvndep	0m 7s	Maven dependency ordering for patch
		_ Other Tests _
		4m 32s

Subsystem	Report/Notes
Docker	Client=19.03.9 Server=19.03.9 base: https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1725/2/artifact/yetus-jdk8-hadoop3-check/output/Dockerfile
GITHUB PR	#1725
Optional Tests
uname	Linux d90c05b04a70 4.15.0-101-generic #102-Ubuntu SMP Mon May 11 10:07:26 UTC 2020 x86_64 x86_64 x86_64 GNU/Linux
Build tool	maven
Personality	dev-support/hbase-personality.sh
git revision	master / `e1d7dc8`
Max. process+thread count	47 (vs. ulimit of 12500)
modules	C: U:
Console output	https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1725/2/console
versions	git=2.17.1 maven=(cecedd343002696d0abb50b32b541b8a6ba2883f)
Powered by	Apache Yetus 0.11.1 https://yetus.apache.org

This message was automatically generated.

Apache-HBase · 2020-05-22T20:15:14Z

🎊 +1 overall

Vote	Subsystem	Runtime	Comment
+0 🆗	reexec	3m 35s	Docker mode activated.
-0 ⚠️	yetus	0m 4s	Unprocessed flag(s): --brief-report-file --spotbugs-strict-precheck --whitespace-eol-ignore-list --whitespace-tabs-ignore-list --quick-hadoopcheck
		_ Prechecks _
		_ master Compile Tests _
+0 🆗	mvndep	0m 20s	Maven dependency ordering for branch
-0 ⚠️	patch	0m 21s	Used diff version of patch file. Binary files and potentially other changes not applied. Please rebase and squash commits if necessary.
		_ Patch Compile Tests _
+0 🆗	mvndep	0m 7s	Maven dependency ordering for patch
		_ Other Tests _
		4m 41s

Subsystem	Report/Notes
Docker	Client=19.03.9 Server=19.03.9 base: https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1725/2/artifact/yetus-jdk11-hadoop3-check/output/Dockerfile
Optional Tests
uname	Linux 13f04b57a9ff 4.15.0-101-generic #102-Ubuntu SMP Mon May 11 10:07:26 UTC 2020 x86_64 x86_64 x86_64 GNU/Linux
Build tool	maven
Personality	dev-support/hbase-personality.sh
git revision	master / `e1d7dc8`
Max. process+thread count	48 (vs. ulimit of 12500)
modules	C: U:
Console output	https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1725/2/console
versions	git=2.17.1 maven=(cecedd343002696d0abb50b32b541b8a6ba2883f)
Powered by	Apache Yetus 0.11.1 https://yetus.apache.org

This message was automatically generated.

HBASE-24297 release scripts should be able to use an existing project…

c67c74f

… checkout

busbey requested a review from saintstack May 19, 2020 17:16

saintstack approved these changes May 20, 2020

View reviewed changes

dev-support/create-release/do-release-docker.sh Show resolved Hide resolved

ndimiduk approved these changes May 20, 2020

View reviewed changes

ndimiduk approved these changes May 22, 2020

View reviewed changes

dev-support/create-release/release-util.sh Outdated Show resolved Hide resolved

dev-support/create-release/release-util.sh Show resolved Hide resolved

dev-support/create-release/release-util.sh Show resolved Hide resolved

HBASE-24297 addendum with review nits

20fed98

busbey commented May 22, 2020

View reviewed changes

asfgit closed this in a9fefd7 May 23, 2020

busbey deleted the HBASE-24297 branch May 23, 2020 03:37

mattf-apache mentioned this pull request Jun 16, 2020

HBASE-23339 Release scripts should use forwarded gpg-agent #1620

Closed

HBASE-24297 release scripts should be able to use an existing project clone #1725

HBASE-24297 release scripts should be able to use an existing project clone #1725

Uh oh!

Conversation

busbey commented May 17, 2020

Uh oh!

busbey commented May 17, 2020

Uh oh!

Apache-HBase commented May 17, 2020

Uh oh!

Apache-HBase commented May 17, 2020

Uh oh!

Apache-HBase commented May 17, 2020

Uh oh!

saintstack left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

ndimiduk left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

busbey commented May 22, 2020

Uh oh!

busbey May 22, 2020

Choose a reason for hiding this comment

Uh oh!

Apache-HBase commented May 22, 2020

Uh oh!

Apache-HBase commented May 22, 2020

Uh oh!

Apache-HBase commented May 22, 2020

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

4 participants