Skip to content

Conversation

@virajjasani
Copy link
Contributor

@virajjasani virajjasani commented Sep 11, 2019

@virajjasani
Copy link
Contributor Author

virajjasani commented Sep 11, 2019

Please review @saintstack
I tried to cover shell triggered and non-shell triggered alter schema update. If we have this metrics, probably good to use it for alter_schema command? Can also be used for other client get status

@Apache-HBase
Copy link

💔 -1 overall

Vote Subsystem Runtime Comment
💙 reexec 2m 13s Docker mode activated.
_ Prechecks _
💚 dupname 0m 0s No case conflicting files found.
💙 prototool 0m 0s prototool was not available.
💚 hbaseanti 0m 0s Patch does not have any anti-patterns.
💚 @author 0m 0s The patch does not contain any @author tags.
💛 test4tests 0m 0s The patch doesn't appear to include any new or modified tests. Please justify why no new tests are needed for this patch. Also please list what manual steps were performed to verify this patch.
_ master Compile Tests _
💙 mvndep 0m 41s Maven dependency ordering for branch
💚 mvninstall 7m 7s master passed
💚 compile 3m 12s master passed
💚 checkstyle 2m 56s master passed
💚 shadedjars 5m 57s branch has no errors when building our shaded downstream artifacts.
💚 javadoc 1m 50s master passed
💙 spotbugs 5m 33s Used deprecated FindBugs config; considering switching to SpotBugs.
💚 findbugs 10m 4s master passed
_ Patch Compile Tests _
💙 mvndep 0m 15s Maven dependency ordering for patch
💚 mvninstall 6m 28s the patch passed
💚 compile 3m 14s the patch passed
💚 cc 3m 14s the patch passed
💚 javac 3m 15s the patch passed
💔 checkstyle 0m 41s hbase-client: The patch generated 9 new + 180 unchanged - 0 fixed = 189 total (was 180)
💔 rubocop 0m 11s The patch generated 3 new + 270 unchanged - 1 fixed = 273 total (was 271)
💚 whitespace 0m 1s The patch has no whitespace issues.
💚 shadedjars 6m 5s patch has no errors when building our shaded downstream artifacts.
💚 hadoopcheck 22m 10s Patch does not cause any errors with Hadoop 2.8.5 2.9.2 or 3.1.2.
💚 hbaseprotoc 2m 59s the patch passed
💚 javadoc 1m 49s the patch passed
💚 findbugs 10m 31s the patch passed
_ Other Tests _
💚 unit 0m 53s hbase-protocol-shaded in the patch passed.
💚 unit 2m 11s hbase-client in the patch passed.
💔 unit 290m 46s hbase-server in the patch failed.
💚 unit 11m 25s hbase-shell in the patch passed.
💚 asflicense 1m 58s The patch does not generate ASF License warnings.
406m 47s
Reason Tests
Failed junit tests hadoop.hbase.master.procedure.TestSCPWithReplicasWithoutZKCoordinated
hadoop.hbase.client.TestAsyncRegionAdminApi2
Subsystem Report/Notes
Docker Client=19.03.1 Server=19.03.1 base: https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-611/1/artifact/out/Dockerfile
GITHUB PR #611
Optional Tests dupname asflicense javac javadoc unit spotbugs findbugs shadedjars hadoopcheck hbaseanti checkstyle compile cc hbaseprotoc prototool rubocop
uname Linux d62b8f9b8295 4.15.0-60-generic #67-Ubuntu SMP Thu Aug 22 16:55:30 UTC 2019 x86_64 GNU/Linux
Build tool maven
Personality /home/jenkins/jenkins-slave/workspace/HBase-PreCommit-GitHub-PR_PR-611/out/precommit/personality/provided.sh
git revision master / a581498
Default Java 1.8.0_181
checkstyle https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-611/1/artifact/out/diff-checkstyle-hbase-client.txt
rubocop https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-611/1/artifact/out/diff-patch-rubocop.txt
unit https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-611/1/artifact/out/patch-unit-hbase-server.txt
Test Results https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-611/1/testReport/
Max. process+thread count 4712 (vs. ulimit of 10000)
modules C: hbase-protocol-shaded hbase-client hbase-server hbase-shell U: .
Console output https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-611/1/console
versions git=2.11.0 maven=2018-06-17T18:33:14Z) findbugs=3.1.11 rubocop=0.74.0
Powered by Apache Yetus 0.11.0 https://yetus.apache.org

This message was automatically generated.

@virajjasani
Copy link
Contributor Author

Please review @anoopsjohn @Reidddddd

@Reidddddd
Copy link
Contributor

Confused, is the JIRA id correct?
alter_status command is broken or Table to RegionStatesCount metrics

@virajjasani
Copy link
Contributor Author

virajjasani commented Sep 23, 2019

Yes, Jira Id is correct, however, the solution is generic i.e. use new metrics, which can be used for broken alter_status command.
Will update the PR title

@virajjasani virajjasani changed the title HBASE-22903 : Table to RegionStatesCount metrics HBASE-22903 : Table to RegionStatesCount metrics - Use for broken alter_status command Sep 23, 2019
@Reidddddd
Copy link
Contributor

Good to know, i'm reviewing another big PR.
Will be back.

@Apache-HBase
Copy link

💔 -1 overall

Vote Subsystem Runtime Comment
💙 reexec 0m 36s Docker mode activated.
_ Prechecks _
💚 dupname 0m 0s No case conflicting files found.
💙 prototool 0m 0s prototool was not available.
💚 hbaseanti 0m 0s Patch does not have any anti-patterns.
💚 @author 0m 0s The patch does not contain any @author tags.
💛 test4tests 0m 0s The patch doesn't appear to include any new or modified tests. Please justify why no new tests are needed for this patch. Also please list what manual steps were performed to verify this patch.
_ master Compile Tests _
💙 mvndep 0m 34s Maven dependency ordering for branch
💚 mvninstall 5m 28s master passed
💚 compile 2m 28s master passed
💚 checkstyle 2m 25s master passed
💚 shadedjars 4m 47s branch has no errors when building our shaded downstream artifacts.
💚 javadoc 1m 29s master passed
💙 spotbugs 4m 20s Used deprecated FindBugs config; considering switching to SpotBugs.
💚 findbugs 8m 11s master passed
_ Patch Compile Tests _
💙 mvndep 0m 15s Maven dependency ordering for patch
💚 mvninstall 5m 7s the patch passed
💚 compile 2m 30s the patch passed
💚 cc 2m 30s the patch passed
💚 javac 2m 30s the patch passed
💔 checkstyle 0m 41s hbase-client: The patch generated 1 new + 180 unchanged - 0 fixed = 181 total (was 180)
💔 rubocop 0m 10s The patch generated 3 new + 256 unchanged - 1 fixed = 259 total (was 257)
💚 whitespace 0m 0s The patch has no whitespace issues.
💚 shadedjars 6m 1s patch has no errors when building our shaded downstream artifacts.
💚 hadoopcheck 20m 17s Patch does not cause any errors with Hadoop 2.8.5 2.9.2 or 3.1.2.
💚 hbaseprotoc 2m 53s the patch passed
💚 javadoc 1m 40s the patch passed
💚 findbugs 10m 7s the patch passed
_ Other Tests _
💚 unit 0m 51s hbase-protocol-shaded in the patch passed.
💚 unit 2m 10s hbase-client in the patch passed.
💚 unit 162m 28s hbase-server in the patch passed.
💚 unit 7m 14s hbase-shell in the patch passed.
💚 asflicense 2m 8s The patch does not generate ASF License warnings.
260m 54s
Subsystem Report/Notes
Docker Client=19.03.1 Server=19.03.1 base: https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-611/2/artifact/out/Dockerfile
GITHUB PR #611
Optional Tests dupname asflicense javac javadoc unit spotbugs findbugs shadedjars hadoopcheck hbaseanti checkstyle compile cc hbaseprotoc prototool rubocop
uname Linux 78f96f1a122f 4.15.0-58-generic #64-Ubuntu SMP Tue Aug 6 11:12:41 UTC 2019 x86_64 GNU/Linux
Build tool maven
Personality /home/jenkins/jenkins-slave/workspace/HBase-PreCommit-GitHub-PR_PR-611/out/precommit/personality/provided.sh
git revision master / 3250a80
Default Java 1.8.0_181
checkstyle https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-611/2/artifact/out/diff-checkstyle-hbase-client.txt
rubocop https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-611/2/artifact/out/diff-patch-rubocop.txt
Test Results https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-611/2/testReport/
Max. process+thread count 4806 (vs. ulimit of 10000)
modules C: hbase-protocol-shaded hbase-client hbase-server hbase-shell U: .
Console output https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-611/2/console
versions git=2.11.0 maven=2018-06-17T18:33:14Z) findbugs=3.1.11 rubocop=0.74.0
Powered by Apache Yetus 0.11.0 https://yetus.apache.org

This message was automatically generated.

@virajjasani
Copy link
Contributor Author

@Apache9 could you please review this PR?

.addAllServersName(metrics.getServersName().stream().map(ProtobufUtil::toServerName)
.collect(Collectors.toList()));
.collect(Collectors.toList()))
.addAllTableRegionStatesCount(metrics.getTableRegionStatesCount()
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I guess we can do a code format here ?

status = @admin.getAlterStatus(org.apache.hadoop.hbase.TableName.valueOf(table_name))
if status.getSecond != 0
puts "#{status.getSecond - status.getFirst}/#{status.getSecond} regions updated."
cluster_metrics = @admin.getClusterMetrics
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think we need a jruby UT to address the alter_status ? Please also provide a Java UT to address the getTableRegionStatesCount interface, Thanks.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Done

table_region_status = cluster_metrics.getTableRegionStatesCount
.get(org.apache.hadoop.hbase.TableName.valueOf(table_name))
if table_region_status.getTotalRegions != 0
updated_regions = table_region_status.getTotalRegions -
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The closed / opening region won't be RIT state ? if so , then we should minus the closed region & opening region etc.

Copy link
Contributor Author

@virajjasani virajjasani Sep 29, 2019

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I believe we can minus closed region(Done), but opening should be part of RIT already

@openinx
Copy link
Member

openinx commented Sep 29, 2019

BTW, please also take a look the HBaseQA, seems checkstyles & ruby has some problem now, please update your PR if it's related. @virajjasani

@Apache-HBase
Copy link

💔 -1 overall

Vote Subsystem Runtime Comment
💙 reexec 0m 33s Docker mode activated.
_ Prechecks _
💚 dupname 0m 0s No case conflicting files found.
💙 prototool 0m 0s prototool was not available.
💚 hbaseanti 0m 0s Patch does not have any anti-patterns.
💚 @author 0m 0s The patch does not contain any @author tags.
💚 test4tests 0m 0s The patch appears to include 2 new or modified test files.
_ master Compile Tests _
💙 mvndep 0m 33s Maven dependency ordering for branch
💚 mvninstall 5m 45s master passed
💚 compile 2m 43s master passed
💚 checkstyle 2m 37s master passed
💚 shadedjars 5m 2s branch has no errors when building our shaded downstream artifacts.
💚 javadoc 1m 26s master passed
💙 spotbugs 4m 53s Used deprecated FindBugs config; considering switching to SpotBugs.
💚 findbugs 8m 45s master passed
_ Patch Compile Tests _
💙 mvndep 0m 15s Maven dependency ordering for patch
💚 mvninstall 6m 19s the patch passed
💚 compile 2m 57s the patch passed
💚 cc 2m 57s the patch passed
💚 javac 2m 57s the patch passed
💚 checkstyle 2m 45s the patch passed
💔 rubocop 0m 16s The patch generated 4 new + 418 unchanged - 1 fixed = 422 total (was 419)
💚 whitespace 0m 0s The patch has no whitespace issues.
💚 shadedjars 5m 38s patch has no errors when building our shaded downstream artifacts.
💚 hadoopcheck 19m 25s Patch does not cause any errors with Hadoop 2.8.5 2.9.2 or 3.1.2.
💚 hbaseprotoc 2m 46s the patch passed
💚 javadoc 1m 33s the patch passed
💚 findbugs 9m 51s the patch passed
_ Other Tests _
💚 unit 0m 47s hbase-protocol-shaded in the patch passed.
💚 unit 2m 5s hbase-client in the patch passed.
💚 unit 174m 8s hbase-server in the patch passed.
💚 unit 7m 24s hbase-shell in the patch passed.
💚 asflicense 1m 35s The patch does not generate ASF License warnings.
273m 21s
Subsystem Report/Notes
Docker Client=19.03.1 Server=19.03.1 base: https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-611/3/artifact/out/Dockerfile
GITHUB PR #611
Optional Tests dupname asflicense javac javadoc unit spotbugs findbugs shadedjars hadoopcheck hbaseanti checkstyle compile cc hbaseprotoc prototool rubocop
uname Linux 7dbacadc9a87 4.15.0-60-generic #67-Ubuntu SMP Thu Aug 22 16:55:30 UTC 2019 x86_64 GNU/Linux
Build tool maven
Personality /home/jenkins/jenkins-slave/workspace/HBase-PreCommit-GitHub-PR_PR-611/out/precommit/personality/provided.sh
git revision master / 1170f28
Default Java 1.8.0_181
rubocop https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-611/3/artifact/out/diff-patch-rubocop.txt
Test Results https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-611/3/testReport/
Max. process+thread count 4289 (vs. ulimit of 10000)
modules C: hbase-protocol-shaded hbase-client hbase-server hbase-shell U: .
Console output https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-611/3/console
versions git=2.11.0 maven=2018-06-17T18:33:14Z) findbugs=3.1.11 rubocop=0.74.0
Powered by Apache Yetus 0.11.0 https://yetus.apache.org

This message was automatically generated.

@virajjasani
Copy link
Contributor Author

Thanks for the review, updated the PR based on comments, please take a look @openinx
Some rubocop warning would be difficult to resolve for any changes in ruby file like Metrics/AbcSize etc, which is usually ignored IMHO.
Thanks

@Apache-HBase
Copy link

💔 -1 overall

Vote Subsystem Runtime Comment
💙 reexec 1m 31s Docker mode activated.
_ Prechecks _
💚 dupname 0m 0s No case conflicting files found.
💙 prototool 0m 0s prototool was not available.
💚 hbaseanti 0m 0s Patch does not have any anti-patterns.
💚 @author 0m 0s The patch does not contain any @author tags.
💚 test4tests 0m 0s The patch appears to include 2 new or modified test files.
_ master Compile Tests _
💙 mvndep 0m 35s Maven dependency ordering for branch
💚 mvninstall 5m 44s master passed
💚 compile 2m 27s master passed
💚 checkstyle 2m 29s master passed
💚 shadedjars 4m 54s branch has no errors when building our shaded downstream artifacts.
💚 javadoc 1m 21s master passed
💙 spotbugs 4m 24s Used deprecated FindBugs config; considering switching to SpotBugs.
💚 findbugs 8m 11s master passed
_ Patch Compile Tests _
💙 mvndep 0m 14s Maven dependency ordering for patch
💚 mvninstall 5m 28s the patch passed
💚 compile 2m 23s the patch passed
💚 cc 2m 23s the patch passed
💚 javac 2m 23s the patch passed
💚 checkstyle 2m 25s the patch passed
💔 rubocop 0m 12s The patch generated 1 new + 418 unchanged - 1 fixed = 419 total (was 419)
💚 whitespace 0m 0s The patch has no whitespace issues.
💚 shadedjars 5m 1s patch has no errors when building our shaded downstream artifacts.
💚 hadoopcheck 17m 2s Patch does not cause any errors with Hadoop 2.8.5 2.9.2 or 3.1.2.
💚 hbaseprotoc 2m 19s the patch passed
💚 javadoc 1m 20s the patch passed
💚 findbugs 8m 41s the patch passed
_ Other Tests _
💚 unit 0m 41s hbase-protocol-shaded in the patch passed.
💚 unit 1m 48s hbase-client in the patch passed.
💚 unit 156m 33s hbase-server in the patch passed.
💚 unit 7m 8s hbase-shell in the patch passed.
💚 asflicense 1m 36s The patch does not generate ASF License warnings.
247m 29s
Subsystem Report/Notes
Docker Client=19.03.1 Server=19.03.1 base: https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-611/4/artifact/out/Dockerfile
GITHUB PR #611
Optional Tests dupname asflicense javac javadoc unit spotbugs findbugs shadedjars hadoopcheck hbaseanti checkstyle compile cc hbaseprotoc prototool rubocop
uname Linux dc7f31295b35 4.15.0-60-generic #67-Ubuntu SMP Thu Aug 22 16:55:30 UTC 2019 x86_64 GNU/Linux
Build tool maven
Personality /home/jenkins/jenkins-slave/workspace/HBase-PreCommit-GitHub-PR_PR-611/out/precommit/personality/provided.sh
git revision master / ce0fbc2
Default Java 1.8.0_181
rubocop https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-611/4/artifact/out/diff-patch-rubocop.txt
Test Results https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-611/4/testReport/
Max. process+thread count 4294 (vs. ulimit of 10000)
modules C: hbase-protocol-shaded hbase-client hbase-server hbase-shell U: .
Console output https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-611/4/console
versions git=2.11.0 maven=2018-06-17T18:33:14Z) findbugs=3.1.11 rubocop=0.74.0
Powered by Apache Yetus 0.11.0 https://yetus.apache.org

This message was automatically generated.

Copy link
Member

@openinx openinx left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks good to me overall, few nits left.

return closedRegions;
}

private void setClosedRegions(int closedRegions) {
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

nit: let's try to align the get & set placement ? all getXX first, then setXX ? or getXX & setXX one by one ?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Done @openinx
Thanks

Copy link
Member

@openinx openinx left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks good to me

@openinx openinx merged commit 944108c into apache:master Sep 30, 2019
@Apache-HBase
Copy link

💔 -1 overall

Vote Subsystem Runtime Comment
💙 reexec 0m 33s Docker mode activated.
_ Prechecks _
💚 dupname 0m 0s No case conflicting files found.
💙 prototool 0m 0s prototool was not available.
💚 hbaseanti 0m 0s Patch does not have any anti-patterns.
💚 @author 0m 0s The patch does not contain any @author tags.
💚 test4tests 0m 0s The patch appears to include 2 new or modified test files.
_ master Compile Tests _
💙 mvndep 0m 35s Maven dependency ordering for branch
💚 mvninstall 5m 54s master passed
💚 compile 2m 36s master passed
💚 checkstyle 2m 32s master passed
💚 shadedjars 5m 6s branch has no errors when building our shaded downstream artifacts.
💚 javadoc 1m 26s master passed
💙 spotbugs 4m 44s Used deprecated FindBugs config; considering switching to SpotBugs.
💚 findbugs 8m 43s master passed
_ Patch Compile Tests _
💙 mvndep 0m 15s Maven dependency ordering for patch
💚 mvninstall 5m 54s the patch passed
💚 compile 3m 6s the patch passed
💚 cc 3m 5s the patch passed
💚 javac 3m 5s the patch passed
💚 checkstyle 2m 54s the patch passed
💔 rubocop 0m 15s The patch generated 1 new + 418 unchanged - 1 fixed = 419 total (was 419)
💚 whitespace 0m 0s The patch has no whitespace issues.
💚 shadedjars 6m 25s patch has no errors when building our shaded downstream artifacts.
💚 hadoopcheck 20m 33s Patch does not cause any errors with Hadoop 2.8.5 2.9.2 or 3.1.2.
💚 hbaseprotoc 2m 53s the patch passed
💚 javadoc 1m 42s the patch passed
💚 findbugs 10m 27s the patch passed
_ Other Tests _
💚 unit 0m 48s hbase-protocol-shaded in the patch passed.
💚 unit 2m 7s hbase-client in the patch passed.
💔 unit 173m 31s hbase-server in the patch failed.
💚 unit 7m 16s hbase-shell in the patch passed.
💚 asflicense 1m 35s The patch does not generate ASF License warnings.
275m 33s
Reason Tests
Failed junit tests hadoop.hbase.master.procedure.TestMasterFailoverWithProcedures
Subsystem Report/Notes
Docker Client=19.03.1 Server=19.03.1 base: https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-611/5/artifact/out/Dockerfile
GITHUB PR #611
Optional Tests dupname asflicense javac javadoc unit spotbugs findbugs shadedjars hadoopcheck hbaseanti checkstyle compile cc hbaseprotoc prototool rubocop
uname Linux 958f7e29d7fd 4.15.0-60-generic #67-Ubuntu SMP Thu Aug 22 16:55:30 UTC 2019 x86_64 GNU/Linux
Build tool maven
Personality /home/jenkins/jenkins-slave/workspace/HBase-PreCommit-GitHub-PR_PR-611/out/precommit/personality/provided.sh
git revision master / ca0d9f3
Default Java 1.8.0_181
rubocop https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-611/5/artifact/out/diff-patch-rubocop.txt
unit https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-611/5/artifact/out/patch-unit-hbase-server.txt
Test Results https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-611/5/testReport/
Max. process+thread count 4225 (vs. ulimit of 10000)
modules C: hbase-protocol-shaded hbase-client hbase-server hbase-shell U: .
Console output https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-611/5/console
versions git=2.11.0 maven=2018-06-17T18:33:14Z) findbugs=3.1.11 rubocop=0.74.0
Powered by Apache Yetus 0.11.0 https://yetus.apache.org

This message was automatically generated.

asfgit pushed a commit that referenced this pull request Oct 8, 2019
asfgit pushed a commit that referenced this pull request Oct 8, 2019
asfgit pushed a commit that referenced this pull request Oct 8, 2019
infraio pushed a commit to infraio/hbase that referenced this pull request Aug 17, 2020
symat pushed a commit to symat/hbase that referenced this pull request Feb 17, 2021
…er_status command (apache#611)

Signed-off-by: huzheng <[email protected]>
(cherry picked from commit 4a37572)

Change-Id: I7aa7bb1646bf1bfc2b8d9bd22c62877a54b0b956
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants