-
Notifications
You must be signed in to change notification settings - Fork 3.4k
HBASE-26311 Balancer gets stuck in cohosted replica distribution #3724
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Conversation
|
💔 -1 overall
This message was automatically generated. |
|
💔 -1 overall
This message was automatically generated. |
|
🎊 +1 overall
This message was automatically generated. |
|
🎊 +1 overall
This message was automatically generated. |
|
💔 -1 overall
This message was automatically generated. |
|
💔 -1 overall
This message was automatically generated. |
|
🎊 +1 overall
This message was automatically generated. |
|
💔 -1 overall
This message was automatically generated. |
|
💔 -1 overall
This message was automatically generated. |
|
🎊 +1 overall
This message was automatically generated. |
|
💔 -1 overall
This message was automatically generated. |
|
💔 -1 overall
This message was automatically generated. |
|
🎊 +1 overall
This message was automatically generated. |
|
💔 -1 overall
This message was automatically generated. |
|
💔 -1 overall
This message was automatically generated. |
|
💔 -1 overall
This message was automatically generated. |
|
🎊 +1 overall
This message was automatically generated. |
|
💔 -1 overall
This message was automatically generated. |
|
Failure only on Jenkins runs. Trying a few things. |
|
💔 -1 overall
This message was automatically generated. |
|
💔 -1 overall
This message was automatically generated. |
|
🎊 +1 overall
This message was automatically generated. |
|
There seems a real regression at TestStochasticLoadBalancerBalanceCluster. looking. |
|
💔 -1 overall
This message was automatically generated. |
|
💔 -1 overall
This message was automatically generated. |
|
💔 -1 overall
This message was automatically generated. |
|
💔 -1 overall
This message was automatically generated. |
|
💔 -1 overall
This message was automatically generated. |
|
🎊 +1 overall
This message was automatically generated. |
|
💔 -1 overall
This message was automatically generated. |
|
🎊 +1 overall
This message was automatically generated. |
|
💔 -1 overall
This message was automatically generated. |
|
🎊 +1 overall
This message was automatically generated. |
|
🎊 +1 overall
This message was automatically generated. |
|
🎊 +1 overall
This message was automatically generated. |
| int[] regionIndexToServerIndex; // regionIndex -> serverIndex | ||
| int[] initialRegionIndexToServerIndex; // regionIndex -> serverIndex (initial cluster state) | ||
| int[] regionIndexToTableIndex; // regionIndex -> tableIndex | ||
| int[][] numRegionsPerServerPerTable; // serverIndex -> tableIndex -> # regions |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Thanks to the refactoring of cached double cost array, I am doing some cleaning up and refactoring here for table skew cost function.
| double diff = (mean - n) * (mean - n); | ||
| totalCost += diff; | ||
| } | ||
| // No need to compute standard deviation with division by cluster size when scaling. | ||
| totalCost = Math.sqrt(totalCost); |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Using the standard deviation instead of linear deviation to assign higher penalty on outliers and therefore unstuck balancer when even region count distribution cannot be achieved with other constraint such as rack/host constraints
hbase-balancer/src/main/java/org/apache/hadoop/hbase/master/balancer/DoubleArrayCost.java
Show resolved
Hide resolved
hbase-balancer/src/main/java/org/apache/hadoop/hbase/master/balancer/DoubleArrayCost.java
Show resolved
Hide resolved
hbase-balancer/src/main/java/org/apache/hadoop/hbase/master/balancer/DoubleArrayCost.java
Show resolved
Hide resolved
hbase-balancer/src/main/java/org/apache/hadoop/hbase/master/balancer/DoubleArrayCost.java
Show resolved
Hide resolved
| numLow * (mean - Math.floor(mean)) * (mean - Math.floor(mean)); | ||
| } | ||
| return min; | ||
| return Math.sqrt(min); |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Same question as above.
|
Please rebase here @clarax ? Thanks. |
|
@clarax, there is a conflict, please rebase. |
make mock rack manager deterministic
This reverts commit 7b90642e85b8f3d193acdb0c5f686d4065501a13.
|
🎊 +1 overall
This message was automatically generated. |
|
🎊 +1 overall
This message was automatically generated. |
|
🎊 +1 overall
This message was automatically generated. |
|
Test failures is unrelated known issue. |
No description provided.