Skip to content

Conversation

@ndimiduk
Copy link
Member

When quotas are enabled, splitting the quota table fails due to the namespace region quotas system being partially initialized.

2024-11-28T14:11:38,672 ERROR [PEWorker-2 {}]
procedure2.ProcedureExecutor(1844): CODE-BUG: Uncaught runtime
exception: pid=7, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION,
hasLock=true; SplitTableRegionProcedure table=hbase:quota,
parent=3c1dbd551845ed2e8033b76570c6d8f6,
daughterA=a8f5e70728d93859d7c3d95aa5a0f63b,
daughterB=c71f49ad7e689a57742a9fe65b918d08
java.lang.NullPointerException: Cannot invoke
"java.util.concurrent.atomic.AtomicInteger.addAndGet(int)" because the
return value of "java.util.Map.get(Object)" is null
at org.apache.hadoop.hbase.namespace.NamespaceTableAndRegionInfo.incRegionCountForTable(NamespaceTableAndRegionInfo.java:101) ~[classes/:?]
at org.apache.hadoop.hbase.namespace.NamespaceStateManager.checkAndUpdateNamespaceRegionCount(NamespaceStateManager.java:95) ~[classes/:?]
at org.apache.hadoop.hbase.namespace.NamespaceAuditor.checkQuotaToSplitRegion(NamespaceAuditor.java:116) ~[classes/:?]
at org.apache.hadoop.hbase.quotas.MasterQuotaManager.onRegionSplit(MasterQuotaManager.java:544) ~[classes/:?]
at org.apache.hadoop.hbase.master.assignment.SplitTableRegionProcedure.preSplitRegion(SplitTableRegionProcedure.java:607) ~[classes/:?]
at org.apache.hadoop.hbase.master.assignment.SplitTableRegionProcedure.executeFromState(SplitTableRegionProcedure.java:298) ~[classes/:?]
at org.apache.hadoop.hbase.master.assignment.SplitTableRegionProcedure.executeFromState(SplitTableRegionProcedure.java:98) ~[classes/:?]
at org.apache.hadoop.hbase.procedure2.StateMachineProcedure.execute(StateMachineProcedure.java:188) ~[classes/:?]
at org.apache.hadoop.hbase.procedure2.Procedure.doExecute(Procedure.java:941) ~[classes/:?]
at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.execProcedure(ProcedureExecutor.java:1825) ~[classes/:?]
at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeProcedure(ProcedureExecutor.java:1503)~[classes/:?]
at org.apache.hadoop.hbase.procedure2.ProcedureExecutor$WorkerThread.runProcedure(ProcedureExecutor.java:2154) ~[classes/:?]
at org.apache.hadoop.hbase.trace.TraceUtil.trace(TraceUtil.java:216) ~[classes/:?]
at org.apache.hadoop.hbase.procedure2.ProcedureExecutor$WorkerThread.run(ProcedureExecutor.java:2181) ~[classes/:?]

See discussion on the dev list, https://lists.apache.org/thread/gdnjb1ys8m93lowykdrh4m7o55mgw3vo

@ndimiduk
Copy link
Member Author

FYI @droudnitsky

*/
@Category({ MiscTests.class, LargeTests.class })
@RunWith(Parameterized.class)
public class TestSplitMergeQuotaTable {
Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I reduced scope of this test class to just cover the quota table. Previously It was for system tables and on branch-2 I was adding coverage for the namespace table. Turns out that we hard-code cannot split the namespace table, so that was futile.


@Parameterized.Parameters(name = "{1}")
public static Object[][] params() {
return new Object[][] { { Map.of(QuotaUtil.QUOTA_CONF_KEY, "false") },
Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Testing both with and without the quotas feature enabled, so that we have confidence that someone who has temporarily disabled the feature is still able to operate on the table.

@Apache-HBase

This comment has been minimized.

@Apache-HBase

This comment has been minimized.

@ndimiduk
Copy link
Member Author

ndimiduk commented Dec 3, 2024

[INFO] Tests run: 2, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 28.001 s - in org.apache.hadoop.hbase.TestSplitMergeQuotaTable

@ndimiduk ndimiduk changed the title HBASE-29005 Cannot split split system tables when quota enforcement is enabled HBASE-29005 Cannot split hbase:quota table when quota enforcement is enabled Dec 4, 2024
@Apache-HBase

This comment has been minimized.

@Apache-HBase

This comment has been minimized.

@Apache-HBase
Copy link

🎊 +1 overall

Vote Subsystem Runtime Logfile Comment
+0 🆗 reexec 0m 27s Docker mode activated.
_ Prechecks _
+1 💚 dupname 0m 0s No case conflicting files found.
+0 🆗 codespell 0m 0s codespell was not available.
+0 🆗 detsecrets 0m 0s detect-secrets was not available.
+1 💚 @author 0m 0s The patch does not contain any @author tags.
+1 💚 hbaseanti 0m 0s Patch does not have any anti-patterns.
_ master Compile Tests _
+1 💚 mvninstall 3m 29s master passed
+1 💚 compile 3m 13s master passed
+1 💚 checkstyle 0m 41s master passed
+1 💚 spotbugs 1m 46s master passed
+1 💚 spotless 0m 52s branch has no errors when running spotless:check.
-0 ⚠️ patch 0m 59s Used diff version of patch file. Binary files and potentially other changes not applied. Please rebase and squash commits if necessary.
_ Patch Compile Tests _
+1 💚 mvninstall 3m 20s the patch passed
+1 💚 compile 3m 20s the patch passed
+1 💚 javac 3m 20s the patch passed
+1 💚 blanks 0m 0s The patch has no blanks issues.
+1 💚 checkstyle 0m 38s the patch passed
+1 💚 spotbugs 1m 50s the patch passed
+1 💚 hadoopcheck 12m 7s Patch does not cause any errors with Hadoop 3.3.6 3.4.0.
+1 💚 spotless 0m 47s patch has no errors when running spotless:check.
_ Other Tests _
+1 💚 asflicense 0m 11s The patch does not generate ASF License warnings.
39m 52s
Subsystem Report/Notes
Docker ClientAPI=1.43 ServerAPI=1.43 base: https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-6501/4/artifact/yetus-general-check/output/Dockerfile
GITHUB PR #6501
Optional Tests dupname asflicense javac spotbugs checkstyle codespell detsecrets compile hadoopcheck hbaseanti spotless
uname Linux 61c0130e47ba 5.4.0-1103-aws #111~18.04.1-Ubuntu SMP Tue May 23 20:04:10 UTC 2023 x86_64 x86_64 x86_64 GNU/Linux
Build tool maven
Personality dev-support/hbase-personality.sh
git revision master / 4a6a514
Default Java Eclipse Adoptium-17.0.11+9
Max. process+thread count 84 (vs. ulimit of 30000)
modules C: hbase-server U: hbase-server
Console output https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-6501/4/console
versions git=2.34.1 maven=3.9.8 spotbugs=4.7.3
Powered by Apache Yetus 0.15.0 https://yetus.apache.org

This message was automatically generated.

@Apache-HBase
Copy link

🎊 +1 overall

Vote Subsystem Runtime Logfile Comment
+0 🆗 reexec 0m 28s Docker mode activated.
-0 ⚠️ yetus 0m 3s Unprocessed flag(s): --brief-report-file --spotbugs-strict-precheck --author-ignore-list --blanks-eol-ignore-file --blanks-tabs-ignore-file --quick-hadoopcheck
_ Prechecks _
_ master Compile Tests _
+1 💚 mvninstall 3m 28s master passed
+1 💚 compile 1m 2s master passed
+1 💚 javadoc 0m 31s master passed
+1 💚 shadedjars 6m 5s branch has no errors when building our shaded downstream artifacts.
-0 ⚠️ patch 6m 14s Used diff version of patch file. Binary files and potentially other changes not applied. Please rebase and squash commits if necessary.
_ Patch Compile Tests _
+1 💚 mvninstall 3m 17s the patch passed
+1 💚 compile 0m 59s the patch passed
+1 💚 javac 0m 59s the patch passed
+1 💚 javadoc 0m 30s the patch passed
+1 💚 shadedjars 6m 1s patch has no errors when building our shaded downstream artifacts.
_ Other Tests _
+1 💚 unit 221m 51s hbase-server in the patch passed.
248m 4s
Subsystem Report/Notes
Docker ClientAPI=1.43 ServerAPI=1.43 base: https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-6501/4/artifact/yetus-jdk17-hadoop3-check/output/Dockerfile
GITHUB PR #6501
Optional Tests javac javadoc unit compile shadedjars
uname Linux e06afcd0942b 5.4.0-1103-aws #111~18.04.1-Ubuntu SMP Tue May 23 20:04:10 UTC 2023 x86_64 x86_64 x86_64 GNU/Linux
Build tool maven
Personality dev-support/hbase-personality.sh
git revision master / 4a6a514
Default Java Eclipse Adoptium-17.0.11+9
Test Results https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-6501/4/testReport/
Max. process+thread count 4991 (vs. ulimit of 30000)
modules C: hbase-server U: hbase-server
Console output https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-6501/4/console
versions git=2.34.1 maven=3.9.8
Powered by Apache Yetus 0.15.0 https://yetus.apache.org

This message was automatically generated.

@ndimiduk ndimiduk merged commit 1880cb3 into apache:master Dec 5, 2024
1 check passed
@ndimiduk ndimiduk deleted the 29005-master branch December 5, 2024 08:46
ndimiduk added a commit that referenced this pull request Dec 5, 2024
ndimiduk added a commit that referenced this pull request Dec 5, 2024
ndimiduk added a commit that referenced this pull request Dec 5, 2024
ndimiduk added a commit that referenced this pull request Dec 5, 2024
ndimiduk added a commit to HubSpot/hbase that referenced this pull request Dec 5, 2024
ndimiduk added a commit to HubSpot/hbase that referenced this pull request Dec 9, 2024
gvprathyusha6 pushed a commit to gvprathyusha6/hbase that referenced this pull request Dec 19, 2024
mokai87 pushed a commit to mokai87/hbase that referenced this pull request Aug 7, 2025
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants