Skip to content

HBASE-23723 Ensure MOB compaction works in optimized mode after snapshot clone #1446

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
busbey merged 4 commits into from
Apr 8, 2020
Merged

HBASE-23723 Ensure MOB compaction works in optimized mode after snapshot clone #1446

Changes from 1 commit
Commits
Show all changes
4 commits
Select commit Hold shift + click to select a range
1de5967
HBASE-23723 Reorganize MOB compaction tests for more reuse.
busbey Apr 7, 2020
4ba9e30
HBASE-23723 Add a test for mob compaction after a snapshot clone.
busbey Apr 7, 2020
22508db
HBASE-23723 note the original table used to write a given mob hfile a…
busbey Mar 9, 2020
3baa8e9
HBASE-23723 checkstyle / javadoc cleanup courtesy qabot feedback.
busbey Apr 8, 2020
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
36 changes: 36 additions & 0 deletions hbase-server/src/test/java/org/apache/hadoop/hbase/mob/TestMobCompactionWithDefaults.java
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -180,6 +180,42 @@ public void baseTestMobFileCompaction() throws InterruptedException, IOException
LOG.info("MOB compaction " + description() + " finished OK");
}

@Test
public void testMobFileCompactionAfterSnapshotClone() throws InterruptedException, IOException {
final TableName clone = TableName.valueOf(test.getMethodName() + "-clone");
LOG.info("MOB compaction of cloned snapshot, " + description() + " started");
loadAndFlushThreeTimes(rows, table, famStr);
LOG.debug("Taking snapshot and cloning table {}", table);
admin.snapshot(test.getMethodName(), table);
admin.cloneSnapshot(test.getMethodName(), clone);
assertEquals("Should have 3 hlinks per region in MOB area from snapshot clone", 3 * numRegions,
getNumberOfMobFiles(clone, famStr));
mobCompact(admin.getDescriptor(clone), familyDescriptor);
assertEquals("Should have 3 hlinks + 1 MOB file per region due to clone + compact",
4 * numRegions, getNumberOfMobFiles(clone, famStr));
cleanupAndVerifyCounts(clone, famStr, 3*rows);
LOG.info("MOB compaction of cloned snapshot, " + description() + " finished OK");
}

@Test
public void testMobFileCompactionAfterSnapshotCloneAndFlush() throws InterruptedException,
IOException {
final TableName clone = TableName.valueOf(test.getMethodName() + "-clone");
LOG.info("MOB compaction of cloned snapshot after flush, " + description() + " started");
loadAndFlushThreeTimes(rows, table, famStr);
LOG.debug("Taking snapshot and cloning table {}", table);
admin.snapshot(test.getMethodName(), table);
admin.cloneSnapshot(test.getMethodName(), clone);
assertEquals("Should have 3 hlinks per region in MOB area from snapshot clone", 3 * numRegions,
getNumberOfMobFiles(clone, famStr));
loadAndFlushThreeTimes(rows, clone, famStr);
mobCompact(admin.getDescriptor(clone), familyDescriptor);
assertEquals("Should have 7 MOB file per region due to clone + 3xflush + compact",
7 * numRegions, getNumberOfMobFiles(clone, famStr));
cleanupAndVerifyCounts(clone, famStr, 6*rows);
LOG.info("MOB compaction of cloned snapshot w flush, " + description() + " finished OK");
}

protected void loadAndFlushThreeTimes(int rows, TableName table, String family)
throws IOException {
final long start = getNumberOfMobFiles(table, family);
Expand Down