Skip to content

HBASE-26583 Make StatisticThread alternative in BucketCache #3956

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
YutSean wants to merge 3 commits into
base: master
Choose a base branch
from
+15 −8
Open

HBASE-26583 Make StatisticThread alternative in BucketCache #3956

Changes from all commits
Commits
Show all changes
3 commits
Select commit Hold shift + click to select a range
987f61f
HBASE-26583 Make StatisticThread alternative in BucketCache
YutSean Dec 16, 2021
eeef882
Make the periodic threadpool final
YutSean Jan 3, 2022
9a7f430
Fix whitespace checkstyle
YutSean Jan 3, 2022
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
23 changes: 15 additions & 8 deletions hbase-server/src/main/java/org/apache/hadoop/hbase/io/hfile/bucket/BucketCache.java
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -216,9 +216,7 @@ public class BucketCache implements BlockCache, HeapSize {
});

/** Statistics thread schedule pool (for heavy debugging, could remove) */
private transient final ScheduledExecutorService scheduleThreadPool =
Executors.newScheduledThreadPool(1,
new ThreadFactoryBuilder().setNameFormat("BucketCacheStatsExecutor").setDaemon(true).build());
private final transient ScheduledExecutorService scheduleThreadPool;

// Allocate or free space for the block
private transient BucketAllocator bucketAllocator;
Expand All @@ -244,6 +242,8 @@ public class BucketCache implements BlockCache, HeapSize {
private static final String FILE_VERIFY_ALGORITHM =
"hbase.bucketcache.persistent.file.integrity.check.algorithm";
private static final String DEFAULT_FILE_VERIFY_ALGORITHM = "MD5";
private static final String STAT_THREAD_ENABLE_KEY = "hbase.bucketcache.stat.enable";
private static final boolean STAT_THREAD_ENABLE_DEFAULT = false;
Copy link
Contributor

@Apache9 Apache9 Jan 2, 2022

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

And better set it to true to keep the old behavior by default?

Copy link
Contributor Author

@YutSean YutSean Jan 2, 2022 •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The purpose to make the log alternative is that the log is only used when heavy debugging is needed. Turn on is a little bit annoying at other time I think. So that I changed the old behaviour. What do you think?

Copy link
Contributor

@Apache9 Apache9 Mar 18, 2022

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think in this PR we'd better keep the old behavior, i.e, set it true. We can file another issue to change it to true on master branch, and mark it as incompatible change. On branch-2.x we'd better keep the old behavior.


/**
* Use {@link java.security.MessageDigest} class's encryption algorithms to check
Expand Down Expand Up @@ -325,11 +325,16 @@ public BucketCache(String ioEngineName, long capacity, int blockSize, int[] buck
}
startWriterThreads();

// Run the statistics thread periodically to print the cache statistics log
// TODO: Add means of turning this off. Bit obnoxious running thread just to make a log
// every five minutes.
this.scheduleThreadPool.scheduleAtFixedRate(new StatisticsThread(this),
// Run the statistics thread periodically to print the cache statistics log if it is enabled.
if (conf.getBoolean(STAT_THREAD_ENABLE_KEY, STAT_THREAD_ENABLE_DEFAULT)) {
this.scheduleThreadPool = Executors.newScheduledThreadPool(1,
new ThreadFactoryBuilder().setNameFormat("BucketCacheStatsExecutor").
setDaemon(true).build());
this.scheduleThreadPool.scheduleAtFixedRate(new StatisticsThread(this),
statThreadPeriod, statThreadPeriod, TimeUnit.SECONDS);
} else {
this.scheduleThreadPool = null;
}
LOG.info("Started bucket cache; ioengine=" + ioEngineName +
", capacity=" + StringUtils.byteDesc(capacity) +
", blockSize=" + StringUtils.byteDesc(blockSize) + ", writerThreadNum=" +
Expand Down Expand Up @@ -1316,7 +1321,9 @@ private void disableCache() {
if (!cacheEnabled) return;
cacheEnabled = false;
ioEngine.shutdown();
this.scheduleThreadPool.shutdown();
if (this.scheduleThreadPool != null) {
this.scheduleThreadPool.shutdown();
}
for (int i = 0; i < writerThreads.length; ++i) writerThreads[i].interrupt();
this.ramCache.clear();
if (!ioEngine.isPersistent() || persistencePath == null) {
Expand Down