Skip to content

YARN-9650. Set thread names for CapacityScheduler AsyncScheduleThread #2665

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
tangzhankun merged 7 commits into from
Feb 9, 2021
Merged

YARN-9650. Set thread names for CapacityScheduler AsyncScheduleThread #2665

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
7 commits
Select commit Hold shift + click to select a range
782fabf
YARN-9650. Added meaningful names for Threads in CapacityScheduler.java
Feb 1, 2021
3414371
adding test for validating CapacitySchedulerThread names
Feb 2, 2021
ddf38b3
Changing thread names to more meaningful name and adding a single test
Feb 3, 2021
09c4c2e
fixing some checkstyle errors
Feb 4, 2021
bd0f343
manually fixing linelength checkstyle
Feb 4, 2021
705546a
fixing indentation checkstyle
Feb 4, 2021
66b2777
fixing indentation checkstyle issue
Feb 4, 2021
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
3 changes: 3 additions & 0 deletions ...a/org/apache/hadoop/yarn/server/resourcemanager/scheduler/capacity/CapacityScheduler.java
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -199,6 +199,8 @@ public class CapacityScheduler extends

private CSConfigurationProvider csConfProvider;

private int threadNum = 0;

@Override
public void setConf(Configuration conf) {
yarnConf = conf;
Expand Down Expand Up @@ -634,6 +636,7 @@ static class AsyncScheduleThread extends Thread {

public AsyncScheduleThread(CapacityScheduler cs) {
this.cs = cs;
setName("AsyncCapacitySchedulerThread" + cs.threadNum++);
setDaemon(true);
}

Expand Down
31 changes: 29 additions & 2 deletions .../yarn/server/resourcemanager/scheduler/capacity/TestCapacitySchedulerAsyncScheduling.java
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -130,13 +130,41 @@ public void testAsyncContainerAllocationWithMultiNode() throws Exception {
testAsyncContainerAllocation(2);
}

@Test(timeout = 300000)
public void testAsyncThreadNames() throws Exception {
conf.setInt(
CapacitySchedulerConfiguration.SCHEDULE_ASYNCHRONOUSLY_MAXIMUM_THREAD,
1);
conf.setInt(CapacitySchedulerConfiguration.SCHEDULE_ASYNCHRONOUSLY_PREFIX
+ ".scheduling-interval-ms", 0);
final RMNodeLabelsManager mg = new NullRMNodeLabelsManager();
mg.init(conf);

// inject node label manager
MockRM rm = new MockRM(TestUtils.getConfigurationWithMultipleQueues(conf)) {
@Override
public RMNodeLabelsManager createNodeLabelManager() {
return mg;
}
};

rm.getRMContext().setNodeLabelManager(mg);
rm.start();

CapacityScheduler cs = (CapacityScheduler) rm.getResourceScheduler();
for (CapacityScheduler.AsyncScheduleThread thread :
cs.asyncSchedulerThreads) {
Assert.assertTrue(thread.getName()
.startsWith("AsyncCapacitySchedulerThread"));
}
}

public void testAsyncContainerAllocation(int numThreads) throws Exception {
conf.setInt(
CapacitySchedulerConfiguration.SCHEDULE_ASYNCHRONOUSLY_MAXIMUM_THREAD,
numThreads);
conf.setInt(CapacitySchedulerConfiguration.SCHEDULE_ASYNCHRONOUSLY_PREFIX
+ ".scheduling-interval-ms", 0);

final RMNodeLabelsManager mgr = new NullRMNodeLabelsManager();
mgr.init(conf);

Expand Down Expand Up @@ -187,7 +215,6 @@ public RMNodeLabelsManager createNodeLabelManager() {
ams.get(i).allocate("*", 1024, 20 * (i + 1), new ArrayList<>());
totalAsked += 20 * (i + 1) * GB;
}

// Wait for at most 15000 ms
int waitTime = 15000; // ms
while (waitTime > 0) {
Expand Down