Arm64: Revisit the heuristics for IO completion poller threads

[kunalspathak]

Today, depending on the number of processors, we create IO completion poller threads. Back then, we did our analysis on older Arm64 machines to come up with the heuristic and they can be different on modern Arm64 machines. We need to revisit them or have some kind of auto-tuning of number of threads creation.

runtime/src/libraries/System.Net.Sockets/src/System/Net/Sockets/SocketAsyncEngine.Unix.cs

Lines 52 to 57 in 0e35b8d

	Architecture architecture = RuntimeInformation.ProcessArchitecture;
	int coresPerEngine = architecture == Architecture.Arm64 || architecture == Architecture.Arm
	? 8
	: 30;
	return Math.Max(1, (int)Math.Round(Environment.ProcessorCount / (double)coresPerEngine));

runtime/src/libraries/System.Private.CoreLib/src/System/Threading/PortableThreadPool.IO.Windows.cs

Lines 41 to 43 in 39b3aac

	int processorsPerPoller =
	AppContextConfigHelper.GetInt32Config("System.Threading.ThreadPool.ProcessorsPerIOPollerThread", 12, false);
	return (Environment.ProcessorCount - 1) / processorsPerPoller + 1;

Related: #67180

I couldn't figure out the best area label to add to this issue. If you have write-permissions please help me learn by adding exactly one area label.

[kunalspathak]

@mangod9

Tagging subscribers to this area: @mangod9
See info in area-owners.md if you want to be subscribed.

Issue Details

---

Today, depending on the number of processors, we create IO completion poller threads. Back then, we did our analysis on older Arm64 machines to come up with the heuristic and they can be different on modern Arm64 machines. We need to revisit them or have some kind of auto-tuning of number of threads creation.

runtime/src/libraries/System.Net.Sockets/src/System/Net/Sockets/SocketAsyncEngine.Unix.cs

Lines 52 to 57 in 0e35b8d

	Architecture architecture = RuntimeInformation.ProcessArchitecture;
	int coresPerEngine = architecture == Architecture.Arm64 || architecture == Architecture.Arm
	? 8
	: 30;
	return Math.Max(1, (int)Math.Round(Environment.ProcessorCount / (double)coresPerEngine));

runtime/src/libraries/System.Private.CoreLib/src/System/Threading/PortableThreadPool.IO.Windows.cs

Lines 41 to 43 in 39b3aac

	int processorsPerPoller =
	AppContextConfigHelper.GetInt32Config("System.Threading.ThreadPool.ProcessorsPerIOPollerThread", 12, false);
	return (Environment.ProcessorCount - 1) / processorsPerPoller + 1;

Related: #67180

Author:	kunalspathak
Assignees:	-
Labels:	area-System.Threading, untriaged
Milestone:	-

[mangod9]

@kouvel as fyi.

[MihaZupan]

cc: @dotnet/ncl

[adamsitnik]

FWIW I am the person who came up with the magic numbers on Unix. Here you can find the data and reasoning behind it.

Recently after a conversation with @kunalspathak I wanted to give it a quick try and see how the number of epoll threads affects the throughput of JSON Platform benchmark on the Ampere machine. Currently more than 80% of time is spent in ConcurrentDictionary.TryDequeue and we need to solve this blocker (#67845, dotnet/aspnetcore#40476) before we try to change the number of epoll threads.

arm64 with 80 cores::

x64 with 28 cores:

[kunalspathak]

@mangod9 - Do you think we will be doing this in .NET 7?

[mangod9]

yeah we hope to investigate this soon. Might be worth validating again after the concurrent queue fix.

[kouvel]

On the Linux Ampere machine, quick tests on JsonPlatform and Json didn't show any significant improvements from decreasing the IO poller thread counts. It still seems like relatively more IO poller threads are necessary on arm64 platforms than on x64 platforms. Probably the heuristics could be more fine-tuned, and I haven't done an exhaustive test.