Project Loom virtual threads investigation

[shakuzen]

I was thinking of Loom and the usefulness of this for virtual thread. Also if it is possible to distinguish between virtual and non-virtual threads.

Originally posted by @jonatan-ivanov in #3562 (comment)

We should look at the latest iteration of the Project Loom virtual threads implementation in second Preview in JDK 20 scheduled for JDK 21 as JEP 444.

User instrumentation

How will users instrument their usage of virtual threads if desired?

Library instrumentation

Are there any changes required to our instrumentation of libraries that will use virtual threads?

Thread metrics

We have existing metrics for threads in JvmThreadMetrics, based on ThreadMXBean. Do these cover virtual threads? Are there any new metrics we should provide for virtual threads?

Virtual thread usage in Micrometer

Lastly, is there anywhere it makes sense for us to use virtual threads when running Micrometer on JDK 21+?

[estigma88]

I run into this feature: I am validating the performance of a JVM Spring Boot service using virtual threads, and the current metrics are useless:

• jvm_threads_deamon
• jvm_threads_live
• jvm_threads_peak

It shows only the min pool thread (I am not sure which pool).

Another interesting metrics could be

• From the ForkJoinPool (JEP-444, section Scheduling virtual threads)
• Pinned threads metrics (JEP-444, section Executing virtual threads).

There are some Flight-Recorder events generated.

[jonatan-ivanov]

Thank you for the report. Those are coming from JMX (ThreadMXBean), see:

micrometer/micrometer-core/src/main/java/io/micrometer/core/instrument/binder/jvm/JvmThreadMetrics.java

Lines 56 to 80 in c85a754

	ThreadMXBean threadBean = ManagementFactory.getThreadMXBean();
	Gauge.builder("jvm.threads.peak", threadBean, ThreadMXBean::getPeakThreadCount)
	.tags(tags)
	.description("The peak live thread count since the Java virtual machine started or peak was reset")
	.baseUnit(BaseUnits.THREADS)
	.register(registry);
	Gauge.builder("jvm.threads.daemon", threadBean, ThreadMXBean::getDaemonThreadCount)
	.tags(tags)
	.description("The current number of live daemon threads")
	.baseUnit(BaseUnits.THREADS)
	.register(registry);
	Gauge.builder("jvm.threads.live", threadBean, ThreadMXBean::getThreadCount)
	.tags(tags)
	.description("The current number of live threads including both daemon and non-daemon threads")
	.baseUnit(BaseUnits.THREADS)
	.register(registry);
	FunctionCounter.builder("jvm.threads.started", threadBean, ThreadMXBean::getTotalStartedThreadCount)
	.tags(tags)
	.description("The total number of application threads started in the JVM")
	.baseUnit(BaseUnits.THREADS)
	.register(registry);

Please notice that there is no thread pool involved here, those are all of the threads that the JVM created regardless if they are part of a pool or not. I think they should be useful even if you use Loom since might not everything will use virtual threads in your application, let me copy here the section from the javadoc of ThreadMXBean for your convenience:

ThreadMXBean#getDaemonThreadCount():

Returns the current number of live daemon threads.

ThreadMXBean#getPeakThreadCount():

Returns the peak live thread count since the Java virtual machine started or peak was reset.

ThreadMXBean.html#getThreadCount():

Returns the current number of live threads including both daemon and non-daemon threads.

If you still found them useless, check if you get the same values from ThreadMXBean as Micrometer reports you. If yes, and you think you should have got a different value, please report an issue back to OpenJDK; if no, please report an issue (with a reproducer) to Micrometer.

We should figure out what to do with virtual threads, not just because as one of the links you pasted says that ThreadMXBean does not support them but we also need to distinguish them somehow in the time series:

java.lang.management.ThreadMXBean only supports the monitoring and management of platform threads.

[shakuzen]

User instrumentation

Manual instrumentation is always an option. I don't think we need to offer automatic Observation instrumentation of an ExecutorService's tasks run on virtual threads. Rather, it would be better in most cases if the component(s) used in the tasks were instrumented. If the components (e.g. HTTP/DB/etc clients) are instrumented, it adds little or no value to instrument the task itself, where we have no metadata on what is being done in the task to give meaning to the Observation. However, feedback from users on use cases will be valuable here. So far we have not got much feedback on open issues like #3881.

Still, even with manual instrumentation in the components, we need scoping so hierarchy works properly across threads. We can use the ContextExecutorService from context-propagation to wrap ExecutorService, including ones that use virtual threads. See existing documentation on Instrumentation of Thread Switching Components.

Library instrumentation

Are there any changes required to our instrumentation of libraries that will use virtual threads?

I could not come up with any changes necessary. Virtual or platform thread is transparent to us in components we instrument. I tried, for example, Spring Boot 3.2 snapshots with virtual threads enabled for Tomcat, and the existing WebMVC instrumentation works the same as before.

Thread metrics

We have existing metrics for threads in JvmThreadMetrics, based on ThreadMXBean. Do these cover virtual threads? Are there any new metrics we should provide for virtual threads?

Existing metrics do not cover virtual threads - ThreadMXBean only provides statistics on platform threads.
New potential instrumentation of virtual threads is only available via JFR. We can count started/finished virtual threads. The other events available are:

• jdk.VirtualThreadPinned indicates that a virtual thread was pinned (and its carrier thread wasn’t freed). This event is enabled by default with a threshold of 20 ms.
• jdk.VirtualThreadSubmitFailed indicates that starting or unparking a virtual thread failed, probably due to a resource issue. Parking a virtual thread releases the underlying carrier thread to do other work, and unparking a virtual thread schedules it to continue. This event is enabled by default.

We should track work for this in a separate issue linked to this one.

Virtual thread usage in Micrometer

Lastly, is there anywhere it makes sense for us to use virtual threads when running Micrometer on JDK 21+?

We could use virtual threads for publishing, but given our current Java 8 baseline, we would have to use reflection to achieve it, and given we only have one (or two for step registries) thread(s) per meter registry for publishing, I don't think the benefits are significant enough to warrant the effort and maintenance.

[jonatan-ivanov]

This also has an interesting aspect: #3973