How to solve the pinning problem in Java virtual threads

The benefits of virtual threads

Unlike traditional OS threads, Java virtual threads are exceptionally lightweight in both memory footprint and context-switching overhead. A Java application can potentially manage hundreds of thousands, even millions, of them without breaking a sweat.

Virtual threads let developers write code that looks sequential and synchronous, even with I/O-bound operations. This often eliminates the need for complex callback-driven or reactive programming styles.

Applications that embrace virtual threads often avoid bottlenecks found in classic Java server architectures. Developers can expect better CPU and memory utilization, which potentially translates to infrastructure cost savings.

The problem of pinning in Java virtual threads

Virtual threads aren't magic wands, however. Under the hood, the Java runtime maps them onto a pool of native OS threads called carrier threads. That's where the problem of pinning comes in.

If a virtual thread needs to execute blocking code, such as traditional synchronization with the synchronized keyword, or native method calls, it cannot simply pause and yield as it does with I/O operations. Instead, the virtual thread becomes pinned to its carrier thread.

A pinned virtual thread prevents other virtual threads from using that carrier thread. Pinning that is too frequent or long-lasting can diminish some of the core scalability benefits of virtual threads.

How to check an application for pinning

Before making a large-scale move to virtual threads, it is critical to assess the pinning risks. The following steps outline how to do this, using Spring.

Enable virtual threads. Often, a simple configuration change will suffice.

spring:
 threads:
  virtual:
   enabled: true

Detect trouble. Next, add the following JVM argument.

-Djdk.tracePinnedThreads=full

Watch for log entries that pinpoint stack traces where pinning occurs. These stack traces reveal which code parts or external libraries might be troublesome. The following code snippet shows what the output might look like.

VirtualThread[#20]/runnable@ForkJoinPool-1-worker-1
VirtualThread[#21]/runnable@ForkJoinPool-1-worker-2 reason:MONITOR
	java.base/java.lang.VirtualThread$VThreadContinuation.onPinned(VirtualThread.java:199)
	java.base/jdk.internal.vm.Continuation.onPinned0(Continuation.java:393)
	java.base/java.lang.VirtualThread.parkNanos(VirtualThread.java:635)
	java.base/java.lang.VirtualThread.sleepNanos(VirtualThread.java:812)
	java.base/java.lang.Thread.sleepNanos(Thread.java:489)
	java.base/java.lang.Thread.sleep(Thread.java:522)
	ca.example.myapp.
.ThreadPinnedExample.lambda$main$0(ThreadPinnedExample.java:20) <== monitors:1
	java.base/java.lang.VirtualThread.run(VirtualThread.java:329)
VirtualThread[#20]/runnable@ForkJoinPool-1-worker-1

Measure the impact. Java Flight Recorder (JFR) is your go-to tool for this. First, start recording.

jcmd <PID> JFR.start duration=200s filename=myrecording.jfr

Then, analyze the pinning duration.

jfr print --events jdk.VirtualThreadPinned myrecording.jfr4

Address the pinning problem beyond library updates

The cleanest solution to the pinning problem may be to update your dependencies to versions explicitly optimized for virtual threads. Projects that thoughtfully embrace Project Loom will replace older synchronization mechanisms with tools to avoid pinning issues, such as ReentrantLock.

If updates aren't readily available, another fix is to temporarily restructure the code. Effective strategies include offloading blocking operations to separate traditional thread pools or refactoring to use non-blocking libraries and asynchronous coding patterns.

Another option is to use semaphores, which can serve as gatekeepers by limiting the number of concurrently executing virtual threads that could perform blocking operations. However, proceed with extreme caution here. If you set the permit count too low, this can severely hinder throughput and counteract the advantages of virtual threads. Ironically, one might constrain an application to run fewer concurrent tasks than was ever possible with classic thread pools.

Virtual threads in Java: Key points to remember

As you evaluate virtual threads, keep in mind several crucial insights.

1. Ecosystem evolution

Many popular libraries are still catching up to virtual threads. They might rely on blocking synchronization primitives that inadvertently pin virtual threads. This will likely improve over time as Project Loom adoption increases, but be mindful of this with the libraries you choose.

2. Resource trade-offs

With virtual threads, the potential to create vast numbers of concurrent threads might inadvertently reintroduce the very resource constraints that classic thread pools aimed to solve. Judicious use of semaphores, thread pools for specific blocking tasks or other limiting mechanisms are essential to prevent uncontrolled resource exhaustion.

3. Ecosystem quirks

Frameworks and languages interact with virtual threads in different ways. Be aware of compatibility concerns. For example, certain Kotlin coroutine patterns might risk pinning under specific conditions. Carefully research how your technology stack dovetails with virtual threads.

4. Metrics tell the story

As you experiment with virtual threads, focus on these KPIs:

• CPU performance. Verify that the expected scalability gains materialize, and detrimental pinning is minimized.
• Memory and garbage collection. Virtual threads might decrease overall process memory, but the larger quantity of thread-like structures might require more attention to garbage collection patterns. Be prepared to adjust heap sizes and potentially garbage collection tuning parameters.
• Latency and throughput. Measure the impact of end-to-end performance on application responsiveness and the number of requests per second the system can handle.

A N M Bazlur Rahman is a Java Champion and staff software developer at DNAstack. He is also founder and moderator of the Java User Group in Bangladesh.