Will ObjectLayout's StructuredArrays finally solve the Java performance problem?

When Java 8 was rolled out, there was a great deal of speculation over whether it would be warmly received. Although there has been a certain amount of grumbling, it appears that much of the Java community is embracing the new, more functional style of programming enabled by the latest version. But forward-thinking developers are even more excited about the changes that could be coming in future releases. Martin Thompson, owner of Real Logic, and Gil Tene, co-founder of Azul Systems, are eagerly anticipating the future of Java. This innovative duo has been working diligently on their favorite github project, StructuredArray, for some time now. They hope that a fresh approach to ObjectLayout will be featured on a grand scale in Java 9 or 10.

We want to close that final gap where Java is inherently slower than C because of memory layout.

Gil Tene, Azul Systems

According to Tene, the motivation for creating a workable ObjectLayout feature has to do with speed and memory. "We want to close that final gap where Java is inherently slower than C because of memory layout. When you program in C, you have arrays of structures that are commonly used in a streaming fashion. Message processing is done in that way, as are computational and memory tasks."

There are two key benefits to using an array of structures compared to using an array of objects. First, the structure offers direct access so there is one less dereference to get to an object. It is possible to compute the address of an object in the array without looking anything up. Gil explained that the other helpful factor is the ability to leverage the streaming memory bandwidth of today's processors. With a flat array, today's software and hardware can do a great job of gathering information and caching it. But all that efficiency is wasted in a typical array of objects due to the randomness inherent in the layout. Implementing Struct-like behavior, without changing the language, as a container for objects would provide enhanced computational speed for low-latency processing.

The StructuredArray versus Value Types

In Tene's informative slideshow on the topic, he listed the key goals of the StructuredArray project:

1. Matching the raw speed benefits C languages get from commonly used layouts
2. Exposing those benefits to normal idiomatic POJO use
3. Achieving this greater speed for regular Java Objects on the heap

Tene stated that there is little overlap between this project and those that focus on Value Types. The one exception is when developers are dealing with an array of small-footprint values. Of course, that small area of overlap is of intense interest. Ben Hutchison, enterprise scale Java software developer at Internet Business Systems, wrote at some length about the benefits of an array of value types. "Value Types power is unleashed when they are stacked in arrays for bulk operations—neat rows of data-bricks lined up in memory. The performance advantage of a large value-type array, relative to a large reference-type array, is massive."

Hutchison mentioned a number of capabilities available when using arrays of value types. For example, less reliance on the GC to interpret and manage heap records, and the ability to create many value types in a single bulk memory allocation. He also pointed out that arrays of value types can be directly mapped onto data from I/O operations.

StructuredArray for Big Data

The StructuredArray project is one that could only have come about in a Big Data world. In fact, it's specifically designed to address the pain caused by managing and accessing such large data volumes. The need for this type of solution has been clear for a while now. As early as 2012, Martin Thompson walked readers through an example of C-style memory management using trade data to demonstrate the improved performance, compact data representation, and reduction of GC pauses. However, Martin also issued the following warning: "If your problem space is dealing with big data, or extreme performance, then there are benefits to be had.  If your data sets are small, and performance is not an issue, then run away now to avoid getting sucked into the dark arts of native memory management."

At the time, Thompson was still talking about going off heap to achieve better performance. The dark art of native memory management is looking much brighter today. StructuredArray is designed to go far beyond the risky workarounds proposed a few years ago. In the near future, it may permit a truly scalable solution to memory allocation on the heap for even the biggest data. Even more important, the standardization and simplification of the process should put it within reach for the uninitiated.

Is there anything more important than the StructuredArray when it comes to Java 9 and 10? Let us know.