Remote Lazy Loading in Hibernate

Types of Lazy Loaders

In Hibernate, there are two main components which can be lazy loaded. They are namely Entities and Collections. An Entity represents a relational data in database, whereas a collection represents collection of children for an entity.

In this article, we'll go through the solutions for lazy loading Entities and Collections separately.

Lazy Loading Entities

Hibernate's lazy loading framework contains two main components namely LazyInitializer and Tuplizer. These components are necessary for creating the reference and initializing the data for this reference. In Hibernate terminology, these reference objects are called Proxy Objects.

LazyInitializer and Tuplizer

LazyInitializer is a Hibernate component that helps to generate proxy objects for Entities and can also be used to fetch the underlying entities of these proxy objects. Hibernate provides a couple of abstract implementations for LazyInitializer such as AbstractLazyInitializer and BasicLazyInitializer, to lazy load or fetch data from inside the Hibernate Session. We'll extend the BasicLazyInitializer and create our own initializer component to fetch data from remote client. Before extending it, let's look at some important methods for lazy loading, in the abstract implementation of LazyInitializer component.

• getImplementation() - this method fetches the entity and returns a persistent object (data object in our case) for the fetched entity.
• getSerializableObject() - this method returns a serializable proxy replacement object for your entity. (We'll discuss more on proxy replacement object in coming sections.)

Tuplizers in Hibernate helps to define contract for certain Hibernate based components. The contract tells how to create such components and play around with the component's property values. These components can either be an Entity or a Mapped component (eg: Composite Identifiers). Hibernate have implementations for these components and are called as EntityTuplizer and ComponentTuplizer respectively. These tuplizers demands you to create Proxy Factory for generating proxy objects of Entities.

As we are using Entity in our solution approach we'll use Entity based tuplizers. Now we need to define own EntityTuplizer and also need to provide a representation called as Entity-Mode for it. Entity modes are actually a representation of the domain model in memory, it can be a POJO, DOM4J or Map based representation. As our data objects are going to POJOs, we'll use POJO based representation.

Hibernate already supports such representation through a class called POJOEntityTuplizer. We'll extend this tuplizer and build a proxy factory which will be used to create proxy object representing our Entity. This proxy object will be of type HibernateProxy as Hibernate would expect.

HibernateProxy is a serializable marker interface that expects you to return a LazyInitializer (as talked above) and a serializable Proxy Replacement object during serialization. This Proxy Replacement object is the one which will be shared with remote client as a lazy entity reference.

Overall our aim is to create a Custom LazyInitializer, a Custom Tuplizer, a Custom Proxy Factory, a Custom Hibernate Proxy object and finally a Custom Proxy Replacement object.

Approach

This flow diagram shows how Lazy initialization component is being extended for lazy loading from a client layer. We'll go through the implementation approach on each component.

CLICK TO ENLARGE IN SEPARATE WINDOW

Flow diagram for remotely lazy loading an Entity

When Hibernate initializes the application configuration for the first time, it creates a CustomTuplizer and then builds a LazyProxyFactory for our Entity. In-order to build a proxy factory we need to create a CustomTuplizer class of type POJOEntityTuplizer and override the buildProxyFactory() method to create and return our own ProxyFactory implementation.

This LazyProxyFactory will implement Hibernate's ProxyFactory interface. The creation of the LazyProxyFactory via CustomTuplizer is shown below.

 @Override protected ProxyFactory buildProxyFactory( PersistentClass persistentClass, Getter idGetter, Setter idSetter) { //Add HibernateProxy as CustomProxyInterface HashSet proxyInterfaces = new HashSet(); proxyInterfaces.add(HibernateProxy.class); //Add any other interfaces ... ProxyFactory cpf = new LazyProxyFactory(); //initialize the ProxyFactory cpf.postInstantiate( getEntityName(), persistentClass.getMappedClass(), proxyInterfaces, idGetter.getMethod(), idSetter.getMethod(), null /*set this if the entity has a composite type identifier */); return cpf; } }

Building a LazyProxyFactory via CustomTuplizer

The getProxy() method of LazyProxyFactory is dynamically called by Hibernate to get the proxy representation of our Entity. Now we need to create a Proxy object to represent our Entity. The createProxyClass() method creates a Javassist based Class Object with HibernateProxy as its interface and its super type as Entity's persistent class. Later this Javassist based Class Object's instance will be returned as a Proxy object.

Next step is to create an interceptor to intercept the calls to this Proxy Object. In-order to do this, we'll add a call back handler that is capable of returning a Proxy replacement object that will be shared with our remote client. As we know our CustomLazyInitializer is capable of returning a replacement object (via its super implementation), we'll add this class as our callback handler.

 //Create Proxy Factory using javassist private void createProxyClass(persistentClass, interfaces) { javassist.util.proxy.ProxyFactory factory = new javassist.util.proxy.ProxyFactory(); factory.setSuperclass(persistentClass); factory.setInterfaces( interfaces ); //comes from buildProxyFactory // set method filters on this factory ... this.customProxy = factory.createClass(); } public HibernateProxy getProxy(Serializable id, SessionImplementor session) throws HibernateException { //create customProxyInstance = this.customProxy.newInstance() //and set the callback/handler to our CustomLazyInitializer ((ProxyObject) customProxyInstance). setHandler(new CustomLazyInitializer(...)); //return customProxyInstance ... }

A LazyProxyFactory creating and returning a CustomProxy object

The setHandler() method accepts only a MethodHandler interface. So we'll add MethodHandler Interface to our CustomLazyInitializer and implement invoke() method in CustomLazyInitializer, which is the actual callback method.

Whenever CustomProxy object's methods are invoked, the method calls are intercepted and the invocation is passed to this invoke() method of the callback handler i.e. CustomLazyInitializer.

 //callback method public Object invoke(final Object proxy, final Method method, final Method nextMethod, final Object[] args) throws Throwable { Object result = super().invoke(method, args, proxy); //if the result is to get proxy replacement object... if (result == INVOKE_IMPLEMENTATION) { Object target = super().getImplementation(); Object returnValue; try { returnValue = method.invoke(target, args); } catch (InvocationTargetException e) { throw e.getTargetException(); } return returnValue == target ? proxy : returnValue; } ... }

CallBack method on CustomLazyInitializer

In the above code the getImplementation() method of BasicInitializer calls the abstract getSerializableProxy() method internally. So, we'll implement getSerializableProxy() method to return our Proxy Replacement Object as shown below. Our Proxy Replacement Object is called as LazyEntityRef here.

 protected Object serializableProxy() { //if initialized already, return from memory ... //if not ... LazyEntityRef lazyEntityRef = new LazyEntityRef(); lazyEntityRef.setClassName(getPersistentClass().getName()); lazyEntityRef.setId(getIdentifier()); ... return lazyEntityRef; }

Implemented method to return Proxy Replacement Object.

The advantage of using Proxy Replacement Object (LazyEntityRef) is that the same class can be re-used for other entities in an application.

Our first part of the implementation is complete. The LazyEntityRef for our Entity is created now and is ready to be shared with the remote client. Now again the remote client has to intercept any calls to this LazyEntityRef object and lazy load the data. One way of intercepting the method calls to LazyEntityRef is to enhance this class again using javassist library and set a handler, whose callback method will lazy load our entity. Let's do this.

Since we need to return an Enhanced Proxy class to the client, we have to do this before the raw proxy object (LazyEntityRef) reaches the client. The best way to do this is to override the de-serialization process and return the enhanced class. Overriding a de-serialization process can be done via readResolve() method of an Serializable Object.

 Object readResolve() throws ObjectStreamException { //create and return an Enhanced Proxy object for client access }

Overriding the de-serialization process of our LazyEntityRef

Here is how to enhance LazyEntityRef object as LazyEntity,

 Class clazz = Class.forName( lazyEntityRef.getClassName(), false, Thread.currentThread().getContextClassLoader()); ProxyFactory pf = new ProxyFactory(); pf.setSuperclass(clazz); pf.setInterfaces(...);//Optionally set a Serializable marker interface to override the serialization process this class pf.setHandler(new ProxyInterceptor(lazyEntityRef)); ...//set Method Filters //Return Enhanced Class = LazyEntity return pf.createClass().newInstance();

Creating Enhanced Proxy Object via ClientProxy

Once our LazyEntity object is created, we need intercept the calls to this object. This interceptor will looks-up our remote bean and fetches the actual data for our entity. So, let's add a call back handler (called ProxyInterceptor) to our LazyEntity object.

Let's create a ProxyInterceptor class implementing javassist's MethodHandler interface and implement the callback method - invoke(). This ProxyInterceptoris then added to LazyEntity object.

 public Object invoke(Object arg0, Method method, Method arg2, Object[] args) throws Throwable { //handle serialization callback if a Serializable interface is added above in the LazyEnity proxy try { if (entity == null) { entity = LazyInitializerRemote.lazyInitializeEntity( lazyEntityRef.getClassName(), ref.getId(), ref.getToken()); } return method.invoke(entity, args); } catch (InvocationTargetException e) { throw e; } }

ProxyInterceptor's Implementation

Finally, the actual implementation to load an entity and fetch the data in the remote bean is shown here.

 Class cls = Class.forName( className, false, Thread.currentThread().getContextClassLoader()) Session currSession = ...//Hibernate Session Object obj = currSession.get(cls, id); //Get persistent data from db ... return ((HibernateProxy) obj).getHibernateLazyInitializer() .getImplementation();

lazyInitializeEntity implementation in LazyInitializerBean.

Testing

We have finally built a simple component to lazy load our data from a remote layer. To test it, we need to make Hibernate, use our Tuplizer component. This can be done by adding a tuplizer tag in hibernate mapping file for our entity.

 <hibernate-mapping ...> <class name=...> <tuplizer entity-mode="pojo" class="package.CustomTuplizer"/> ...

Adding Tuplizer Component to Hibernate Mapping File.

Now deploy the application, and run it. You can create a test client, that invokes a remote bean to load data from Hibernate Session, as shown below.

 TestHome home = ... //remote look-up TeastBean TestRemote remote = home.create(); TestObj obj = remote.getLazyEntity(); //calls session.load(Object) on remote bean //obj is a lazy initialized object whose calls will be intercepted by above ProxyInterceptor ...

Adding Tuplizer Component to Hibernate Mapping File.

Lazy Loading Collections

Unlike Entities, lazy loading collections are pretty straight-forward ,also Hibernate provides direct support for initializing collections of any type. Let's look at Hibernate's collection initialization mechanism first.

Every time Hibernate encounters a collection during initialization, it creates a CollectionType object to map the actual collection values. Examples of such CollectionType supported by Hibernate are BagType, SetType, ListType, MapType, ArrayType and these types are self explanatory.

These collection type objects hold a value object containing the actual collection values and hence acts as a wrapper for the actual collection. The reason for creating wrappers is that these wrappers help to track the state of the collection and even provide lazy initialization support.

Hibernate provides in-built wrappers for some collections, they are listed below,

PersistentBag - representing an Unordered List,
PersistentSet - representing a Set,
PersistentList - representing an Ordered List,
PersistentMap - representing a Map,
PersistentArrayHolder- representing an Array, etc.

In order to create a lazy loadable collection, we need to create a collection type object and define it via Collection-Type attribute in Hibernate Mapping file. We'll use Set to represent our collection and add a lazy loading capability for it.

CustomCollectionType and UserCollectionType

When Hibernate finds a collection-type attribute in Hibernate Mapping file, it creates a CustomCollectionType object, which maps the collection representation with a user written collection type.

The value for collection-type attribute is our own implementation of collection type and that must implement UserCollectionType interface. This collection type object instance is created and added to CustomCollectionType object created earlier.

Approach

The flow diagram shows our approach for lazy loading the collection from a remote client.

CLICK TO ENLARGE IN SEPARATE WINDOW

Flow diagram for loading the collection from client

In order to achieve this implementation, we need to create a collection type called LazyCollectionType, a wrapper for our collection called LazyPersistentSet, a replacement object for this wrapper called LazyCollectionRef (this will be shared with the client) and finally a Collection reference at the client side called LazyCollection.

This LazyCollection when accessed will invoke the remote Bean to initialize and load the underlying collection (from persistent storage) and returns the set. This LazyCollection once initialized will be cached at the client side for future access.

Now let us walkthrough through the details of the mentioned components.

First we need to tell Hibernate to use our Collection Type implementation.

 <set name=... collection-type="package.LazyCollectionType"> <key column=.../> ... <one-to-many class=.../>

Add collection-type attribute to Hibernate Mapping File.

Now that Hibernate creates a LazyCollectionType and adds it to CustomCollectionType component (as talked above). Once this collection type is added, Hibernate invokes the instantiate method on this collection type, to get the wrapper for the underlying collection.

As we are using Set as the collection, we need to return a Set wrapper called LazyPersistentSet of type PersistenSet.

 @implemented public PersistentCollection instantiate(SessionImplementor session, CollectionPersister persister) throws HibernateException { return new LazyPersistentSet(session); } @implemented public PersistentCollection wrap(SessionImplementor session, Object collection) { return new LazyPersistentSet(session, (Set) collection); } @implemented public Class getReturnedClass() { return Set.class; } @implemented public Object instantiate(int anticipatedSize) { // HashSet to hold collection values return anticipatedSize <= 0 ? new HashSet() : new HashSet(anticipatedSize); }

LazyCollectionType implementation.

The last instantiate method creates an empty Set for holding the collection values, in this case its going to be an HashSet to hold our collection values.

The next step is to override the Serialization process for LazyPersistentSet wrapper.Upon Serialization, the wrapper objects returns a replacement object called LazyCollectionRef, which will be shared with the remote client. LazyCollectionRef contains the identity of the actual collection (like collection id, role etc).

 Object writeReplace() throws ObjectStreamException { //get from cache, if already initialized ... //if not... LazyCollectionRef ref = new LazyCollectionRef(); ref.setId(super.getKey()); ... return ref; }

Overriding the Serialization of LazyPersistentSet.

When LazyCollectionRef reaches the remote client, it will be de-serialized to return an AbstractSet implementation called LazyCollection.

Since LazyCollection is an AbstractSet implementation , we'll implement abstract methods that will look-up the remote bean and initializes the underlying collection. For example, an implementation of LazyCollections's iterator method is shown below.

 //iterator method implemented public Iterator iterator() { if (this.collection == null) { //look-up remote bean and load collection this.collection = (Set) LazyInitializerRemote.lazyInitializeCollection( reference.getRole(),reference.getId()); }//add exception handling return this.collection.iterator(); }

Load collection via Remote bean

This collection once initialized can be stored internally and can be retrieved from cache without needing to load it again.

Finally, the actual implementation to load collections in the remote bean is shown here.

 PersistenceContext context = ((SessionImplementor)currSession).getPersistenceContext(); CollectionPersister persister = ((SessionFactoryImplementor) sessFactory) .getCollectionPersister(role); CollectionKey key = new CollectionKey(persister, id, EntityMode.POJO); PersistentCollection collection = context.getCollection(key); // collection - contains set containing actual data

lazyInitializeCollection implementation in LazyInitializerBean

Conclusion

So far we have seen a way to load entities and collection from remote client. When I say remote client, it just need not be thin/fat client that runs on a remote machine, even components in other layers can act as a client. Bottom line is that, this approach helps to initialize data from anywhere out of the Hibernate Session with JDBC or ODBC.

Next thing to bring to the table is, how good the performance of remote lazy loading. Typically for entities the memory foot print can be considered low and applying remote lazy loading will reduces the performance in terms of number client/server calls, more proxy objects in heap etc. But just the memory foot print should not be the deciding factor for lazy loading entities.

Remote lazy loading can yield a better performance for collections. Even other fetching strategies of Hibernate can be applied to increase the JDBC performance of remote lazy loading, like using pagination, lazy fetching object hierarchies, etc.

About The Author

Premkumar Rajendran is a Technical Lead in Architecture and Technology Services Division of HCL Technologies, India with over 6 years of experience in Java Programming, over 4 years in Web Services Development and about 2 years in architecting solutions to customer. He is a IBM certified SOA Solution Designer and also Sun Certified Developer in Java Web Services. His interest includes exploration of RIA, Web 2.0, iPhone. He is one of the reviewer of iPhone in Action (Manning) book.