Keel - The Next Generation Meta Framework

An application framework is a bundle of common processes and services ready-made for use. You can think of frameworks as a skeleton upon which you flesh out your business logic. The end result is a complete, fully functional product or information system. There is a plethora of available server-side Java frameworks to choose from, each filling some niche or area of concern. Some Java frameworks are proprietary and tied to a particular container or server; however, some of the best frameworks are open source and freely available.

In the open source domain, some frameworks are stable, popular, and easy to use. Others are unfinished, lacking in documentation, and extremely complex to configure. The level of quality and utility varies widely, and the number of choices can seem overwhelming. It can be extremely perplexing for the developer tasked with selecting the right framework for a given project. He or she has to answer questions such as, "Will the framework provide all the services my project requires?" "Will it be flexible and extensible?", "Will it work with my existing environment?", "Will it be standards-based?" Most importantly, "Will it be stable, reliable, and well-supported?" Keel is one such framework that provides positive answers to these and other questions. This article explains the technical architecture of Keel and its core functionalities and services so that one has a better idea why Keel is best suited to answer the above stated important questions for any project.

Keel could be described as a "Framework of Frameworks", or a meta-framework, a highly extensible backbone for integrating Java components and services. A meta-framework can be defined as a specialized framework that allows easy integration of other functional specific frameworks that work together seamlessly. Meta-frameworks act as a bridge and connect together multiple application frameworks and tend to extract the best out of them. Like the keel of a ship, the Keel framework provides direction for server-side Java projects. With Keel, every service is accessed through an interface, which insulates your application code from becoming dependent on an implementation of that particular service. With Keel, you can avoid tightly coupling your code to a particular service or component, giving you the freedom to "unplug and plug in" services without needing to change your business logic.

In addition to providing this structure for integrating services and components, Keel comes pre-configured with many "best of breed" implementations for each service right "out of the box". For example, there are currently four "user interface" implementations available in Keel, (with plans to add Java SWT and others in the future): Struts/JSP, Cocoon, Velocity, and a Command Line Interface (CLI). The same application code will run completely unaltered using any of the four implementations of the user interface. Moreover, a developer can choose a new interface option to fit his or her needs.

Keel is based on the popular Jakarta Avalon project. Avalon is a lower-level framework that demonstrates some groundbreaking design principles such as:

• Component Oriented Programming (COP) - a design of breaking down the system and its functionalities into components. Each component is tied by contract to an Interface. This allows for easy replacement of components or services without impacting other parts of the project. Flexibility is always preferred in a development lifecycle.
  
  
• Inversion of Control (IOC) - a design wherein a component is always externally managed throughout its lifecycle. Doing so exercises strict control over the behavior of the component.
  
  
• Separation of Concerns (SOC) - a pattern that deals with demarcating the concerns or issues in a problem space and tackling them as independently as possible. For example, a given component can be LogEnabled, Securable, Recyclable, Poolable and Configurable. Each of these is a discreet area of concern, and each is dealt with separately by implementing separate interfaces.

While Avalon provides an excellent infrastructure for developing server-side Java applications, it was designed in a rather abstract form to promote the greatest degree of flexibility. It is actually a collection of interfaces and design patterns rather than a fully implemented server application. Keel attempts to "fill in the blanks" by building on Avalon and providing at least one pre-configured implementation for every service.

Keel consists of various modules or projects. The two most important modules are keel-core and keel-server. Keeping true to the COP design pattern, Keel consists of many high level services or roles. Each role is a Java interface that defines a service. Functionalities of the service block are encompassed in the method signatures that define the contract. Some of the fundamental roles or services in Keel include Persistence, Security, Scheduling, Mail and Crypto. These core roles are defined in the keel-core module. Every role defined in Keel has to be entered in a configuration file, and all of the core roles are entered in a file named "roles.xconf." Roles in Keel can be thought of as the main building blocks, as they layout the blueprint of the system. Each high-level business process should be defined as a role. All related business processes are then implementations of a broader role.

Implementations of roles are components. Components are the flesh and blood of the system. They do the actual work by processing the requests they receive through the Keel interface. The keel-server module is comprised of default implementations of these core roles. And this might be considered the default backbone of the framework. It should be stressed that these are merely Keel's default implementations. If desired, you could certainly define your own implementations of these roles/services. For example, the JDBC datasource is the default implementation for the Persistence service. However, you could replace it with another datasource type such as LDAP, XML or a simple filesystem database. This highlights the flexibility and configurability of Keel, and it is apparent throughout.

Just like their roles, each of the components in Keel must be defined in a configuration file. The default components are configured in a configuration file called "system.xconf". These configuration files are the driving parameters for the KeelContainer; the repository of all Keel components. Before delving further into the container itself, it's prudent to cover the deployment options in Keel.

Figure 1: Keel Roles and Components

Keel can be deployed in one of two ways. The first method is with a single JVM. As you can imagine, this method is best suited for small applications that require a moderate level of scalability and run on a single instance of a web server. The second method of deployment is with multiple JVMs that either run on one server or on multiple servers in a cluster. This is more suitable for high-volume web applications that require load balancing due to steep scalability. In the latter case, Keel uses Java Messaging Services (JMS) as the communication protocol between each of the clients and the server(s). Each client talks to one or more JMS brokers, and each broker in turn is serviced by one or more Keel servers. Of course, this deployment scenario is more complex to configure and deploy, but it yields much more flexibility and scalability than the single JVM option. For development purposes, the single JVM deployment is recommended because of its simplicity. It's worth emphasizing here that either way, it is truly a deployment-time decision. Deployment options are defined in external configuration files, and application code is not affected by the deployment options that you choose.

Figure 2: Single JVM Deployment

Figure 3: Multiple JVM Deployment

Now that we've gone over the details of deployment, we're ready to dive right into the KeelContainer. To keep things simple, we'll focus on the single JVM deployment scenario. The KeelContainer is the top-level container for all Keel components, and it is a sub-class of Avalon Fortress' Container class. When a request is received from a client for the first time, the Keel server comes to life, and all the classes are loaded by specialized ClassLoaders in Keel. This process instantiates the KeelContainer which in turn, readies the components for instantiation. Components can be instantiated in this fashion, during KeelContainer startup or on demand later, depending on the configuration specification in the "roles.xconf" file. Components can be Poolable, wherein instances are collected by the container. They can also be ThreadSafe, where the same instance is handed over to every request. Lastly, they can be factory serviced wherein a new instance is created for every request. The KeelContainer thus initiates, manages and destroys all components. Any class - whether it's another component or a plain old Java object (pojo) - can use any component, simply by obtaining an instance of it from the container. The KeelContainer has sole control over the component and its lifecycle. This illustrates the Inversion of Control design pattern in Avalon. Direct instantiation of a component by another class is illegal, and will result in a component that can not be used.

The Avalon Framework implements the Separation of Concern (SOC) design pattern by way of Lifecycle interfaces, and Keel makes full use of Avalon's available interfaces. Some of the popular Lifecycle interfaces are as follows:

• LogEnabled - essential logging mechanism.
• Configurable - required for access to configuration files.
• Servicable - required for access to other components via the KeelContainer.
• Securable - required for authorization on the component.
• Poolable - required for the component in the KeelContainer for faster turnaround.

There are other standard Lifecycle interfaces that add even more functionality to the component. Each component goes through its Lifecycle processes at sometime during its existence; either during initialization or when it is accessed. It's the responsibility of the KeelContainer to make sure that the component undergoes its Lifecycle processes before it is handed over to the requester. Lifecycle processes are a powerful way of plugging different functionalities into a component. Most importantly, only those functionalities that are required should get used.

All of the services in Keel are created for the roles and the component-based design pattern discussed so far. Some of the services and their implementations available in Keel are as follows:

• Model - This service deals solely with business or application logic. There is a default implementation that contains a mechanism to extract a request object from the User Interface (UI), run necessary application logic using the parameters passed, and send the resultant response back to the UI. The crux of the business logic is implemented by Model objects.
  
  
• Persistence - As you guessed, this is the service for storing objects and data. The default implementation is based on JDBC datasource. Other options are Entity EJB, Java Data Objects (JDO), LDAP, Hibernate etc.
  
  
• Mail - This is the Messaging and Email service. The default implementation is based on the JavaMail API.
  
  
• Authentication/Authorization - This is the Security service. The Authentication piece is based on JAAS and is flexible enough to incorporate multiple layers of authentication processes. The Authorization piece works down to the instance level of components and authorizes functionalities by verifying against security rules specified in the Persistence layer. Method level authorization is in the works.
  
  
• Scheduling - This service deals with scheduling jobs in the background. Currently the default implementation is based on Quartz - the popular open source scheduling framework. Any type of component, including Models, can be scheduled to run at fixed, regular intervals or some specific moment in time, once or multiple times. It's more prudent to run jobs on a different instance of the JVM to keep it separate from the real time application. This makes for easy maintainability. Keel's scheduling service also has a provision for this.
  
  
• Crypto - This is the Encryption service for security of transmitted data. Currently there are implementations based on BASE64 and DES.
  
  
• Query - This service deals with efficient querying against the Persistence layer to produce both key list and complete results list. There is a general purpose implementation of this based on queries specified in the configuration files.
  
  
• Text Indexing and Searching - Based on the Query service there is an indexing and searching facility in Keel which works on both files and database tables. This service is built upon the popular Apache Lucene opensource project.

Yet another important feature of Keel is the total decoupling of the server side code from that of the User Interface (UI). Logic and presentation are totally independent. The benefit is that any UI implementation can be plugged into Keel, and several were mentioned at the beginning of this article. Keel strictly adheres to the Model View Controller (MVC 2) architecture and as such, any MVC based UI is a perfect candidate for Keel.

There are many useful applications that come bundled with Keel, so you can get started immediately and learn Keel best practices while doing so. By studying these sample applications, you can learn the majority of Keel's features and functionalities. Although they serve quite well as sample applications, they are certainly production-ready. Keel's bundled applications include:

• Poll Application - a full-featured voting and polling application.
• CRUD - a generic database maintenance tool that allows for creating, reading, updating and deleting data elements
• Registration - a JAAS enabled Login and Registration of users
• Navigation - a general purpose workflow process
• Security Maintenance Application - a tool to create and maintain security rules for Authorization.

It is common knowledge that the presence of proper documentation in many open source projects is lacking, to say the least. Sadly, it is for this reason alone that many people have resisted open source projects. The Keel developers addressed this barrier early on, as they themselves are veterans in the open source domain. Keel comes with a good amount of documentation for newbies and more is being added regularly by the Keel users themselves. The Keel web site contains an extensive online documentation Wiki, and an active mailing list that is extremely responsive. Professional training and support are also available through The Keel Group Ltd., and other organizations. In the very near future, a book will be released titled "Java Frameworks and Components", which will have Keel explained in some of the chapters.

As mentioned earlier, Keel comes bundled with some of the essential services - Persistence, Models (Business Logic Layer), Authentication/Authorization(Security), Scheduling, Querying, Searching and Indexing, etc. All these services are abstracted away from the actual implementations. Although default implementations of all these services are ready made for use, you can have your own implementation of any service. For example, if the default implementation of the persistence layer is not suited for a project it can be easily replaced by other persistence layers such as Hibernate. Or if you have a project already using a persistence layer then that layer can easily be plugged into Keel with minimal code change. This is the main power of Keel - it is very flexible and easily extensible. Keel, being a meta-framework based on the Apache Avalon project, is stable and reliable. It uses industry proven opensource projects (Struts, Cocoon, Hibernate, etc.) to implement its services.

Keel was developed over a span of 15 months and it is the result of rigorous brainstorming as well as serious, hands-down coding. Keel is already being used successfully in commercial applications within various industries and government. As more and more companies are looking at open source solutions, it's worth considering Keel for product/system development. It's never made sense to reinvent the wheel. With Keel, you get the entire car for free.

Links

• Keel Home Page - http://keelframework.org
• Keelgroup mailing list - [email protected] http://lists.keelframework.com/listinfo.cgi/keelgroup-keelframework.com
• Apache Avalon Project - http://avalon.apache.org