OSGi: A Proven Approach for Developing Modular Applications

OSGi is about how to create a useful application by installing a set of (preferably reusable) modules in a framework. Basically, we set out in 1998 to solve the component programming model; amazingly I think we largely succeeded looking at the many successful OSGi projects. Mission accomplished? 

Not really, looking at the often heated debate around OSGi in the Enterprise world: Spring, Jigsaw, etc. Many love it, some seem to hate it with a vengeance (Come on guys, get a life, this is only technology). My explanation is that OSGi violates some Enterprise developers' basic expectations that it should just be a library on your class path, providing some useful and convenient functions but otherwise staying out of the way.

 Modularity is Archictecture

Modularity is all about your code base; OSGi is just a tiny layer that provides the runtime environment for well behaved modules and yells when they misbehave. OSGi is not a secret sauce that will make the applications instantaneously modular. Just like Object Oriented forced structured developers to think differently, modularity forces today's developers to shift paradigm (and code base) to reap its benefits. Modularity is architecture!

OSGi is not a secret sauce that will make the applications instantaneously modular. [...] Modularity is architecture!

As we've been doing modularity for a long time inside the OSGi, we found that many existing software patterns are fundamentally non-modular and developed modular alternatives. Alas, we then found that these modular solutions were only really understood (and loved) by developers that actually work in a modular environment. For others it just seemed like gobbledygook.

Just like all next-next problems (the problems you get when you solve the next problem), modular OSGi solutions are often felt to be overly complex and unnecessary, which is logical if one has not yet felt the pain of the next-next problem. In this article I will try to explain the unexpected consequences of strong modularity for some of today's best practices.

The Unexpected Consequences of Strong Modularity

Modularity in Java is about (simplistically said) turning your JAR into a module. Modules can depend on other modules and they can hide implementation details. The space between modules is then only about explicitly shared packages. As implementation details are kept private we can change them at will in future versions, the primary advantage of modularity. What are the effects of this hiding on our programming model?

One of the absolute best practices in our industry is interface based programming. Instead of using implementation classes we use interfaces. The consumer of the interface is then not coupled to any implementation details of the provider. This decoupling gives us reuse and substitution; Testing becomes easier, etc. I've yet to meet an Enterprise developer that does not understand these advantages. The type coupling model is shown in the following picture:

Interfaces allow us to paint a rosy view of the coupling in our system as there is no coupling between consumer and provider in the UML. Unfortunately, this UML is not a realistic reflection of the reality; inevitably somewhere deep in the system lurks a coupling to the implementation class. Our industry has been struggling with this problem for a long time as demonstrated by the large number of well known patterns that address this issue. The bad news is that virtually all these patterns break modularity.

For example, we have a Consumer (C) and a Provider (P) module, the Provider provides the FooImpl class that implements the interface Foo. In the classic way the Consumer just does "new FooImpl()" to create an implementation for the Foo interface:

As the red arrow points out, this violates the module boundary as the Consumer requires access to an implementation detail of the provider, the class FooImpl. Even without modules this was painful to use and resulted in the most successful patterns in the Gang of Four's Design Patterns book: The Factory pattern. Instead of the Consumer grabbing an implementation class, it calls a FooFactory.create() method that then knows how to create the desired implementation.

Indirection, Hiding and Decoupling

This indirection is good news for our Consumer as it becomes truly decoupled from the Provider. However, as the picture shows, the Factory still has to violate the Provider's module boundary by knowing the FooImpl class. Yes, it can be hidden it in a text/XML resource but that does not mean the coupling does not exist. It is also kind of an awkward pattern because it has all the problems of globals and singletons. 

In the early 2000's Rod Johnson wrote a very successful book about Inversion of Control (IoC) and demonstrated how useful this pattern was for Enterprise applications. He then popularized this model with the open source Spring framework. IoC, or the Hollywood principle: "Don't call us, we'll call you," means that the Consumer loses any control it had. Instead of creating the object when it needs it, it gets the object ahead of time injected through a set method or constructor, a.k.a. Dependency Injection. The arrows in the IoC picture clearly show the direction of the control:

The Problems with IoC

Unfortunately, IoC containers make the coupling to implementation details worse! The IoC container needs to know the implementation details of the Consumer (where to inject) and the Provider (what to create). Still, in our UML diagrams it looks fantastically decoupled! The culprit is the IoC configuration as it concentrates (and thus couples) the implementation details, for example the XML in Spring.

The problem is that IoC requires us to give up control in our code, the only place where we actually have full knowledge of the internal details. Is there a pattern where we can have all the advantages but that does not require the violation of the module boundaries? Where neither the Consumer nor the Provider have their boundaries penetrated?

As we're unwilling to know how sausages are made but still want to eat them, sausages can provide us with a metaphor. Can we get a Bratwurst without knowing anything about its production? Yes, in the real world, we just go to a local shop and find them in the freezer.

Does the shop then know how they're made? Nope, it is not a factory, the butcher delivers them fresh every morning, only they know the gory details. The trick here is the intermediary: The shop. Both the Consumer and the Provider have control and the shop is the passive broker. This is the classic publish, find, bind pattern.

If a Provider module can publish its instances and a Consumer can find these instances then the broker can bring the parties together. In OSGi this pattern is implemented as "µservices" and it also underpins the Java Service Loader.

So we're almost in nirvana, except that we now have a pesky dynamic dependency problem. As both the Consumer and the Provider have independent control we might end up going to the shop and not finding any sausages. Using Java code to control this dependency is very awkward, resulting in a lot of boiler plate code.

OSGi and Dependency Injection

Early OSGi releases forced you to do this more or less by hand; this was hard and unpleasant. However, this is a perfect job for Dependency Injection. Each module can specify its dependencies; a central controller can then monitor these dependencies and activate/deactivate the component as necessary.

In OSGi, DI of services is provided by Spring DM, Blueprint, iPojo, Declarative Services, and several other fully interoperable mechanisms. The Broker pattern is the only pattern I know that gives control to consumers and providers and does not force them to expose their private parts gratuitously.

Long story but I hope it became clear why modularity breaks so much code: our industry is using many patterns that are not compatible with a modular world. OSGi is surprisingly agile when the module boundaries are respected but it is brutal when your code tries to violate them. This is as it should be because without this enforcement modularity is an empty shell.

Interested in articles and opinion pieces from Cameron McKenzie? Check these out:

• Why the Amazon S3 outage was a Fukushima moment for cloud computing
• Software ethics and why ‘Uber developer’ stains a professional resume
• It was more than user input error that caused the Amazon S3 outage
• Don’t let fear-mongering drive your adoption of Docker and microservices?
• Stop adding web UI frameworks like JSR-371 to the Java EE spec

Recommended Books for Learning Spring and IoC

Spring in Action  ~ Craig Walls
Spring Recipes: A Problem-Solution Approach  ~ Gary Mak
Professional Java Development with the Spring Framework ~ Rod Johnson PhD
Pro Java EE Spring Patterns: Best Practices and Design Strategies ~ Dhrubojyoti Kayal

You should follow Cameron McKenzie on Twitter: @cameronmcnz