Adding MVEL to the Java Developers Toolkit

What is MVEL?

By Mark Proctor - Drools Project Lead (Red Hat) and
Mike Brock - MVEL and Errai Project Leads (Red Hat)

MVEL started out as an expression evaluator for the Valhalla project by Mike Brock. Valhalla itself was an early Seam like framework for automating "out of the box" web applications, and while Valhalla itself is now dormant, MVEL continues forward as an actively developed project. It was originally compared to projects such as OGNL, JEXL and JUEL; however, it has since far surpassed them in both terms of performance, features and ease of use – particularly with regards to integration. It does not try to be Yet Another JVM Language, but instead focuses on solving the problems of embedded scripting

MVEL is particularly ideal for restrictive environments that can't use bytecode generation due to memory restrictions or sand boxing. Instead of trying to re-invent Java, it instead aims to provide a familiar syntax for Java programmers while also adding syntactic sugar for short and concise expressions.

MVEL is an integral part of the Drools rule engine, http://www.jboss.org/drools/, and many of the tight integration points were developed in partnership with the Drools team. At the time other scripting languages where reviewed, but exhibited the following issues:

• Lack of optional type safety
• Poor integration, normally via a map populated context.

• Inability to work without bytecode

  • Those with bytecode generation had slow compilation times, plus scalability problems (perm gen)
  • Those without bytecode generation where very slow at runtime execution.
• Too much memory consumption.
• Large jar/dependency size

Shameless plug: Come and learn about Drools & jBPM, and other AI technologies at Rules Fest, http://rulesfest.org/html/home.html. Red Hat will be co-located at Rules Fest and are providing two free days - one dedicated Drools and & jBPM technology, the other focused on Healthcare and Open Source. For more details on those two days see, http://community.jboss.org/wiki/DroolsJBPMBootCampHealthcareFocusSanFranciscoApril2011.

Property Accessors and Data Structures

MVEL supports a JavaScript-like syntax that we know and love from other expression languages, with support for simplified property accessors, inline maps and lists:

[1, 2, 3, 4] // array

["key1" : 1, "key2" : 3 ]   // map

person.name   // simple property accessor

person.pets["rover"].age   // map accessor

person.pets["rover"].?age   // null safe accessor

Using the 'with' statement and inline constructor setters, the following are functionally equivalent:

with ( new Person() ) { 

  name = "Bobba Fet", age = "50" 

}

new Person().{ 

  name = "Bobba Fet", age = "50" 

}

This can be used with the inline maps and lists syntax for complex data structure creation:

[ people : [ 

  new Person().{ name = "Bobba Fet" },

  new Person().{ name = "Darth Vader" }

 ],

places : [ 

  new Place().{ name = "Death Star" }

 ]

]

Like some other languages the return values in MVEL default to the last value in any routine. However, you can also use the 'return' keyword to return from a routine manually.

if (val) { “foo”} 

else 

{ “bar”};

Is the same as:

if (val) { return “foo” } 

else 

{ return “bar” };

This example returns a Map with the key's "room" and "doors", the Door references two previously created instances from the room Map:

rooms = [

 "basement" : new Room("basement"),

 "lounge" : new Room("kitchen")

];

// return is implicit, 

// but can be explicitly added if preferred

doors = [ 

 new Door( 

   rooms["kitchen"], rooms["basement"] 

 ) 

];

[ "rooms" : room, "doors" : doors ];

'switch', 'for', 'foreach', 'if', 'do while', 'do until', 'while' and 'until' block statements are all available and examples can found at the MVEL website.

External Variable Dictionaries

MVEL can evaluate a given expression using a Map to provide the dictionary of available variables:

Map vars = new HashMap();

vars.put( "person", p ); 

// p is a previous created instance

// p.pets references a Map.

Dog dog = 

  ( Dog ) MVEL.eval( "person.pets['rover']", vars );
 

The above will compile and execute the String in a single pass, using MVEL's "interpreted" mode. It is also possible to pre-compile scripts for faster execution.

MVEL was developed with a performance-optimized parser, which keeps the jar size small and means MVEL does not require any additional external dependencies. The jar is around just 700kb, including debug information. The compiled statement is still interpreted, using an optimized internal execution graph. And for performance-sensitive code, MVEL's optimizer can emit bytecode to accelerate field and method access. Work is under-way for an optional secondary layer to fully compile executable statements to bytecode for “fast as Java” execution.

Example of compilation:
 

ExecutableStatement stmt = 

  MVEL.compileExpression( "person.pets['rover']" );

MVEL.executeExpression( stmt, vars );

A context object can be used to execute scripts against that provided object. Notice in the example below we can access the 'pet's property directly, this would work for method's on Person too:

Dog dog = 

  ( Dog ) MVEL.eval( "pets['rover']", p );

Context objects and variable dictionaries can be used together:

vars.put( "otherPet", patch ) 

// patch is another pet

MVEL.eval( 

  "pets['rover'].age == otherPet.age", p, vars 

);
 

Import statements are possible and '*' package imports can be used too:

"import org.domain.Dog; pets['rover'] == new Dog('patch', 7)"
 

Class imports can be added programmatically, via a re-usable ParserConfiguration, which is ideal if the imports can be shared with a large number of expressions during compilation (I'll explain ParserContext in a moment).

ParserConfiguration pconf = 

   new ParserConfiguration();

pconf.addImport( "Dog", Dog.class );

ParserContext pctx = new ParserContext( pconf );

MVEL.compileExpression( 

   "pets['rover'] == new Dog('patch', 7)", pctx 

);

Type Information and Type Safety

The ParserContext can be used to optionally provide type information at compile time for external variables. Where type information is available the compiler can make better assumptions, resulting in much faster execution performance. Local variable assignments support type inference and will also be optimised if they can infer type.

ParserContext pctx = new ParserContext( );

pctx.addInput( “p”, Person.class );

MVEL.compileExpression( "p.pets['rover']", pctx );
 

So far, MVEL has been executing dynamically in a non type safe way -- in a similar manner to other expression languages, like OGNL or JEXL. But compilation can be made to ensure full type safety, using optional strong typing. If type information is not available or cannot be inferred compilation will fail:

ParserContext pctx = ParserContext.create()

.stronglyTyped()

.withInput(“p”, Person.class);

MVEL.compileExpression( "p.pets['rover']", pctx );
 

Setting the type of the context object can be done via the 'this' input:
 

ParserContext pctx = ParserContext.create()

.stronglyTyped()

.withInput( “this”, Person.class );

MVEL.compileExpression( "this.toString()", pctx );

Full type inference is provided for local variables, so type declaration or casting while supported are optional. MVEL will also infer type information from generics where available:
 

public static class Person {

private Map<String, Pet> pets;

...

}

...

ParserContext pctx = ParserContext.create()

 .stronglyTyped();

 .withInput( “person”, Person.class );

//Both the following MVEL statements are valid and type safe:

MVEL.compileExpression( "Pet rover = (Pet) person[“rover”];\n”+

“return rover.age;", pctx );

MVEL.compileExpression( 

   "rover = person[“rover”];\n”+“return rover.age;", pctx 

);

An interesting feature that MVEL provides is the ability to analyse an expression and tell you the external inputs, i.e. variables not assigned locally. It can also be used to report on the inferred types for local variables. When determining external inputs strong typing cannot be used (for obvious reasons) and the inputs types will be resolved as java.lang.Object.
 

ParserContext pctx = ParserContext.create();

MVEL.compileExpression( "person.pets['rover']", pctx );

// Map returns [“person” : Object.class]

Map<String, Class> inputs = pctx.getInputs();

And to get the type of the local variables:

ParserContext pctx = ParserContext.create()

  .stronglyTyped()

  .withInput( “person”, Person.class );

MVEL.compileExpression( "pet = person.pets['rover']", pctx );

// Map returns [“pet” : Pet.class]

Map<String, Class> inputs = pctx.getVariables();

Indexed Variable Dictionaries

Using Maps for external variable dictionaries is the normal way to embed scripting languages, but not optimal for performance. It consumes more memory and every read or write is a Map put/get. Recent versions of MVEL have brought pre-computed indexed variables, via the configuration property 'indexAllocation”, so that reading and writing variables is as fast as accessing an array, and consumes no more memory than that needed for the array matching the length of the variables. The indexing is also used for local variables, ensuring high performance through out.
 

ParserContext pctx = new ParserContext( )

  .stronglyTyped()

  .withInput(“person”, Person.class)

  .withInput(“otherPet”, Person.class)
 

String[] varNames = new String[] { “person”, “otherPet” };

pctx.addIndexedInput(varNames);

pctx.setIndexAllocation(true);

String exp = “person.pets['rover'].age == otherPet.age”;

SharedVariableSpaceModel model = 

  VariableSpaceCompiler.compileShared(expr, ctx);

ExecutableStatement stmt = MVEL.compileExpression( expr, pctx );

// Notice the values order must 

// match the indexedInput variable name order

Object[] values = new Object[] { person, patch };

MVEL.executeExpression(stmt, model.createFactory(values) );

Other Features

• Property Handlers

  • Allows registration of getter/setter handlers for a given class type. Once registered all getter/setter invocation is delegated to this instance.

• Pluggable Data Converters

  • Data converters can be registered to handle to/from data conversions between any class type.

• Function and Lambda Definitions

• Projections and Folds

• Macro's

  • Macro support in MVEL is a basic facility to allow the replacement of a token with an expanded source output.

• Interceptors

  • Provide the ability to place interceptors within a compiled expression. This can be particularly useful for implementing change listeners or firing external events based from within an expression.

• Shell

  • Command line MVEL scripting shell

• Templating

  • Forget Velocity or other templating systems. MVEL provides comprehensive templating capabilities with great performance and no additional dependencies, all contained within the same 700kb jar. The full MVEL expression language is available inside of the templating tags, providing consistency between your templating and scripting environments, as well as higher peformance and less memory usage.

Performance Enhancements

• Optional Bytecode generation

  • Current “spot” generation for accessors
  • Full optional bytecode generation planned to get “as fast as java” execution

• Highly-optimized real-time interpreter

• Write Optimisations

• Escape analysis and Redundancy Optimisations

Summary

By now you should be thoroughly convinced that as a Java developer you cannot afford to leave MVEL out of your toolbox, and that it fills a unique gap between complete programming environments like Groovy and Scala and expression languages like OGNL and JUEL. So while the world debates the various merits of Gavin King's Ceylon project, they miss JBoss' secret weapon being loaded into the torpedo's for launching... MVEL :)

The type safety aspect, combined with property accessors, type inference and compact data structures makes MVEL an ideal testing language for Java developers – we just need someone to develop the tooling, which can support re-factoring, hint hint IDEA ;)