Sending attachments with SOAP - A simple JAX-WS development example in Java

Sending attachments with SOAP prerequisites

This article uses the WSO2 Web Services Application Server (WSAS.) It is recommended that you download and install WSO2 WSAS 2.0. The article uses the servlet edition installed on Apache Tomcat. Any application server can be used with the servlet version, just follow the installation instructions included with WSO2 WSAS. You don't have to use an application server at all, as WSO2 WSAS works great in a standalone format. WSO2 WSAS requires Java 1.4 or 1.5 but there are no other prerequisites for it. Of course web services and SOAP especially is used, so familiarity with that will help.

When XML is not enough: binary data

There are endless ways to send data over the network, which means there are also many ways to send attachments with SOAP. There are numerous protocols and data formats. Standardization around SOAP has taken away a lot of the guesswork in sending data between systems. SOAP standardizes the protocol (HTTP) and the data format (XML.)

One of the main criticisms of SOAP is its use of XML, and that complaint often carries over when it comes to sending attachments with SOAP. XML is text based. This not only makes for large messages, but makes it incompatible with binary data. For example, let's say your message needs to include an image. This poses a problem when your message format is text.

Combining binary data with SOAP attachments

Ok, so you need to send binary data between applications. You'd like to use SOAP attachments, but it's limited to text. So should you just give up on sending attachments with SOAPall together? Of course not, there are too many advantages to SOAP. You just need a way to combine it with binary data. You see web pages do this all the time; it can't be that hard, right? Let's explore some solutions to this problem.

Naive approaches to sending attachments with SOAP

One way you might try is to simply dump the binary into a text node. It might look something like Listing 1.

Listing 1. XML with Binary Data: First Try

<?xml version="1.0" encoding="UTF-8"?>
<message>
 <data><!-- binary data goes here --></data>
</message>

Remember that characters are also bytes, just like binary data. An XML parser, whether it's a DOM, SAX, or StAX parser, must use the character set encoding of the document to interpret all the bytes in the document as characters. So our binary data could easily have characters that correspond to reserved XML characters, like < or > or &. Any such byte sequence in text node above will cause the parser on the other side to break. So this approach will not work.

But wait, maybe there's a way to fix this approach. What about using a CDATA block? That will tell the parser to ignore the characters inside the block. This modified approach might look like Listing 2.

Listing 2. XML with Binary: Using CDATA

<?xml version="1.0" encoding="UTF-8"?>
<message>
 <data><<![CDATA[ binary data goes here ]]></data>
</message>

Now if we have bytes that would be interpreted as a > (for example,) they will be ignored. However, the parser has to figure out where the CDATA section ends. It does this by looking for the byte sequence corresponding to the characters ]]> . It might seem unlikely, but our binary data could have just such a byte sequence in the middle of it. That would cause any parser to think that the CDATA section had ended and the subsequent characters would be interpreted just like in our first attempt. So that's not going to work either. We need a way to make sure those bytes aren't interpreted at all.

Base 64 encoding: Works but bloated SOAP attachments

There is a solution to this variant of our problem. One common way to do this is to use Base 64 encoding. This technique has been around (as a standard) since the 80's. It involves using a 64 character “alphabet” consisting of the lower case characters, a-z, the upper case characters, A-Z, the numbers 0-9, and the + and / symbols. Every byte gets mapped to these characters, so there's no way for any byte to get misinterpreted as anything that would choke an XML parser.

So there, problem solved, right? Yes, but … it's a rather inefficient solution. Base 64 encoded binary winds up being, on average, 37% larger (number of bytes) than the raw, non-encoded binary data. In addition, the parser on the other side needs to know about the encoding so that it can decode the payload. One could imagine that if Base 64 encoding was part of the SOAP standard, then there would be some standard way to indicate this SOAP message processors. This is not the case, though. It may be a solution, but it is both inefficient and non-standard. We need something that is both more efficient and standardized.

SOAP with Attachments: Works but flawed design

One solution to the problem is to use what is known as SOAP with Attachments. The idea here is to just put the binary data outside the SOAP message completely. Figure 1 provides a nice visualization of this.

Figure 1. SOAP with Attachments

This is very similar to how binary files can be attached to emails. The SOAP message contains a reference to the binary file that is attached to the message. This is both more efficient and standardized, but it has some flaws in its design. The binary attachment is not part of the SOAP message at all. It's similar in a lot of ways to just passing a URI for the binary data and leaving it up to the message processor to retrieve the actual binary data. It presents some real problems for things like WS-Security. Still, this is what was used for a while, until a better solution was proposed: MTOM.

MTOM: Best of both worlds

MTOM stands for SOAP Message Transmission Optimization Mechanism. It combines the efficiency of SOAP with Attachments, but it does so without having to break the binary data outside of the SOAP message. How can this be? The key is a technology called XML-binary Optimized Packaging or XOP.

XOP Explained

XOP allows binary data to be included as part of the XML Infoset. In fact the XML Infoset becomes a superset of the traditional Infoset known as the XOP Infoset. It allows for the binary data to be stored outside of the XML document, just like in SOAP with Attachments. It uses a special XOP:include element to tell the processor to replace the contents with the referenced binary data, thus encapsulating the logic of discrete storage and retrieval of the binary data. This logic becomes inherent to the XML parser, and allows the SOAP parser to treat the binary data as part of the XML document, with no special retrieval logic. Similarly it allows for a SOAP server to create a SOAP message in a uniform way, no special logic for breaking out that binary data from the SOAP message.

MTOM in WSO2 WSAS

We've talked a lot about the need for MTOM and how it should work in theory. It doesn't do us a lot of good without a real implementation. Luckily there's an easy way to get a great MTOM implementation, just use WSO2 WSAS. WSO2 WSAS is built on top of tried and true technologies, including Apache Axis2. Axis2 gives WSO2 WSAS its MTOM implementation. Let's take a look at how to tap into the power of WSAS/Axis2's MTOM implementation.

Sending an MTOM Message from a Web Service with the Axiom API

MTOM support on Axis2 builds on the same classes used throughout Axis2. It uses Axis2's Object Model (OM). Axis2 supports both Base 64 encoding and MTOM, and makes it relatively simple to switch between them. Why? Well for very small files, it can actually be more efficient to use Base 64 encoding.

To achieve this seamless switching between optimized and non-optimized transport, Axis2 treats binary data as an XML text node. The only difference is that you need to pass in a javax.activation.DataHandler for accessing the data, as shown in Listing 3.

Listing 3. Adding Binary Data with the Axiom API

OMFactory messageFactory = OMAbstractFactory.getOMFactory();
OMNamespace ns = messageFactory.createOMNamespace("urn:foo" , "foo");

OMElement binaryElement = messageFactory.createOMElement("payload",ns);
FileDataSource src = new FileDataSource("temp.dat");
DataHandler handler = new DataHandler(src);
OMText textNode = messageFactory.createOMText(handler, true);
binaryElement.addChild(textNode);

In the example in Listing 3, a javax.activation.FileDataSource is used to provide the DataHandler with access to the binary data. You can use any class that implements the javax.activation.DataSource interface. For example, when working with images, the org.apache.axis2.attachments.ImageDataSource can be used. It implements the DataSource interface and can be more convenient when working with images.

So how does Axis2, and thus WSO2 WSAS, know to use MTOM to optimize the binary data? That is actually what Axis2 will do by default. You can manually override this by adding just one line of code, as shown in Listing 4.

Listing 4. Turning off MTOM

textNode.setOptimize(false);

That single line of code will tell Axis2 to not optimize, i.e. don't use MTOM. Thus Axis2 will use Base 64 encoding of the binary data, and it really will be a text node. Otherwise, MTOM will kick in, and an XOP include will be used to optimize the transport of the binary data within the SOAP message.

Enabling MTOM on the server

Of course to get all this wonderful, automatically optimized behavior, you do need to enable MTOM. You can do this through your axis2.xml file very easily, as shown in Listing 5.

Listing 5. Enabling MTOM in axis.xml

<parameter name="enableMTOM">true</parameter>

It can't get anymore painless than that, right? This is a global setting, and is the default setting on WSO2 WSAS. You can actually enable MTOM at four different levels: global, service group, service, and operation. You use the same semantics for each level. You can use the Management Console to manage MTOM at each of these levels. For example of this take a look at Figure 2.

Figure 2. Managing MTOM at the Service Group Level

Here we see MTOM being managed at the Service Group level. Each service in the group can also be individually managed, as shown in Figure 3.

Figure 3. Managing MTOM at the Service Level

Of course each service can have one or more operations. WSAS lets you manage MTOM at that level too, as shown in Figure 4.

Figure 4.

Notice that at each level, MTOM has three possible values: true, false, and optional. If the property is set to true, then the service will send an optimized message when necessary, i.e. when binary data is included. If the value is set to false, then optimization will never be used, and Base 64 encoding will be used for any binary data. If it is set to optional, then WSAS will optimize if and only if the request came in was optimized. The type of request will indicate to WSAS if it should use MTOM or not.

Why do we need this kind of flexibility? As mentioned earlier, it is often advantageous to use Base 64 encoding on small files. Thus you could decide that certain operations should use MTOM and others should not. Or you could make it optional on an operation, programmatically do a check for the size of the data being sent, and then choose to override the default MTOM if the file is small. Then you're sending MTOM. Let's take a look at how easy WSAS makes it to send an MTOM message from a web service client.

Creating a SOAP client that sends MTOM messages

Sending a MTOM message from a client is just as easy as sending an MTOM message from a web service. Axis2 provides several convenient APIs. An example is shown in Listing 6.

Listing 6. Client code for sending MTOM message

ServiceClient wsClient = new ServiceClient();
Options clientOptions = new Options();
clientOptions.setTo(new   EndpointReference("http://some/service"));
clientOptions.setAction("foo");
       clientOptions.setProperty(Constants.Configuration.ENABLE_MTOM, Constants.VALUE_TRUE);
wsClient.setOptions(clientOptions);

As you can see in Listing 6, the key thing to do is to enable MTOM in options for the web service client. Once you do that, then Axis2 will optimize any binary data you send to the web service using MTOM automatically. We've seen how to send MTOM messages from a web service and to a web service, now let's look at how to work with data that has been sent using MTOM.

Handling an MTOM Message in a Web Service

Now let's assume you have a web service that accepts binary data as part of a SOAP message from a client. If your web service is running on WSAS, you don't need to do anything special to be able to handle optimized binary data from your clients. Your clients can send SOAP messages that use MTOM or Base 64 encoding. It's all seamless with WSAS. Listing 7 shows an example of receiving optimized data.

Listing 7. Web Service receiving Optimized SOAP

OMElement dataElement; // Optimized element
OMText dataNode = (OMText) dataElement.getFirstOMChild();
DataHandler handler = (DataHandler) dataNode.getDataHandler();
InputStream stream = handler.getDataSource().getInputStream();
// read from the stream, proces data, save to file or database, etc.

As we saw earlier, the Axiom API treats the binary data as a text node. This allows a single API for dealing with optimized and non-optimized (Base 64 encoded) data. You simply access the DataHandler associated to the text node (that contains the binary data) and use that to obtain an InputStream. Once you have the InputStream, you can read all the bytes and process them however you need to. WSAS makes it easy to handle SOAP messages with optimized binary data payloads. Let's take a look at how easy it is to work with MTOM on clients.

Handling an MTOM Message in a Client

There's no magic to handling an MTOM web service response. We've already seen how to setup the request. In Figure 8 you see how to deal with a response that contains binary data optimized with MTOM.

ServiceClient wsClient = new ServiceClient();
Options clientOptions = new Options();
clientOptions.setTo(new EndpointReference("http://some/service"));
clientOptions.setAction("foo");
clientOptions.setProperty(Constants.Configuration.ENABLE_MTOM, Constants.VALUE_TRUE);
wsClient.setOptions(clientOptions);
OMElement payload;
OMElement response = wsClient.sendReceive(payload);
OMElement dataElement = response.getFirstElement();
OMText binNode = (OMText) dataElement.getFirstOMChild();
DataHandler dataHandler2 = (DataHandler) binNode.getDataHandler();
InputStream stream = dataHandler.getInputStream();

Again the key here is using the Axiom API. It lets us treat the binary data as a text node, and then use the DataHandler to get an InputStream to the data. Again, once you have the InputStream, you can process the data however you need.

Sending attachments with SOAP summary

We've seen how MTOM provides the perfect combination of SOAP standardization and efficiency for transporting binary data within a web service message. We've seen how WSO2 WSAS implements the MTOM specification using Axis2. We've taken a look at how to setup both web service servers and clients to both send and receive MTOM optimized messages. Now we have everything we need for adding binary data to our web services using WSO2 WSAS.

About the Author

Michael Galpin is an architect at eBay in San Jose, CA. He's been developing software since 1998, and holds a degree in mathematics from the California Institute of Technology.