Use case: Testing integration of complex web services

Lets say you have an application which uses an external parties web service. You have no control over the design of the third party service.  In addition its not possible to have a copy of the application.  The organization providing the service will provide a user acceptance version but that is all.  The service wraps something complex, its difficult to setup, and requires many other dependencies to operate correctly.  It would be to expensive an option to have a development instance.  Lets call this service xorg-ws, and our application xorg-integration.

This situation gives the developers of xorg-integration a problem. They want to write integration tests, but a full end to end test is not possible because we don’t have a suitable xorg-ws environment for testing against. Our contract with xorg-ws is in the form of the WSDL definition. We have some understanding of that contract and how to use the data returned. At the very least we would like to test, how our application integrates against the contract.

Conceptually the system might look something like this sketch.  Yea! its pretty rough, so I hesitate to call it UML   In this simple example there is a Computation object that needs to access data from xorg-ws.  Our design hides the xorg system behind a service layer, XorgService.  This is responsible for making calls to the xorg system via whatever means is available.  I have not expanded the details but lets assume it knows how to correctly marshal and un-marshal from a Java-Ws web service configured against the WSDL contract that xorg-ws provides.

Example use of xorg web service

If there is no Integration Test, then what?

In an ideal world we would have a development copy of the xorg-ws and we would write end to end integration tests against that.  As stated in the use case this is not possible so what is the next best thing?  Is it possible to get closer to an integration test?  We could write some tests against the Computation object, and pass it a mock XorgService using something like EasyMock.  We could then assert that the correct calls are made against the XorgService.  Its a good test, but its only testing service layer logic and the wiring.  Any data in the test comes from the test itself when building the mock object.  Its time to think about what we want to test.  One way of looking at things is to think about what is not tested in the mock test.

What is not tested?

1. The service has to marshal the call to the web service.  On occasion this will involve packing some data or formation of a message.
2. Once the call returns it has to unpack and marshal anything returned into the correct objects and types.
3. Web service calls have to be made, proxies used etc.  Over the wire transmission of the web service.
4. The xorg system has to respond to the request, perform any actions required after its own marshalling processes.

From this list there are a few things we are not interested in testing:

• Point 3.  This would be like testing the implementation of Java-WS.  No point in testing something that has tests in its own project.
• Point 4.  This should be tested by the xorg team.

So little remains, points 1 and 2 require tests that prove that data is marshals correctly.  That could be done using units against the marshal routines but given that an XML system like JAXB would be used there may not be much to test.  Ok, so its possible to unit test everything but why do I still feel nervous about the test coverage suggested.  We still do not have an end to end test.  Its my experience that without that your still badly exposed despite high coverage.  What if we could build a cheap xorg-ws.  One that provided a deep mock of the actual application?  Well, it would enable us to link everything together and see the full system operating including all the behavior in xorg-integration.  The only issue, is that the behavior mocked, would be our version, so if we get that wrong, things could still break when the system enters the manual UAT stage against a real xorg-ws.  Lets take a look at what SoapUI could offer, if its very cheap to build it might be the next best thing to a full integration test.

Using SoapUI to deep mock a web service

I have a wish list for a mock that tries to replace a real system in an integration test.

1. The tests should be data driven.  This means tests can be written and changed very easily.  Its important that the mocks are static, and return the same response every time they are called.  If the responses are non deterministic you can’t rely on them, its a test anti-pattern.
2. It is important to be able to capture and generate mock operations from the WSDL files.
3. The result should be deployable in any test integration environment.
4. There should be support for running the mock and tooling with an IDE.

From the documentation on the SoapUI pages it looks like the product will do all of this.  It does much more too.  If your developing your own web services and are in control of both sides of the coin, it has all kinds of useful tooling to decouple development dependency within the team.

For our use case these pages show how it could be done:

Web Service Mocking
Getting Started with Web Service Mocking

There are several points of note:

• Groovy can be used to modify the response.  Its easy to see how groovy could get static response data from files, or even a database.
• There is a command line for running the mock service.  Essential if you want to provide a service for continuous integration.
• SoapUI can generate mocks from a WSDL or WADL for REST.
• The tool is very powerful and includes support for functional testing, load testing and performance testing so could prove useful in other areas of the organisation.

Conclusions

Its become a large article your probably asleep by now, but it boils down to two points.  Yes SoapUI provides excellent test tooling for web services, and could provide rich mocks that are cheap to develop.  No its not a replacement for integration tests against a real live system, but in some cases where this is not possible SoapUI would be the next best thing.  If you do suffer UAT bugs after all this, at least you can add or modify a test case so that it never re-occurs, that is an embarassment I would put effort in to avoid!

                          

The tests against the service should return with configurable and adaptable behavior. They should be quick to run, easy to understand and simple to develop. This might be a solution:

http://www.soapui.org/gettingstarted/mocking.html

Has anyone used this or something similar? Comments welcome.

                          

The spring integration project embodies principles found in the Spring Framework and Enterprise Integration Patterns (Hohpe and Woolf, 2003). The book and project are worth a look for anyone working in that space.

• Spring Integration Project
• Spring Integration in Action, Manning Early Access Program