Service-oriented architectures (SOA) are all the rage. Instead of tightly bound interfaces that are subject to constant change and possible breakage every time changes are made or new functionality is introduced, SOA promises loosely-coupled connections that allow unforeseen applications to take advantage of ever-expanding capabilities.
From an enterprise architect's point of view, you gotta love it.
Yet from a tester's point of view, this is scary stuff.
For one thing, services have no user interface at all. Typically encoded XML messages being shipped around by various transports, they interface with other applications but not with end users. Because they are headless (this evokes images of the headless horseman -- also scary) they are hidden from testers.
Granted, services can be tested indirectly by exercising the provider and consumer applications -- those that enable or use the services -- but this approach comes too late in the development cycle and is ineffective in isolating the issue at its source. An incorrect or failed result could be in at least five places: the provider application, the encoding, the transport, the message content or the consumer application.
Another drawback to testing through the applications is that the promise of SOA, especially as it manifests in Web services, is that it may be used by applications yet to be developed or by consumers outside the enterprise. In other words, it is boundless.
Headless.. Boundless... Yikes!
Build or Buy
Companies that are rushing headlong into SOA are forced into one of four choices. The first we have already mentioned -- let the testers test the content indirectly through the providing or consuming applications. But, as already pointed out, this comes too late, covers too little and is ineffective at pinpointing problems.
Another option is to buy SOA-specific testing tools. As you would expect, products are emerging to enable developers to exercise service infrastructures, and this is all good. The problem is that these are focused on the implementation and not the content. What is missing is the input from the business side, and these tools are too technical for most testers and users to use.
For example, a message may be sent from an application that processes insurance policy applications to an underwriting application that assesses the risk and returns a response, either rejecting the application or accepting it, then pricing the premium and defining any endorsements or limitations. Development could assure that the message was properly sent and received in the correct format, but would not know whether the policy should have been accepted or rejected, or what the terms should be.
And while the testers could apply for policies and verify the response, they would not know whether an improper result was caused because the content in the message had missing or incorrect parameters; the response was intended for a different policy request altogether, or the policy application or underwriting business logic was wrong... or what. And of course, if their company decided to expose its underwriting services to independent brokers, they would not have access -- or time -- to test those applications as well.
Another option is for the development group to write a user interface specifically for the testers. This has the benefit of moving testers back into the development cycle to test earlier, allowing them to contribute their domain expertise to the content, and also for helping them pinpoint the source of any problems within the infrastructure itself.
The downside, though, is that it places another burden on developers to create and maintain yet another interface. For programmers already crushed by crazy schedules, writing code for testers is likely to come at the end of the line. And the interface itself could introduce issues, of course.
The last and best option is a fully integrated testing solution that enables testers to test at each point and end-to-end. This would include these key capabilities:
Creating and sending a message to a provider application;
Receiving and verifying a message to a consumer application;
Either sending or receiving messages to or from files when the applications aren't available or to isolate the infrastructure, and
Testing end-to-end through the user interface of the provider and consumer applications and the messages in between.
The benefit of this package of functionality is that it allows testers to test both in isolation and in integration, and to participate before applications are ready, as well as after. This building-block approach will enable them to reuse test cases in each combination, thus pinpointing issues early and specifically. Best of all, it exploits their domain expertise throughout the entire services architecture, making them a partner of development instead of another customer.
No matter which option you choose, the best advice is that doing something is better than doing nothing. You can't revolutionize your enterprise architecture without making radical changes in your testing technology. Otherwise, you may end up out of luck.
IE 7
Firefox 2.0
