Service Oriented Architectures: The Real Deal

Service oriented architectures, or SOAs, are just another in a long line of software systems engineering means of dealing with complexity. In software, the trail of abstraction goes from basic bits through functions, objects, modules, components, until today where we are trying to abstract entire systems. The history of this increasing abstraction is fascinating and perhaps worth talking about, but I won't here, for now. I want to talk about where we are today and what I believe has really happened with our notion of abstraction and how that has affected entire industries (okay, so that sounds grand and sweeping, but I believe it's not far off the mark).

Service orientation is the view that things provided by software and hardware can be made available as a "service" to be consumed by those who need whatever capability the service offers. Services are not necessarily applications but can be as simple as some function that is made available for other uses beyond the original intended use. We have identified categories of services that include basic or simple services (think of a function that simply looks up and returns an address) and composite services. A composite service can be thought of as occurring in 2 flavors (a bit of an oversimplification); orchestrated and otherwise. Orchestrated services are services that require the use of multiple other services in some sequence that must be adhered to. The term orchestration indicates that there is some entity outside of the service that directs the sequencing and ensures that the service completes successfully or not. Unorchestrated composite services are services requiring use of other services but no special sequencing is required. An example of an orchestrated service might be the end-to-end process of acquiring a loan. An example of an unorchestrated service might be the delivery of a complete customer profile (used in making a loan determination). This is just a brief highlight of some service basics. I mention this because it's important to understand how I think about services for the rest of this to make any sense.

The notion of services seems to have come about with the increasing complexity of IT systems which we knew about but weren't really overly concerned with until...businesses started having to deal with systems other than their own. This naturally occurs when there is a significant amount of merger and acquisition activity and when a need is identified for something that already exists but is not readily available. Merger and acquisitions create businesses with multiple IT systems that are not in general compatible. Leveraging investment in these systems has taken the form of identifying ways to use the system capabilities without a significant expenditure on rework. If I can take existing capabilities and make them available without completely rewriting the original software that implements that capability, then I have used resources to make that capability available, but it's less and different resources than I would have used if I had to start from scratch.

This is nothing new, it's just that we have started thinking about how to really deal with this complexity in a way that doesn't involve enormous outlays of time and other resources. And yet, it's still hard and many organizations that undertake to "implement" a SOA, seem to fail or give up. Why? Much has been written about the why but I want to state my observations about the "why." So what follows are my thoughts on SOA success or failure.

1. SOA is not a product! There are companies that sell SOA technologies but these are not SOA; they are enablers of SOA, perhaps, but the market hype around them leads you to believe they do everything. NOT SO!
2. SOA isn't for everyone. If you have several incompatible legacy systems, applications, and organizations offering different types of capabilities and you need to make them appear as a somewhat unified whole, then you should probably consider taking a SOA approach. Otherwise, it remains to be seen that the investment will be worth the ensuing chaos.
3. SOA is not just a technical solution. This is the biggie from my experience. Technically, there are some hard things to do when implementing a service oriented approach. However, they are not things that are entirely new. In fact, many of them have been done many times over in different ways. The real challenge in SOA is that it will nearly always incur a significant investment in social change within the participating organizations. This social change cannot be mandated from the senior executive level nor can it be entirely effected by a grassroots effort from the lower levels. The change must occur as a natural change to the way the entire organization thinks about itself and in the way the various parts interact with each other. This is significant! Traditional IT organizations may well see this as a threat to their very existence. Middle management may see it as a loss of control or power and senior level management may believe that it will be possible by just saying it will be so. The end result is that the effort will fail because expectations and understanding follow all of the things about human nature that make anything involving more than 1 human, difficult. This failure is important to understand because at all levels of involvement, SOA will be perceived as not useful since we tried and it didn't work. This is something that I want to explore in other posts, even though it's not overly new or innovative. It's still interesting.
4. SOA isn't the end. We are still abstracting and thinking about how to reduce overall costs in using software and the hardware that makes software possible. The most current trend is to go beyond software services and consider everything as a service. Thus, we have Software As A Service (SaaS), Platform As A Service (PaaS), and Infrastructure As A Service (IaaS). These are loosely grouped together and termed the "cloud." In each of these, the level of abstraction looks at software, hardware, and support as something that is offerable (not a great word) as a service. This means that I no longer have to consider spending small budgets on equipment and infrastructure that I can now outsource. Beware, though. This approach is not to be taken lightly and serious organizational consideration should be put into understanding the tradeoffs and risk with taking this approach.

None of this is groundbreaking and as I mentioned, there are copious quantities of publications on the subject. What I believe, though, is that SOA is the first in a line of abstractions where we as engineers cannot be oblivious to the "people" issues that surround the technical work. I also believe there are integrity and ethical issues that surround this work that we as engineers must step up to and not just peddle a technical solution because it's sexy technology. Pitching SOA as the answer to all problems is something at which marketing has excelled. As engineers, we haven't stepped up to answer this blatant nonsense with anything other than just sword rattling. We need to do a better job of answering incorrect marketing claims and letting those without the knowledge drive the industry. This means we have to educate those making the claims, we have to speak out loudly as a cohesive group when these claims are made, we need to better educate future engineers to taking a contrary stand when necessary, and we need to make the average engineer more articulate.

Feel free to disagree. These are my opinions and observations from having worked in this area for the last 10 years and having seen the progression since I started writing software in 1976.