We've Moved...

The sun has set on Chaotic IT. I am now posting on The SOA Coach blog. Same content, stickier name. Hope you can drop by.

Message to Corporate Dinosaurs: The Meteor Has Landed

65 million years ago something unexpected happened that altered the course of Earth's history forever. Dinosaurs had reached the apex of evolutionary fitness and had ruled over land-based animals for millions of years. They had become indomitable masters of their surroundings. Then, suddenly, in a geological instant, they vanished.

Geologists and paleontologists call this the "K-T event." Whatever happened 65 million years ago was so devastating that it caused an abrupt change in the fossil record, marking the transition between two major geological eras, the Cretatious (K) and Tertiary (T). The K-T event wiped out half of all land-dwelling creatures on the planet, eradicating the strongest, fittest, and most powerful species but sparing many smaller, weaker ones. Among the weaker ones were mammals, which of course eventually gained dominance. What is most curious about the K-T event, or any mass extinction, is why inferior species survive while superior ones perish.

The answer is agility, and the fit often sacrifice it in favor of sheer mass.

The dinosaurs as a species did not turn to dust in an instant. Rather, they gradually died off over a period of 10,000 years or so, most likely of starvation in the wake of severe global climate change. All creatures were subject to these conditions of extreme scarcity, but the dinosaurs were at a distinct disadvantage because their size required them to find comparatively more resources to survive. From an evolutionary standpoint they got careless and allowed their size to compromise their ability to adapt in the face of change. Meanwhile, millions of smaller species thrived because of their much more economical consumption of energy. I believe we are witnessing a similar extinction event in today's business ecosystem.

The untouchable corporate monoliths of the industrial age may be headed toward a similar fate as the dinosaurs, threatened by the global impact of the Internet Era. For these 8000-pound T-Rexes, what used to be uncontested dominance over resource-abundant business territories is withering into a struggle to fend off swarms of new and nimble competitive breeds encroaching from all directions. The digital age has declared open season on traditional business models and has little appreciation for historical aristocracies. It's anyone's game, and everyone is stepping up to compete.

Michael Hugos of CIO magazine blogged about this phenomenon, citing how Kodak, GM, and Ford have sustained deep wounds in the face of new competitive threats. He says of today's corporate heavyweights,

"They are rigid hierarchies through and through; it’s in their genes; it seems they are not and cannot be agile."

If this is indeed the case, then these companies' existence is endangered. Today's business environment increasingly favors organizations that can self-organize quickly and dodge the effects of constant change. Heavyweights cannot maneuver around change, so they become big targets that sustain constant damage. Lacking the ability to adapt, time will eventually erode the pedestals out from underneath them.

Some heavyweights are attempting to inherit agility through acquisition, such as News Corp. via its jaw-dropping purchase of MySpace. Yahoo! and Google (yes, even they are threatened) have strategically acquired web-based social networking prodigies Flickr and YouTube, respectively, to remain a step ahead in the game of evolution.

No industry is immune from this arms race of innovation, though. It is corporate Darwinism on an epic scale. Newspapers, movie companies, record labels, retail outlets, auto makers, schools, restaurants, every business is affected by today's rapid pace of technological change. And many large, market-dominating predators are most at risk because of the sheer magnitude of effort required for them to suddenly do things differently.

I believe we are on the cusp of a massive corporate extinction event similar to the K-T event. The blast has struck and it will take years for the dust to settle. Meanwhile, the chaos has disrupted the economic balance of power, offering new advantages to the small and nimble. Like the dinosaurs, large companies may not perish overnight. But for those not able to adapt, the death knell has already begun to toll.

Long Tail SOA and the Mythology of Reuse

I have been preoccupied for the last several months with traveling and becoming a new dad, but I managed to cobble together an article for the latest issue of SOAWorld magazine. It presents and contrasts two rather opposed SOA strategies--Mainstream SOA and Long Tail SOA--and argues that organizations embracing SOA tend to focus too much energy on building Services for reuse. Highly reusable Services perform business functions that are in high demand, hence the "mainstream" label, but these Services are scarce compared to the accumulated demand for one-off Services. In general, only 20% of a company's Services are highly reusable. Targeting only these neglects the other 80% of the company's Service market. To truly become agile, IT organizations should build Service portfolios at the Long Tail end of the spectrum, because Services do not need to be reusable to be valuable. . .they just need to be available.

http://soa.sys-con.com/read/557348_1.htm

How to Ruin a SOA Program and Bankrupt IT

I spent the first half of last week in New York city at the SOAWorld 2007 conference, where I had the privilege of presenting my thoughts on the ROI of SOA. It was a fun session with a very inquisitive audience. I have received a lot of valuable feedback on the presentation and the supplemental article in SOAWorld magazine and would like to address one topic that has come up frequently, and that is whether simply choosing SOA over point-to-point (P2P) integration is enough to achieve agility.

I explained in my article that a fundamental characteristic of SOA is its linear cost curve and contrasted it with P2P's non-linear cost curve. The "bottom line" is that SOA inherently keeps costs linear, predictable, and scalable over the long haul while the cost of P2P accelerates uncontrollably. The net effect is that service-oriented networks are enterprise assets that appreciate in value as they grow, while P2P networks are depreciating liabilities. SOA, by its nature, produces positive ROI. P2P, by its nature, produces negative ROI. Therefore, if you embrace SOA and forsake P2P you should be set, right? Wrong.

To be sure, taking a stand against traditional P2P integration is a big leap in the right direction. However, it is one thing to say you are service-oriented and quite another to be service-oriented. And if you're talking the talk but not walking the walk you can end up in worse shape than if you had opted against SOA in the first place.

SOA often requires a paradigm shift -- a shift in attitudes and thinking away from project-based, short-term objectives toward enterprise-based, long-term objectives. When the people doing the talking are not the ones doing the walking, the mental shift may not occur everywhere it needs to. This can produce a situation where an organization has obtained sponsorship and funding for an SOA program but lacks the discipline to adhere to its core principles during implementation. When SOA has been promised but the implementers are only comfortable with P2P, the result is often point-based SOA.

From a financial standpoint, point-based SOA combines the short-term cost hikes of SOA with the long-term, accelerating cost accumulation of P2P. What is left is an overly complex network of tightly-coupled services linked together through physical connections and proprietary interfaces. (Imagine a network of web services in which each service can be used by only one consumer.)

Relating this phenomenon back to the Bottom Line analysis, the resulting cost curve (P-SOA) resembles the P2P cost curve. The only difference is that it has been shifted upward by a coefficient that represents the extra infrastructure costs of SOA. In the end, point-based SOA costs more than traditional P2P integration and yields no gain in IT flexibility or agility.

IT organizations that are adept at selling SOA at the podium need to also have the tactical ability to execute on SOA missions. This requires shared vision, passion, and deep commitment to the fundamental principles of SOA at all levels of the IT ranks. Companies that have difficulty producing this cultural unity are at a distinct disadvantage in ever realizing the benefits of SOA. In fact, they may be better off just ignoring the hype and walking away.

In Defense of Bottom Line SOA

Just when I thought I had sold the SOA community on my perspectives on the ROI of SOA, Alastair Bathgate, a fellow tech blogger whom I respect, threw down the gauntlet and took some jabs at my recent whitepaper. Two thoughts filled my head as I read his retort:

1. He's right
2. I'm glad he spoke up

His first point:

"Although, as Marc points out, the cost of connecting all the components in a P2P architecture is undoubtedly high, I wonder if that would ever happen in reality..."

In my paper, I based my ROI analysis on the assumption that total infrastructure connectedness is required to achieve maximum agility. However, it would be foolish to think that every last component in an IT environment would need to be connected to every other in order to produce appreciable return on integration investments. In fact, it may be altogether impossible. However, I do contend that business-critical segments of the infrastructure can be configured into fully connected integration sub-networks.

Bathgate continues:

"[Marc's] target of zero latency...is again appealing, but unlikely to be achieved...even with the most sophisticated SOA."

Again, he's right on the money. No company can ever hope to operate in absolute real-time, even with a perfectly managed service-oriented infrastructure. There are countless forces at play constantly that contribute friction to even the most streamlined business processes. (It is these forces that produce the chaos that is the theme of this blog.) In order to home in on the essence of SOA and P2P, though, I had to eliminate these variables from my equations, otherwise my analysis would have been so muddied up with extraneous data that the fundamental properties of P2P and SOA wouldn't have shined through.

Then he adds:

"...the enterprise is always dreaming up new business processes...which means that maintaining a Service Oriented Architecture can be just as painful as building P2P solutions."

"...the orchestration layer of any SOA needs constant reconfiguring to meet new business requirements."

SOA certainly can be just as painful, or more, than the P2P approach (I assume that by "painful" Mr. Bathgate means costly), but this is a function of the quality of a particular SOA implementation and is not an immutable property of SOA itself. Some SOA initiatives will succeed and some will fail. That's just the way it is, but we shouldn't fault or avoid SOA just because success is not guaranteed.

Also, I agree with Bathgate on the point that SOA is not immune to change. In fact, my ROI model is based on the assumption that service-based networks are subject to at least the same rate of change as P2P networks. The bottom line, though, is that SOA isolates an instance of change to a single location rather than allowing it to rip through the network like a shock wave.

Finaly, Bathgate closes with:

"I think that Marc has written a highly interesting and thought provoking paper that considers the “pure” arguments. He points to a fork in the road where a decision must be made to take the P2P or SOA paths. I just think that in the real world there is some dirty middle ground that we have to accept exists..."

Again, I could not agree more. My ROI model is a starting point for understanding the raw characteristics of SOA as compared to P2P. It is not meant to be construed as a methodology or formula for success. It places SOA and P2P into a vacuum, free from the influences of external, real-world forces, systematically dissects them and analyzes their DNA under a microscope. Indeed, it is a highly purified view of two extreme approaches to enterprise integration. Perhaps overly scientific for many practical purposes, but necessary to reveal their most fundamental characteristics.

Mr. Bathgate has reminded us that enterprise system integration is not a black-or-white affair. It's not all or nothing. It's not P2P or SOA. In reality, every IT organization must strike a balance between the extremes -- a blended strategy that applies the right philosophy to the right context at the right time. Use my ROI model to understand where the boundaries lie and navigate your own path to agility using sound judgment and an understanding of your business that could not possibly come from a paper you downloaded from the Internet.

Gartner Bullish on SOA (Tell Us Something We Don't know!)

Once again, Joe McKendrick out at ZDNet published a very quotable blog article. He cites a few notable sentiments from Gartner research director, L. Frank Kenney, about the outlook of SOA and throws in a few of his own.

Kenney suggests firms "aggressively invest in SOA as it will rapidly become the architectural foundation for virtually every new business-critical application." McKendrick adds that, according to Gartner, 80% of mission-critical operational applications and business processes will be SOA-driven by 2010. (I assume these are IT solutions as opposed to "over-the-counter" ones.)

This message shouldn't shock anyone who has been following Gartner's SOA forecasts over the last few years. But what is unique about McKendrick's article is its cautionary tone against using Gartner's bullish outlook as an excuse to go on an SOA spending spree. He states that "the downside [of SOA initiatives] is that too much is being expected of SOA in too short of a time" and adds a few more quotes from Kenney supporting the notion that it can take a long time, years even, for SOA to show appreciable ROI.

The lesson here is that companies should be very careful not to run up their SOA tabs simply because SOA is becoming big business. SOA is an investment in the future of the company. Like any good investment, SOA will operate in the red for a while before it matures and begins to show real returns. (I've never heard of a passenger jet that paid for itself on its maiden voyage or a brand new office high-rise that broke even before all its floors were built.)

Companies should invest in SOA, but not foolishly. A dollar spent on SOA today is a dollar that won't be seen again until sometime after our next president is sworn in. It's a realization that every SOA stakeholder should be comfortable with. So, by all means, go shopping. But take a list! Know exactly what you're buying and how long it's going to take to pay it off.

(See my recent whitepaper for more of my thoughts on how much SOA really costs.)

Free Whitepaper on the ROI of SOA

One of the most difficult aspects of Service Oriented Architecture (SOA) is demonstrating tangible return on investment (ROI). Many folks concede that SOA is more of a long term value proposition than traditional architectural approaches, but that doesn't do much to lower the blood pressure of the business sponsors footing the bill. SOA programs need strong business backing to be successful, so SOA advocates need to be able to convince the business, and keep them convinced, that it is the right way to go.

I recently wrote a whitepaper that compares the fundamental economics of SOA versus classic point-to-point enterprise architecture. It puts a few simple but powerful concepts and mathematical formulas behind the oft discussed but seldom quantified differences between the two disciplines. My goal was to offer the IT community an objective analysis of the economics of SOA and a simple model for understanding and tracking its ROI over time. These concepts have helped me through many strategy meetings and I hope they can do the same for others. Here is a link to the free download (no registration required):

Bottom Line SOA: The Economics of Agility

All I ask in return for the free download is a bit of constructive feedback, either as a comment on this blog or directly to me at marc@marcrix.com. Also, this paper will be appearing in the June issue of SOAWorld magazine and I will be presenting it in person at the SOAWorld 2007 conference in New York City in late June. I look forward to your comments.