Chaotic Innovation: Give Mavericks the Keys to the Castle

This post is inspired by a recent article by Joe McKendrick, one of the more fair and balanced SOA bloggers in the 'sphere, entitled, "Is 'rogue' IT always a bad thing?" He points out that, on occasion, employees outside the ordained chain of IT command are guilty of developing ad hoc technology solutions that do not conform to prevailing corporate standards. In the two examples he provides, the rogue solutions provided clear benefits to the business. However, recognizing that such vigilantism by maverick techies runs the risk of undermining important governance controls, he asks his readers whether IT departments should crack down on this behavior. This question cuts to the heart of my theme of Chaotic IT, so I'll proudly offer my take, arguing that IT should not only allow it but encourage and reward it.

First, we must understand that businesses are complex systems; each is composed of myriad independent "parts" (employees, information systems, teams, customers, partners, and so on) that work together to meet business goals. What makes them complex is their endless variety of parts (e.g., no two employees are the same) and the sophisticated web of relationships they make up. This inherent complexity makes the behavior of the business largely unpredictable, and generally uncontrollable by those relatively few parts in leadership positions (just as ant colonies or bee swarms are not controlled by their respective queens). When we are at peace with this, we are at peace with the fact that no one individual, or group of individuals, runs our businesses--we all do, collectively.

One important business theme of the modern era is agility. We must innovate and adapt faster than ever before because, if we don't, our competitors--or a twenty-year-old with a wild hair and a Ruby on Rails book--will. So how do we make our businesses more fluid, adaptive, responsive? It would be great if the answer came in a box. Then we could simply buy it, plug it in, go to the beach, and giggle into our Treos as we watch our stock blow the roof off of Wall Street. But, alas, there is no shrink-wrapped silver bullet. (Any CIO that has invested in SOA technology should know this first hand by now.) The good news, though, is that many of the resources and infrastructure companies need to become agile are already baked into their businesses. They just need to be discovered and exploited.

Some notable thought leaders lately have put forth the concept of initiating "change from within" or "inside-out" transformation to achieve new levels of competitive advantage. While the terminology differs slightly among them, the idea is consistent: that the solutions to a company's strategic innovation challenges already exist within the organization, most likely within the minds of employees down in the tranches, not the leaders at the helm. Whereas the leaders may have a mental picture of the solution and an inclination to spark a project to bring it to fruition, employees at the front lines may already possess finished, tested products that are ready to put before real customers. It stands to reason that if this is indeed the case, companies are better off seeking out this innate knowledge and cultivating it rather than attempting to invent it from scratch. Tim O'Reilly calls these key individuals "alpha geeks" that exist at the "edges" of their companies. Malcom Gladwell, in his book The Tipping Point, calls them "Mavens." Richard Tanner Pascale and Jerry Sternin, in a May 2005 Harvard Business Review article, call them "secret change agents" and "positive deviants." (It is this last term that, to me, resonates most closely with McKendrick's notion of "rogue" engineers.) These authors regard individuals outside mainstream IT as critical to the success of corporate change initiatives, even though the knee-jerk reaction to their efforts often is to apply negative labels to them, such as "deviant," "rogue," "vigilante," "subversive," or "illegal." These individuals, while they seldom occupy positions of high authority, have a deep understanding of both technology and the business drivers that fuel innovation. What's more, they exist at all levels of the organization and can significantly outnumber the mainstream leaders. It is as if every company has a secret underground reservoir of extreme innovation talent that, when tapped, can bring about significant positive changes in agility levels. By leveraging the efforts of these so called mavericks, a company, at the very least, is able to cast a wider net over its sea of innovation possibilities.

Finally, I would like to add my own point of view to this topic--the chaotic take. As I mentioned before, businesses are complex systems. As such, they behave much more like living organisms than machines and are constantly influenced by forces beyond anyone's direct control. (See my previous post entitled Snowball Oriented Architecture in which I introduced the concept of corporate physics. I'll also expand on this in more detail in future posts.) Since businesses exhibit the fundamental characteristics of living things, I look to the natural world for clues about how businesses--and their IT departments--may operate more competitively. One observation I have made is that living things--all of them-- are inherently agile. If we learn what makes them agile perhaps we can apply the same principles to our businesses.

From cells to bacteria to people to herds to flocks to swarms to entire ecosystems, all living systems, like businesses, must undergo constant transformation in response to changes in their environments for the sake of maximizing their odds of survival. It's a game of evolution, no less real for corporations than it is for Galapagos finches or the influenza virus. We all play by the same rules.

So what makes living things naturally agile? It is simply the ability of their constituent parts to rapidly self-organize in the face of some external threat. This is a two step process. First, the threat must be detected quickly. Second, a positive transformation must take place quickly to counteract the threat. The first step is made possible by the awareness of each part to its immediate environment, and the second by its ability to react without first obtaining "permission." For example, when a person touches a hot burner on the stove, sensory cells in the hand register the threat and react immediately by sending an electrical impulse to the brain. Also, in order for evolution to work, individual strands of DNA must be allowed to mutate on their own, without direction from the brain or other external source. These small, local deviations from the norm are then detected by nearby elements and propagated if beneficial or rejected if not. This cycle of "selection" continues until all components have been transformed or the variation has been eradicated, depending on its overall usefulness. The important things to remember are that 1) adaptive changes are nearly always initiated by components at the edges of complex systems, without the knowledge of a central source of authority and 2) that positive change sweeps through these complex systems from the bottom up.

So, "rogue" initiatives are healthy, natural processes that foster a spirit of constant experiementation and reinvention within our businesses, which is crucial for building agility. Sometimes the results of this experimentation will be beneficial, sometimes not, which is why governance models should incorporate feedback mechanisms to discover and evaluate the "fitness" of innovations that flow from the bottom up. In my opinion, this practice is much more contructive than cracking down on rigid policies that keep the flow of creativity traveling down a one way street from the top down. So empower the mavericks, turn them into sensory receptors and change agents for the business, and let them know that the leaders are listening.

I didn't originally intend for this post to be so lengthy, but I believe the decision to allow or disallow "rogue" innovations will have a critical impact on most companies' quest for agility. It may seem counterintuitive to grant employees the autonomy to act according to their own judgment based on cues perceived only by them. In fact, some would probably say that it invites chaos into the business. My point exactly.