Organizational Shifting

As an engineer, I'm familiar with the fascination the IT world has with emerging decentralizing technologies like SOA and cloud computing.

And, I know how foreign the world of organizational theory, structures, and behavior is to most IT folks...but IT leaders are going to have to become literate in this area if they're going to be effective in matching the new IT to business needs.

The basic reason is that all network technologies trigger massive non-technological shifts. This was true for railroads, telecommunications, and electricity.

John Hagel, John Seely Brown, and Lang Davison recently began a blog entitled "The Big Shift" to discuss how IT is triggering a fundamental shift in organizations from push-oriented to pull-oriented. They've written about the topic before (e.g., "From Push to Pull" in 2005), but the blog entries are perhaps more accessible.

Is push-pull a more catalyzing contrast than exploit-explore? than complex-complicated?

That's probably the wrong question...more relevant for systems engineers and architects is do you understand how these concepts highlight key aspects of this shift?

The Power of Asymmetric Knowledge

This recent post by Joshua Porter at Bokardo comparing a symmetric social network (Facebook) with one that enables asymmetric social networks (Twitter) is worth reading.

Though the authors don't say it quite this way, my snap reaction was:

Asymmetric relationships are more powerful because they enable much more agile and adaptable in matching information to a decision context.

Symmetric relationships require a much richer shared context...which allows for deeper exploration of a specific context, but is much less agile/adaptable.

Porter links to James Governor's Monkchips blog on the topic which states "Asymmetric Follow is a core pattern of Web 2.0"...which sounds about right.

Why Services Are Hard...

...or at least one reason why... :-)

I ran across a quote from "Who Says Elephants Can't Dance?", Lou Gerstner's book about how he changed IBM:

“I came to see, in my time at IBM, that culture isn't just one aspect of the game—it is the game. In the end, an organization is nothing more than the collective capacity of its people to create value....I have a theory about how culture emerges and evolves in large institutions: Successful institutions almost always develop strong cultures that reinforce those elements that make the institution great. They reflect the environment from which they emerged. When that environment shifts, it is very hard for the culture to change. In fact, it becomes an enormous impediment to the institution’s ability to adapt.”

I suppose this has always been true, but was less noticeable when organizations were smaller and the world changed more slowly.

Today, most large organizations are fundamentally complex transaction-oriented information ecosystems, or they specialize in the design, development, or maintenance of systems within such ecosystems.

Service orientation (not technology per se) transforms a transaction-oriented information ecosystem into a hybrid that both (a) maintains its transaction processing capabilities and characteristics, and (b) exposes services, enables the relatively unstructured bottom-up/middle-out wiring together of services, and creates innovative new information transformation capabilities (along with new management capabilities).

The culture that produces transaction-processing excellence tends to focus on such qualities as predictability, reliability, and lack of deviance from a standard, and tends toward a bureaucratic organization with well-defined roles, rights, responsibilities, and processes to create and maintain that excellence.

Such a culture is in most ways poorly suited to thrive in a service-oriented ecosystem with its dynamic, emergent, and relatively unstructured exploration of potential information transformation capabilities...many of which are also potentially disruptive.

If you work in a large organization, you may have seen a "deer in the headlights" response to this new "dance" challenge, which seems to be nearly universal these days for elephants since large organizations today are usually information-intensive.

If it's hard for the elephants, it seems even harder for the elephant keepers...who don't know how to do anything but design, develop, and maintain elephants.

And, what makes this especially difficult is that it seems to be as much about turning elephants into composable "ropes, spears, fans, etc." as it is about getting the elephant to move learn new steps.

Seems like more than a minor identity crisis is emerging...

Self-Creation and Knowledge

For the past couple of decades, the term "autopoiesis" has been popular in certain scientific and organizational circles. Its literal meaning ("self-creation") hints at the possibility of some sort of emergent behavior that results in "information for free." This diagram from emergentfool.com is typical.

From a philosophical perspective the term is potentially problematic since it seems to imply that something can emerge from nothing...that there are uncaused effects (or that the assertion that all effects have causes is false). A part of the brilliance of Cynefin is that it largely dodges the ontological issue and focuses on the epistemological one.

Regardless, autopoiesis has been used as a concept to explore how organizations know and how they translate knowledge into action. "Exploring the Foundations of Organisational Knowledge" (Vines, Hall,and Naismith) is a recent paper that takes this approach.

They have a nice discussion of Karl Popper's "three worlds", and apply evolutionary theory to KM in a metaphorical fashion...though I was a bit confused by the conflation of evolution's "random mutation" with an organization's individual intelligent agents engaged in purposeful activity.

As a rough metaphor, there's a lot I like about highlighting the emergent aspect of knowledge. However, I'm not sure how useful it is when it comes to specific decisions and actions. Having slogged through Stuart Kauffman's "Origins of Order" a decade ago, I understand the attraction. And, I'm intrigued by ongoing laboratory efforts reproduce the origin of life and the emergence of complex subsystems in living organisms.

Agent-based approaches that decentralize attractor/constraint-oriented behavior clearly have value. But I wonder whether we've hit "peak emergence" (ala "peak oil") in our understanding of this phenomena...we don't seem to be much closer to a good understanding of the underlying causes of emergent information and behavior than we were before the topic became a staple of popular science 20+ years ago.

Whether we have or not, it seems inevitable that the rate of new emergent behavior will continue to accelerate for the foreseeable future as connectivity and interoperability continue to increase at warp speed.

Side Trails from Empirical Knowing

A couple of references from "Imperial Secrets" I found interesting:

• Persistent Surveillance and Its Implications for the Common Operating Picture" - Kelley's critique in "Imperial Secrets" was of such statements as "Once achieved, persistent ISR coverage will, in theory, deny the adversary sanctuary, enabling coherent decision making and action with reduced risk." I'm reminded of Dave Snowden's highlighting the fact that we filter ~95% of the raw data that hits our senses, and that the sense we make of what gets through our filters is shaped by a variety of factors, including which narrative fragments have been recently activated. For a nice summary of the limits/biases of individual sensemaking, see "The Psychology of Intelligence Analysis." A Panopticon-style capability would dramatically improve decision making in some ways...but it would also shift attention to the limits imposed by basic cognitive and social constraints on translating data into action, and, perhaps more important, would have significant and unpredictable effects on the activities of those under surveillance. Overall, it would seem that the net result is a more Complex space where prediction becomes more, not less, difficult.


• Critical Thinking and Intelligence Analysis - Another NDIC publication that thoughtfully considers the art and science of transforming data into decisions. I especially liked Table 7 (Analysis: Past, Present, and Future), which highlights various aspects of the shift from in-depth slice/dice analysis to agile sensemaking.

The challenge of knowing irregular, ambiguous, and rapidly changing threats has resulted in an emphasis on capabilities that are more exploratory/agile than exploitative/structured. A couple of articles in Joint Forces Quarterly (Q1 2009) highlight two aspects to this trend. Both reflect a maturing understanding of the limits of exploratory and exploitative capabilities.

• "Hybrid Warfare and Challenges" (Frank Hoffman, pp. 34-39) - "hybrid warfare"is the term associated with what seems to be the next phase of an ongoing transformation to make the U.S. military more agile. It emphasizes the need to operate in multiple "modes" simultaneously to effectively engage an opponent using conventional, irregular, and terrorist tactics in a coordinated fashion. Hoffman acknowledges the difficulty of such an endeavor...one that is similar in some ways to that discussed by Cash, Earl, and Morrison in the Nov 2008 HBR, which I discussed here.
• "Systems versus Classical Approach to Warfare" (Milan Vego, pp. 40-48) is a good summary of the concerns that have been raised about NCW, EBO, and SOD. Attempts to apply Complicated (expert, analytical) tools to a Complicated domain have, unsurprisingly, provoked reactions ranging from skepticism to hostility among many students of warfare. If you're unfamiliar with recent critiques of Systems of Systems Analysis, Systems Thinking, and other mechanistic approaches to warfare, you might find this article interesting. The bottom line is "Uncertainty in war is not only a result of a lack of information, but is also often caused by what one does not comprehend in a given situation." Analytical approaches (crafted for Complicated situations) may actually degrade the ability to comprehend in Complex situations.

Finally, John Arquilla has been writing about network/swarming tactics for years. "The Coming Swarm" is a recent summary of the current situation...it reminded me of Israel's approach to dealing with swarm terrorism.

Empirical Knowing

A recent book by Patrick Kelley explores how empires know. Entitled "Imperial Secrets: Remapping The Mind of Empire", it's a fascinating and non-traditional exploration of the challenges and limitations empires face in translating data into action.

Although most organizations are not empires, many dominate a specific context in ways that create the same kind of challenges in knowing. So, it's worth at least a look if you're a student of organizational behavior.

Kelley focuses on three empires facing challenges in knowing: Rome in the first and second century AD, Ottoman Turkey, and Britain in India. He draws lessons from specific narratives, including:

• As an empire comes to dominate its time and place, it struggles to detect signals that are weak or don't fit the frames that made the empire successful. An additional complication is that an empire's dominant presence warps the information space it inhabits.


• Informal networks are often critical and unrecognized in filling in an empire's blind spots.


• Empires are occasionally surprised when actions it perceives as being ordinary provoke an extraordinary response.

Although much of this will be familiar to students of epistemological approaches to organizational sensemaking (e.g., Karl Weick), the narratives provide a framework for a nuanced explication of the concepts. And, Kelley is a thoughtful analyst. Here's a few excerpts I liked:

• "I wish to contest the unidirectional read of how 'knowing' works; i.e., that an observing subject gains ever more knowledge of some given object and consequently, power over the latter accrues to the former."
• "I propose that the Panopticon actually operates in reverse in the imperial context....empires will nearly always operate at an information deficit in relation to their subjects....The overseer is there for everyone to observe—what the empire believes, does, wants, and will do is laid out in imperial media, legal codes and judicial decisions, the conduct of its agents, and the architectural and scientific 'performances' of power—all in contrast to the 'inscrutable oriental' who resists observation physically, linguistically, and epistemologically in his capacity rather than inability to 'imagine that....' "
• "...imperial policy makers faced the problems of 'slippery' knowledge—data points collected, analyzed and presented in a context different from where they originally resided by virtue of imperial process."
• "Empire consists of a certain set of patterns that characterize the meaning and function of physical reality—Empire is information, which may almost sound like a definition, but my intent is rhetorical emphasis. Consequently, the management of information—disseminating the preferred patterns, and identifying competing patterns for elimination—becomes the core function of imperial administration."
• "The risk of drawing virtual lines and imposing order on the grand confusion of reality is not simply that our lines may be imperfect, our approximation inexact. Rather, these virtual creations can become so comfortable and accessible that we come to inhabit them as real, producing 'knowledge' about things and places that don’t actually exist. Reality as such then enters into the 'species of alterity'; it literally becomes something foreign and other—opaque, if not irrelevant, to our attempts to understand it."
• "... the strategic information most fundamental to a successful imperial order—information about beliefs, identity, authority and allegiance—does not appear as some 'golden nugget' at the end of an operational or tactical tasking order. Rather, it circulates along networks of exchange, akin to an economy in which information serves as currency."
• "The unique intelligence problem of empire, in contrast, is precisely that such distinctions [foreign-domestic] do not exist—or rather that they exist in over-abundance, with inside/outside fractures splintering and overlapping ad infinitum within and between spaces, communities and individuals under the imperial umbrella."
• "If absolute power corrupts absolutely, it also tends to isolate completely—twin tendencies any executive authority risks as it ascends to the heights of imperial power. Bureaucracies rise in tandem with that isolation, providing the intellectual equivalent of walls and gates; but subverting that intellectual structure by act of will can prove nearly as impossible as escaping from the physical walls for reasons of either status or security. "
• "Modern strategic discourse is full of discussions about decision cycles and getting inside an opponent’s OODA loop (Orient, Observe, Decide, Act). Less well explored is how to get inside or engage another’s time, as such, which may be moving at a different rate and in a different direction."
• "A key aspect of the British performance, and an enduring problem of imperial intelligence, I suggest, is not to correctly 'know' foreign minds, but to read backward, and accurately perceive how those foreign minds come to understand the apparently familiar and domestic."
• "As I have argued throughout this text, imperial intelligence is less a problem of determining the truth or falsehood of specific facts, and more an issue of negotiating how truth is constituted and what 'knowing' means. These issues leap to the fore in the contest over education, which is fundamentally a fight to frame how meaning can and will be constituted in both the past and future."
• "None of these [narrative] products would suitably respond to any conceivable state information request, and certainly would not fit into the almost infinitely replicated 'intelligence cycle' model, but they might well answer information requirements. The rhizomatic nature of information, particularly in an imperial environment, indicates that there will nearly always be connections and contexts not immediately evident to any decision-maker posing specific questions....Better still, they are quite obvious in what they leave out, and in this are less deceptive than traditional maps."
• "Th e language and image of empire is universally visible and available to its nominal subjects, while the reverse is not usually the case."
• "...the rise of imperial power creates the demand for a whole host of new government functions and institutions, which cannot be created, except with great difficulty and foresight, ex nihilo. Far easier, and more common, is to simply adapt existing institutions to new roles..."
• "Modern Americans have come to believe that the norms and values encapsulated in their form of government and their ways of conducting foreign relations are the birthright and open options for men everywhere. In accordance with this persuasion there simply can be no 'others.' " [Adda Bozeman quote]
• "...new information—in order to make sense—must fit into a narrative, a history, and this history amplifies the problem of the other/same dichotomy. 'As distinct from the present, the past is alien, exotic, or strange; as continuous with it, this past is familiar, recognizable and potentially fully knowable.' [Hayden White quote] "
• "Empires thrive despite, rather than because of, their information institutions."
• "In early days, say just after you have sacked Carthage or defeated the Nawab of Bengal, you might know very little, but you also know what you don’t know, i.e. basically everything. This makes the 'known unknowns' a very large category, but also makes the more dangerous 'unknown unknowns' very small. As imperial experience progresses, as administrators learn the languages and jurists incorporate local legal principles, the 'known known' flourishes in a rhizomatic way...For as 'known unknown' shrinks with every census and cartographic survey, the 'unknown unknown' consequently expands proportionately. While the analogy may be inexact, imperial power becomes more expert over time, and consequently more subject to the patterning and heuristic biases associated with the 'Paradox of Expertise.' "
• "My general assessment has been that empires are always at an information deficit—telling more than they hear—and the deficit over time becomes associated with a lost capacity to listen."
• "A perspective which presumes the role of an objective machine loses twice over by deliberately abjuring to assimilate the specifically human factors that only humans can collect, while making inevitable bias more difficult to detect in bureaucratically neutral discourse."

As someone who is more critical realist than postmodern, I think Kelley occasionally overstates his case. But, given the growing importance of Complex contexts and the growing number of "virtual empires", he may be right to do so.

Complex Knowing

A few miscellaneous items from the past couple of weeks that highlight the limits to and evolution of analytical approaches to decision making and decision support:

• "When Knowledge Management Hurts" - This short article by London Business School professor Freek Vermeulen discusses a study asserting that incorporating formal knowledge into decision making can actually result in worse decisions. I don't have time to discuss the actual study (link in article), but it's worth looking at if you are a KM practitioner.
• Unintended consequences are inevitable when services are made mashable. This slashdot posting discusses Google having to shut down free SMS message after a popular iPhone app overwhelmed Google's servers.
• "The Rise of The Social Network System" - a short article on social hyperconnectivity...it's about linking contexts. I'm always slightly surprised by articles like this in light of how much science fiction has explored this area in the past few years. Also, Tim O'Reilly's discussion of the article.
• Lots has been written recently about the decline of newsprint. Clay Shirky has a nice overview of the rise and decline of this particular asynchronous hardcopy Few-to-Many media channel. It's nice to see some historical perspective in something as ephemeral as a blog post.
• Finally, Deloitte has posted a study of mining safety in South Africa that was done using Cynefin. If you've found this framework a bit abstract, this study might help you better understand one way it can be used.