SOA Generations?

In the commentary section of recent post on causes of IT complexity, Roger Sessions (ObjectWatch) described 3 eras of Enterprise Architecting:

• Need to align IT and business - Zachman
• Need a process - FEA, TOGAF
• Need something thinner - VPEC-T, AEA, SIP

This sounds a lot like the SOA discussions I've seen recently...with most of the current focus being on the pre-process phase (e.g., how to carve up the objects in "SOA-space" so that we can begin to talk about a SOA-centric process). Or maybe I'm just seeing connections where there really are none... :-)

Does NCW Include Social-Cognitive?

After 3+ years of reading NCW literature, I'm starting to wonder if I just imagined that "The Implementation of Network Centric Warfare" has diagrams delineating four domains: Information, Social, Cognitive, and Physical.

Much of what is written either ignores the Social and Cognitive, or decries its absence. The latest item I've run ("Unintended Consequences of the Network Centric Decision Making Model") across falls into the latter category. It's actually a couple of years old, but it's a good critique of why NCW must include the Social and Cognitive domains. If you're still unconvinced, go read the various online books by Garstka and/or Alberts which provide a more abstract rationale (e.g. "Understanding Command and Control").

Communities of Exploration

Dion Hinchcliffe has a new post discussing "online customer communities", which triggered these thoughts:

• It seems like there's two broad online community types: top-down and bottom-up.
• The top-down types are often associated with formal organizations that have a specific mission and associated goals. The two types I think of (often associated with Knowledge Management) are Communities of Practice (cut across multiple organizations, but the Practice tends to be a formal organizational function and usually has a formal body of knowledge) and Communities of Interest (see, for example, how the US DoD has defined this).
• The bottom-up types are less well-defined, but Dion describes 3 broad categories: Social Networks, Grassroots, and Customer. I realize that Dion calls Customer "top-down", but I'm using "top-down" in the governance sense of the word. Since Customers are by definition not governable in a traditional sense, I'm calling them bottom-up...but they could eventually evolve into something that's a mixture of both.
• Finally, it would seem that both types (bottom-up and top-down) are struggling with structure.
  
• The top-down communities have too much structure to achieve the adaptability and agility organizations desire. So, they're introducing bottom-up concepts (e.g., tagging) and hoping that will do it. I personally think these concepts have some value, but that the real issues revolve around agile/adaptable governance and what might be called composable community objects (process fragments, data/info objects, etc.), not some "emergence magic."
• The bottom-up communities have too little structure to reliably allow coherent action toward a specific goal. Dion's post offers a number of helpful insights into this issue with some interesting suggestions. But, this feels like a topic that's still in the forming stage.
  
• I'm a little uneasy with the Deloitte SlideShare that proposes a "tribalism" metaphor. I like the implication of adaptability and shared purpose. But, "tribalism" also seems to imply a structure (governance and community objects) that is too static for most online communities.
• How soft issues like trust and identity interact with structural issues is still very unclear...but since both areas are important, how they interact/relate seems like an important question that should yield valuable insights.

I like the Exploitation-Exploration contrast, so I wonder if the right balance of top-down and bottom-up might be called a "Community of Exploration." Regardless, I don't expect to see a mature understanding of this concept in the near future.

Whither ESBs?

I've used ESBs only as a workflow/messaging infrastructure for small-scale prototypes, and haven't thought much about how ESBs fit into the rapidly evolving web-oriented terrain.

So, I was intrigued by Joe McKendrick's latest post "Enterprise Service Busted", in which he summarizes a recent dust-up about the place & value of ESBs. I really learn a lot from these periodic spats since they seem to occur when concepts surrounding a key technology have matured to the point that a robust understanding is emerging (the last one of these I saw was around REST/WOA a few months back).

In reviewing the genealogy of Joe's post, I came across a blogger I had not read before. Todd Biske, in this post from October 2006, has a list of 18 functions that should be external to a web service. He then maps those functions into 3 overlapping spaces (software infrastructure, networking & security, and system management) and places various vendors/products on the map. One of the most interesting aspects of the emerging hyperconnectivity is why/how IT should be partitioned and the implications the resulting structure has for the developer, operator, and user. This is one of the better commentaries I've seen. Highly recommended.

Here's some of the key forums in the genealogy:

• Joe's blog
• David Linthicum's blog
• SearchSOA
• Todd Biske's blog

Constructing Meaning

I try to spend a few hours at the library every 3-4 months to skim several periodicals I'm interested in. Since I've let my Harvard Business Review subscription lapse, it's on the list.

A couple of HBR articles I ran across last week in my latest skimming prompted this post (these articles are from over a year ago, since I've been fairly busy the past 18-24 months).

The first is from May 2007 ("Inner Work Life", Amabile & Kramer). The authors divide the "inner life" into Perceptions, Emotions, and Motivation. Most of my interest has been in the arena of perceptions. This article reminded me that I probably tend to underappreciate the centrality of emotions and motivation. I suppose I'm no different from most folks in that I assume most people are more like me than they really are. Those differences may be more difficult to understand and bridge when the differences are grounded mostly in emotions and/or motivation...perhaps because emotions and motivation tend to be deeply entwined with key personal narratives and the mystical/mythical terrain of innate personality structure.

I suppose that I need to be more aware of the potential danger and difficulty of treading on emotive/motivational issues. We've all seen managers and organizations that were amazingly tone-deaf on some issue. However, this landscape also offers tremendous opportunity (i.e., the intense devotion of some Apple users). See the article for an in-depth discussion.

The second article is "Promise-Based Management" (Sull & Spinosa) from April 2007. It has a box entitled "A Primer on Speech Act Theory" which discusses how speech and actions are related. My initial reaction was to recall Weick's "how can I know what I mean until I see what I say?"...a question that calls attention to the degree to which we talk/act ourselves into meaning. This topic has a way of generating lots of heat w/ little light, so I'll limit myself to a few comments:

• We probably expect (and often demand) more agreement than is required for coherent decisions and action. Since this blog is not about philosophy/religion, I'll just note in passing that we seem to have an innate need for this that often results in unnecessary conflict...though separating this from some folk's apparent love of conflict may be difficult... :-)
• The descriptive-active spectrum of statements is mentioned. I'm not sure what to make of this. While descriptive statements may be purely aesthetic, even they seem to often have strong undercurrents of action. I guess I'm wondering if even the purest of descriptive statements aren't usually grounded in some sort of telos...ok, enough philosophy.
• Over the past few centuries there's been a whipsawing between the philosophical poles of "it's all objective & knowable" and "it's all subjective and unknowable" that continues to reverberate. Most people tend to act (whether they admit it) as if certain things are objective, knowable, and true. However, most of us probably spend more time arguing about "what's real" than is really needed (again, see the first bullet).

In a late modern (or postmodern) age, we perhaps underappreciate the power of words to create action. This article helps remind us of that linkage.

Why SOA is Orienting

On a morning run this week, I listened to a recent Dave Snowden podcast of a presentation given to IT professionals at an Agile Conference in Limerick. Some Dave's points about sensemaking triggered a few thoughts about how SOA is more aligned with sensemaking than traditional IT.

Before I get into those, I think it's worth noting that architecting frameworks/styles have "orienting" or sensemaking needs. Whether it's Zachman or MODAF or TOGAF or etc., a useful framework highlights certain fundamental distinctions and relationships that are used by an architect as a map to organize the the architecting territory for architecting decisions.

Although the SOA "style" is obviously different from frameworks that involve an architecting process, there are similarities in that both styles and processes fundamentally shape how architecting is performed.

The process of applying an architecting framework/style to a real world context involves at least three sensemaking/orienting activities:
(a) the architect orienting to the context (in light of the framework/style),
(b) the architect orienting to the framework/style (in light of the context), and
(c) the architect orienting to the (never completely clear and continually evolving) relationship between the context and the framework/style.

In traditional IT architecting, we tend to complete most of this orienting work by late in the requirements phase or early in the design phase, and the bulk of the work is usually associated with some mixture of (a) and (c).

In SOA, significant chunks of this work are potentially not completed until the user "composes a CONOPS" by discovering and weaving together previously deployed services. Allowing key architectural binding decisions to be delayed until the user interacts with developed capabilities is a significant departure from traditional systems which "give you any color you want as long as it's black."

Since SOA is a relatively immature style, IT architects have just begun to understand its strengths, weaknesses, basic patterns, etc...especially when compared to traditional architecting frameworks/styles.

And, it would also seem that this difference in how sensemaking occurs in SOA's will result in major changes in how IT architects orient to an architecting context and, finally, the interplay between context and framework.

Hence my assertion that SOA is as much about orienting as Oriented.

Back to the thoughts triggered by Dave...

As my tagline ("the intersection of decisions and technology") implies, there are decision making influences on technology and technological influences on decision making. Here's some of the human aspects that Dave highlights, along with my observations about traditional IT and SOA:

• Humans filter out all but 2-3% of the data they're exposed to; IT has very constrained input capabilities that usually do very little filtering.
• Humans orient using (a) fragmented patterns that were (b) recently activated; IT "orients" by exhaustively searching all data (often by using sophisticated, but static, pre-built indices/patterns).
• Human memory morphs every time it is activated, and tends to adapt/evolve to a context; IT data is static.
• Humans detect "weak signals" by noticing anomalies that trigger patterns. Weak signal detection is improved by keeping inputs ambiguous and "priming" memory by activating memories of high-risk/low-probability narrative fragments. IT has nothing like this, and tends to "fight against" human sensemaking by flooding a decision maker with large amounts of data.

Although SOA is still more IT than human in these dimensions, SOA does seem to be significantly more amenable to filtering, fragmented & morphing than traditional IT. To the degree that's true, it implies that architecting, design, development, deployment, and use of SOA will involve much more orienting throughout all phases than traditional IT.

Bottom line: Traditional IT is a typical tool in the sense that we don't interact much with it to solve a problem. As with a hammer/saw/etc., our interactions tend to be in a prescribed (and proscribed) manner with most of the adaption occurring outside the tool. SOA is more like a rope than a hammer in that we can use the tool to explore our understanding of needed capabilities and then use the tool to develop/evolve those capabilities.

Is KM Dead?

Over the years, I've occasionally participated in discussions about whether knowledge can be managed. The traditional understandings of management include activities like Plan, Organize, Control, Direct, and Staff.

Since I've come to see knowledge as being strongly contextual, I've tended to characterize "Knowledge Management" as a bit of an oxymoron. However, Dave Snowden has undertaken the quixotic task of returning the word "management" to its origin, which is more Complex than the current understanding (which is mostly Knowable).

That, and a number of other fascinating insights into KM are found in this interview of Snowden and Larry Prusak by Patrick Lambe. Highly recommended.

Techno-SOA = Failure

In the latest review of SOA successes and failures, Joe McKendrick discusses a Burton Group finding that techno-centric SOA projects are at best a collection of JBOWS that fail to create new value.

SOA successes focused on the business aspects (People, Processes, and Organizations), especially where a culture shift was required, and ensured that robust governance was part of the transformation....technology as a tool...a concept to conjure with....

Yet more evidence that perhaps we should say Service "Orienting" Architectures to emphasize that the shift to SOA is more epistemological (i.e., involves how SOA changes how we know) than it is ontological (i.e., involves the "being"/structure of the architecture).

Unfortunately, most engineers are so grounded in a Known/Knowable framework that the Complex/Chaotic is often greeted with a blank stare. SOA provides a serious opportunity for these individuals to broaden their horizons.

Notable quote from the SearchSOA.com article: Manes found repeatedly in her research that the "if you build it, they will come" approach to SOA wound up a failure and Roberts confirmed that, saying, "We stopped trying to build business cases for SOA, it wasn't working. Instead use SOA to strengthen the existing business case."

Traditional architects may find little (if any) basis for such an approach in the standard architecting frameworks. That's no excuse for failing to learn from these SOA failures.

Erlang, Cloud Computing, and CEP

This slashdot post on Erlang caught my eye when it mentioned that both Amazon and Google are using the programming language as the foundation for their utility computing offerings.

I've always been intrigued by functional programming languages, so I clicked through several of the links....here's the ones I found interesting:

The post is about a blog entry that provides some additional context, although you'll need to click the "History of Erlang" link (in the blog entry) to get a good description of why Erlang makes sense for the cloud.

Which kind of made me wonder if Erlang had been used for Complex Event Processing...a Google search turned up this comment on Erlang and CEP.

Will cloud computing make functional programming mainstream? I kind of doubt it, but Google and Amazon's vote may make FP more widely known and used than it is today.

Emergence and Enron

I know little about Enron beyond what I've read in the paper, but this HBS Working Knowledge interview with Malcon Salter triggered a few thoughts about hyperconnectivity, emergence, and risk.

Much of the coverage of Enron has focused, with reason, on the ethical lapses of Enron's leaders. However, a contributing factor seems to be the ability of "financial engineers" to create sophisticated structures faster than the market could create tools to effectively evaluate and govern those structures. More recently, the securitization of debt seems to have catalyzed a similarly unstable structure in the mortgage markets.

All of which seems reminiscent of the structural governance challenges that are emerging from Web/Enterprise 2.0. As with Enron, there are significant opportunities to create value via disintermediation. However, there are also significant risks that emerge when the complexity of information processing structures outstrips our ability to understand and govern those structures.

The Upside of Disruptive

According to a new article in the current HBR, IT enables process agility that provides significant (albeit potentially fleeting) competitive advantage ("Investing in IT That Makes a Competitive Difference", co-authored by Andrew McAfee who has a blog entry linking to a free (until late July 2008) copy of the article).

Quibbles aside (e.g., does the emergence of China and India as major players on the IT stage contribute to some of the discussed effects? (vs. generic globalization, which is dismissed as a factor)), the case made seems plausible given the potential disruptive effects of an emerging hyperconnectivity and the relentless march of Moore's Law.

It raises interesting (unanswered) questions about why some organizations are evidently very effective at (a) identifying local innovations that can easily be replicated across the enterprise, and (b) deploying and governing those changes. Nor does it address the potential risk of deploying innovations that are much less relevant to the enterprise than they are to a specific local context (part of the governance challenge).

It strongly endorses an approach to SOA that would (a) deploy a common infrastructure across the enterprise, and (b) encourage a bottom-up thread-based application of SOA to business needs. And, it clearly recognizes that this approach introduces some significant governance challenges/opportunities.

However, it also makes it clear that the primary competitive advantage comes in the Execution domain, not in the Exploration domain (except where Exploration is looking for greater efficiency/effectiveness in Execution-oriented processes). The good news is that there appears to be plenty of room for just that kind of innovation. The challenge of catalyzing IT-enabled innovation in areas other than Process remains.

Finally, the case for Web/Enterprise 2.0 seems much more anecdotal...perhaps in another 10 years we'll have enough data to make a clear case in that area.

All of which makes me wonder...if we're seeing this kind of immediate effect from Process improvements, what will emerge once this type of improvement approach spreads to the People and Organization layers of the stack?

Cloud-hopping

Nick Carr summarizes some of the latest writing on cloud computing. I don't have anything to add, but do wonder if 50-100 years from now this development will be seen as the primary factor shaping the world's legal framework for property.

If you find this topic interesting, you might want to check out William Gibson's cypberpunk novels and Neal Stephenson's "Cryptonomicon."

Enterprise 2.0 & Turf Wars

Dion Hinchcliffe has a nice summary of the recent conference. Dion mentions that turf wars are beginning to pop up. I see this as perhaps the most valid evidence of the real emergence of E2.0.

The internal hyperconnectivity that E2.0 creates tends to undermine and disintermediate all low-value-added individuals, groups, and processes. To the degree these entities have power/influence and low ROI (simultaneously), "turf war" may radically misrepresent the conflicts that E2.0 catalyzes since much of the resulting conflict may be more asymmetric than traditional.

The Downside of Disruptive

Lots of ink has been spilled about the risks & opportunities of disruptive innovation, but much of the analysis has been more techno-centric than full-spectrum (e.g., People, Organization, Process, and Technology). This was one of the criticisms leveled at Christensen's initial writing on disruptive innovation that he addressed in subsequent discussions (e.g., "The Innovator's Solution").

Joe McKendrick discusses one slice of this vis-a-vis SOA in this post. The opportunity (and the challenge) of SOA is that it enables (and requires) what I've called "full-stack innovation" (i.e., People, Organization, Process, and Technology) to generate disruptive innovation and the associated ROI.

If you just do the Technology, you've spent a lot to do the same thing you're currently doing. While you may theoretically now have a business foundation that supports more agile and innovative capabilities, you haven't begun to address the hard stuff (e.g., People & Organization).

On the other hand, if you tackle the full stack, you have to constrain your efforts to narrow threads since full-stack efforts inherently create a lot of internal non-technical disruption.

I've not seen anyone with a good story on how to handle the business tension between low-disruption low-ROI infrastructure-centric SOA and high-disruption high-ROI business-thread-centric full-stack SOA. This seems especially difficult in light of what Joe discusses...that there tends to be a significant disconnect between the IT and business views of technology and its potential.

Is Most SOA Really JBOWS?

In a fun post, Joe McKendrick provides Ten Ways to distinguish Just a Bunch Of Web Services from SOA.

I take this mostly as a measure of the relative immaturity of current service-oriented implementations...however, it does help illuminate the possibility that full-blown SOA may be overkill for some needs. In two related posts (here and here), Joe discusses the immaturity of SOA (a topic that has generated a fair amount of talk recently).

Joe also references a "15 Ways to Tell It's Not a Cloud", which prompts me observe that it seems like new domains are often easier describe negatively ("what it's not") than positively ("what it is").

Controlling the Value Chain/Net

Since Micheal Porter's popularization of value chain analysis in the mid 1980's, a common theme is analyzing margins along a chain (from raw materials to consumer) to determine where margins are highest and why. For example, Wal Mart is widely seen as controlling a large chunk of the retail value chain (although they're a bit of an anomaly since they've used their influence to increase market share instead of earning well-above-average margins)....similar in some ways to the influence Sears wielded when retailing was dominated by department stores.

As SaaS, PaaS, etc. swirls in the Web 2.0 world, this topic is beginning to emerge in considering the value chains traditional IT supports and enables. This post by Josh Greenbaum is a nice recent discussion of the topic as applied to the Salesforce.com market. Control of value chains/nets would seem to be inherently more difficult to achieve and hold in the information domain, but the potential opportunity/risk means that a lot of maneuvering is inevitable.

If you're unfamiliar with value chain analysis, this link provides a short intro.

IT as Dialogue Systems

Nigel Green has augmented VPEC-T with an assertion that information is perhaps more about Context and Dialogue than the data/info/knowledge/wisdom "stack."

As I've said in previous posts, when considering Information, I think it is more productive to focus on Context than a "stack" structure that, for better or worse, tends to evoke visions of increasing aggregation.

I've not said much explicitly about dialogue, but references to Snowden, Klein, and Weick should point toward the fact that sensemaking in a Context always involves the use of Language in a Conversation (with yourself and others).

Although I had not stumbled across these folks at Capgemini UK until recently, they are some of the very few architects I know of that seem to fully appreciate why IT ends (not begins) with technology.

Are Processes Dangerously Superficial?

Nigel Green (Executive Enterprise Architect at Capgemini UK) has an interesting discussion asserting that a "wiring diagram" approach to business processes is likely to falter when it spans department or external boundaries.

He touches on factors I've mentioned before (e.g., values, policies, events, content, trust), and rightly notes that these must be explicitly addressed in architecting.

These factors have been formalized into an acronym (VPEC-T), and discussed in a book ("Lost in Translation", by Nigel Green and Carl Bate) that "focuses on outcomes and adoption, not on technology, processes, and function." This sort of thing is exactly right in my opinion, but it seems that many techno-centric companies find it not so much heretical as incomprehensible.

His Beads-Threads metaphor may not be especially catalytic, but the basic message is an important one that rarely appears in the techno-centric world of IT architecting.

Can Innovation be Taught?

Victor Newman has begun blogging, and has an interesting post asserting that trying to teach innovation is a cargo cult sort of endeavor....it reminds me of the knowing-doing gap discussion.

This reminds me of the debate about whether leadership is innate or can be taught.

For both leadership and innovation, I suspect that both innate ability/predisposition and education are required...but that some sort of "innateness" is probably the primary enabling/gating factor.

Hyperconnectivity and Micromanagement

Tyler Boudreau has an interesting article on how Network Centric Warfare has created a real risk of being micro-managed. This is nothing new...but if you've never considered the issue, his experience in the Marine Corps is well worth reading.

The story is both comical and frightening. My initial reaction is to dismiss it as the sort of thing that occurs during a technology transition phase....once folks become accustomed to being hyperconnected, they'll understand the need to change the way they operate.

However, I may be naively optimistic...the temptation to Monday-morning quarterback and scape-goat may ultimately be the factor that gates the potential value of NCW and hyperconnectivity.

Learning 2.0?

I recently ran across a blog entry that referenced an articled co-authored by John Seely Brown on education entitled "Minds on Fire: Open Education, the Long Tail, and Learning 2.0."

As a former homeschooling parent who has done some research into educational philosophies and frameworks, I found this fusion of social media/technology and main-line educational theory engaging and occasionally a bit simplistic.

The authors rightly emphasize the value of active social engagement during learning. However, their discussion of a "Cartesian View" vs. a "Social View" of learning seems like another variation of the whole Direct Instruction (DI) vs. Constructivist debate....something I find a bit tiresome since both have value, depending on the learning context. Their "learning about" vs. "learning to be" has a similar flavor; this time they recognize it's "both-and", not "either-or."

Processes in large organizations tend to focus almost entirely on capturing knowledge in an artifact (e.g., design document), and ignore the equally important need to get artifact-based knowledge into a decision maker's head. This article helps illuminate the challenge and importance of the latter activity.

To the degree that this kind of article encourages a better understanding of where and how to apply DI and Constructivism (including socially-driven variants), it's a good thing...however, I 'm a bit concerned that the casual reader with little understanding of the how pedagogy is contingent on context might not fully appreciate the need for a contextually-driven balance.