Thursday, September 23, 2010

If all you have is an Elephant and 6 Blind Men

I was in a presentation the other day where (once again) a speaker invoked the Blind Men and the Elephant parable to explain some misunderstanding


I'm sure you know the one:
  • There's a large IT project with a lot of different people involved.
  • There are differences of opinion about what is important
  • According to the speaker, the reason we can't agree is because everybody is concentrating solely on their portion of the system rather than looking at the system as a whole.
I can't remember how many times I've heard this story used and I'm sure you've also heard it plenty of times. In fact, as architects in the IT world it seems to be one of those Metaphors We Live By.

Unfortunately it also seems to be the only analogy we use to explain this situation. As such it is a good example of another analogy that is used (and abused) in our discipline - If all you have is a hammer, everything looks like a nail.

Here's an alternative so we can have more than a single hammer to explain different perspectives on large projects:
Fly, Bat, Worm.

This was described in Neal Stephenson's book Anathem and it goes something like this. Instead of 6 blind men groping different parts of the elephant, imagine a Fly, a Bat, and a Worm perceiving the entire elephant and trying to communicate with each other about it. The (ideal) Fly perceives everything solely through sight, the (ideal) Bat perceives everything solely through hearing, and the ideal Worm solely through touch. How can these three discuss what they have perceived? How does a deaf Fly describe something to the blind Worm, or vice versa?
"[The Fly] says 'the worm seems to be relating some kind of account of its wormy doings, but since I don't squirm on the ground and can't imagine what it would be like to be blind, I haven't the faintest idea what it's trying to tell me!'"
While different people working on a large IT project don't perceive with different senses, they do conceive their mental models of the proposed solution differently. We're not so starkly different as to be communicating like a blind Worm to a deaf Fly, but the people working on a large project have different types of knowledge through which they both understand the problem domain and devise solutions using IT. Even people with nominally similar roles can have quite different detailed knowledge models of their area.

For example, Enterprise/Solution Architects are not completely divorced from the "wormy doings" of the Operations people who "squirm on ground"[*], but they are culturally different. Even if you have experience across all the different facets of Business and IT, different levels of expertise result in different levels of detail in the conceptual models we use to understand our problems and solutions. We might use the same terms in discussions but that does not mean we are immune to cultural differences in understanding. In fact the way we use the same terms often hides the differences in understanding that we have. We may think we have reached consensus, but internally we can have quite different interpretations of those words - i.e., the contexts that govern their meaning - and therefore the consequences for how they interact to provide a solution. "Service" anyone?
[*] No offense to the excellent Ops people I know. s/Operations/Development | TechArchitect | BusinessArch/ etc as required.

Designing a system requires constructing an understanding - a conceptual model of the solution. I.e., the concepts, relationships and interactions that need to exist for the solution to work as intended. It is a constantly evolving process that combines models of the problem domain with possibilities presented by the technology in our solution domain. We need to iterate on this process to improve our knowledge of both during design. And on large projects we also need to collaborate and reach consensus with others while performing this iterative learning process.

But how we understand the problem domain and how we devise solutions within it is subjective, not objective. How we identify concepts in the domain is theory-laden. How we devise explanatory models is influenced by our culture, language, and expertise. The definition of culture when we consider its influence on developing models can be as broad as "western culture" or as narrow as "java developers", "declarative language developers", "scrum practitioners", "enterprise architects", "EAI integration expert", "enterprise architect", "DevOps", etc.

We don't perceive the same reality and simply create different models for it. We understand that reality through our existing sets of concepts. To bastardize another metaphor. We see our domains, and solutions, through culture-tinted glasses. Whether we are explicitly aware of this bias or not.

I'm not going deep into the theory in this post, but the fields of metaphysics, epistemology, cognitive psychology, neurophilosophy, philosophy of science, philosophy of social science (and others) all deal with this issue in their respective fields. And while there are many debates in those fields about the extent to which subjectivity influences how we gain, use, and communicate knowledge, its certainly broadly agreed that it does influence it. If you're interested, here's a popular recent article in the NY Times on the role of language and how it shapes how we conceive reality.

The influence of subjectivity in system development is not new in IT. Some researchers and practitioners have been discussing its effects since the 60s. People such as Peter Naur, Terry Winograd, Meir M Lehman, and Bruce Blum. The most interesting aspect is that it is now slowing filtering through to the mainstream IT world's books and conferences. E.g., Domain Driven Design alluding to the influences of modelling and Kevlin Henney's JAOO presentation on the role of Metaphor (pdf) just to name two.

But we should be considering it more explicitly. As Gotleb Frege wrote,
The objects in the world are already delineated to some extent by the classifications embodied in socially inherited language. In fact, learning a language essentially means learning to grasp objective thought concepts.
We should be spending more effort understanding how this influences large-scale system design and development.

Which brings me back around to the situation that usually results in the Elephant and Blind Men analogy being deployed:
  • There's a large IT project with a lot of different people involved
  • There are differences of opinion about what is important
  • The reason they can't agree is because it is impossible for everyone to see the same holistic picture. Not because we are concentrating solely on individual portions of the system rather than looking at the system as a whole.

We observe and use the same terms to talk about the overall design but we conceive it differently based on our cognitive apparatus. We're not all looking at different parts of the same elephant. On a large IT project we can all see the whole elephant, we just see it differently - like looking at an Andy Warhol picture of an elephant.

The only way to overcome these differences is through interaction and feedback. Not to say, "but you aren't looking at the big picture", but rather, "what do you mean by that?", and "can you give me a specific scenario?".




Image references:

Blind Men and the Elephant: WikiMedia Commons
Fly, Bat, Worm: Cafepress.co.uk
Calvin and Hobbes on Relativism
Warhol Elephants



3 comments:

  1. Hi Jason,

    Great post about cognitive models and their impact on software development. I don't know if you've gotten a chance to look at Chapter 7 of my book, but it's on the same topic. It tries to provide a synthesis of several competing cognitive models of software architecture. And it's one of the chapters available for free download here: http://rhinoresearch.com/content/software-architecture-book

    Regards,

    George Fairbanks
    Author of Just Enough Software Architecture

    ReplyDelete
  2. Thanks George. I'm slowly working through your book. Looking forward to Ch.7.

    Jason

    ReplyDelete
  3. I work for a large Aerospace company and have to say that, within that environment, the blind men analogy by far resonates more strongly with reality than the fly, bat and worm analogy . Perhaps this just shows a difference in problem set between Enterprise IT and Aerospace. The blind men analogy is just another word for "silo-ing" and the other is more of a tower of Babel problem.

    The problem really is one of turf wars -- each manager is well established in a tight command and control style structure and absolutely must show progress with whatever it takes. Including manipulation and twisting of metrics and their interpretation, choosing lesser solutions to drive their own milestone completions over solutions that they know would be better and cheaper overall. This attitude reflects poorly down to the teams that practically work on isolated islands via a 'toss-it-over-the-fence' engineering style. This industry continues to try the waterfall-style big planning, big design and big requirements up front and are shockingly surprised when they see the same result when they do their testing and integration at the end fall apart like it always has.

    Interestingly, the same prescription is needed as part of the cure for this illness -- interaction and feedback. It needs to go beyond these, though, and into true collaboration and an abolishment of the strict control and command institution.

    ReplyDelete