Intuition Vs. Analysis

“Not everything that can be counted counts, and not everything that counts can be counted,"
A sign in Albert Einstein's office.

"Does comprehending the powers and perils of intuition matter?," notes the social psychologist David G. Myers, and concludes "I contend that it matters greatly."

I concur with him. Intuition is a very powerful tool which everyone of us possess. Some can use it stronger than others but we all have it. Intuition helps us many a times, especially in crisis. When the left-brain stops functioning, it the right-brain which comes to our aid. Understanding its powers and its perils is important for us. To make useful decisions in our personal, social and business life we need to understand and differentiate the power of intuition and analysis and the ability to know when to use what.

As James says "...the real challenge is figuring out which problems can be solved by rapid cognition and which are better solved by a calculating, rational approach."

Like the case I made for Data and Patterns, it is equally important to understand Intuition and Analysis.

I have been noticing the book, "Blink : The Power of Thinking Without Thinking" by Malcolm Gladwell. Gladwell is a respected author because he has written one of the good books in the past few years, The Tipping Point. This is a book which is worth reading twice, thrice and more. A lot of good ideas and lessons to be learned from this book.

I have not read Blink yet. However, I have come across a review of the book by David Brooks in the NY Times and a e-mail exchange between Gladwell and James Surowiecki, author of The Wisdom of Crowds on the subject What Do We Mean When We Talk About Intuition? in Slate.

“Intuitive thinking is perception-like, rapid, effortless," notes Princeton University psychologist Daniel Kahneman. By contrast, “deliberate thinking is reasoning-like, critical, and analytic."

Lets look at the arguments put forth by Brooks, Surowiecki and Gladwell and look at what we can learn from them. We will also touch upon the wisdom of David G. Myers from his book, Intuition : Its Power and Peril.

Myers provides a list of valuable quotes on intuition and analysis from various historic figures in his book which I will sprinkle in this post. You can find his book on Amazon or read it online.

What is Intuition? (from Myers and Webster)

Intuition is our capacity for direct knowledge, for immediate insight without observation or reason. “Intuitive thinking is perception-like, rapid, effortless," notes Princeton University psychologist Daniel Kahneman. By contrast, “deliberate thinking is reasoning-like, critical, and analytic."

What is Blink all about, David Brooks in the New York times gives his impressions.

THERE is in all of our brains, Gladwell argues, a mighty backstage process, which works its will subconsciously. Through this process we have the capacity to sift huge amounts of information, blend data, isolate telling details and come to astonishingly rapid conclusions, even in the first two seconds of seeing something. '' 'Blink' is a book about those first two seconds,'' Gladwell writes.

Gladwell says we are thin-slicing all the time -- when we go on a date, meet a prospective employee, judge any situation. We take a small portion of a person or problem and extrapolate amazingly well about the whole.

''We are innately suspicious of this kind of rapid cognition,'' Gladwell observes. We assume that long, methodical investigation yields more reliable conclusions than a snap judgment. But in fact, ''decisions made very quickly can be every bit as good as decisions made cautiously and deliberately.''

MY first impression of ''Blink'' -- in blurb-speak -- was ''Fascinating! Eye-Opening! Important!'' Unfortunately, my brain, like yours, has more than just a thin-slicing side. It also has that thick-slicing side. The thick-slicing side wants more than a series of remarkable anecdotes. It wants a comprehensive theory of the whole. It wants to know how all the different bits of information fit together.

Gladwell and Surowiecki provide a insightful discussion on Slate and create a format for understanding the various forms and ways of decision making.

James Surowiecki notes that "Blink is a book about the phenomenon you call "thin-slicing," which is "the ability of our unconscious to find patterns in situations and people based on very narrow 'slices' of experience." His main contention is the idea that rapid cognition is often as good as more careful deliberation.

James quotes Van Piper "When we talk about analytic versus intuitive decision-making, neither is good or bad. What is bad is if you use either of them in an inappropriate circumstance," This suggests that the real challenge is figuring out which problems can be solved by rapid cognition and which are better solved by a calculating, rational approach. So, the questions I'll start with are: First, is thin-slicing really how decisions get made in complex environments? And second, and more important, if it is, can we really count on it to reliably produce good decisions?

Malcolm challenges the assumptions of James in 'The Wisdom of Crowds' about collective decision making. He says "The first is that you are explicitly challenging what might be called the Standard Model of decision-making. We have an awful lot invested, as a culture, in the notion that the best results in complex environments come from centralizing authority in the hands of a single, highly expert, and deliberative individual. But that, you would argue, is wrong. We're actually better off decentralizing decision-making into the hands of the many—even if they are relatively nonexpert and even if their decision-making process is much less deliberate."

He continues :

One of the key arguments in my book is that human beings think in two very different ways. Sometimes we consciously and carefully gather all facts, trationally sort through them, and draw what we take to be a rational conclusion (the Standard Model). And sometimes we reach conclusions unconsciously—our mind quickly and silently sorts through the available information and draws an immediate judgment, which may be done so quickly and so far below the level of awareness that we may have no understanding of where our conclusions came from. I call this Rapid Cognition.

Malcolm now breaks the argument into the various ways of decision making. The collective or distributed model, the individual, rapid cognition and standard model or deliberate decision making.

James expounds the virtues of group decision making and at the same time lists the appropriate places where it makes sense not to use this kind of decision making.

NASA, for instance, recently ran an online experiment called "clickworkers" to test whether the collective judgment of ordinary people would be of any use finding and classifying craters on Mars. You could go to the site, get trained (for a couple of hours, I think), and then click away. The result, in NASA's words: "the automatically computed consensus of a large number of clickworkers is virtually indistinguishable from the inputs of a geologist with years of experience in identifying Mars craters." And these people weren't even being paid.

Why does this work? The key is that even though each person in the group is making mistakes (often, lots of them), as long as the group is large enough and diverse enough, the errors people make effectively cancel themselves out. And what remains, remarkably often, is the information you're looking for. This sounds—at least to some people—implausible, or pseudo-mystical, or both. But, as you mention, in my book there are myriad examples of this phenomenon at work, solving problems from the simple (guessing the number of jellybeans in a jar) to the mindbogglingly complex (finding a lost submarine on the basis of a few fragments of information).

You're right, though, that "nonexperts groups" aren't always better than expert individuals. In the first place, in some cases—like situations where decisions need to be made in a matter of seconds—collective decision-making is impractical. Other situations—like flying a plane or performing surgery—seem to be tailored pretty well to individuals. More important, there are problems where you need to know a lot just to understand the question you're trying to answer. In those cases, relying on a group of laypeople may be futile.

What's important, though, is that even in those situations where expert knowledge seems necessary, you're better off relying on the judgment of a group of experts rather than a single expert, no matter how brilliant.

James states two reasons on why relying on single individuals is not correct.

True experts—that is, the real titans—are surprisingly hard to identify.

The second, and more important, problem is that even brilliant experts have biases and blind spots, and so they make mistakes.

James sees Blink and Wisdom..on the same page. "Blink and The Wisdom of Crowds intersect quite nicely. A lot of your book, as I said, is about how biases and prejudices and inexperience can lead us astray when we rely on rapid cognition. My book suggests that in lots of cases, if you aggregate those flawed judgments, you can get rid of the flaws and keep the benefits of rapid cognition."

James believes that since we spend more time on the information available in the Standard model of decision making there are more chances of recognising our mistakes and taking a better decision. Malcolm thinks "that it's very hard—even within the most formal decision-making structures—to separate out the influence of unconscious bias...my final point in this round would simply be that there are good ways of fixing the individual decision-maker as well. We can put up the equivalent of screens. We can find ways of editing out nonessential information."

Malcolm makes this very important point that more information is not always better and that, no matter what, there are essentially blind spots and biases within every individual and this can cause wring decisions whether it is a spontaneous decision or a deliberate decision. The best way to solve this problem is to provide screens or help to sort out the "useless information". James questions this exact same thing. He thinks that to realise which information is central to the outcome of a decision and which is irrelevant is not an easy one and "It often requires the careful study of data to see what factors are and aren't correlated with each other, and it requires an analytic approach that seems to belong more to the Standard Model."

“The first principle," said Einstein's fellow physicist Richard Feynman, “is that you must not fool yourself---and you are the easiest person to fool."

James beleives that this is the deep paradox of Blink, which is that successfull rapid congnition requires a well-defined rules or structure to guide the people using it. In other words, "we need a structure for spontaneity". The other important aspect of thin-slicing is that it requires self-awareness.

I am more skeptical, though, of the idea that everyone can learn to thin-slice consistently well, and that therefore it's a good idea for most people to rely on rapid cognition. When I look at the kinds of experts you write about—from firefighters to art critics to George Soros—I'm struck by two things: First, these people have spent an enormous amount of time working in their given fields; and second, I think they're probably naturally exceptional at pattern recognition. I'm just not sure that most people, even with more experience and even with the right structures in place, are going to make consistently better decisions via rapid cognition. In fact, as you point out, there are some situations in which it isn't even clear you want individuals making decisions: Cook County Hospital dramatically improved its record of distinguishing patients who were having heart attacks from patients who weren't by replacing the judgment of individual doctors with a simple algorithm.

My doubts aren't about the virtues of thin-slicing. I've thought for a while now that one of the reasons why the collective decision-making mechanisms I write about in my book—like, for instance, betting markets—work well is that in part they aggregate intuitions and impressions that people can't necessarily articulate, but that are nonetheless real and valuable. That's why I think Blink and The Wisdom of Crowds really do fit together. My qualms, really, have to do with the idea that for most people, the combination of rapid cognition and individual decision-making does make it harder to spot potential pitfalls and to correct mistakes in time to make a difference. I'm sure the collective product of our rapid cognitive judgments will usually be excellent. But when it comes to the average individual, I still wonder.

Malcolm concludes the discussion with saying that "In order to make for a successful snap judgment, you have to carefully and deliberately intervene in the decision-making environment."

After all this have we come any further with understanding with decision making process? Yes and No. I have a deeper understanding of the decision making process. I understand that decisions can be made collectively, individually and can be instant or deliberate. We need to be more self-aware to be successfull in our snap decisions and at the sametime self-awareness builds up over a period of time and cannot be learnt quickly and successfully. In fact, structuring the process of data collection should help us in using our intuition. In the end, we all need help in our decision making process. And, in order to help others make better decisions (that is one of the jobs of the leader) we need to "carefully and deliberately intervene in the decision-making environment". However, I am nowhere near understanding where to use what type. For launching new products, for making organization changes, for revising the pay structure, for joining a MBA course, for trusting people and for varius other myraid thing I guess I have to learn through experience.

Business, and life in general, is a series of choices. This involves decisions. To do or not to do. If yes, which one? Till what time? Where? How? When? Why?. As Drucker says, Making good decisions is a crucial skill at every level and that not to take a decision is also a decision. In this scenario we will be better prepared if we are trained to use our intuition (using patterns) as well as critical analysis (using data) and hopefully be aware so as to gain the wisdom to use the right model in the right situation.

---

Books listed in this post.