Intelligence In Numbers

Wisdom of CrowdsThat crowds are dumb is a given – an immutable truth that’s not worth arguing about. Meetings are to be sneered at, group-think deserves contempt, committee and procrastination are synonyms. As Nietzsche puts it, “Madness is the exception in indiviudals but the rule in groups.” And Thoreau: “The mass never comes up to the standard of its best member, but on the contrary degrades itself to a level with the lowest.” (etcetera: We are good at googling).

The smartest (ok, ok, funniest) philosopher of them all, “Dave Barry sums up our collective feelings in one (slightly long) line, “If you had to identify, in one word, the reason why the human race has not achieved, and never will achieve, its full potential, that word would be meetings. When on quotes, here’s another one: “Anyone taken as an individual is tolerably sensible and reasonable – as a member of a crowd, he at once becomes a blockhead.” – Friedrich Schiller, through Amit Varma.


New Yorker
columnist James Surowiecki wanted to prove Barry wrong, and wrote a whole book about the value of meetings. Ok, half a book maybe. Seriously though, Surowiecki’s erudite, well-researched, and interesting book provocatively takes on a widely held notion – that individuals are better decision makers than groups – and proves how wrong it is. Not conclusively, but there’s enough empirical evidence in the book to force a rethink. The book was an instant bestseller and won a lot of acclaim – but most importantly it is an enjoyable read.

So, why are groups good? A long time back, somewere in England, a philosopher decided to run an experiment involving a few hundred men and a dead ox. The philosoper strongly believed that most people were dorks, and that crowds of dorks were dorkier and this experiment was going to be conclusive proof of his belief. The dead ox was placed in a shandy , and people were asked to guess the weight of the ox. At the end of the day, the estimates were collected and averaged. The expected outcome was that the mean guess would be far off, thus proving that crowds were stupid.

Unfortunately for the philosoper though, the mean of all the guesses was almost exactly equal to the weight of the ox, and much better than most individual guesses. In additon to thorougly confusing the philosopher, the experiment had a point to it: It was an early indicator of the idea that forms the basis of Surowiecki’s book: A collective decision made by a group of people, each of whom thinks independently, will almost always be better than the best individual decision. This is not the same as consensus, which is the dumbing down of individual opinion to satisfy the group. You cannot find a better example of the Wisdom of Crowds than the Internet: Google, Linux, Firefox and Wikipedia are examples of decentralized mechanisms that have delivered outstanding results. The Blogosphere, in conjunction with Google’s Pagerank is quite possibly the best example of a perfect democracy: the more people that like a blog, the more links it gets, increasing its relative importance and readership.

Drawing on examples from everyday life – stock market bubbles and traffic jams; the Colombia crash and overcrowded bars – Surowiecki paints a compelling picture of smart groups, and provides suggestions for possible ways to harness the potential. One of the methods discussed is a “decision market” like the Iowa Electronic Market – a stock market like system where you bet on a certain person winning an election. The accuracy of the IEM’s predictions are better than even polls – which is surprising considering the small number of people that participate in it. Fascinating. By the way, what do you think was the most reliable lifeline on Who Wants to be a Millionaire?

Get a couple of people in a room. Give one of them $10, and ask him to share it with the other guy. If the other guy accepts his offer, they both get to keep the money, otherwise no money for either of them. Most of the time, the offer was $5 apparently, while people rejected lowball offers (anything less than $2), even though it was free money. Surowiecki attributes this to an innate sense of what constitutes fair that we all have – and uses it to explain why we vote in elections and why we tip at restaurants in distant towns that we know we are not going to go back.

The book has been accused of not being rigorous enough (as in providing statistics for why crowds are smarter), and sometimes you get the feeling that the author is hand waving his way to a pre-determined conclusion using anecdotes, but you know what, if you judge a book by the amount of pleasure you get from reading it, then this is a good book. I’ve not enjoyed non-fiction more (if you don’t count Starry Nights, that is). Plus I now know all about the virtues of a decentralized, diverse corporate power structure and ways to run productive meetings.

Excerpts can be found here and here.

4 thoughts on “Intelligence In Numbers”

  1. this topic is interesting and only because everytime i read an article about it, it presents an opposing view (almost a natural “not gate”!)
    for example, the last time i read abou this, i was convinced by the author that crowds and averages are simply not the best thing. a simple (though some may say, silly) example: if 5 people of varying sizes were to vote on the right size for a shirt, the average shirt produced would fit no one!(hey i warned you abou thte silly factor!) the author made a strong case for the individual – for example using your examples above, don’t forget that google, linux et all were all the concepts/ideas of one or atmost two people and it remained so for awhile. over time, many contributed and made it popular, but they were still add-ons remember? the core of what made it great never came from a meeting – and in my belief never would have. somewhere inbetween the discussion of whether linux should be a micro-kernel based architecture, a homer-like character would have said they should break for doughnuts and that would have been the end of that)
    i have seen this personally too: when i come across a computing problem at work and think up a solution, i am asked to sit in a meeting to see if it is the best choice – an hour later nothing happens, actually i am now more confused then ever given teh choices (which reminds me: you should read “The Paradox Of Choice: Why More Is Less” by Barry Schwartz). then the next day, my friend, who was also at the meeting but drew a blank then, would have thought about it alone overnight and would present a radical solution to me.)
    as for your restaurant example, i say tip most to the restaurant out of town since you might never see the poor guy again. at a regular place, reduce the tip since you are going to keep visiting him anyway 🙂

  2. There are definitely two sides to the story. How does google think a search result is important – by determining how many people link to tha site. That’s the “democracy” Surowiecki describes, but you have a great point – the idea behind google itself came from a couple of individuals.

    Surowiecki argues that the meetings you describe here are badly run meetings – well run meetings with a diverse group (intellectually diverse, people holding different opinions and no impediments to people expressing their opinions) will always come up with better solutions most of the time.

    Of course, there are limits to what groups can do – writing a good book, a painting etc. are probably things that only individuals can do (although editors, reviewers might play a role).

    I will add The Paradox of Choice to my growing ‘to-read’ list…

Comments are closed.