The Information Trap: How excess information can compromise decision quality

Another week, another episode of Location Location Location.

We meet Emma, a conscientious first time buyer whose journey into the housing market becomes an epic saga. Armed with her spreadsheet and an unwavering spirit, Emma embarks on her quest for the perfect home. She pores over every detail, from mortgage rates to crime stats to the number of paces to the nearest tube stop. As the piles of data grow, she narrows her choices with the help of a colour-coded 17-point checklist. She’s adamant she knows what she wants… but hasn’t viewed a property in 3 months. Enter Kirsty and Phil stage left to rescue the situation in a comforting three-act narrative.

It raises the question at what point does too much information make a decision harder?

To answer this, let us turn our attention to a fascinating experiment by Paul Slovic, a distinguished psychologist based at the Oregon Research Institute. Slovic investigated professional horse bettors. Slavic found that beyond a certain threshold (5 pieces of information), additional information stopped adding to the bettor’s decision accuracy. New information didn’t improve the decision – it only served to make them more confident about the decision they had already made (see figure below).

The lesson? Don’t go hunting for new data forever. Beyond a certain threshold more information can bog you down. In fact it can put you on a path to overconfidence and confirmation bias.

Why is this? When the volume of information increases, our ability to separate signal from noise becomes compromised. We lose sight of what’s really important. This is a paradox: a “less-is-more” effect.

I first came across the study in the Central Intelligence Agency’s book “The Psychology of Intelligence Analysis“. CIA trainers use the case study to help new analysts make the right calls because it exemplifies the stresses caused by information overload.

How do we know what is “the signal”, and what is “the noise”?

If you’re reading this, you’re probably required to plough through lots of information on a daily basis. The study poses some interesting questions about human judgment.

The irascible philosopher Nassim Taleb distinguishes between “the signal”, which represents valid predictive information, and “the noise”, which encompasses extraneous or random data.

To recap on our horse racing example, the experts who had 20 years of experience handicapping horse races have deep expertise: they know which variables matter. Their hard-won experience has developed their intuition: an ability to acquire knowledge without recourse to conscious reasoning. An unconscious pattern-recognition guides them to what matters.

“Come on Emma… The outside space or the second bedroom – pick one!”

Novices can fall into the trap of building abstractions upwards from data – leading to erroneous conclusions. We’ve all been flummoxed by decisions, like our friend Emma – needing the perspective of a seasoned advisor to reduce, simplify and clarify.

When is the right time to stop thinking and start acting?

Decisions are always a matter of weighing probabilities.

Client teams launching new products should need to accept they operate in complex, ambiguous marketplaces. Aiming to make the best decision for the situation they face today is sufficient if they are primed to pivot tomorrow.

Because let’s face it, how many decisions are truly one-way doors? In more and more domains you can tweak your service or proposition on the fly. We saw this first hand recently on a product launch. Our client gave us a great schooling in agile innovation, gathering enough data to soft launch for one audience, then expand the offering in a live environment.

A rule of thumb

Former US Secretary of State Colin Powell developed a useful rule of thumb here: the 40-70 rule. His view? You need between 40 and 70 per cent of the total information to make a decision. Too little information and you’ll make a poor choice – but if you seek complete information it will end up taking too long. Decisions end up being made for you.

In summary: despite uncertainty, we must act.