Written by Robin Hogarth

Everyone has been affected – one way or another – by the financial crisis that hit the world in 2008. To me, the most amazing phenomenon has been the inability of the world’s leaders (whether social scientists, government officials, or business executives) to predict what was going to happen and to take protective actions in time. Sure, there had been many warnings that house prices were overvalued in many parts of the world and that people had been borrowing too much. However, no one really foresaw how quickly everything would unravel and the mammoth consequences that occurred. Indeed, if you had been given an accurate forecast about a year ago, it is quite likely that you would have not known what to make of it. After all, there is always somebody who is making “outlandish” predictions.

It is illuminating to step back and ask why our intellectual elite failed to make accurate predictions for 2008. How could it come about that large – and apparently successful enterprises like Lehman Brothers – would be forced into bankruptcy and that huge financial institutions in the USA and the UK were only “saved” by massive injections of public funds? A year ago, institutions like the Royal Bank of Scotland and AIG represented what many thought of as highly successful modern corporations. Today, they are partly “nationalized” but, in other times, would have gone out of business.

You could argue, of course, that 2008 was just an exceptional year in which there was a conjunction of several highly unlikely events and, thus, it is not surprising that no one foresaw what was going to happen. Clearly, such events are very rare. You just have to live with them and, since by definition, they occur rarely we need not worry too much in predicting what will happen in 2009 and beyond.

You might take solace from this thought but, I believe, it is also relevant to ask how well our intellectual elite was able to predict major socio-economic events prior to 2008. In other words, how good is the predictive track-record for major events across time? For example, how accurate were the predictions for 2007? How accurate were they for 2006? And so on…. If you check, you will find that many major events were not predicted – bankruptcies, frauds, political regime shifts, and even huge successes. For example, consider the impact of Google in our lives. It’s so important we’ve even coined a verb “to goggle” to signify what we do on a daily basis. You’ve probably almost forgotten what life was like bG – or “before Google.” However, it’s humbling to realize that when the founders of Google tried to sell their business idea for $1.6 million at the end of the 1990s, the best offer they received was $750,000. At the end of 2008, Google was worth more than $173 billion!

The “bottom line” is that accurate prediction just does not exist for most important outcomes in the socio-economic domain. And for those of us working in decision research, this is both bad news and good news. The bad news is that most of the models we work with – whether descriptively or prescriptively – assume that people can characterize the future by probability distributions over sets of possible outcomes. Sure, there are issues as to how well people are able to do this but the basic paradigm remains intact. The specific bad news is that the paradigm is incomplete. You also need to worry about events that are not included in the imagined sample space of outcomes. By failing to do so, you have not modeled all the relevant uncertainty.

The good news is that, once aware of the importance of this situation, we can do something about it. Indeed, we can look upon it as a great stimulus for research. It has the potential to change our paradigm.

Here are some ideas. Let’s try and take some inspiration from other domains. Consider, for example, the prediction of earthquakes. For millennia, people have been trying to predict the occurrence of major earthquakes. There are, however, two important facts that have now been established. The first is that the timing and location of major earthquakes are virtually impossible to predict. The second is that – worldwide – the occurrence of earthquakes follows an almost perfect power distribution. In other words, each year we can be almost sure that somewhere in the world there will be one major earthquake (greater than 8 on the Richter scale), and smaller ones increasing in frequency in a lawful way. So if you live in an earthquake-prone region, you cannot predict when a “big one” will hit. On the other hand, there are precautions you can take, for example, living in earthquake-proof buildings, always keeping stores of water and food at home, and so on. What I am suggesting then is applying the logic behind this kind of protective model to handle what we don’t know might happen. This is a particularly important potential area of research that could be applied to many domains of activity.

A second idea is to apply systematically some quite mechanical rules to augment conventional estimates of uncertainty. For example, if you assess subjectively a 95% confidence interval, double it to account for the possible events you don’t even know about.[1] The use of heuristic rules for decisions is well-accepted in decision research.  Specifying and “validating” the positive use of heuristics for assessing the uncertainty we cannot imagine is clearly a great topic for research.

The state of social science knowledge today is such that we cannot predict many important future events accurately. The worrying fact is that this is not a new phenomenon but, by failing to recognize uncertainty, we have failed to learn from history.  It seems that each generation believes in its uniqueness to the extent of wanting to repeat the errors of the past. In short, we don’t really know what is going on out there – but neither did our predecessors and, in the short term at least, it is unlikely to get much better for our children and grandchildren. I see this as both a challenge and an opportunity for our field.  There is a need to develop new paradigms that can extend our existing knowledge of both how people make decisions and how we can help them to make better ones. [2] If we don’t do this, our society is bound to repeat the errors made by our and preceding generations.

[1]See the interesting work on this topic summarized in P. Juslin, A. Winman, & P. Hansson (2007). The naïve intuitive statistician: A naïve sampling model of intuitive confidence intervals. Psychological Review, 114 (3), 678-703.

[2]The themes in this note were alluded to in the author’s Presidential Address at SPUDM21 in Warsaw. They are also addressed and developed further in a book to be published in 2009,  S. Makridakis, R. M. Hogarth, & A. Gaba. Dance with Chance: Making Luck Work for You. Oxford, UK: Oneworld Publications.