Economists need their own uncertainty principle

Bad risk management contributed to the current financial crisis. Two economists believe the situation could be improved by gaining a deeper understanding of what is not known, as Philip Ball explains.

Former US defense secretary Donald Rumsfeld is an unlikely prophet of risk analysis, but that may be how posterity will anoint him. His remark about 'unknown unknowns' was derided as a piece of meaningless obfuscation, but more careful reflection suggests he had a point. It is one thing to recognize the gaps and uncertainties in our knowledge of a situation, and another to acknowledge that unforeseen circumstances might entirely change the picture.

The economy is as prone to any human endeavour to unknown unknowns - but economic decision-making is commonly misled by confusing them with known unknowns. Financial speculation is risky by definition. Yet the danger is not that the risks exist but that the highly developed calculus of risk in economic theory — for which Nobel prizes have been awarded — gives the impression that the risks are under control.

The reasons for the current financial crisis have been picked over endlessly, but one widespread view is that it involved failures in risk management. Facing up to these failures could prompt the bleak conclusion that trying to anticipate the economic future is an impossible task. That's the position taken by Nassim Nicholas Taleb in his influential book The Black Swan¹, which argues that big disruptions in the economy can never be foreseen, and are much more common than is evident from conventional theory.

How should those still working in financial markets absorb this pessimistic message? In a preprint on arXiv², Andrew Lo and Mark Mueller of the Sloan School of Management at the Massachusetts Institute of Technology, in Cambridge, suggest that what economists grappling with uncertainty need is a proper taxonomy of risk — not unlike, as it turns out, Rumsfeld's infamous classification. In this way, they state, risk assessment in economics can be united with the way uncertainties are handled in the natural sciences. It may then become clearer where conventional economic theory is a reliable guide to planning and forecasting, and where its predictive value fails.

Physics envy

The current approach to uncertainty in economics, write Lo and Mueller, suffers from physics envy. "The quantitative aspirations of economists and financial analysts have for many years been based on the belief that it should be possible to build models of economic systems that are as predictive as those in physics," they point out.

Much of the foundational work in modern economics took its lead directly from physics. One of the principal architects of modern economics, Paul Samuelson, acknowledged that his seminal book Foundations of Economic Analysis³, published in 1947, was inspired by the work of mathematical physicist Edwin Bidwell Wilson, who was a protégé of the pioneer of statistical physics J. Willard Gibbs.

When Samuelson formulated his ideas, physicists had come to recognize that the uncertainties of random thermal noise could be described by a normal, or Gaussian, distribution of fluctuations (the classic bell curve).

Economists should recognise the existence of uncertainty that their models can't capture.

Economists have known since the 1960s that fluctuations in commodity prices are different. They don't fit a Gaussian distribution but are 'fat-tailed', meaning that they have a greater proportion of big deviations, compared with a bell curve. Even so, many standard economic theories have failed to accommodate this deviation from the Gaussian form, most notably the celebrated Black–Scholes formula used by traders to calculate the price they should pay when trading with the financial instruments known as options.

Incorrect statistical handling of economic fluctuations is a minor issue compared with the failure of market traders and managers to distinguish fluctuations that can in principle be modelled from those that are more qualitative — to distinguish, as Lo and Mueller put it, trading decisions (which need maths) from business decisions (which need experience and intuition).

Quantifying uncertainty

The conventional view of the origin of economic fluctuations — that they are caused by 'external' shocks to the market, delivered, for example, by political events — has truth in it. And these external variables can't be meaningfully factored into the equations. Irreducible uncertainty, write Lo and Mueller, "cannot be modeled quantitatively, yet has substantial impact on the risks and rewards of quantitative strategies".

They propose a five-tiered categorization of uncertainty in any system, whether it be physical, economic or political. The classification ranges from complete deterministic certainty, exemplified by Newtonian mechanics (where once you have the equations, you can predict the future perfectly), through noisy systems and those that must be described statistically because of incomplete knowledge about deterministic processes (as in coin tossing), to 'irreducible uncertainty'.

They describe this last type of uncertainty as "a state of total ignorance that cannot be remedied by collecting more data, using more sophisticated methods of statistical inference or more powerful computers, or thinking harder and smarter" — something like the "unknown unknowns". Physical systems tend to have more upper-level, easily modeled uncertainty; social systems more of the lower-level, imponderable kind.

The authors think that this is more than just an enumeration of categories, because it provides a framework for thinking about uncertainty. "It is possible to 'believe' a model at one level of the hierarchy but not at another," they write. It's rather like saying we can place some trust in daily weather forecasts, but not much in monthly ones. And they sketch out ideas for handling some of the more challenging unknowns, for example when qualitatively different models may apply to the behaviour of markets at different times or under different conditions.

They call for more support of postgraduate economic training to create a cadre of better informed practitioners, who are more alert to the limitations of the current economic models, such as those used to calculate expected daily returns on investments in a 'business-as-usual' market. They point out the dangers of devolving business management decisions to financial analysts who have become accustomed to thinking that their models capture all there is to say about economic risk. But to truly eliminate the ruinous false confidence engendered by the clever, physics-aping maths of economic theory, why not make it standard practice to teach everyone who studies economics at any level that their models apply only to specific and highly restricted varieties of uncertainty?