According to certain Internet sites, this week's lottery numbers can be predicted by identifying hidden patterns in previous draws. Probability theory, of course, suggests otherwise.

Standard economic theory similarly pours cold water on the idea that the behaviour of stock markets is affected by past market movements. Future behaviour of a market, say the textbooks, depends only on events in the real world, such as the profits and losses made by individual companies. So studying patterns in today's trading will not reveal the course of tomorrow's.

But many economists have long suspected that the textbooks are not telling the whole story. Some have argued that past trading does seem to have subtle effects on future fluctuations. Others in the new field of 'behavioural economics' suggest that the irrational psychology of investors lies behind these trends.

Now these dissenters have gained some surprising allies: physicists intent on revealing hidden patterns in market behaviour. The jury is out on whether the field they have created — known as econophysics — will change mainstream economics, but their work is stirring debate. “Economics could use a little shaking up,” says Blake LeBaron, an economist at Brandeis University in Waltham, Massachusetts. “There is no question that there is some interesting work coming out of econophysics.”

Telling tails

For much of the twentieth century, economists believed that the probability of a given change in the value of most markets — such as stock or foreign-currency exchanges — followed a pattern known as a bell curve (see right). This curve has two important properties: averaged across time, the most likely change is zero, and, because the curve tails off rapidly at extreme values, the probability of large fluctuations occurring is very low. Average stock prices may increase or decrease over the long-term, but day-to-day fluctuations seem to follow the bell curve.

For many years, economists believed that market fluctuations could be described by the bell curve.

Analysis of the data, however, shows that market behaviour is subtly different. In the 1960s, mathematician Benoit Mandelbrot, then at IBM's Thomas J. Watson Research Center in Yorktown Heights, New York, together with economist Eugene Fama of the University of Chicago, showed that markets are better described by power-law distributions1,2. Power-law curves look superficially similar to bell curves, but their tails — the regions that cover large fluctuations — are different. Big jumps in market value are more common in power-law systems, giving rise to power-law curves' characteristic 'fat tails'.

But there are many different kinds of power-law curve, each of which has a differently shaped tail. The tail's shape is described using a parameter 'v' — the higher the value of v, the faster the curve falls away, and the thinner the tail.

Over the past 20 years, several economists have shown that, in the case of fluctuations on the German stock exchange, v is greater than three3,4,5. This has important consequences, because statistical theory strongly suggests that systems with v greater than three cannot be random. For stock markets, this would mean that past performance can indeed offer clues about future fluctuations. Other work seemed to back this idea up. Economists already knew, for example, that large jumps in market value tend to cluster together over time6.

The results pointed to a new kind of market statistics. But with the precise value of v unknown, the exact nature of the statistics remained unclear. It is here that physicists found they could contribute. In 1995, Gene Stanley of Boston University, together with former colleague Rosario Mantegna, undertook what was then the most exhaustive study of market fluctuations ever. They analysed movements between 1984 and 1989 of the Standard & Poor's Index of the New York Stock Exchange (NYSE), which covers the 500 largest companies in the United States.

Coming from a physics background, Mantegna and Stanley were used to working with very large data sets. By examining nearly five million data points, they were able to back up the earlier work with a far larger body of data7. Together with colleagues from Boston University, Stanley went on to pin down the value of v at four for the Standard & Poor's index, as well as for the movements of the stocks of a 1,000 individual companies on the NYSE and other exchanges8. Other physicists have reported a similar value for foreign-exchange markets9. By using these truly gigantic data sets, physicists have been able to put much stronger constraints on the possible values of v, says Luís Amaral, one of Stanley's colleagues at Boston University.

Deviant behaviour

Market value: Doyne Farmer believes physicists are having an influence in the economic sphere. Credit: C. RESNICKE

But is any of this work, as the physicists involved believe, making a significant contribution to economic theory? Most economists remain sceptical. Deviations from random behaviour have been documented since the 1980s, they say. Economists already know, for example, that stocks that perform exceptionally well over one year tend to underperform relative to the market over the next10, suggesting that investors irrationally overvalue stocks that have risen in the recent past. “Most of the econophysics papers that I've seen are not particularly innovative,” says Andrew Lo, an economist at the Massachusetts Institute of Technology. “In many cases they are reinventing things that economists have done many years ago.”

But Doyne Farmer, a physicist at the Santa Fe Institute in New Mexico who has worked on problems in economics, defends the contributions of those in his field. Ten years ago, he says, trading firms characterized risk using bell-curve techniques. Many investment companies now use power-law statistics when analysing markets, and econophysicists feel this is in part because of their influence.

Physicists also point out that other techniques they have developed now form part of successful investment strategies. One ingredient in a good strategy is the selection of groups of assets that tend not to go up or down together. Diversifying holdings in this way helps investors to protect themselves against unexpected fluctuations in the market, as a drop in one group of assets tends to be cancelled out by an increase in the value of another. But this simple plan is surprisingly hard to implement, as underlying correlations between groups can be obscured by day-to-day price fluctuations.

Called to account: Luís Amaral (left) and Jean-Philippe Bouchaud believe that analytical techniques commonly used in physics can be deployed to understand and even predict market dynamics. Credit: J. KIRBY, map

Jean-Philippe Bouchaud is one of several physicists who have turned to random matrix theory — a mathematical technique developed to tackle problems in nuclear physics — to help solve the correlation problem. The theory can be used to identify the level of correlation that would be expected even if two stocks were not linked. This can then be used as a baseline for avoiding stocks that really do tend to vary in value together. In 1991, Bouchaud, who was then at the French Atomic Energy Commission in Paris, helped to establish Science & Finance, a Paris-based company that uses random matrix theory and other techniques to help to design investment portfolios.

Science and Finance is now used by several major investment companies. Despite this, many economists insist that physicists have done little more than analyse unusually large data sets. To silence their critics, econophysicists may need to make a more radical contribution to economics. A tantalizing link between behavioural economics and the statistics of power-law distributions may offer them the chance to do just that.

Law enforcement

Power laws in physics are often seen in systems where the 'principle of universality' applies. The overall behaviour of such systems depends only weakly on the precise characters of the individual elements. Take the example of a substance that is heated until it is a mixture of a liquid and a gas. Under certain conditions, this system can be described using only the dimensions of the space it occupies, and the most basic geometrical details of how its constituent molecules interact with one another. All other aspects, such as the types of molecule involved and their precise size and mass, can be ignored. Consequently, crude models can capture important aspects of the system's behaviour.

As markets are also characterized by power laws, econophysicists hope to model them in a similar way. Markets are made up of independent traders, each of which assesses many economic factors before deciding whether to invest in a stock. But if the power-law conjecture is correct, market models need only consider a few crucial details of traders' behaviour, leaving out many economic variables. LeBaron has already used traders' behaviour as the basis for studies of markets11, but the power-law connection encouraged econophysicists to create simpler versions of his model. Initial results suggest that the new approach may have something to offer.

In 1997, for example, physicist Guido Caldarelli, then at the University of Manchester in England, together with researchers from the University of Fribourg in Switzerland, created a computer model of a hypothetical market that contained just one stock. In this model, traders use simple mathematical strategies to predict future market changes, and base decisions to buy or sell the stock on these predictions. One trader might believe that recent trends will continue, for example, whereas another may bank on month-long cycles in market prices. Every so often, Caldarelli and his colleagues removed the least successful trader and introduced a new one with a random strategy in their place.

Despite the model's simplicity, and the use of arbitrary strategies, Caldarelli found that the stock's price behaved similarly to those on real markets — it followed an irregular pattern of rallies and crashes that were distributed according to a power law12. Neil Johnson, a physicist at the University of Oxford, claims to have used a variant of the same model to predict the movements of real markets, and says he hopes to commercialize his findings.

Trade secrets

Some economists have been working along similar lines. In 1999, Thomas Lux, then at the University of Bonn in Germany, teamed up with Michele Marchesi, an electrical engineer at the University of Cagliari in Italy, to build a model in which the actions of one trader can directly influence the opinions and actions of others13. Lux and Marchesi divided traders into several different types. 'Optimists', for example, are more likely than 'pessimists' to buy if the market is rising.

The model shows the kind of behaviour predicted by fat-tailed distributions and hints at how big fluctuations might arise. Rising markets, for example, benefit the optimists, and so prompt some pessimists to switch strategies to become optimists. This increases overall investment, forces prices up and converts more pessimists into optimists. The result is a speculative bubble that eventually ends much as it started: a momentary drop in prices prompts a few optimists to become pessimists, and triggers a wave of conversions to the pessimistic strategy.

Power laws might also be active at even larger scales. In almost every nation, a small fraction of individuals has a large fraction of the wealth, and the distribution follows a simple power law. No one is sure exactly why this should be, but Bouchaud, together with Marc Mézard of the University of Paris South, has produced a model in which 1,000 individuals attempt to make money by trading among themselves or by investing14. The researchers found that the percentage return of investments — with the wealthy on average gaining or losing larger amounts than the less wealthy — inevitably leads to the power-law pattern.

Do these ideas hint at a new way of looking at economics? Physicists admit the models are crude, but argue that they provide valuable insights into financial systems. Others agree. Robert Axtell, a social scientist at the Brookings Institution in Washington, has successfully modelled the way in which companies are distributed by size in the United States15. “My model is very minimal,” he says. “It is so spare as to seem quite unrealistic.” Nevertheless, it produces intriguingly accurate results.

But so far, much of the work has failed to resonate with mainstream economists. Advocates of the models say that entrenched ideas may be to blame. “The idea of explaining statistical characteristics with behavioural models is alien to most economists,” says Lux. “There is a hard core of conservative economists who will never accept these new ideas,” Farmer adds.

Hard sell: physicists must be more innovative to win economists' respect, says Andrew Lo. Credit: M. LUTCH

The trading models go against the grain of standard economic thinking, which assumes that traders make judgements by considering company performance and the larger economic climate, and are not influenced by the actions of others. The models also ignore many details of the market that economists consider important. “Unless physicists are willing to make more of a commitment to understanding the intricacies of markets and economic dynamics, their work will not be taken very seriously by economists,” says Lo.

Lo says that poor communication between the two camps is partly to blame. “There is very little common ground for exchanging ideas in a productive manner,” he says. Farmer, meanwhile, admits that physicists need to catch up on what economists have already achieved. “The time is rapidly approaching when physicists who want to do serious work in finance need to interact more closely with economists,” he says.

If econophysics is to make its mark, it seems there will have to be more mutual understanding.