Over the past few years, several independent teams of researchers have noticed something surprising in historical data on a broad set of technologies. Everyone, of course, knows about Moore's Law — for decades, the density of transistors on integrated circuits has doubled every two years, with computational speed advancing even faster. This spectacular record of improvement shows up in just about any metric. Much less known, however, is that this pattern of exponential advance isn't actually limited to electronics; it applies just as well to technologies ranging from cars or batteries to beer or nuclear power.

One study, for example, looked at data for these and other technologies, in 28 domains in all, stretching back over 50 years, and considered the rate of improvement for measures such as performance and energy efficiency. The data for all these technologies follows a similar exponential improvement, although the timescale for doubling can be very different, with annual improvement rates ranging from 3 to 65 per cent. LED performance, for instance, has been improving by a factor of about ten each decade, whereas batteries have improved more slowly — ten-fold improvement requiring about 40 years. Even so, it's the same pattern: Moore's Law is a law of technology in general (see C. L. Magee et al., available at http://go.nature.com/sxJbnv; 2014).

Technology is close to discovery and invention, and we're used to thinking of these as largely unpredictable processes. They're chancy, success being furthered by effort and investment, of course, but also requiring some luck. So the regularity of this pattern across the whole domain of technology is surprising. It's not yet clear what it means. But it may make it possible to make actual predictions of technological progress using historical data alone, rather than relying on experts.

It's possible to make predictions of technological progress using historical data alone.

That idea comes from the work of physicist Doyne Farmer and economist Francois Lafond, who have built on these empirical studies. Their idea was to model the data on technology improvement as a stochastic process, using a geometric random walk, which naturally exhibits an exponential, long-term growth (or drift) decorated with erratic noise. Fitting such a process to the data on a broad class of technologies, they then tried to stand at one moment in time and to use past data to predict future trends. They found good success in making such out-of-sample forecasts. They don't claim these are the best forecasts possible; only that this simple method works quite well.

Anyone can make predictions, of course. And historically, many predictions of technological futures have been spectacularly wrong. The really important thing is having some knowledge of the likely accuracy of a prediction. Indeed, as Farmer and Lafond point out, predictions of low accuracy can even be dangerous if they're trusted. Because the mathematical properties of the geometric random walk are well studied, they were able to go further in deriving a closed form expression for each particular technology and to get an estimate of the expected distribution of forecast errors for projections over any interval of time. For the specific case of solar photovoltaic modules, for example, the method suggests that the price of such modules will most likely continue to drop at about 10% annually, but that there's still a 5% chance that prices in 2030 will be higher than today.

This new capability should be valuable to anyone charged with making decisions on which technologies to invest in, especially policy makers aiming to invest public funds wisely in response to problems such as climate change. In pursuing alternative energy sources, for example, we should expect a lot more from investments in photovoltaics than from alternatives such as biofuels or wind. It's possible to make such claims not because we know anything about why technologies work this way, only because the data implies that they do. Yet this data may also offer hints regarding the mechanisms that make one technology grow faster than another.

Many people have likened technological advance to an evolutionary process, advancing as older techniques, components or ideas get combined in new ways. Biologists know that some organisms evolve and adapt more rapidly than others due to features that make it relatively easy to alter some elements — cell surface receptors in bacteria, for example — without undermining other underlying functions. Such independent flexibility enables fast, profitable experimentation, and creates the capacity for rapid evolution. In a recent study, Subarna Basnet and Chris Magee at MIT find evidence that something very similar seems to be true with technologies (preprint at http://arxiv.org/abs/1601.02677; 2016). The faster evolving ones seem to have fewer interactions or complex interdependences between their elementary components.

In the case of technology, interactions among components or properties happen in many ways. A spring mechanism might improve a device's performance if it were made stiffer. Making it bigger might do that, yet would also increase the device weight, which might be bad. Steps to speed up transistors may be good for computing rates, yet also create more heating and the need for cooling systems. Engineers encounter these interactions all the time. In a clever way, Basnet and Magee found a way to get a rough ranking of technologies by the number of such interactions they involve. They looked at the patent literature and measured the frequency of six specific keywords — prevent, undesirable, requirement, fail, disadvantage and overcome — they suspected might reflect important interactions among components or processes. Across the board, in technologies ranging from milling machines to superconductors to 3D printers, they found that the rate at which a technology has improved is significantly correlated, inversely, with the frequency of these words in relevant patents. The more interactions described in the reports, the slower the technology advances, apparently because finding beneficial changes is just harder to do. Complexity slows discovery and advance.

This doesn't explain, of course, why some technologies are more complex than others. Naively, one might think that older technologies, after long development, would grow more complex, but that's not consistent with Moore's Law holding for individual technologies over many decades. For now, we just don't know.