An impressive book argues that we should embrace failure in economic and social progress, finds Matt Ridley.
Charles Darwin's big idea — that blind trial and error can progressively build a powerful simulacrum of purposeful design — got pigeonholed under biology. Yet it always had wider implications in economics, technology and culture. Darwin probably drew some elements of his bottom-up thinking from the political philosophers of the Scottish enlightenment, notably Adam Smith and Adam Ferguson. Biology is now returning the favour.
Books such as Steven Johnson's Where Good Ideas Come From (Allen Lane, 2010), Kevin Kelly's What Technology Wants (Viking, 2010) and Brian Arthur's The Nature of Technology (Free Press, 2009) are suffused with concepts from natural selection, as is my own, The Rational Optimist (Fourth Estate, 2010). Tim Harford's Adapt follows this tradition, focusing on the key role of failure — the 'error' in trial and error — in economic and social progress.
Harford's thesis is that “trial and error is a tremendously powerful process for solving problems in a complex world, while expert leadership is not”. Whether designing computer games, improving foreign aid or discovering how to knock out genes, the heroes and heroines of Harford's book get results not by designing clever solutions and imposing them, but by trying variations and selecting the few that work from among the many that don't. Intelligent design is just as bad at explaining politics and business as it is at explaining evolution.
The Google company — headed by Sergey Brin (left), Eric Schmidt (centre) and Larry Page — encourages employees to spend time toying with their own ideas.
Harford's case histories are well chosen and artfully told, making the book a delight to read. But its value is greater than that. Strand by strand, it weaves the stories into a philosophical web that is neat, fascinating and brilliant. Like the best popular science, it advances the subject as well as conveying it, drawing intriguing conclusions about how to run companies, armies and research labs.
The book's message will be music to the ears of many scientists, for Harford exposes the dismal inefficiency of the preconceived, top-down grant-giving that funds much of modern academic research. He celebrates instead the power of prizes and blue-sky funding, and even molecular geneticist Mario Capecchi's documented Nobel-prizewinning decision to use grant money given for one purpose for another. Innovation and discovery come from pluralism and serendipity, not command and control.
Yet this will also be an uncomfortable book for some scientists who read it carefully. For however much they celebrate bottom-up, emergent, evolutionary order in the genome or an ecosystem, most scientists embrace intelligent design as soon as they turn to politics or economics, with government planning playing the part of God. The messy, competitive, pluralistic, unplanned nature of the marketplace is too often anathema to the scientific mind.
A good example is climate policy. Harford shows how exhortations from on high to people to cut their carbon footprints, or winner-picking by governments for advancing certain technologies, is ineffective and counterproductive. Why? Because, he explains (citing chemist Leslie Orgel), “evolution is smarter than we are, and economic evolution tends to outsmart the rules we erect to guide it”. A planning rule that forces British developers to install a minimum amount of on-site energy generation in new office buildings has led to the lunatic spectacle of convoys of diesel-drinking trucks taking carbon-rich wood from forests to biomass boilers in city centres because solar and wind power cannot meet the requirements on such small scales.
Trial and error cannot be used for everything. Nuclear power stations and banks must work without melting down lots of times first. Harford's analysis of what went wrong at the Piper Alpha oil-rig explosion in the North Sea, the Three Mile Island nuclear meltdown in Pennsylvania and the Lehman Brothers bankruptcy in the United States is illuminating and intelligent. Ineffective safety systems, latent errors and overlooked whistle-blowers are common in all such disasters, but the key ingredient is tight coupling: the systems were designed so that if one part failed, others went with it. To eliminate errors, writes Harford, is “an impossible dream. The alternative is to try to simplify and decouple these high-risk systems as much as is feasible.”
Harford provides some evidence that a new era of evolutionary business is dawning, although the basic idea is as old as the limited-liability company — “a safe space within which to fail”. Companies such as Google have taken the model of the 'skunk works' — the name for the trial-and-error division of Lockheed that came up with aircraft such as the U2, the blackbird and the stealth bomber — and rolled it out through the whole firm, by encouraging employees to spend 20% of their time on their own projects. Google's products, as well as its ideas, are designed so that they evolve by trial and error.
Yet vast swathes of the world are resistant to the implications of this selective approach. Government agencies, academic institutions and financial behemoths are not allowed to fail. “Government regulations,” writes Harford drily, “by their very nature, tend to be somewhat impervious to the possibility of improvement.”
It would be hard to improve Harford's outstanding book. If pressed, I might say that the focus on variation and selection leaves no room for discussion of the other elements of evolution, especially replication and recombination. Nonetheless, Adapt is fine, funny and fluent.