The United States currently spends at least twice as much per capita on healthcare as most of Europe. It is often thought that US taxpayers are, in effect, bankrolling drug innovation across the globe. But a report from the IMS Institute for Healthcare Informatics in mid-April on the use of medicines in the United States should give pause to anyone who considers that the American healthcare system still genuinely values innovation.

The IMS data indicates that overall US drug spending is reaching a plateau: the year-on-year increases that the industry has depended on for its growth are getting smaller each year, amounting to just 1% in real terms for 2009–2010. In total, spending on generic drugs increased 21% in 2010, while spending on patented medicines fell 0.7%.

And it isn't just a switch from existing brand products to generics; there is a pronounced migration away from newly approved, patented drugs. Whereas in 2006, the sales of newly approved brand drugs accounted for $11 billion in sales (an average of $114 million per drug), figures from last year show that they are down to only $4 billion in sales (an average of just $62 million per drug).

More and more, payers are embracing cheaper, older drugs or generics and de-emphasizing expensive new ones. Troublingly for innovation in key public health areas, this trend appears to be particularly prevalent for chronic treatments of common diseases (such as cardiovascular and neurodegenerative disorders). Chronic disease accounts for nearly half of all US drug spending. And yet, in 2010, two out of three patients who received chronic drug treatment started out on, or were switched to, generics.

In Europe, the mechanism by which drug prices are squeezed has recently been laid bare. A new European Parliament report, Differences in Costs of and Access to Pharmaceutical Products in the EU, shows that the vast majority of national governments in the European Union are not particularly interested in the medical value of medicines when it comes to setting prices. This conclusion emerges because 27 out of 30 countries in Europe use the system of reference lists to set drug prices. In this system, each country sees what neighboring countries are prepared to pay and then sets its own prices accordingly. Sometimes this means the average of all prices, sometimes the average of cheap countries, and often simply the lowest price around. Whatever the national equation, the outcome is that prices are driven inexorably lower.

Price squeezing, of course, pays little heed to the innovation part of the medicines equation. And it is only one of several factors that are steering companies away from truly innovative medicines, not least the spiraling cost and length of human testing and increasingly exacting regulatory requirements. Entering the world of the really new molecular entity is beginning to look like a risky journey for most companies, at least without a push from behind or a pull from the market.

In chronic diseases for which there are already marketed drugs—those with 'partially met needs' rather than 'unmet needs'—the problem of incentivizing industry is particularly acute. The marketed treatments clearly help many patients. They are not ideal, but they are good enough to make developing a better drug a regulatory challenge. And, with generic entry, the downward price pressure on newly marketed products is intense.

Industry still tries to improve on the good, but it has proved difficult. For example, Pfizer tried to extend its cholesterol-lowering franchise by developing torcetrapib as an adjunct to its bestseller Lipitor. Unfortunately, torcetrapib had a tendency to kill more patients than Lipitor. Consequently, when Lipitor slides into the generics pit this year, Pfizer will find a massive hole in its revenues. With increasingly stringent regulatory requirements, these kindS of failures are becoming more common and the huge trials involved more expensive.

Historically, drug companies have been keen to address R&D to unmet medical needs. Government encouragement through orphan drug legislation, for instance, has been profoundly successful. According to a PhRMA report issued in January, there are around 425 orphan drugs in development at US companies. Payers, too, have encouraged a focus on unmet needs. According to BioCentury, specialty drugs now account for ∼15% of US drug spending despite representing only 0.5% of prescriptions. No wonder all the large companies want to adopt orphans.

However, if we really want to encourage innovation that will affect most patients, a mechanism is needed that works as well as orphan drug legislation but addresses diseases that are not rare and for which there are already some treatments.

This could be achieved in several ways. Much as they have used incentives in the fields of neglected disease or homeland security, governments could make advance purchase commitments—essentially agreements to buy a predetermined amount of a novel medicine—for products targeting chronic diseases that meet criteria of improved efficacy or safety.

In Europe, where governments need to reconcile spiraling health system budgets with innovation, more funding could be pushed toward translational research on common ailments. Across the Atlantic, the mission of the proposed National Center for Advancing Translational Sciences could also include a specific program that funnels money into innovative translational R&D on candidates with novel mechanisms in complex diseases with the potential for improved safety or efficacy.

The Food and Drug Administration could help too, by putting an upper limit on the number of people who could be included in a trial. That would allow companies to better control clinical development costs.

Blockbuster products for chronic conditions have ruled supreme for so long in drug development that calling for increased investment in those areas seems counterintuitive. But the fact is that myriad scientific, regulatory and reimbursement challenges are now radically shifting industry's incentives toward rare diseases. Let's not make the mistake of neglecting innovative translational R&D on common Western diseases. In the coming years, these will represent the largest burden on healthcare not just in the West, but everywhere.