Widespread regional variation in how health care providers treat some conditions in the United States reflects the sobering fact that, for many interventions, there is no consensus about what constitutes effective clinical care. Physicians and health care providers must try to make sense of innumerable and conflicting guidelines in order to choose the best available intervention for their patient.

Scientific, systematic review of data from medical literature and clinical trials is crucial to forming a reliable evidence base of what actually works in health care. With this in mind, professional medical organizations, patient advocacy groups, government agencies and others have synthesized available data on the efficacy of particular interventions and have produced guidelines recommending certain courses of action for specific conditions.

The problem is that there is no consensus among the approaches to systematic review, and, more troublesome, no clear understanding of the best methods for assessing the evidence. The result is a minefield of conflicting information. For example, the National Guideline Clearinghouse contains clinical guidelines from 360 different organizations and lists 471 guidelines relating specifically to hypertension.

A 24 January Institute of Medicine (IOM) report entitled “Knowing What Works in Health Care: A Roadmap for the Nation” recommends that the government step in and create a framework—which the report's authors enigmatically term “The Program”—to impose order on this chaos. [See page 226]

The Program would do three things. First, as resources are limited, it would identify interventions that are priorities for evaluation. Second, it would develop standardized and reliable methods for performing systematic reviews of all the available data about a given intervention. And third, it would develop standards for producing clinical guidelines.

Change on this scale is fraught with potential obstacles, but many of these have been preempted by the report's authors. Conflict of interest is an obvious pitfall, but the IOM recommends the creation of a Priority Setting Advisory Committee to identify high-priority topics for systematic review. To avoid potential bias, the committee would include patients and consumers, payers and guideline developers as well as those with the appropriate clinical expertise. Further, the IOM recommends that a separate Advisory Board—also containing representatives of diverse public- and private-sector interests—be created to oversee the Program.

Getting existing organizations on board is another challenge, particularly as the new structure would be likely to render their guidelines obsolete. The IOM therefore proposes an approach that builds on the expertise of these societies. Professional organizations that currently produce guidelines would still be best placed to do so—not least because guidelines produced by a respected professional society are likely to engender trust in the users; the guidelines would simply have to be produced according to standards developed by the Program.

So far, so good. The actual business of assessing the evidence, however, is much more murky. For a start, as the authors of the report note, systematic review is itself an evolving science, and it is not known how many researchers in the United States are qualified to conduct or develop such reviews. Recruiting sufficient numbers of scientists with the appropriate expertise to develop methods, and providing adequate training in these techniques, will be crucial to the success of the Program.

To complicate matters further, the evidence to be assessed is not always easily available. A study reported in the 17 January issue of the New England Journal of Medicine (http://content.nejm.org/cgi/content/full/358/3/252) highlighted the problem of publication bias. The authors examined trials relating to 12 antidepressants. Based on a review of the published literature, efficacy looked good: 94% of trials had a positive outcome. Perusal of both published and unpublished data collected by the Food and Drug Administration revealed a different story—now only 51% of trials reported positive outcomes.

Recent media reports also suggest that pharmaceutical companies have suppressed negative clinical trial data. For example, lawsuits are being brought against GlaxoSmithKline, which stands accused of holding back early trial data linking its antidepressant Paxil with an increased incidence of suicide attempts.

The IOM proposal is an important step towards providing more reliable evidence of what works in health care, but complete transparency in clinical trial reporting will also need to be legislated to support this goal.