Pharmacoeconomics is a relatively new science that has become increasingly important in recent years. Pharmacoeconomics may be defined as the assessment of the costs of clinical outcomes associated with alternative drug treatments (Walley and Haycox 1997). Results of pharmacoeconomic studies are crucial to decisions about governmental approval for new drug indications, formulary decisions in large healthcare organizations, as well as national health-care systems and programs. However, many concerns have been raised about investigator independence and possible bias in the design, analysis, and reporting of pharmacoeconomic studies (Hillman et al. 1991; Kassirer and Angell 1994).

Unfortunately, many researchers and decision makers are not sufficiently familiar with principles of pharmacoeconomics, research design, and data analytic techniques, and, therefore, have difficulty in evaluating the veracity and implications of findings of pharmacoeconomic studies. This is particularly true in the area of psychopharmacology. Although several authors have reviewed pharmacoeconomic methodology in the area of psychopharmacology (Hargreaves et al. 1998; Revicki 1999; Revicki and Luce 1995), impact on the general research community has been modest. To address this issue, the American College of Neuropsychopharmacology (ACNP) established a Task Force to Develop Methodologic Standards for Pharmacoeconomic Research Regarding CNS Compounds. The Task Force comprises members of the ACNP, non-ACNP scientists, and representatives of the pharmaceutical industry with expertise in pharmacoeconomic research. All non-ACNP members had publications in either CNS or related health economics outcomes research. The Task Force developed a set of criteria to assist in the evaluation of the quality of the methods and reporting of a pharmacoeconomic study involving CNS products. This editorial briefly summarizes the deliberations of the Task Force and presents these criteria. We refer the reader for further information to the full text of the ACNP Task Force Report at HYPERLINK http://www.acnp.org/citations/Npp03140192

DESCRIPTION OF THE CRITERIA

Task Force members compiled a pool of over 200 rating criteria and standards from existing pharmacoeconomic research literature, guidelines, and standards. These involved general research concerns, such as whether the hypothesis was clearly stated and whether the methods used tested the hypothesis adequately. Also included were specific pharmacoeconomic concerns, such as whether direct, indirect, and intangible costs were identified and whether the clinical outcomes were defined and linked to pharmacoeconomic outcomes. Direct costs refer to expenses incurred, including the cost of diagnosis, treatment, prevention, rehabilitation, and long-term care. Indirect costs refer to reduced earnings from decreased productivity and absenteeism. Intangible costs refer to a patient's impaired quality of life, including pain, suffering, and stress (Greenberg et al. 1993; Russell et al. 1998).

Redundant criteria were either eliminated or consolidated into 33 individual criteria, which were grouped into seven categories (Scope of Study; Study Objectives; Sample; Methods; Definitions; Results and Discussion; and Conclusions). Each criterion was evaluated on an 8-point Likert scale, ranging from 1 (unreasonable/no support) to 8 (reasonable/full support). Three articles (Boyer et al. 1998; Rosenheck et al. 1997; Sclar et al. 1995), varying in study type (cost identification and cost effectiveness) and disease entity under investigation (depression and schizophrenia) were selected to test the feasibility of the draft criteria. The results of the feasibility study were used to come up with 29 revised criteria (see Table 1), which were regrouped into five domains (Study Scope and Objectives; Pharmacoeconomic Definitions and Assumptions; Design; Results; Discussion and Conclusions). This set of 29 criteria was re-evaluated on a 6-point Likert scale using three new psychopharmacologic articles. These studies were chosen, because they represent modeling, cost-effectiveness, and cost-identification methodologies in schizophrenia or depression pharmacotherapy and pharmacoeconomic research (Ginsberg et al. 1998; Revicki et al. 1997; Simon et al. 1996).

Table 1
Full size table

EVALUATION OF THE CRITERIA

The criterion ratings of the Task Force members on the three articles provided assistance in evaluating the usefulness of the publications. Ratings of an article from each individual criterion were summed within each of the five domains. Then, an over-all summary score was calculated for each article by averaging the means of each of the five domain scores.

Reports with an over-all score of equal to or greater than 5 are likely of high quality. Those in the range of 3.0 to 4.9 are likely acceptable. Those with a score of less than 3.0 are likely unacceptable and should be viewed with skepticism.

COMMENT

The ACNP Task Force has developed and tested a set of 29 criteria that can assist researchers, clinicians, and policy makers in the assessment of the quality of published reports of pharmacoeconomic studies in the psychopharmacologic area. These criteria should be considered guides in assessing quality, and should not be considered a true scale, which would require a significantly larger sample of independent ratings of studies.

Guideposts for reading and evaluating reports have successfully improved the comprehensiveness of scientific studies in other areas. An example is when methodologic variables for studies in depression were formulated at the International Dahlem Conference published by Angst (1983) and then adopted and republished in several other journals (Kupfer and Rush 1983a; 1983b; 1983c).

The present set of criteria is consistent with and extends previously proposed guidelines for economic evaluation of pharmacological products and is developed to evaluate efficiently the quality and credibility of research reports. Because pharmacoeconomics is a young and emerging field, and this set of criteria has not been evaluated using large samples of clinical and health policy researchers, continued development and refinement is warranted, and encouraged by the ACNP Task Force.

ACNP Task Force:

R.M.A. Hirschfeld, Department of Psychiatry & Behavioral Sciences, University of Texas Medical Branch, Galveston, TX,USA; J. Russell, Department of Psychiatry & Behavioral Sciences, University of Texas Medical Branch, Galveston, Texas, USA; M. Buatti, Wyeth-Ayerst Laboratories, Philadelphia, PA, USA; T. Croghan, Health Services and Policy Research Department, Eli Lilly and Company; D.L. Dunner, Department of Psychiatry and Behavioral Sciences, University of Washington, Seattle, WA, USA; G. Fulop, Medical Policy and Programs, Merck-Medco Managed Care, L.L.C.; J. Greden, Department of Psychiatry, University of Michigan Medical Center; A.N. Grudzinski, Pfizer, Inc., New York, NY, USA; G. Hess, Care Sciences, Inc., Philadelphia, PA, USA; T. Hylan, Pfizer, Inc, New York, NY, USA; M.B. Keller, Department of Psychiatry and Human Behavior, Butler Hospital, Brown University, Providence, RI, USA; J.A. Lieberman, Department of Psychiatry, University of North Carolina School of Medicine; R.E. Meyer, Best Practice, L.L.C.; R. Miceli,Pfizer, Inc., New York, NY, USA; D. Revicki, MEDTAP International, Bethesda, MD, USA; J.F. Rosenbaum, Department of Psychiatry, Massachusetts General Hospital, Harvard Medical School, Boston, MA, USA; A.F. Schatzberg, Department of Psychiatry and Behavioral Sciences, Stanford University School of Medicine, Stanford, CA, USA; J. van Loon, Quathos, Advanced Health Outcomes Solutions, The Netherlands.