A recurring mantra among articles published in nephrology journals is “randomized controlled trials are needed”. The assumption that underlies this statement is, of course, that randomized controlled trials (RCTs) are the pre-eminent source of clinical evidence. This established wisdom has recently been questioned by the Chair of the UK's National Institute for Health and Clinical Excellence (NICE), the body that has the controversial task of deciding, by rigorous appraisal of clinical data, whether new medicines will be reimbursed by the government. Speaking at the Royal College of Physicians in London, Professor Sir Michael Rawlins surprised his audience by suggesting that RCTs have been placed on “an undeserved pedestal” and calling for a broader approach to the testing of therapeutic interventions. His words warrant particular consideration by nephrologists.

...the cost of conducting an RCT has now reached almost US$15,000 per participant

In 2004, the Cochrane Renal Group reported that fewer RCTs had been published in nephrology than in any other subspecialty of internal medicine. Reliance on this limited number of RCTs to inform the management of kidney disease has resulted in notoriously wide regional variation in treatment guidelines, protocols and outcomes. The Cochrane researchers pinpointed several types of renal disease, most notably glomerulonephritis, where the paucity of trials is particularly marked. Ranking the quality of the few RCTs that had been published as “low”, they drew the damning conclusion that “clinical research in nephrology, and trials in particular, is in crisis”.

The reasons for the dearth of trials in nephrology are manifold. According to one estimate, the cost of conducting an RCT has now reached almost US$15,000 per participant. To this already astronomical figure can be added other nephrology-specific expenses such as the fees imposed by dialysis companies for the use of their facilities. In the face of these costs, pharmaceutical companies—the primary instigators of clinical trials—are obliged to focus their research efforts on disease areas that can provide proportionally large returns. Although nephrologists would argue that chronic kidney disease has reached epidemic status, it nevertheless represents a small market compared with diabetes or hypertension. Subsets of patients (for example, children or individuals with a particular type of kidney disease) are an even less attractive proposition for drugmakers.

In many situations, RCTs are difficult or impossible to conduct. Some renal diseases affect too few people to make an RCT feasible, and ethical considerations limit randomized comparisons of certain treatments (for example, hemodialysis and peritoneal dialysis). Ironically, individuals with kidney disease—who are more likely to die of cardiovascular disease than of end-stage renal disease—are generally excluded from trials of cardiovascular interventions. Even when an RCT can be conducted, there is no guarantee that the findings will be replicated in the real world. Safety issues—such as the adverse cardiovascular effects of coxibs—do not always become apparent within the tightly controlled setting of a trial; similarly, small but clinically meaningful therapeutic effects might not be detected.

Of course, RCTs have undeniable strengths. Freedom from the bias associated with observational studies gives RCTs a unique capacity to debunk sometimes long-held misconceptions. Until the results of the 4D trial (and subsequently, the AURORA study) were published, many clinicians erroneously assumed that the beneficial effect of statins in the general population would also be seen in patients on dialysis. Similarly, the CHOIR and CREATE trials surprised many members of the nephrology community by revealing that high hemoglobin targets are not just unnecessary in individuals with chronic kidney disease, but can even be harmful.

Notwithstanding the successes of some RCTs in nephrology, the fact that such trials are imperfect and often unattainable cries out for regulators to reassess the validity of other sources of clinical evidence, including observational studies, historical controlled trials and case-control studies. The Dense Deposit Disease Focus Group has, for example, proposed the use of a centralized repository of treatment and outcomes data to guide the management of this rare disease. In addition, they suggest that the number of participants required to evaluate novel interventions for the condition could be aided by a Bayesian approach, whereby assumptions are made in advance about the likely effect of treatment. Such flexible thinking is likely to benefit patients more than a rigid adherence to the current hierarchy of clinical evidence. Perhaps it is time to question, rather than simply reiterate, the dogma that RCTs are necessary.