The wait for the study’s launch has been prolonged. Credit: David Ellis/Aurora Photos/Corbis

Like proud new parents, US researchers had high hopes for the National Children’s Study (NCS). It would track 100,000 children from birth to age 21, provide a wealth of data about environmental effects on health and yield a greater understanding of health disparities between different ethnicities and income levels. It might even reveal links between exposures and conditions such as asthma, autism and attention deficit hyperactivity disorder, which are increasingly common in children.

European agency to mandate transparency of clinical trials A giant, global, orbiting, quantum-entangled clock Genomes reveal surprising diversity of pre-Columbian Mexico

But 14 years after planning began, with more than US$1 billion spent and 5,050 children enrolled in a pilot phase, the study still lacks the scientific grounding it would need to be fully implemented next year as scheduled, a review by the US National Research Council has found.

“The study has great promise,” says Greg Duncan, an economist at the University of California, Irvine, and chair of the review committee. “But we did identify a number of problems that need to be addressed.”

History of setbacks

The council’s report, released on 16 June, is the latest blow to a study mired in controversy. During the 2000s, the administration of then-president George W. Bush repeatedly attempted to cancel the NCS, only for Congress to restore its funding. In 2012, the study was scaled back in the face of projections that it would cost more than twice the initial estimate of $3 billion over 25 years.

The latest report finds that the study’s protocols for data collection have not yet been finalized or tested, and administrators failed to back up important decisions with scientific documentation. The review panel also says that the scientific hypotheses used to guide study design were poorly defined. “The hypotheses were just silly,” says Nigel Paneth, an epidemiologist at Michigan State University in East Lansing, who was involved with the study before it was scaled back. “They bore no relationship to any public-health goal that I could recognize.”

The panel traces many of the study’s problems to a lack of expertise in the programme’s management office at the US National Institute of Child Health and Human Development. The authors note that the office does not seem to have incorporated feedback from scientists on key decisions, and they highlight a series of ensuing concerns, including an insufficient model for comparing the effectiveness of different study designs. “The panel is concerned that the Program Office may not have sufficient in-house expertise in relevant scientific and survey research disciplines to enable it to function effectively,” the committee writes.

Funding problems and design issues have plagued the study since it was authorized by the Children’s Health Act of 2000. When the pilot study suggested that the initial strategy of going door to door to enlist participants would be too expensive and slow, study planners began recruiting volunteers through group health-care providers. That raised concerns that the study would exclude rural areas not served by such groups. The drive to cut costs also led the programme to contract out data collection to private consulting groups instead of academic investigators.

Under review

In March 2013, amid concerns about the effects of these changes, Congress requested that the National Research Council and the Institute of Medicine review the NCS and withhold payment on contracts related to the study until 60 days after the review was completed.

The resulting report raises valid issues, says Francis Collins, director of the US National Institutes of Health: “They had substantive concerns about the study design and oversight and we ought to take that seriously.”

Collins plans to convene a panel of experts to assess the study’s next steps and to gauge whether it is time, given the project’s long history, to update its design to incorporate technological advances in electronic medical records and ways to assess environmental exposures. “If we’re going to be doing this for 21 years, let’s makes sure we’re making the very best use of everything that’s available to us,” he says.

Duncan declines to speculate on how long it will take the study’s organizers to incorporate the committee’s recommendations, which include soliciting input from outside researchers and incorporating a scientific review-and-approval process. But given the information provided to the review committee, he says that the study was already unlikely to start on time. “We expected to see a lot of completed protocols for sampling and early data collection,” he says. “We didn’t.”