Infection rates dropped slightly during a large HIV-vaccine trial. Credit: K. KASMAUSKI/SCIENCE FACTION/CORBIS

Paris

More than 1,000 researchers in Paris last week rapturously applauded formal results from the largest-ever HIV vaccine trial. In a preliminary announcement in September, the trial, which included 16,402 people in Thailand, was said to show that a vaccine combination reduced the risk of HIV infection by nearly one-third (see Nature doi:10.1038/news.2009.947; 2009).

But some scientists are sceptical, arguing that the response of the HIV research community, long deprived of any good news from vaccine trials, is based more on hope than on rigorous science.

The US$119-million phase III trial, sponsored by the health ministry of Thailand and the US Army, started in 2003. Half of the recipients served as a control group; the other half were given four shots of ALVAC-HIV, an attenuated canarypox virus carrying HIV genes, and two shots of AIDSVAX, a recombinant form of the gp120 HIV surface protein. The trial's results were published on 20 October to coincide with the AIDS vaccine meeting in Paris (S. Rerks-Ngarm et al. N. Engl. J. Med. doi:10.1056/NEJMoa0908492; 2009).

The results are a "milestone in HIV vaccine research", says first author Supachai Rerks-Ngarm, of the Thai Ministry of Public Health.

"Because the history of preventive interventions against HIV has been so poor, the HIV research community has seized on this," counters Peter Smith, a tropical epidemiologist at the London School of Hygiene & Tropical Medicine. "There is not much evidence from the data that it protects at all."

The trial was set up to measure the number of people in each group who became infected with HIV, and the amount of the virus that was circulating in the blood (the viral load) of those who became infected during the trial.

The teams analysed infection rates in three ways (see table). Two of those methods (known as intention to treat (ITT) and per-protocol) found a 26% drop — not statistically significant — in infection rates in the vaccine group compared with the control group. The third method (mITT), the only one presented on 24 September, excluded participants who had contracted HIV between the time they enrolled in the trial and their first vaccination. In this analysis, the reduction between the vaccine and control group infections was 31%, which just scraped into statistical significance.

This showed that statistical significance was highly dependent on whether very small numbers of individuals were excluded from either group. The small numbers of infected individuals in the trial — 132 across both groups — also meant that none of the subgroup analyses was statistically significant.

The ITT analysis is generally considered the main yardstick of the outcome of drug clinical trials, although an mITT analysis is acceptable if agreed by independent experts. For vaccine trials, the per-protocol analysis — in which patients are excluded if they don't strictly adhere to the vaccine schedule — is the most valid, says Adel Mahmoud, former president of Merck Vaccines and now a molecular biologist at Princeton University in New Jersey.

"The results of this trial should be treated with caution and some scepticism," says Tim Peto, a researcher in tropical diseases and clinical medicine at the University of Oxford, UK. "My view is that a more balanced interpretation of the data is that the results show some evidence that the vaccine might be effective and that, unlike previous vaccine studies, this study cannot clearly rule out that the vaccine is ineffective."

Nelson Michael, a researcher at the Walter Reed Army Institute of Research in Silver Spring, Maryland, and director of the US Military HIV Research Program, which co-organized the trial, argues that including people who didn't stick to their shots reflects a real-world vaccination scenario, and that excluding those who were HIV-positive before the trial began was justified.

The trial also failed to detect any difference between the viral load in the two cohorts. Mahmoud calls this "very, very disturbing", because an effective vaccine would be expected to at least reduce the viral load in people who become infected. However, Michael suggests that this finding might still prompt new vaccine leads, and that scientists should see whether they can uncover the immunological origin of any possible vaccine protection.

Dan Barouch, a HIV vaccine researcher at Harvard University in Cambridge, Massachusetts, says that the results are ultimately positive. "We don't understand why we saw the protection that we did, and the results are only modest, but nevertheless the Thai trial provides the first evidence of vaccine protection in humans," he says.

"Everyone is saying let's try to have hope, and this is a hope that the results mean something," says Mahmoud. "But raising expectations with no fundamental scientific base is dangerous."

figure b

figure c