Power and significance: clinical trial benefits with advanced covariate adjustment

Success is hard to come by in drug development. “On average, 50% of phase III trials with novel therapeutics fail,” says Subha Madhavan, vice president and head of AI/ML at Pfizer. “A primary reason for failure is that many clinical trials do not reach statistical significance.” There will always be some risk and uncertainty when undertaking a trial, of course, but the odds can be improved. A well-known statistical procedure called covariate adjustment can improve the success rate — if applied in an optimized way.

All clinical trials collect data about each patient, from their age and gender to disease location and status. Any of these covariates could have an impact on the patient’s survival. “There are many different types of noise that data scientists have to tackle in clinical trial data,” Madhavan explains. “From a patient-centric perspective, there can be biological variability among trial participants, which can make it challenging to detect subtle treatment effects across the population.”

The key to improving trial success rates is to distinguish the genuine treatment effects from variability that can be attributed to other factors. “Covariate adjustment is about increasing the signal-to-noise ratio by removing the noise that is explained by prognostic variables,” says Félix Balazard, director of optimized development at Owkin, an AI-based biotechnology company. Prognostic variables are baseline characteristics (such as age) that affect health outcomes even in the absence of a treatment. If unadjusted for, highly prognostic variables risk masking the signal in the analysis, and therefore the probability of success. “By investing in advanced covariate adjustment, you can stack the odds in your favour.”

It is crucial to determine which covariates to adjust, to improve the quality of the signal and therefore improve the significance of results. Owkin’s research has helped it develop an AI-powered approach that identifies the best covariates and how to adjust for them. As a result, Owkin believes it can help the pharmaceutical industry improve the success rate of its drug pipeline, and bring more proven treatments to patients, faster.

Overcoming subjective methods

Covariate adjustment is regularly used now, although not always well, says Sean Khozin, an impact adviser at Owkin who has led an incubator on real-world evidence at the US Food and Drug Administration (FDA). One of the most common uses of covariate adjustment is in oncology, where patients tend to be heterogeneous, with different stages of malignancies. The problems, he notes, arise in terms of how it is used.

The initial step is selecting the covariates to adjust for. “Not a lot of analytical effort goes into identifying those variables,” Khozin says. “The method can be rather subjective.” For example, many trial coordinators might choose to adjust the same covariates that were adjusted for in previous trials. Such an expert-driven approach misses the opportunity to optimize the covariate adjustment based on external data matched to the data that will be collected in the upcoming trial (see ‘The different routes to covariate adjustment’).

“Better choice of adjustment covariates leads to improved statistical power and therefore reduced probability of trial failure,” Madhavan notes. Technology, specifically artificial intelligence (AI), helps improve those choices. “AI provides new sources of prognostic information that were unattainable using previous methods,” Madhavan says.

High-powered histology

Owkin is pioneering an AI and machine learning (ML) approach, part of which relies on image processing. For example, Owkin scientists have developed HCCnet, a deep-learning model that can evaluate histology slides of resected hepatocellular carcinoma (HCC). It proved so accurate they started using it to produce additional covariates¹. The researchers evaluated the impact of covariate adjustment in terms of either gain in statistical power or reduction in sample size required to achieve a desired statistical power. With HCCnet, they were able to increase statistical power from 80% to 85%, or alternatively to reduce patient enrolment by 12% while retaining 80% statistical power².

HCCnet, and other similar models, are expanding the types of data that can be used to find covariates. “Histology is not a new modality, but with a data-driven approach using deep learning for covariate adjustment, we were able to extract information on the patients,” Balazard says. Deep learning can conduct a much more thorough examination of the histology images than most experts can achieve. “We can capture information that wasn’t obvious before,” he adds, such as details of the complex microenvironment of tissue around a tumour.

These prognostic models are an extension of the deep-learning technology Owkin has used to build other models. Another iteration is MSIntuit CRC, a CE-marked, in vitro diagnostic medical device used to pre-screen for a specific molecular alteration in colorectal cancer (CRC) from digital pathology slides.

Reaching for better regulation

Expanding the reach of advanced methods of covariate adjustment in clinical trials depends in part on increased regulatory support. That’s improving.

In April 2023, Owkin received a Letter of Support from the European Medicines Agency (EMA) concerning its innovative approach to building prognostic covariates using deep learning of histology slides for oncology trials.

And more broadly, in May, the FDA released guidelines around using covariate adjustment in randomized clinical trials.

“With the new focus on this topic provided by the FDA guidance, we are eager to go beyond that and look at how data collected from prior trials and real-world evidence can help improve the choice of covariates,” says Madhavan.

The combination of regulatory endorsement and interest from industry will accelerate the optimized use of covariant adjustments in clinical trials, which should ultimately benefit patients. As Khozin says, “Integrating data and AI-based analytical tools will have a direct correlation with expediting current trials and making them more efficient.”