In 1991, David Townsend, a physicist then at the University of Geneva in Switzerland, built a low-cost positron emission tomography (PET) scanner. The design left some spaces in the instrument's structure, and Townsend wondered whether he could fill them, and improve the machine, by squeezing a second scanning technology into the gaps. A doctor friend told him that surgeons were more familiar with the anatomical information provided by computed tomography (CT), so he added that, and the PET–CT scanner was born.

Photoacoustic microscopy reveals the microvasculature of a mouse brain, while ultrasonic microscopy gives a glimpse of the skull (grey). Credit: JUNJIE YAO AND LIHONG WANG

At first, many in the medical establishment were sceptical about the instrument's potential. “They all thought David Townsend was kind of loony,” says Michael Vannier, a radiologist at the University of Chicago. “There was a lot of foot dragging associated with PET–CT.”

But what was unfamiliar 15 years ago has since become the norm. These days, “any sensible person would not use PET alone,” says Vannier, adding that the PET–CT scanner has “vastly improved doctors' ability to identify the stage of a lymphoma, allowing more nuanced treatment decisions.” Simon Cherry, a biomedical engineer at the University of California, Davis, agrees. “It's almost impossible to buy a PET scanner without a CT scanner attached to it now,” he says.

That success has spurred other efforts to combine imaging modalities. None of the many ways of looking inside the human body is perfect, but merging the strengths of two or more technologies may allow physicians to see details they have never seen before and improve the detection, diagnosis and treatment of ailments ranging from cancer to heart disease or Parkinson's disease.

Imaging techniques mainly show either structural information — the physical shape of an organ or a tumour — or functional information, such as which molecules are present or what metabolic activity is occurring. But no single technique is optimal for both. Coronary angiography, for example, in which X-rays image a contrast agent flowing through the heart's blood vessels, provides “exquisite detail” of those vessels, but doesn't show whether the cells in the heart muscle are living or dead, says Vannier. If they're dead, coronary bypass surgery would be ineffective. “You could improve the circulation,” Vannier says, “but the heart wouldn't beat any better.”

Similarly, X-ray tomography can spot tiny breast tumours but has trouble telling which are benign and which are malignant. This lack of functional information leads to a lot of false positives, followed by unnecessary surgery, says Lihong Wang, a biomedical engineer at Washington University in St Louis. “If you can tell the function, you can get more information,” Wang explains. “The end result is higher accuracy.”

Wang is trying to improve endoscopy by combining conventional ultrasound with a technology called photoacoustics. Ultrasound endoscopy provides high-resolution images of structures and is widely used to look for oesophageal or colorectal cancer, for example. But its contrast is low, so it does not readily distinguish between blood vessels and lymphatic vessels, or healthy and diseased soft tissue.

Photoacoustic tomography fills that gap by helping to identify different types of molecules1. To obtain a photoacoustic image, Wang delivers pulses of light through an optical fibre to the tissue being examined. When the tissue absorbs the light, it heats slightly and expands, sending out a pressure wave that can be detected by the same sort of receivers that pick up the ultrasound signal. Different substances in the body — water, lipids, melanin — absorb different wavelengths of light, so by selecting the wavelengths, investigators can identify what they're looking at. Two particular wavelengths of visible light, 562 nm and 584 nm, can distinguish between haemoglobin that's saturated with oxygen and haemoglobin that lacks oxygen. This can reveal a tissue's rate of oxygen consumption, which is a measure of metabolic function. Near-infrared light with a wavelength of about 1,200 nm can pick out lipids and allow doctors to tell if a plaque inside an artery is vulnerable to breaking off and causing a clot. Wang suggests that the absorption of photons by DNA could allow photoacoustic imaging of cell nuclei, making it possible to perform a sort of on-site biopsy without removing any tissue.

Both photoacoustic and ultrasound imaging systems use an ultrasonic transducer to process the signals, so they can easily be incorporated in the same machine, Wang says. He is currently working with Philips Healthcare to build a commercial version.

The structural information provided by ultrasound can also be combined with the molecular information available through confocal microscopy, an optical technique that provides high-resolution images, allowing scientists to identify molecules when a fluorescent tag latches onto them. “They're perfectly complementary techniques,” says Christopher Contag, a microbiologist at Stanford University's Molecular Biophotonics and Imaging Laboratory. He suggests going even further in combining techniques: adding a third, such as photoacoustics, would make the images even richer. “Every modality gives you a little bit different information, so a combination of them gives you some varieties to choose from,” he says.

Contag's approach could provide enough information for doctors to perform histopathology inside a living patient, instead of on a piece of excised tissue in a lab. This would not only reduce the need for biopsies, but could also improve the accuracy of the diagnosis. Tissue that has been removed and stuck to a microscope slide might be dried out, reshaped or otherwise changed by the process. Leaving it in place avoids those problems.

Combining modalities in this way could also extend the use of molecular probes. Using dyes already approved by the US Food and Drug Administration to stain tissue inside a patient could make Contag's vision of in vivo biopsy possible — and new molecular probes might make it possible to detect cancers at an earlier stage. One probe, developed by Stanford's Matthew Bogyo, tests for legumain, a protease produced by most cancers in their early stages. Such a probe would make it easy to spot the early signs of cancers in the microscope, Contag says. “If there's a tumour, it's activated and it lights up like a beacon.”

Comparing contrasts

Another approach to making images richer is to mix together not data from different technologies, but different results from the same technology. Multispectral magnetic resonance imaging (MRI) is a case in point. Conventional MRI uses radio-frequency pulses and changes in the magnetic field to create an image. Each specific sequence of pulses provides its own type of image, or contrast, making different types of tissue easier to see.

A group led by neuroscientist Suzanne Corkin of the Massachusetts Institute of Technology combined four of these contrasts to look for early signs of the brain damage caused by Parkinson's disease2. They wanted to measure the volume of a brain region called the substantia nigra, which is known to deteriorate in Parkinson's. Post-mortem examinations of the brains of Parkinson's patients have led to a theory of how the damage progresses, but doctors had been unable to see the changes in a living brain.

Two of the contrasts provided by different pulse sequences show the boundaries of the substantia nigra, but individually each is a bit fuzzy, says David Ziegler, a neurologist at the University of California, San Francisco, who worked with Corkin on the MIT project. A third contrast shows white matter appearing very bright, but cannot distinguish it from cerebrospinal fluid, whereas a fourth makes the fluid look black, so putting those two contrasts together makes it clear which is which. The team took images of volunteers using the four contrast methods in sequence before combining them into a single image. “Now the boundaries of the substantia nigra and the red nucleus [another brain structure] just sort of pop,” Ziegler says. Although such a small study (29 Parkinson's patients and 27 controls) could not provide definitive results, the images seemed to confirm what the post-mortem data had suggested about the way the disease progressed.

We can start going deeper and deeper.

Multispectral MRI could lead to the earlier detection of Parkinson's, and perhaps earlier treatment, and might provide a greater understanding of the mechanisms of the disease. “We can start going deeper and deeper to detect the earliest losses that might be associated with Parkinson's,” Ziegler says. They might learn even more by marrying that ability to PET scans or electroencephalograms.

The trick to multispectral MRI is to take all the scans in the same session. Any MRI image has some distortions, which can be corrected for. But with multiple scans, the images must be aligned correctly for that to work; scans from separate sessions would be hard to line up. “You can't take differently distorted images and combine them in any meaningful way,” Ziegler says. The MIT team scanned the patients with the four contrast methods back to back, then immediately repeated the sequence, in a process lasting about an hour. They then averaged the two sets to correct for artefacts from movement, and then combined the four contrasts into one image.

Clear guidance

The latest combined technology is the PET–MRI scanner, designed by Cherry and only just coming onto the market. Many major US hospitals don't yet own one. Using PET, which is useful for finding biomarkers, adds functional information to the MRI image. The combination could improve imaging of the brain and of the soft tissue in the pelvic area, helping to identify such diseases as bladder or brain cancer. Cherry says the combination provides the best of both worlds: “PET is the most sensitive technology out there. MRI is the highest-contrast technology.”

At roughly US$5 million per machine (roughly double the cost of high-end MRI), PET–MRI faces a high hurdle to widespread adoption. And some scientists are sceptical. “We don't yet have examples where PET–MRI is clearly superior to PET–CT,” Cherry admits.

But Vannier is afraid that being too cautious in developing new combinations of imaging techniques could hobble medical progress. As medicine gets better at detecting more cancers and other abnormalities, he says, it's important to distinguish those that are potentially lethal from those that are unlikely to become life threatening. And better identification of vulnerable plaques could revolutionize coronary care and minimize unnecessary treatments that can do more harm than good. “The ultimate goal is to not over-treat or do one size fits all,” he says.

Imaging that combines structural and functional information might not only increase early detection, but also guide doctors about what to treat and what to leave alone. “The imaging,” Vannier says, “will provide the tool that in clinical practice will allow you to confidently make the decision.”