Modern diagnostic medicine involves countless life-and-death decisions taken on the basis of images formed without normal acts of seeing — using non-visible emissions, artificial perceptual systems and computerized cognition. Layered on top of artificial procedures are our own natural processes for seeing and knowing. Clinicians and technicians who use machine-generated images understand the physical means of image formation, and gain an intuitive sense of how to see what they seek, but they are unlikely to be aware of the mathematics behind image processing.

The more elusive a clear image, the more significant will be mediating procedures. Nowhere is this more apparent than in the scanning of women's breasts for tumours — an urgent matter given that breast cancer accounts for almost a fifth of cancer deaths in women.

X-ray mammography is beset by a series of difficulties, which stock imaging packages do not solve reliably. For example, an image can be sharpened in such a way that the fuzzy edges of a malignant growth acquire the defined boundary of a benign tumour.

Complex structures in breast tissue result in poor signal-to-noise ratio (always remembering that one person's noise is another person's signal). In particular, the tiny signs of crucial microcalcifications are easily lost. The low levels of ‘safe’ doses limit clarity. Scattering of X-ray photons blurs definition. Features at different depths in the breast are confusingly superimposed. And the compression of the breast during the procedure results in non-linear relocations of internal features.

Brady and Highnam's juxtaposed images of an actual X-ray mammogram of a tumour in the left breast (right) and a computer-modelled tumour in the right breast (left).

Michael Brady and Ralph Highnam of the Department of Engineering Science at the University of Oxford are approaching these problems through the building of a highly sophisticated model of the physics of X-rays passing through breast tissue, to such good visual effect that they can closely simulate actual images and replicate the way that different imaging processes transform the information.

They can filter out the particulate noise that masks microcalcifications, augment zones of interest through ‘hot light’ simulation, ape the definition that could theoretically be achieved with mono-energetic emissions, imitate the effect of an antiscatter grid (which in normal practice requires a doubling of the dose), and investigate the 2D effects of 3D structures in normal and compressed configurations.

Such algorithmic modelling is a powerful tool for understanding — and a handy device for teaching. It also permits the processing of a standard image so that those features deemed to be of clinical significance can be highlighted, with a developed awareness of the parameters of what is being done. The approach is potentially applicable to any image-processing system, and is being extended at Oxford specifically to MRI breast scans, for women of any age.

Ultimately, even such meticulously controlled images have to be scrutinized by our own infinitely more complex apparatus, and judgement by the ‘educated eye’ of a clinically-trained connoisseur remains crucial. Indeed, it could happen that such eyes may feel uneasy working with images that look virtually identical but instinctively seem not quite the same.

Experienced and perceptive radiographers acquire a clear sense of how to work with ‘their’ radiologists, operating the machine to produce images that their radiologists can ‘read’, but may experience difficulty in seeing precisely what they want in images taken by another radiographer. This variability does not necessarily mean that one will see a malignant tumour while another sees a benign one, or even sees nothing unusual, but it does indicate the subtly personal factors involved in the making and interpretive viewing of images that to the untrained eye reveal no useful information. Why we see certain things and overlook others remains a highly complex business.

Not only do theoretical models of different types of machine perception and cognition serve as enormously valuable aids within the technical field of diagnosis — permitting the systematic exploration of image formation and processing in a way that is not possible with actual X-rays and actual breasts — but they can also alert us to the way that the perceptual systems of individual observers work to extract meaning from the resulting radiographs.

And, beyond their medical utility, they serve to underline the selective and purposeful nature of all kinds of image processing — artificial and human — and to highlight the marvellously refined way that human vision learns to attune itself to worlds that lie quite outside the compass of our normal seen experience.