What do you do for a living?

Look in the eyes: Hany Farid is "obsessed" with the spot of light reflected in a person's eye.Dartmouth College

I'm an applied mathematician, but I work in a computer science lab [at Dartmouth College in New Hampshire]. My primary research area is developing computational and mathematical techniques to detect tampering in digital media.

Most of my funding comes from law enforcement: I have a grant from the FBI. As well as doing research, I am often approached by people and organizations to help to authenticate digital media — so I've become something of a digital detective.

How's business?

I get so many requests that I've had to start charging for my time. This dissuades many people, but many enlist my help. I've worked on an amazing array of cases — a prisoner had me analyse images of a crime scene, a man accused of adultery had me analyse images purportedly showing him with another woman, a doctor had me analyse images of a patient who claimed that the images had been doctored to cover up botched work. The list goes on and on. I also do quite a bit of expert-witness testimony.

Your software can pick out a number of problems with a photo — including whether the people in it were actually photographed in different places. How does that work?

Our eyes are like mirrors — you can see a lot in them, in the reflected dot of light. I'm obsessed with this. Every time I look at someone I get obsessed with their eyes.

By examining the shape and colour of these dots, you can determine the shape and colour of the light. In addition the location of these dots on the eyes can be used to determine where the light was positioned.

What other tricks do you use?

We can tell if an image or part of an image has been re-sized or rotated. When an image is re-sized, the computer has to fill in the missing pixels. This process leaves behind a very specific pattern, which we can quantify and detect.

We can also detect if one part of an image has been copied and pasted into another part of the image. It is fairly easy to determine if two regions are similar, but it is very hard to determine in what two locations and what shapes these regions might be. There are an enormous number of possibilities to consider.

We have developed an algorithm to find such regions. The basic idea is that we break the image into small regions, and sort these regions according to their numeric values of colour. Sorting is very fast and easy. Then we perform 'region-growing' to see if larger regions are duplicated. Since natural images don't typically have large duplicated regions, their presence is suggestive of tampering.

How accurate are these techniques?

That depends. We can estimate the direction of a light source to within 15 degrees, which is much better than the naked eye. Other techniques depend on the quality of the image and the amount of compression. When an image has been heavily compressed, the compression sometimes destroys the statistical pattern that we are trying to measure.

Now everyone has 5-megapixel cameras and high-quality pictures, that makes it easier for us. It's much harder to tamper with that many pixels. The noise in a poor-quality picture is our curse - noise gets mixed up with tampering.

And do you find much tampering?

The vast majority of images are real. I've found only about five serious fakes in the past few years. I predict, however, that this number will increase.

This is a growing field and more and more people are beginning to work in the area. The Institute of Electrical and Electronics Engineers has a new journal dedicated to this area of research, and The Society for Industrial and Applied Mathematics is creating a new journal as well.

Have you ever had a consultation about scientific photos?

Yes, that's been fascinating. After stem-cell fraudster Woo Suk Hwang, some people are very nervous about photo manipulation in journals (see 'CSI: Cell Biology'). A scientist hired me to check some images generated by his postdocs and graduate students. He was worried that they may have been manipulated. I did detect some tampering, but the types of manipulations were OK with him.

I've also worked with the editor of the Journal of Cell Biology on a few cases where tampering was suspected. In one case, there was a real problem with the images but the authors were able to resolve it.

So what should science journals be doing to guard against this?

I think the real problem is that there isn't a strict policy about what you can and can't do to an image. Forget a strict policy; there isn't even a clear policy. I think they should insist on high-resolution, high-quality images that are harder to manipulate. [_Nature 's policy can be found here

Will forgers still get away with it?

It's an arms race, and I can tell you how it will end: they'll win. It will always be easier to forge a photo than to detect a forgery. But we can take it out of the hands of the amateurs.

Visit our newsblog to read and post comments about this story.  

Dartmouth College