Joe Amon/Denver Post/Getty
In an unusual twist on biometrics research, US computer scientists have joined with law-enforcement officials to find new ways to automatically detect tattoos on people in photographs. The work aims to replace time-consuming, manual searches through body-mark databases with faster, computer-driven image matching to help identify crime victims and criminal suspects.
Under the standard, word-based system of classifying tattoos, two suspects might both be labelled as having a tattoo of a cat. In reality, as a photograph would show, one has a tattoo of a snarling tiger whereas the other has a beaming Hello Kitty cartoon design. Pattern-recognition algorithms can recognize the difference by extracting information about shapes, features and colours in tattoo photos, researchers reported at an 8 June workshop in Gaithersburg, Maryland.
Six research teams are competing in a 'tattoo recognition technology challenge' to see which algorithms are best at various tasks. These functions include identifying the same tattoo in different photos, such as a swastika photographed at different times on the same person; identifying similar tattoos on different people, such as a gang sign; or picking out one element of a tattoo and realizing it is part of a larger design — for instance, a partial spiderweb on the neck of a bank robber caught on a security camera.
Tattoo matching will never be as precise as fingerprints in identifying a particular person — but it does not have to be, says Stephen Fischer, chief of the multimedia division at the US Federal Bureau of Investigation's (FBI's) Criminal Justice Information Services division in Clarksburg, West Virginia. “It only needs to provide some investigative benefit to aid in narrowing down suspect lists or providing investigative intelligence related to a crime,” he says.
One in five US adults has a tattoo. Many are decorative or have an innocuous personal significance, but others reflect membership in a gang or other possibly criminal group the person has chosen to identify with. That makes tattoos a useful source of information if law-enforcement officials have to make a quick judgement, such as on the street or at an international border. “They need to know, does this person represent a threat, and they need to know that really quick,” says Daniel Olson, a cryptanalyst at the FBI in Quantico, Virginia.
For now, law-enforcement officials must rely on a classification system that has 70 possible ways to describe a tattoo in words. The descriptors are as generic as 'male face' and 'female face', and provide no way to capture the abstract and symbolic designs that are becoming more popular, says Mei Ngan, a computer scientist at the US National Institute of Standards and Technology (NIST) in Gaithersburg. In addition, how a tattoo gets classified can vary; one investigator might label a scary clown face as a human face, whereas a second person might call it abstract and a third might label it as a symbol.
Because of the huge variation in designs, tattoo recognition is more complicated than face recognition, says Landis Huffman, an engineer at MITRE, a research company in Bedford, Massachusetts. To get around that complexity, his team uses a common image-matching technique called a scale-invariant feature transform. It hunts for key visual points in a pair of photos, then calculates how many of those points match up. For the NIST challenge, the researchers added tattoo-specific features such as searches for skin-coloured areas around patches that are not skin-coloured.
Other challenges faced by the researchers include the fact that poorly executed tattoos fade or become blurry over time. Body hair can get in the way, obscuring some designs. And tattoos can be added to, so that what started as a rose tattoo, for instance, becomes part of a much more elaborate pattern.
Computers can recognize the same tattoo in different photos of one person more easily than they can match similar-looking tattoos on different people, says Peter Lo, senior research manager of the biometrics research firm MorphoTrak in Anaheim, California. Even harder is matching sketches or other graphics of tattoos against actual tattoos on the skin.
One day, the NIST project could help law-enforcement officials to standardize the way they collect and identify tattoo images, Ngan says. It could also inform policing in real time. At Purdue University in West Lafayette, Indiana, Edward Delp leads a team that has concentrated on analysing images of gang graffiti. Now he is applying the same image-analysis techniques to tattoos, and putting the information in a map-based mobile app so that police officers can work out where similar tattoos and graffiti are popping up across a city.
- Journal name: