NEWS

How an ordinary camera can see around corners

Digital cameras have been used to reconstruct rough images of hidden objects just by analysing light that bounces off a wall.

Search for this author in:

Experimental setup for Computational Periscopy

Researchers reconstructed an image on a computer screen that was out of sight of a camera with the help of an obstacle.Credit: Charles Saunders

Good news for those who want to know what lies around the corner: electrical engineers have used an ordinary digital camera to reconstruct rough images of hidden objects just by analysing light that bounces off a wall, as if there were a mirror there.

Several teams of researchers have demonstrated devices that could do similar things, but the latest technique is entirely based on an algorithm the team built and requires no special equipment.

“It was thought to be practically impossible to reconstruct an image from only scattered light from a wall without any advanced instruments,” says Allard Mosk, an optical physicist at Utrecht University in the Netherlands.

“It’s surprising to see that you can treat a wall as if it’s a mirror,” says study co-author Vivek Goyal, an electrical engineer at Boston University in Massachusetts. The paper appears on 23 January in Nature1.

Mirrors enable us to see an object even when we do not have a direct line of sight to it. They reflect images faithfully because light bounces off them only at precise angles (at least in the classical, non-quantum explanation, which is accurate for practical purposes).

A mirror sorts the rays of light and redirects them so that our eyes get a faithful image. But when light hits a white wall or another non-shiny surface, it scatters off in random directions. The information is still available, but it is scrambled. Researchers have devised various techniques to unscramble it, which until recently have all required either special lighting (such as scanning by laser), special cameras or both.

Mirror, mirror on the wall

Goyal and his collaborators reconstructed images with a different trick. With a digital camera, they took pictures of light reflected off a wall that emanated from the computer screen hiding around the corner.

But they also placed an obstacle — a dark screen or a chair — in between the computer screen and the wall, and that obstacle blocked some of the light from reaching the wall. Counter-intuitively, the obstacle helps to keep the rays of light from getting too scrambled, which then makes it possible for the algorithm to reconstruct the images from information contained in the light particles that reach the camera. The principle is analogous to the one that allows pinhole cameras (and the eyes of certain molluscs) to produce sharp images without a lens: they block most of the light rays, except for those that go through a narrow hole.

Goyal’s team algorithm starts by reconstructing the position of the obstacle from its shadow, before proceeding to reconstruct the hidden target image itself. Goyal and his collaborators demonstrated that the algorithm could then reconstruct various simple images displayed on the screen, and even an animation.

Mosk calls it “a beautiful example of how you can sometimes turn an obstacle into an advantage”. Other groups last year demonstrated a similar image-reconstruction technique with ordinary cameras, but their approach relied on knowing the shape and position of the obstacle beforehand2.

In principle, a future version of the latest algorithm would also be able to reconstruct the shape of the obstacle without having to know it in advance.

Fun for all

Because the team’s technique does not require special hardware, once perfected, it would be straightforward to turn it into a consumer product. “It’s completely plausible that this becomes a mobile-phone app,” Goyal says, although he has no plans to do so himself.

“Admittedly, it has some fun ‘gee-whiz’ applications, and it probably has some creepy applications, too,” Goyal says.

The authors suggests that the algorithm could be used to monitor hazardous environments, such as burning and collapsed buildings, in navigation and for detecting “hidden adversaries”.

doi: 10.1038/d41586-019-00267-x
Nature Briefing

Sign up for the daily Nature Briefing email newsletter

Stay up to date with what matters in science and why, handpicked from Nature and other publications worldwide.

Sign Up

References

  1. 1.

    Saunders, C., Murray-Bruce, J. & Goyal, V. K. Nature https://doi.org/10.1038/s41586-018-0868-6 (2019).

  2. 2.

    Baradad, M. et al. IEEE/CVF Conference on Computer Vision and Pattern Recognition 6267–6275 (2018).

Download references