Skip to main content

Inpaint by Numbers

An algorithm to automate the repair of moving images


On supporting science journalism

If you're enjoying this article, consider supporting our award-winning journalism by subscribing. By purchasing a subscription you are helping to ensure the future of impactful stories about the discoveries and ideas shaping our world today.


Astute moviegoers may have noticed a traveling car and its trail of exhaust in a scene from Peter Jackson's The Lord of the Rings: The Fellowship of the Ring. Current tools, such as Adobe Systems's After Effects "healing brush," can readily camouflage, or inpaint, small blemishes. To fix larger gaffes, though, the user must tirelessly cut small patches of image and paste them over the unwanted object. The effort typically yields mediocre results for all but the smallest repairs: on the DVD of Fellowship, a blurry spot is visible where the car was.

But new software using advanced mathematics may soon enable video editors to automatically inpaint a moving object quickly and seamlessly. It can even massage away large moving objects that hide other action, according to the software's principal developer, computer scientist Guillermo Sapiro of the University of Minnesota.