Software: Picture perfect

Journal name:
Nature
Volume:
545,
Page:
S12
Date published:
DOI:
doi:10.1038/545S12a
Published online

Improvements in image capture and viewing experience.

SSIMWave: Seeing the screen

University of Waterloo, Canada

“We want to become synonymous with good user experience for video,” says Abdul Rehman, co-founder and chief executive of SSIMWave. The start-up, which was spun out of the University of Waterloo in Canada in 2013, wants to improve how people watch videos online.

The company's technology is based on a family of algorithms developed by Zhou Wang, a computer engineer at Waterloo and Rehman's PhD supervisor. The structural similarity, or SSIM, algorithms can predict how someone will perceive the technical quality of a video, and thus help to ensure that viewers have the best possible experience. Software scoring based on the SSIM algorithms reflects the ratings given by real viewers with 95% accuracy, says Rehman.

Getting to that point took a long time, and a huge amount of data. Wang and his students — Rehman was the fifth doctoral researcher — spent ten years conducting hundreds of subjective experiments to study the human visual system and how it responds to video content. For example, how different parts of the video capture the viewers' attention, and how variations in contrast affect how the images are perceived.

The algorithm also takes into account how various technical properties of a video affect the perception of quality. Higher resolution does not mean higher video quality by default, says Rehman. “Jamming more and more pixels in won't do you any good if you're not doing a better job at preserving overall quality,” he explains. Resolution, frame rate and dynamic range — the ratio between the darkest and lightest areas of the picture — all need to be taken into account.

The software also looks at the impact of parameters such as screen size, brightness and pixel resolution on different devices and the viewing conditions. Watching a video on an iPhone is very different from watching it on a television, he says. “If you do not take that into account, then you're introducing that much error into your quality measurement.”

Rehman compares SSIMWave's knowledge of audience perception to the data accumulated by another giant of the technology industry. “Google understands people's search patterns,” he says. “We understand their perception of video quality.”

SSIMWave's software helps companies to measure, manage and optimize the viewing experience of videos streamed over the Internet by adjusting the speed of transmission, known as the bit rate, as well as technical aspects of the video such as resolution.

But bit rate does not necessarily mean higher visual quality — using the same bit rate to encode different video content can result in dramatically different visual quality. By adjusting other parameters, SSIMWave can achieve the same perceptual quality at a lower bit rate, saving bandwidth and money. “We felt we had to commercialize the software because the industry needed to move from data quality to perceptual quality,” says Rehman.

The company's clients include the online auction site eBay, the Spanish telecommunications company Telefonica and the computer graphics card manufacturer AMD. SSIMWave is also working with the American Society of Cinematographers.

Rehman attributes the company's success to the university's support. The university takes no cut of the intellectual property developed by its faculty and graduate students, he says. Researchers own 100% of it.

“A university's 'product' is their intellectual property, so for them to say they don't want it is unique,” he says. “I don't know of any other university that does that.”

Artmyn: Smart art

Swiss Federal Institute of Technology in Lausanne

Julian Cassady Photography

The Artmyn scanner incorporates 60 LEDs that each go on and off in a fraction of a second.

Artmyn was born from the marriage of art and science. While working at the auction house Christie's in London, Alexandre Catsicas, who has a background in finance and fine art, saw the potential for a technology that could provide high-resolution displays of artworks to potential buyers. In 2015, Catsicas discovered that a lab at the Swiss Federal Institute of Technology in Lausanne was developing ultra-high-resolution digital-image capture technology. The following year, Catsicas formed Artmyn along with computational imaging researcher Loïc Baboulaz and image-processing engineer Julien Lalande from the institute.

Artmyn begins by taking tens of thousands of pictures of a piece of art under different light sources to produce a computer model of the object that users can virtually manipulate. They can move it around to see 3D details of brush strokes, for example, and view it in different light conditions. The system uses “a combination of new-generation camera technology and highly specialized software algorithms to process the data”, says Catsicas, who is Artmyn's chief executive.

Once the founders had hit on a concept for the company, they had to move quickly, Catsicas says. Potential customers were already knocking on their door. Initial interest came from specialist auction houses, museums and professionals such as art restorers and insurers. Catsicas hopes eventually to also market the system to members of the public who are simply interested in viewing art.

“Carrying an academic project to the real-world is truly an exciting adventure,” says Baboulaz. “It is very rewarding to see what was once just an idea becoming a company driven by a team of highly motivated people.” But he cautions that a company needs someone dedicated to the business side — these tasks cannot be reasonably combined with technical research and development.

Author information

Affiliations

  1. Brian Owens is a science writer in New Brunswick, Canada.

  2. Andrew R. Scott is a science writer in Perth, UK.

Author details

Additional data