Nature | Toolbox

Computers on the reef

Software tools that digitize and annotate underwater images are transforming marine ecology.


Article tools

The Project Twins

Blue spires seem to pop out of the photograph in one place; a patch of bushy forms of pinkish-purple in another. To the untrained eye, each is distinct and clearly a coral. Then, Manuel González-Rivero points to a third cluster. The bulbous shapes look like coral, except the smooth grey surface isn't quite right. “The texture and colour suggest that you are probably not looking at a coral,” he says. “More likely you are looking at a crustose coralline alga.”

The image is a high-resolution panoramic photograph collected by the XL Catlin Seaview Survey, a scientific initiative that began to catalogue the world's reefs in 2012. To understand how coral reefs are responding to overfishing, pollution, global warming and ocean acidification, the Catlin team — ecologist González-Rivero among them — is documenting coral abundance, health, structure and biodiversity in millions of underwater snapshots.

XL Catlin Global Reef Record

Neural networks are learning to classify snapshots of coral reefs collected by the XL Catlin Global Reef Record.

It would take decades to go through all these images manually, even with González-Rivero's expert eyes. But the Catlin team is using a neural-networking algorithm: a deep-learning system in which a computer learns to classify what it sees in coral-reef pictures. The project was led by computer scientist Oscar Beijbom at the University of California, Berkeley, and the software can zip through Catlin's gigantic photo album — currently around one million photographs — in a matter of months.

The software is just one example of how coral researchers are embracing advances in computer science and software to speed up under-sea mapping of reefs around the world. Combined with high-quality imagery and sensors that collect standardized biological data about reefs, these tools could unleash an era of semi-automated data collection and monitoring, freeing up ecologists to spend less time processing data and more time doing research.

“It's a tremendous step forward,” says Mark Eakin, who manages the Coral Reef Watch programme for the US National Oceanic and Atmospheric Administration (NOAA) in College Park, Maryland. “When you aren't limited by the speed of people going through and manually processing images, the yield of information is just so much greater.”

Ocean of data

Coral researchers' entry into the world of big data comes none too soon. Long limited by the size of their diving fins and the capacity of their oxygen tanks, marine ecologists are racing to expand their surveys to document and understand the longer-term impacts of rising ocean temperatures and acidification. The bleaching of corals around the world that has accompanied the epic 2015–16 El Niño warming event in the tropical Pacific Ocean has only heightened concerns.

González-Rivero's goal is to cover as much territory as possible to get a sense of how different corals and reefs are responding to these stresses. Computers will never replace the human eye, nor will they obviate the need for detailed underwater investigations and laboratory research, but they can speed up the basic surveys, he says. “What we are trying to do is find a compromise where we get enough information to understand the reef, but at a much faster pace and in a much cheaper way.”

The quality does not have to be compromised: according to Beijbom's unpublished results, the deep-learning system agrees with the human eye on features in coral photos about 81% of the time — impressive considering that even two experts are likely to agree only 84% of the time.

Beijbom plans to launch the algorithm in a few months' time for anyone submitting pictures to his website CoralNet, which already uses computer-assisted systems to help the automated analysis of images. The service is free thanks to funding from NOAA, and 420 users from a variety of institutions, including NOAA, have already uploaded nearly 269,000 images to the site. The best results seem to come from use of a semi-automated program in which the computer does simple analyses and alerts human experts to cases that it's not confident about, Beijbom says.

In many ways, González-Rivero says, marine science is catching up with the terrestrial sciences, which have been developing tools to gather and process copious amounts of data from satellites and aircraft for decades. The software and hardware can't be directly translated to analysing seas, however: the ocean swallows light, so it is difficult to study anything but the shallowest reefs from above.

That has pushed coral researchers to adapt the tools. At Michigan State University in East Lansing, for example, biophysicists David Kramer and Atsuko Kanazawa have modified a handheld sensor originally designed for agricultural research.

When used on land, the sensor measures information such as fluorescence in plants, the carbon content of the soil, the temperature of the air and the humidity. Around 300 sensors are in use in 18 countries, and every time a researcher or a government official takes a reading, the data are uploaded to a central server for analysis.

The modified system, dubbed CoralspeQ, pings reefs with different kinds of light and records the returning spectral signal in 256 wavelengths, from ultraviolet to infrared. These data can be used to measure a reef's photosynthetic activity, for instance, by measuring the fluorescence of chlorophyll in symbiotic algae that provide their host corals with oxygen and nutrients. Knowing how much photosynthetic activity is taking place, and where, could help researchers to identify stressed systems, Kramer says.

The devices use commercially available sensors and are built with the help of 3D printers. Kramer and Kanazawa hope to bring down the cost of the underwater version from its current US$500 and get it into the hands of as many scientists as possible. “We need an army of people making high-quality measurements,” Kramer says.

David Kramer and Atsuko Kanazawa

The ‘CoralSpeq’ sensor measures the photosynthetic activity of coral reefs.

Computer-assisted vision

Marine microbiologist Arjun Chennu has developed an underwater imaging system called HyperDiver to collect even more detailed data across a greater radiation spectrum. Coral ecologists then annotate the images, and the information is fed into an algorithm that is based on open-source machine-learning software and is similar to the one currently used by CoralNet. The machine's 'hyperspectrum' means that it can capture much more information than can the human eye, says Chennu, who works at the Max Planck Institute for Marine Microbiology in Bremen, Germany. This makes it easier to differentiate between corals that look similar in standard images. “For example, we resolve the often-used 'other coral' categories into their proper taxonomic types, and also include sponges, macroalgae and seagrass in our predictions,” he says.

Others have adapted commercially available software that is already used to map landscapes and analyse landslides by overlapping 2D images into 3D models. PhD student John Burns at the University of Hawaii at Manoa's Institute of Marine Biology uses a program called Agisoft PhotoScan, which costs $549 for an educational licence for the professional edition. Free software is available, but it is less sophisticated, Burns says.

The models — which can achieve a resolution of just 1 millimetre when used with good cameras — can be analysed by people or computers to identify coral species and quantify reef coverage. But, because they're 3D, they can also be used to track structural changes as reefs bleach and break down owing to high ocean temperatures — a new kind of ecological information.

For Burns, the beauty of the method is its simplicity: data can be collected quickly and with minimal training. “This method just lets you take hundreds of thousands of single-lens images with your camera, and then you are essentially stitching them together,” he says.

  1. A 3D model of a coral reef can be created by combining overlapping 2D images


  2. A 3D model of a coral reef can be created by combining overlapping 2D images

    Google Earth

  3. A 3D model of a coral reef can be created by combining overlapping 2D images

    John Burns

  4. A 3D model of a coral reef can be created by combining overlapping 2D images

    John Burns

  5. A 3D model of a coral reef can be created by combining overlapping 2D images

    John Burns

  6. A 3D model of a coral reef can be created by combining overlapping 2D images

    John Burns

Standardized store

Sophisticated technologies aren't the only answer, says Emily Darling, a marine ecologist with the Wildlife Conservation Society in New York City. Because separate research efforts are collecting ever-greater quantities of data on coral reefs, it is important that they collect standardized data sets and store them in a repository that can be accessed by the entire community.

In an effort to collect systematic data on the recent global bleaching event, for example, Darling and her colleagues came up with a very simple technology — an Excel spreadsheet that scientists around the world can use to register various data on reef conditions. The value is that when scientists come out of the water, they can immediately import and analyse their data, and Darling now has uniform results from more than 61,000 reef colonies in 13 countries. Roughly 58% showed bleaching.

Ultimately, Darling says, coral ecologists need to converge on some kind of a central repository for the full suite of information that they are collecting around the world. “We need places where data are accessible, where they are telling stories, and where people can go and figure out whether conservation actions are working or not,” she says. “We need to be able to answer those questions a lot faster.”

Journal name:
Date published:



This article omitted to give the name of the system developed by Arjun Chennu and wrongly said that is based on a neural-network algorithm. It is called HyperDiver, and it uses a machine-learning algorithm similar to the one used by CoralNet. The text has been updated.

For the best commenting experience, please login or register as a user and agree to our Community Guidelines. You will be re-directed back to this page where you will see comments updating in real-time and have the ability to recommend comments to other users.

Comments for this thread are now closed.


Comments Subscribe to comments

There are currently no comments.

sign up to Nature briefing

What matters in science — and why — free in your inbox every weekday.

Sign up



Nature Podcast

Our award-winning show features highlights from the week's edition of Nature, interviews with the people behind the science, and in-depth commentary and analysis from journalists around the world.