Wide Field-of-View Fluorescence Imaging of Coral Reefs

Abstract

Coral reefs globally are declining rapidly because of both local and global stressors. Improved monitoring tools are urgently needed to understand the changes that are occurring at appropriate temporal and spatial scales. Coral fluorescence imaging tools have the potential to improve both ecological and physiological assessments. Although fluorescence imaging is regularly used for laboratory studies of corals, it has not yet been used for large-scale in situ assessments. Current obstacles to effective underwater fluorescence surveying include limited field-of-view due to low camera sensitivity, the need for nighttime deployment because of ambient light contamination and the need for custom multispectral narrow band imaging systems to separate the signal into meaningful fluorescence bands. Here we describe the Fluorescence Imaging System (FluorIS), based on a consumer camera modified for greatly increased sensitivity to chlorophyll-a fluorescence and we show high spectral correlation between acquired images and in situ spectrometer measurements. This system greatly facilitates underwater wide field-of-view fluorophore surveying during both night and day and potentially enables improvements in semi-automated segmentation of live corals in coral reef photographs and juvenile coral surveys.

Introduction

Coral reef ecosystems are in a state of crisis, suffering massive global declines over the last three decades including up to 80% loss of coral coverage in the Caribbean¹ and approximately 50% in both the Indo-Pacific² and the Great Barrier Reef³. These declines are occurring rapidly over large spatial scales, often at over 1% loss of coverage per year^2,3 due to both local (e.g. pollution, nutrient enrichment, overfishing and sedimentation) and global stressors (global warming, ocean acidification and sea level rise)^4,5,6. Thus, new non-invasive, rapid underwater ecological monitoring tools are urgently needed to better understand how coral physiology and reef ecosystems are responding to these stresses on spatial and temporal scales relevant for management.

Coral fluorescence imaging can complement standard underwater imaging for both laboratory and in situ monitoring of corals, but current fluorescence camera systems are of limited practical use for ecological surveys. Methodological obstacles include low camera sensitivity to the full relevant spectral range of coral fluorescence emissions, the need for powerful illumination and small field-of-view (FOV) due to low sensitivity and the need for nighttime deployment in order to avoid signal contamination by ambient light. Thus, there is a need to develop an in situ fluorescence imaging system capable of performing large-scale coral reef surveys (Fig. 1a).

Figure 1

Fluorescence imaging basics.

(a) Daytime deployment of FluorIS in Moorea, French Polynesia. (b) Characteristic excitation (dashed-green) and emission spectra (green) of GFP⁸ and in vivo chlorophyll-a excitation⁹ (dashed-purple) and emission⁸. Peaks are normalized to 1 for illustration. The excitation spectrum of the filtered strobe is depicted in dashed blue, with a peak at 467 nm. The transmission spectrum of the barrier filter is depicted in dashed yellow (data from Tiffen Inc.). (c) The physics of fluorescence imaging during night and day and system components required for fluorescence imaging. On the left, a reflectance image taken under white illumination and on the right, fluorescence images taken during night and day are shown. The coral images were taken in Bocas Del Toro, Panama at a depth of 8 m. The color of the arrows in the schematic represents the light spectrum of each respective component. (1) In order to image the chlorophyll-a fluorescence a camera that is sensitive to long wavelengths (>650 nm) is required. (2) The filtered excitation strobes emits short wavelengths, typically <500 nm. (3) A long-pass barrier filter is mounted on the camera lens to block reflected light. During nighttime, only fluorescence excited by the strobes is imaged, assuming the barrier filter fully blocks the excitation spectrum. During daytime, additional reflectance and fluorescence from ambient light are recorded. The reflected ambient component that is not filtered by the barrier filter appears green or yellow in the image, as seen in the sand to the left of the corals in the daytime fluorescence image.

Fluorescence is defined as the reemission of photons with longer wavelength than the absorbed photons⁷. Most scleractinian coral tissue contains two primary groups of fluorophores (characteristic spectra are depicted in Fig. 1b). The first major group are photosynthetic pigments from the symbiotic dinoflagellate algae (Symbiodinium spp.) living within the coral tissues, with chlorophyll-a the predominant pigment. Chlorophyll-a emits with a primary peak at ~685 nm and a secondary peak at ~730 nm^8,9. The second group are the fluorescent proteins (FPs), found in the coral animal tissue with several different classes exhibiting typical emission peaks between 482 nm to 609 nm^10,11,12,13. Within the varied groups of existing FPs, the FluorIS was developed to image green fluorescent proteins (GFPs), as their excitation range partially overlaps with chlorophyll-a, enabling simultaneous imaging of GFP and chlorophyll-a with a single excitation source. The excitation of cyan FPs overlaps with this excitation range as well¹². However, they appear green in the images as the commercial yellow barrier filter commonly used for fluorescence imaging blocks the short wavelength part of their emission. While cyanobacteria were found in association with corals¹⁴, the fluorescence of their phycobiliproteins does not seem to contribute significantly to the fluorescence of the coral host¹⁵.

FPs can comprise up to 14% of the total protein content of some coral species¹⁶, potentially contributing to important biological functions in corals, that are not yet entirely understood. Recent studies have shown that FP levels are directly related to light levels¹², resiliency to bleaching stress^17,18, photoprotection^19,20 and photoacclimation^12,13. Changes in GFP fluorescence can indicate heat stress²¹ and express as an early sign of coral bleaching prior to visible paling of the tissue^13,22. Additionally, GFPs have been shown to play a role as light-induced electron donors, affecting photochemical reactions²³. Measurements of chlorophyll-a fluorescence are often used to quantify photosynthetic health, through pulse amplitude modulated (PAM) fluorometry²⁴ and by measuring chlorophyll-a concentrations in tissue samples²⁵ or through photographic estimation²⁶. Over larger spatial scales, observations of coral fluorescence (both GFP and chlorophyll-a) can aid benthic cover classification^27,28, contribute to identification and surveying of cryptic coral juveniles^29,30,31,32 and might be able to contribute to understanding sub-lethal and pre-bleaching physiological responses to stress.

A fluorescence imaging system has three main components: 1) an excitation source that emits light in the excitation range, 2) a camera with adequate sensitivity to detect even the weak fluorescence signal and 3) a barrier filter on the camera that transmits a high proportion of the fluorescence emission while blocking the excitation illumination (Fig. 1c). Ideal excitation and emission filters have zero spectral overlap, such that when the strobes are the sole illumination source (i.e., during nighttime), only the fluorescent emission is imaged. During daytime, ambient reflected light in the wavelengths not blocked by the barrier filter is recorded along with the fluorescence signal, that can be orders of magnitude smaller (Fig. 1c). This mixing of the fluorescence signal with the ambient light has been a significant obstacle in establishing fluorescence imaging as a surveying tool because nighttime work in reef environments is often logistically difficult and potentially dangerous.

Consumer cameras have previously been used for fluorescence imaging with commercial excitation and emission filters, but have lacked adequate sensitivity for wide scale deployment and some had limited sensitivity to chlorophyll-a fluorescence. Mazel (2005)³³ recorded daytime coral fluorescence with a consumer camera by using very short exposure times (2 ms). The short exposure time reduces the level of ambient light relative to strobe-induced fluorescence intensity, as the strobe duration is fixed and usually not longer than 1 ms. This is a very effective solution for daytime fluorescence imaging, but is not always applicable as many consumer cameras are unable to synchronize with a flash at such short exposure times. To overcome ambient light contamination, imaging can be done at sunset or sunrise when ambient light levels are lower^29,32. A black cloth shroud (Supplementary Fig. 1) can be used to cover the camera rig and subject, but this is often impractical underwater in field situations as it requires very calm conditions. Additionally, several other systems have previously been used for recording coral fluorescence, primarily based on either spectrometers³⁴ or multispectral imaging systems with narrow band filters^22,27. However, the acquisition process with spectrometers is slow as they measure only a single small (~3 cm diameter) location at a time and multispectral cameras require expensive custom fabrication and are more cumbersome in the field.

In this paper we present the design for an underwater wide FOV Fluorescence Imaging System (FluorIS) that primarily utilizes off-the-shelf components, is easy to deploy by divers and provides high quality fluorescence imagery during both night and day (Fig. 1a). We aim at imaging typical sizes of quadrants used for benthic coral ecosystem surveys, e.g., at least 50 cm × 50 cm. The FluorIS is based on a consumer camera modified for increased infrared (IR) sensitivity. We show that the green to red ratio of FluorIS images is highly correlated to in situ spectrometer measurements, allowing quantitative measurements including relative ratio comparisons of these two important spectral channels and we demonstrate fluorescence imaging during both daytime and nighttime in a scale effective for ecological surveys. In addition, there is currently an immense demand to screen corals and other marine invertebrates for novel FPs for bioimaging applications, especially those emitting in the red and the far-red wavelengths^35,36 and FluorIS can be useful in screening corals for presence of FPs. This low cost and easy to operate system will enable both new physiological processes studies as well as more rapid large scale ecological surveys.

Results

For both reflectance and fluorescence imaging, our system is comprised of a Canon 5D Mark II professional grade off-the-shelf camera with a wide-angle lens (Canon 17–40 mm or Sigma 20 mm), a Sea&Sea underwater housing with the Fisheye Dome Port 240, fitted with a 40 mm extension ring for better alignment of the dome port with the lenses to reduce distortions. In FluorIS, the barrier filter was a Tiffen #12 yellow filter mounted on the lens (Fig. 1). The 5DII camera that is used in FluorIS was additionally modified by removing the internal IR filter. This internal IR filter strongly attenuates long wavelengths above 650 nm, limiting chlorophyll-a fluorescence imaging by off-the-shelf cameras (Supplementary Fig.2a). Internal IR filters are used in consumer cameras to optimize the sensor's sensitivity to color based on the sensitivity of the human visual system. Such modifications are done for IR imaging in the astronomy community³⁷ and for artistic purposes by photographers.

For illumination we used Sea&Sea YS250, Inon Z240 and Ikelite DS161 underwater strobes. The excitation filters in FluorIS were blue NightSea filters (spectrum in Supplementary Fig.2b). As the modified camera has an expanded spectral range in the long wavelengths, an additional filter (Schott BG39) was used for the strobes to block IR wavelengths that pass through the primary excitation filter (Supplementary Fig.2b). Our goal was to achieve wide scale imaging and thus a framer with a field of view of 50 cm × 70 cm was used and the camera was placed 70 cm above the substrate.

We demonstrate that: 1) the FluorIS is superior to a fluorescence imaging system using an unmodified camera body, 2) the relative intensities of FluorIS are highly correlated to in situ spectrometer data, thus effectively acting as a quantitative instrument, 3) effective daytime fluorescence imaging is possible with this system and 4) fluorescence information can be used for improved automated analysis of benthic survey images in coral reef ecosystems.

Spectral Range and Sensitivity

The modified camera demonstrates increased sensitivity as a result of removing the internal IR filter, especially in the long red wavelengths (>650 nm), which facilitates chlorophyll-a fluorescence imaging and thus enables instantaneous imaging of both GFP-like fluorescence and chlorophyll-a fluorescence. Coral fluorescence images taken with both a standard and a modified camera body reveal that the image taken with the modified camera has 20 times higher intensity in the red channel, a signal nearly absent in the image taken with the standard camera body (Fig. 2). The signal in the green channel in the modified camera was also 1.4 higher than in the standard camera (green and red channel values obtained from averaging over the entire image). The modification expands the spectral range of the red channel up to 1200 nm, the long wavelength limit of the sensor (LifePixel Inc., personal communication). As the modified camera has increased sensitivity and spectral range making it superior to standard fluorescence imaging systems, it is used for constructing FluorIS and in all further experiments and results.

Figure 2

Fluorescence camera comparison.

Fluorescence images of the same corals taken inside a black cloth shroud that eliminated ambient light taken with(a) an unmodified standard camera and (b) a modified camera that is used for FluorIS, with the same illumination and filters. (c) Y-axis indicates pixel value comparisons of the green and red channels along the horizontal blue lines marked in (a,b), with X-axis indicating pixel count. On average, the red channel signal was 20 times larger in the image taken with the modified camera, while the green signal was stronger by a factor of 1.4 (values averaged over the entire image). Both images were taken with the same strobes, filters, strobe geometry and camera settings (ISO 640, exposure time 1/200s, f#8) and image values were normalized to be between 0 and 1. The red values are almost negligible in the standard camera, but significant in the modified one.

Spectral Intensity Measurements

To determine if FluorIS can provide quantitative information about the concentration of fluorophores, we conducted an experiment where we imaged fluorescence of 35 tagged corals from three species (Orbicella franksi- previously known as Montastrea franksi, Siderastrea siderea, Stephanocoenia michelinii) in the field at Bocas Del Toro, Panama. For the purposes of this study we restricted the measurements to corals that only expressed GFP and chlorophyll-a fluorescence. During the same dive, fluorescence was measured on each individual colony using an underwater spectrometer³⁴ at three locations on each coral, marked with colored tacks (examples in Fig. 3a).

Figure 3

Comparison of FluorIS to a spectrometer.

Nighttime images taken at 5 m in Bocas Del Toro, Panama. (a) Images show an Orbicella franksi coral head in reflectance and fluorescence and the three color ellipses depict spectrometer measurement locations that were marked on the corals with a colored tack. In the plots, the solid and dashed lines depict the spectrometer and the camera measurements, respectively, with line colors corresponding to the tack color. All values are normalized to 1 at 685 nm, for comparison. Spatial differences in relative fluorescence between the three areas captured by the spectrometer closely match the camera measurements. (b) The wide field-of-view of FluorIS enables visualization and quantification of changes in fluorescence across colonies and reveals that the GFPs were only present in significant concentrations in the lower part of the Montastrea franksi colony shown, even though this difference is barely perceptible in the reflectance image. (c) Scatter plot comparing the ratio between green fluorescence intensity to red fluorescence intensity (GFP/CHL), for both the FluorIS system and the spectrometer, revealing highly correlated measurements (r = 0.898, with p < 0.001, Spearman rank correlation coefficient). Data points include all 105 measurements (3 locations for each of the 35 corals). The red line depicts the result of robust regression on the data, while the green line indicates a slope of 1.0.

The absolute intensity of the fluorescence emission depends on the intensity of the excitation. We imaged a wide area underwater and excited the fluorescence with wide-angle non-uniform strobes, creating a non-uniform illumination pattern across the FOV. In addition, water attenuates light with distance and therefore the illumination pattern depends on the 3D structure of the corals, that varied from image to image (image formation model is detailed in Supplementary Note 1). Thus, the intensity of the incident lighting was unknown and difficult to calibrate. Nevertheless, the ratio between the green emission (corresponding to GFP) and the red emission (corresponding to chlorophyll-a) in the camera measurements at each pixel does not depend on the intensity non-uniformity, as this non-uniformity cancels out when looking at ratios. This ratio might change as the fluorescence emission propagates from the coral to the camera as water attenuates the red long wavelengths stronger than the shorter green ones. Assuming imaging at constant distances (as with our framer) and coral size small relatively to the framer distance, this change is constant and does not harm correlation, but it should be taken into account in other cases. We termed this ratio GFP/CHL and compared it between FluorIS images and spectrometer measurements. Red and green image channel values were averaged across ellipses adjacent to the tack location. Spectrometer measurements were averaged across the spectral range corresponding to the fluorophore emission (Supplementary Fig. 3).

For example, values from a fluorescence image of a tagged Orbicella franksi were compared to the corresponding spectrometer measurements (Fig. 3a). At one of the locations (marked with a red tack), the spectrometer measurement revealed a lower GFP fluorescence intensity (500–630 nm) relative to the 685 nm peak at the other two locations. The same intensity difference is also demonstrated in the image values. As the camera captured a larger field of view than the spectrometer, it enabled comparison of fluorescence changes across an entire coral colony and even between different colonies (Fig. 3b). The correlation of the spectrometer and FluorIS GFP/CHL ratio measurements (n = 105) is significant (r = 0.898, p < 0.001, Spearman rank correlation coefficient, Fig. 3c). Thus the use of FluorIS for generating quantitative ratios for fluorescence quantification was validated.

Daytime Fluorescence Imaging

Here we show that a daytime fluorescence image can be obtained by subtracting the pixel values of an image taken with the excitation strobes off (I_ambient) from an image taken during daytime with the excitation strobes on (I_day) resulting in the desired signal (F_strobes), i.e., the fluorescence excited by the blue strobes. We term this method ambient light subtraction: F_strobes = I_day – I_ambient and show that the resulting daytime fluorescence image approximately matches a fluorescence image of the same corals taken during nighttime (Fig. 4). In practice, the two images I_day and I_ambient should be acquired with the same camera settings (ISO, aperture, shutter speed, focus), position and with minimal delay, so they can be aligned with minimal motion between images and to avoid changes in ambient illumination such as clouds and wave caustics. Our camera was mounted on a framer to minimize movement between frames. Compared to nighttime GFP/CHL ratios, the accuracy of the daytime GFP/CHL ratio was 85% for the left coral and 90% for the coral on the right (Fig. 4). These daytime images were taken with an exposure time of 1/250s, the fastest available flash synchronization speed, to allow the minimal amount of sunlight in the image. In order to imitate what happens when I_ambient is stronger relatively to F_strobes we increased the exposure time in steps. The exposure time could be increased up to 1/80s yielding similar accuracies, but when the exposure time was increased to 1/50s the accuracies dropped significantly, suggesting that fluorescence values relative to ambient light dropped below noise levels in this case.

Figure 4

Ambient light subtraction.

When imaging fluorescence during daytime, subtraction of the pixel values of an image that contains only ambient light from an image taken with the blue strobes on produces a derived fluorescence image, similar to a nighttime image taken directly with no ambient light. As the strobe duration is fixed (on the order of a few ms), increasing the exposure time increases the intensity of the ambient illumination in the image, while keeping the same fluorescence intensity. Thus, imaging the same scene with different shutter speeds allowed us to test how varying contamination levels from ambient light affected the resulting fluorescence image. In the daytime images shown, the shutter speed was 1/80s, three times the minimum possible shutter speed (1/250s), to allow high levels of ambient light in the image. For both these exposure times (1/80s and 1/250s) the accuracy of the ambient light subtraction method was 85% compared to nighttime GFP/CHL ratios for the coral on the left and 90% for the coral on the right. However, when the exposure time was increased to 1/50s the accuracies dropped significantly, suggesting that there was a point where the fluorescence values relative to ambient light dropped below noise levels. Images were taken in Bocas Del Toro, Panama at a depth of 5 m.

The variance of noise in images increases linearly with the image signal because of photon noise and that becomes a limiting factor when an image contains a large additive signal over the signal of interest³⁸. This is a concern for using the ambient light subtraction method, because the fluorescence signal is usually much weaker than the reflectance image. Thus, the accuracy of this method depends on F_relative = F_strobes/I_ambient, the relative fluorescence intensity. After subtracting I_ambient, the remaining signal F_strobes has to be above noise levels. The noise level in an image depends on the camera and the ISO setting. Define the minimum required signal-to-noise-ratio (SNR) as α and ρ as the sensor gain. Following these definitions yields the condition for the minimum value of F_relative (derivation is in Supplementary Note 2):

Note that by definition³⁹ ρ decreases with the ISO, so for higher ISO values, the minimum value of F_relative increases, as expected.

We calculated F_relative for two ISO values based on measured noise levels in the modified camera (Supplementary Fig. 4). In well-exposed images (where the maximum image values are close to the maximum possible intensity), the fluorescence signal can be 50 times smaller than ambient light contribution for ISO 160 and 25 times lower for ISO 640 and still yield a meaningful signal. This novel derivation underpins our ability to estimate a valid fluorescence signal when the recovered signal is greater than the sensor noise threshold.

As we observed, the quality of daytime fluorescence imaging increases as F_relative increases. Even in very clear water, only 13% of the red light (600 nm) is transmitted from the surface to a depth of 10 m due to strong absorption of long wavelengths by water⁴⁰. This gives an inherent advantage to using long wavelength chlorophyll-a fluorescence information, because due to the water absorption, F_relative for chlorophyll-a fluorescence is high. Moreover, FluorIS integrates over the entire emission spectrum of chlorophyll-a (photosystem I and II) and thus is highly sensitive for chlorophyll-a fluorescence. In our experience performing daytime imaging deeper than 5 m, the red channel of I_day was mostly the result of chlorophyll-a fluorescence rather than reflectance, even without using the ambient light subtraction method (Figs. 4, 5b, 6), since water absorption attenuates ambient red light. Thus, deeper than 5 m the fluorescence signal has high contrast over the background in the red channel, even in a single fluorescence image without ambient light subtraction. This is useful for automatic annotation of live coral tissue in images, as described in the following section.

Figure 5

Computer-aided segmentation of the live areas of coral colonies using fluorescence images.

In (a) and (b), yellow lines indicate user-designated foreground strokes, blue lines indicate user-designated background strokes and red-green lines indicate automatically generated contours. (a) Supervised segmentation done with the same strokes on (left to right) reflectance, nighttime fluorescence and daytime (mixed with ambient light) fluorescence images, compared to manual segmentation that was done by outlining the colony on a reflectance image. The automated segmentation of the reflectance image mistakenly includes some part of the sand as the coral, along with other anomalies, while the nighttime fluorescence image enables accurate automatic segmentation. The daytime fluorescence image (without ambient light subtraction) also demonstrated improved contrast between the coral and its background due to the red chlorophyll-a fluorescence component. The automatic contour is almost optimal and it only missed a small part of the colony in the lower right corner. These images were taken in Bocas Del Toro, Panama, at 5 m depth. (b) Segmentation of a wide angle scene during daytime, using reflectance and the red channel of I_day (daytime fluorescence image mixed with ambient). As in (a), segmentation using the reflectance image included part of the background as a coral. These images were taken in Moorea, French Polynesia, at 10 m depth, at mid-day. Red arrows point to a coral that blends in with the background rock in the reflectance image, but stands out in the fluorescence image. (c) Segmentation of four additional corals (actual images not shown for clarity). For each coral, semi-automated segmentation was done using a fluorescence image (blue curves) and compared to a manual segmentation done using a reflectance image (black curve). Images were not registered so the contours were manually aligned. The contours closely match, but the semi-automated segmentation using fluorescence images required only 20% of the time of manual segmentation.

Figure 6

Fluorescence to assist in locating juvenile coral in images.

In the red fluorescence channel, small corals have significantly stronger contrast than in reflectance images, making them easier to detect either by human operator or with computer algorithms. Juvenile corals are marked in both images with polygons of similar shape and color. (a) Reflectance image of a quadrat, taken at 10 m depth in Moorea, French Polynesia. The quadrat covered an area of 50 cm × 70 cm and was imaged from a distance of 70 cm (for better display, we show an area that is 2/3 of the original area). (b) The red channel of a daytime FluorIS image of the same quadrat (with values are stretched to be between 0 and 1 for display), showing clear chlorophyll-a fluorescence even at relatively shallow depths in clear water, as the red spectrum in the ambient light from the surface is strongly attenuated in the water column. The image was taken in full sunlight and no ambient light subtraction was done.

Fluorescence for Automated Annotation

Fluorescence images can provide high contrast between living benthic coral substrates that contain fluorescent pigments and non-living substrates. This differentiation is useful because live corals are often similar in color to their background (brown sand, sediments or dead coral substrate), making them difficult to differentiate with standard color photography⁴¹. With the improved contrast in fluorescence images, segmentation of living tissue areas from dead ones can be achieved in less time, making fluorescent images potentially superior to standard reflectance images for rapid automated quantification of live coral area. To test this possibility, semi-automated segmentation was done on reflectance and fluorescence images taken during both daytime and nighttime, using publicly available software⁴² that requires a few input strokes from the user to indicate foreground and background (Fig. 5). Fluorescence images required less user assistance resulting in an 80% reduction in the time required for the reflectance images to achieve similar segmentation contours (Fig. 5). On reflectance images, the automated segmentation was often confused between corals and their background. This source of error was greatly reduced using fluorescence images.

Discussion

The Fluorescent Imaging System presented in this paper greatly facilitates wide field-of-view, high sensitivity in situ fluorescence imaging of coral ecosystems during night and day using off-the-shelf components. We demonstrate that FluorIS enables simultaneous imaging of two of the primary fluorescent spectral bands of interest for studies of scleractinian corals, (GFP and chlorophyll-a) in a single exposure and over a larger area than previous methods, creating the potential for application across larger time and spatial scales. Thus, FluorIS potentially enables expanded physiological and ecological research applications utilizing in vivo fluorescence, addressing pressing issues in coral physiology, ecology and conservation.

The strong correlation between relative intensity measurements of the green and red channels (GFP/CHL) obtained from raw FluorIS data and co-located data from a fluorescence spectrometer validates the ability of FluorIS to produce quantifiable fluorescence intensity data in specific bands (Fig. 3). Zawada and Jaffe²² have shown that relative fluorescence levels of green and orange bands can serve as predictors for bleaching and Mazel et al.²⁷ used three bands centered at 520, 580 and 685 nm as an aid for automatic classification²⁷. In both of these reports^22,27 custom underwater multispectral cameras were built that were complex and relatively expensive. Here we demonstrate for two out of the three bands used in Mazel et al.²⁷ that similar information on relative intensities can be extracted from a single wide-band image with our relatively low cost and easy to operate system. Therefore for some applications, FluorIS can be used instead of a spectrometer, saving considerable time and expense, enabling comparisons across whole images and allowing for large-scale coverage.

The role of FPs in corals is still not entirely clear. Our system will potentially add a new perspective over lab experiments about this question by enabling large-scale in situ surveys of FP spatial abundance and biodiversity. In addition, it will enable in situ studies of the diel, seasonal and annual changes of FP expression, the correlation of this signal to coral resilience, pathology and recovery in the field. Previous studies have already demonstrated changes in FP expression in response to light and temperature stress, coral growth, wound regeneration and innate immune responses^12,13,22,43. Thus, the onset of risk for coral bleaching in the field might be observable using our system before any visible bleaching or pigment changes occur.

Having a camera system that can capture accurate fluorescence information during the day simplifies the logistics of conducting fluorescent surveys in the field. We showed that the fluorescence signal can be recovered from daytime images by subtracting a frame that was taken with the excitation strobes off, greatly facilitating large-scale daytime fluorescence surveys. The accuracy of the recovered fluorescence increases with the SNR of F_strobes (strobe-induced fluorescence), that increases with F_relative (ratio of strobe-induced fluorescence to ambient light). In low F_relative values, the fluorescence signal is distinguishable and can be used for applications that do not require accurate fluorescence intensity, such as coral segmentation; with higher F_relative values, the signal itself can be also recorded quantitatively. Although the ambient light subtraction method can potentially be applied on most consumer cameras, FluorIS has a few advantages over standard cameras due to higher bit depth, lower noise for each ISO level and increased sensitivity enabling the use of lower ISO levels for the same intensity. Specifically, the increased long-wavelength sensitivity gives FluorIS a major advantage over other unmodified high quality cameras.

During daytime there is an advantage to imaging long-wavelength fluorescence, which is easy with FluorIS, since these wavelengths are heavily attenuated in seawater and therefore the contribution from ambient light is minimal. In our experiments, the red channel of I_day (the single daytime fluorescence image mixed with ambient light) contained mostly fluorescence from photosynthetic pigments, as ambient red light is relatively weak deeper than ~5 m. Although the red channel still contains a small amount of reflected ambient light, it still yields high contrast for live organisms over the non-living substrates such as dead corals, sand and bare rocks, even without ambient light subtraction. In addition, having a weak signal of ambient reflectance provides context to the image in contrast to nighttime images where the background is black. In deeper mesophotic reefs the green ambient light becomes weaker and we expect to get higher contrast for the GFP fluorescence in single daytime images as well. Thus, even without ambient light subtraction, single images using FluorIS provide notable advantages for automated image annotation and segmentation, as applied to both physiological and ecological research.

Coral image annotation is currently a bottleneck⁴⁴ for photographic coral ecology surveys^45,46, especially with increased use of autonomous underwater vehicles (AUVs) for coral reef surveys⁴⁷. Recent automated analysis methods^41,48 have shown potential to increase the speed of point-based processing of images from large-scale surveys. Nevertheless, as they are based mostly on shape analysis, these automated methods have difficulties in distinguishing between live and dead corals that retain the coral skeletal structure⁴¹. As non-living substrates generally do not fluoresce, fluorescence can overcome this issue for automatic recognition. GFP fluorescence has been used for better detection of corals (mainly recruits and juveniles) by divers and in images^29,30,31,32. However, FP fluorescence alone might not be enough for this purpose since FP expression varies across coral species and individuals. Some corals do not express FPs and among the FPs that are expressed, different excitation wavelengths are required, i.e., imaging should be done with different light source spectra. For example, Baird et al.³⁰ point to a Porites spp. recruit that was not visible in the fluorescence image. In addition, the contrast of GFP over ambient illumination in daytime is low. Nevertheless, all live hermatypic corals have chlorophyll-a in their zooxanthellae and thus fluoresce in the long red wavelengths that can be readily imaged with FluorIS. We showed that chlorophyll-a fluorescence is readily visible in daytime FluorIS fluorescence images below ~5 m depth, even when it is mixed with ambient reflectance and we showed improved performance of user-assisted automatic segmentation of corals using FluorIS images. Algae growing on dead substrates also have chlorophyll-a fluorescence, but it is generally weaker and has a different texture, as demonstrated in Fig. 5b. Therefore, we believe that FluorIS is particularly well suited for improving automated analysis of coral reef surveying photos.

One of the most important future applications in this context is surveying field abundance of juvenile corals, as reproduction is critical for ecosystem recovery and has been shown to be negatively affected in some reef ecosystems under future climate change conditions⁴⁹. However, photographic surveys of juvenile corals are rarely done as it is difficult and time consuming to locate the juveniles in a reflectance image because their texture and color are very cryptic. The intensity of chlorophyll-a fluorescence in an image is not a sufficient indicator, as algae fluoresce as well. However, our preliminary work (Fig. 6) suggests that chlorophyll-a fluorescence images taken with FluorIS can serve as a better input for human annotators or an automatic algorithm looking for juveniles, because their strong chlorophyll-a fluorescence results in high contrast versus the background in the red channel of FluorIS. Note that the fluorescence itself is not enough to identify a juvenile vs. algae and the candidate fluorescing organisms will need to be further sorted out according to shape and texture, that appear different in the images we acquired. Further development has the potential to allow for large scale, rapid surveys of coral juveniles which would be a major advance to coral reef science.

For future work, we plan several improvements to increase the functionality of our system: automated acquisition of daytime pairs (strobes on and off for the same scene), calibration of intensity for more quantitative measurements, deployment on underwater vehicles, custom light sources with additional excitation wavelengths and integrated reflectance and fluorescence imaging. Thus we believe that wide field-of-view fluorescence imaging of corals can be integrated into rapid, automated and synoptic coral reef surveys by humans and by autonomous underwater vehicles and it could contribute to improved understanding of these vital and iconic ecosystems.

Methods

Imaging System

A fluorescence imaging system is composed of three main components: camera, emission filter and an excitation source (Fig. 1c). Here we describe our considerations for choice of these components and the specific components used for FluorIS. Similar systems can be built from other components complying with the spectral and sensitivity considerations described below.

Using a Consumer Camera for Chlorophyll-a Fluorescence

The relative spectral responses of the Canon 5D Mark-II we used (www.astrosurf.com/buil/50d/test.htm) are depicted in Supplementary Fig. 2a (blue, green and red curves). The sensitivity in the long visible wavelengths is very weak. Specifically it is ~5% at 685 nm, the peak of chlorophyll-a fluorescence emission. Even though CMOS and CCD sensors are sensitive to long wavelengths, most consumer cameras include an infrared (IR) filter on top of the color filters. Thus the measured color sensitivities (Supplementary Fig. 2a) are the product of the color filter sensitivities and the IR filter transmission. The standard IR filter cuts out wavelengths above ~650 nm (www.astrodon.com/products/filters/astrodon_inside_dslr), minimizing sensitivity to the desired chlorophyll-a fluorescence. Therefore, we modified the camera by removing the IR filter and replacing it with a filter that transmits the entire spectrum. The modification was done by LifePixel inc. (www.lifepixel.com), using their “full spectrum” option. There are several additional labs that offer this service.

This modification results in a camera with an increased sensitivity to the long wavelengths. Note that although the sensor sensitivities depicted in Supplementary Fig. 2a are specific to the camera we used, the characteristics of the IR filter are similar in many consumer cameras. There are reports regarding lenses that perform poorly with a modified camera due to IR reflections. We have not encountered problems with the lenses we used, but if using other lenses, it is advisable to check with reports such as the one in dpanswers.com/content/irphoto_lenses.php.

Camera Linearity

Our image analysis assumes that the camera response is linear, a common case for raw format images. The system's linearity was verified by imaging an Xrite ColorChecker chart at six exposures. Then the mean intensity of each of the three color channels in each of the 24 squares in the chart was calculated. This yields 72 functions of the color channel intensity as a function of the exposure. The R² coefficient for these 72 functions ranged between 0.9967 and 1.0, where 1.0 indicates ideal linearity. Note that in order to maintain camera linearity, the raw images need to be converted in a mode that preserves the linearity. Such modes are usually available in the options of raw conversion software. We used Dcraw open source code (www.cybercom.net/~dcoffin/dcraw) and Matlab.

Developing Proper Illumination

A typical Xenon strobe emits IR wavelengths that are not fully attenuated by the blue NightSea filter (Supplementary Fig. 2b). This is not a problem with most standard cameras that have an IR block filter, but images taken with the modified camera under the illumination of a Xenon strobe filtered with just the Nightsea filter include significant IR reflectance component.

There are several solutions to this problem. These IR wavelengths can be filtered at the camera, by using a filter with a steep cutoff above 720 nm (peak of chlorophyll-a photosystem I). Such high precision filters have narrow acceptance angles and thus are not available for SLR lenses. There are commercial filters that are designed to be mounted over the camera sensor (e.g., Astrodon filter), but we chose not to use them because of some reports that they reduce image quality. Instead, we filter the infrared wavelengths at the strobe, where a steep cutoff is not needed, as the green and red wavelengths are already attenuated by the NightSea filter. To block the IR wavelengths we used a Schott glass BG39, in addition to the blue NightSea filter. Its transmission curve is depicted in Supplementary Fig. 2b. This filter attenuates the undesired long wavelengths above 744 nm by more than 99%, with very small (3%) attenuation in the wavelengths that are needed from the strobe. The standard Nightsea strobe mount is slightly modified to accommodate this extra filter, that is at the same diameter as the blue filter (87 mm). In addition, black electrical tape was placed around the strobes and the filters to block any unfiltered light leaking from the mounts. Spectral leakage with this system was tested by imaging various non-fluorescent targets with camera settings typically used in the field and was found to be negligible (e.g., below noise levels). Note that theoretically adding an IR block filter is not needed when using blue LED strobes.

The intensity required from the strobes depends on the distance of objects from the strobes, the size of the imaged area, the camera sensitivity and the fluorescence efficiency of each fluorophore (image formation model in Supplementary Note 1). In general, the fluorescence efficiency is weak and the most powerful strobes available are recommended for use. We used off-the-shelf Xenon strobes: for reflectance imaging we used Ikelite DS-161 (guide number 24) strobes (one or two strobes); for fluorescence imaging we used two Sea&Sea YS-250 (guide number 32) strobes during nighttime and two additional Inon Z-240 (guide number 24) strobes during daytime. These two models are the strongest commercial underwater strobes currently available and also have a fast recharge rate. For daytime fluorescence we used 4 strobes positioned in the 4 corners around the camera. The strobes were attached with two strobe arms each, to enable placement as close as possible to the corals, while illuminating the entire field of view. This yielded well illuminated images with camera settings of f/8 and ISO 640. Note that target lights, that are sometimes used to focus when imaging at night also need to be filtered by the excitation filters to avoid contamination of white light, unless they turn off automatically when the strobes are triggered, or when using short exposures time such that the contribution from the target light is very weak relatively to the fluorescence.

Spectral Intensity Measurements

To compare quantitative information measured by the camera to spectrometer measurements, we conducted an experiment in Bocas Del Toro, Panama, where a total of 35 corals from three different species (Orbicella franksi, Siderastrea siderea, Stephanocoenia michelinii) with FluorIS were imaged at night in depths ranging from 1 m to 10 m. To validate the data from FluorIS, we measured the fluorescence intensity with a custom made underwater spectrometer³⁴ (NightSea, benthic spectrofluorometer, with an Ocean Optics USB2000 spectrometer) during the same dives when images were acquired. This spectrometer measures an emission area of approximately 12 cm², whereas an image with our framer covered an area of 3500 cm². The spectrometer measurements were therefore done at three locations on each coral, each marked with colored tacks indicating the measured position in the image. The tacks had a line marked on them, that indicated the direction next to the tack where the spectrometer measurement was taken (Fig. 3a).

In a linear camera, the image intensities are related to spectrometer measurements (Supplementary Note 1, equation 10). To compare between the sensors, we manually marked on each image the approximate area next to the tack that was measured by the spectrometer (example of annotated ellipses in Fig. 3a). In the images, we calculated the average pixel values within the ellipse in each color channel (R & G). We compare these values to the integral of the intensities measured by the spectrometer for the relevant wavelength range that corresponds to the fluorescence emission detected by the camera:

where I_spectrometer (λ) is the intensity measured by the spectrometer at a wavelength λ. Integration ranges are Λ_G = [520, 630] nm and Λ_R = [630, 800] nm. The integration for the GFP emission starts at 520 nm as the yellow barrier filter attenuates shorter wavelengths. Since the peak of the GFP emission in the spectrometer measurements is outside the transmission of the yellow barrier filter (Supplementary Fig.2), the camera only measures the long wavelength side of the GFP fluorescence. Analysis of our spectrometer measurements revealed that the integral over the long wavelength shoulder of the GFP emission is strongly correlated to the peak intensity (r = 0.992, p < 0.001, Spearman rank correlation coefficient). Thus the integral of the GFP emission over the spectral region longer than the barrier filter is highly indicative of the peak intensity (Supplementary Fig.3).

Fluorescence Imaging During Daytime

To have registered images we used a framer and took subsequent images while manually turning the strobes on and off. A user of our system reported experimenting with leaving the strobes on and taking subsequent images with no delay. In most cases the short delay between the frames did not allow for strobe recharge, voiding the need to turn the strobes off.

Interactive Segmentation

For segmentation experiments we used the method from Gulshan et al. (2010)⁴² (code available online in www.robots.ox.ac.uk/~vgg/software/iseg/). This method performs interactive segmentation, where the user indicates foreground areas and background areas using a few strokes. The algorithm then expands the strokes to find the contour separating the foreground and background. The user can refine the segmentation by adding strokes. To compare segmentation results between reflectance and fluorescence images, we used the same minimal set of user strokes.

Statistical Analysis

The Spearman rank correlation coefficient was calculated to assess the correlation presented in Fig. 3. This non-parametric test was used because the data were not normally distributed, as determined from p < 0.001 using both the Kolmogorov-Smirnov goodness-of-fit against normal reference distributions and the Shapiro-Wilk normality test. The Spearman alpha value was 0.05 and the test was two tailed. The PASW Statistics 17 package was used for these analyses (Rel. 17.0.3. Sep 2009. Chicago: SPSS Inc.).