Coral reefs globally are declining rapidly because of both local and global stressors. Improved monitoring tools are urgently needed to understand the changes that are occurring at appropriate temporal and spatial scales. Coral fluorescence imaging tools have the potential to improve both ecological and physiological assessments. Although fluorescence imaging is regularly used for laboratory studies of corals, it has not yet been used for large-scale in situ assessments. Current obstacles to effective underwater fluorescence surveying include limited field-of-view due to low camera sensitivity, the need for nighttime deployment because of ambient light contamination, and the need for custom multispectral narrow band imaging systems to separate the signal into meaningful fluorescence bands. Here we describe the Fluorescence Imaging System (FluorIS), based on a consumer camera modified for greatly increased sensitivity to chlorophyll-a fluorescence, and we show high spectral correlation between acquired images and in situ spectrometer measurements. This system greatly facilitates underwater wide field-of-view fluorophore surveying during both night and day, and potentially enables improvements in semi-automated segmentation of live corals in coral reef photographs and juvenile coral surveys.
Coral reef ecosystems are in a state of crisis, suffering massive global declines over the last three decades including up to 80% loss of coral coverage in the Caribbean1 and approximately 50% in both the Indo-Pacific2 and the Great Barrier Reef3. These declines are occurring rapidly over large spatial scales, often at over 1% loss of coverage per year2,3 due to both local (e.g. pollution, nutrient enrichment, overfishing and sedimentation) and global stressors (global warming, ocean acidification and sea level rise)4,5,6. Thus, new non-invasive, rapid underwater ecological monitoring tools are urgently needed to better understand how coral physiology and reef ecosystems are responding to these stresses on spatial and temporal scales relevant for management.
Coral fluorescence imaging can complement standard underwater imaging for both laboratory and in situ monitoring of corals, but current fluorescence camera systems are of limited practical use for ecological surveys. Methodological obstacles include low camera sensitivity to the full relevant spectral range of coral fluorescence emissions, the need for powerful illumination and small field-of-view (FOV) due to low sensitivity, and the need for nighttime deployment in order to avoid signal contamination by ambient light. Thus, there is a need to develop an in situ fluorescence imaging system capable of performing large-scale coral reef surveys (Fig. 1a).
Fluorescence is defined as the reemission of photons with longer wavelength than the absorbed photons7. Most scleractinian coral tissue contains two primary groups of fluorophores (characteristic spectra are depicted in Fig. 1b). The first major group are photosynthetic pigments from the symbiotic dinoflagellate algae (Symbiodinium spp.) living within the coral tissues, with chlorophyll-a the predominant pigment. Chlorophyll-a emits with a primary peak at ~685 nm and a secondary peak at ~730 nm8,9. The second group are the fluorescent proteins (FPs), found in the coral animal tissue with several different classes exhibiting typical emission peaks between 482 nm to 609 nm10,11,12,13. Within the varied groups of existing FPs, the FluorIS was developed to image green fluorescent proteins (GFPs), as their excitation range partially overlaps with chlorophyll-a, enabling simultaneous imaging of GFP and chlorophyll-a with a single excitation source. The excitation of cyan FPs overlaps with this excitation range as well12. However, they appear green in the images as the commercial yellow barrier filter commonly used for fluorescence imaging blocks the short wavelength part of their emission. While cyanobacteria were found in association with corals14, the fluorescence of their phycobiliproteins does not seem to contribute significantly to the fluorescence of the coral host15.
FPs can comprise up to 14% of the total protein content of some coral species16, potentially contributing to important biological functions in corals, that are not yet entirely understood. Recent studies have shown that FP levels are directly related to light levels12, resiliency to bleaching stress17,18, photoprotection19,20 and photoacclimation12,13. Changes in GFP fluorescence can indicate heat stress21 and express as an early sign of coral bleaching prior to visible paling of the tissue13,22. Additionally, GFPs have been shown to play a role as light-induced electron donors, affecting photochemical reactions23. Measurements of chlorophyll-a fluorescence are often used to quantify photosynthetic health, through pulse amplitude modulated (PAM) fluorometry24 and by measuring chlorophyll-a concentrations in tissue samples25 or through photographic estimation26. Over larger spatial scales, observations of coral fluorescence (both GFP and chlorophyll-a) can aid benthic cover classification27,28, contribute to identification and surveying of cryptic coral juveniles29,30,31,32, and might be able to contribute to understanding sub-lethal and pre-bleaching physiological responses to stress.
A fluorescence imaging system has three main components: 1) an excitation source that emits light in the excitation range, 2) a camera with adequate sensitivity to detect even the weak fluorescence signal, and 3) a barrier filter on the camera that transmits a high proportion of the fluorescence emission while blocking the excitation illumination (Fig. 1c). Ideal excitation and emission filters have zero spectral overlap, such that when the strobes are the sole illumination source (i.e., during nighttime), only the fluorescent emission is imaged. During daytime, ambient reflected light in the wavelengths not blocked by the barrier filter is recorded along with the fluorescence signal, that can be orders of magnitude smaller (Fig. 1c). This mixing of the fluorescence signal with the ambient light has been a significant obstacle in establishing fluorescence imaging as a surveying tool because nighttime work in reef environments is often logistically difficult, and potentially dangerous.
Consumer cameras have previously been used for fluorescence imaging with commercial excitation and emission filters, but have lacked adequate sensitivity for wide scale deployment and some had limited sensitivity to chlorophyll-a fluorescence. Mazel (2005)33 recorded daytime coral fluorescence with a consumer camera by using very short exposure times (2 ms). The short exposure time reduces the level of ambient light relative to strobe-induced fluorescence intensity, as the strobe duration is fixed and usually not longer than 1 ms. This is a very effective solution for daytime fluorescence imaging, but is not always applicable as many consumer cameras are unable to synchronize with a flash at such short exposure times. To overcome ambient light contamination, imaging can be done at sunset or sunrise when ambient light levels are lower29,32. A black cloth shroud (Supplementary Fig. 1) can be used to cover the camera rig and subject, but this is often impractical underwater in field situations as it requires very calm conditions. Additionally, several other systems have previously been used for recording coral fluorescence, primarily based on either spectrometers34 or multispectral imaging systems with narrow band filters22,27. However, the acquisition process with spectrometers is slow as they measure only a single small (~3 cm diameter) location at a time, and multispectral cameras require expensive custom fabrication and are more cumbersome in the field.
In this paper we present the design for an underwater wide FOV Fluorescence Imaging System (FluorIS) that primarily utilizes off-the-shelf components, is easy to deploy by divers, and provides high quality fluorescence imagery during both night and day (Fig. 1a). We aim at imaging typical sizes of quadrants used for benthic coral ecosystem surveys, e.g., at least 50 cm × 50 cm. The FluorIS is based on a consumer camera modified for increased infrared (IR) sensitivity. We show that the green to red ratio of FluorIS images is highly correlated to in situ spectrometer measurements, allowing quantitative measurements including relative ratio comparisons of these two important spectral channels, and we demonstrate fluorescence imaging during both daytime and nighttime in a scale effective for ecological surveys. In addition, there is currently an immense demand to screen corals and other marine invertebrates for novel FPs for bioimaging applications, especially those emitting in the red and the far-red wavelengths35,36, and FluorIS can be useful in screening corals for presence of FPs. This low cost and easy to operate system will enable both new physiological processes studies as well as more rapid large scale ecological surveys.
For both reflectance and fluorescence imaging, our system is comprised of a Canon 5D Mark II professional grade off-the-shelf camera with a wide-angle lens (Canon 17–40 mm or Sigma 20 mm), a Sea&Sea underwater housing with the Fisheye Dome Port 240, fitted with a 40 mm extension ring for better alignment of the dome port with the lenses to reduce distortions. In FluorIS, the barrier filter was a Tiffen #12 yellow filter mounted on the lens (Fig. 1). The 5DII camera that is used in FluorIS was additionally modified by removing the internal IR filter. This internal IR filter strongly attenuates long wavelengths above 650 nm, limiting chlorophyll-a fluorescence imaging by off-the-shelf cameras (Supplementary Fig.2a). Internal IR filters are used in consumer cameras to optimize the sensor's sensitivity to color based on the sensitivity of the human visual system. Such modifications are done for IR imaging in the astronomy community37, and for artistic purposes by photographers.
For illumination we used Sea&Sea YS250, Inon Z240 and Ikelite DS161 underwater strobes. The excitation filters in FluorIS were blue NightSea filters (spectrum in Supplementary Fig.2b). As the modified camera has an expanded spectral range in the long wavelengths, an additional filter (Schott BG39) was used for the strobes to block IR wavelengths that pass through the primary excitation filter (Supplementary Fig.2b). Our goal was to achieve wide scale imaging, and thus a framer with a field of view of 50 cm × 70 cm was used, and the camera was placed 70 cm above the substrate.
We demonstrate that: 1) the FluorIS is superior to a fluorescence imaging system using an unmodified camera body, 2) the relative intensities of FluorIS are highly correlated to in situ spectrometer data, thus effectively acting as a quantitative instrument, 3) effective daytime fluorescence imaging is possible with this system, and 4) fluorescence information can be used for improved automated analysis of benthic survey images in coral reef ecosystems.
Spectral Range and Sensitivity
The modified camera demonstrates increased sensitivity as a result of removing the internal IR filter, especially in the long red wavelengths (>650 nm), which facilitates chlorophyll-a fluorescence imaging and thus enables instantaneous imaging of both GFP-like fluorescence and chlorophyll-a fluorescence. Coral fluorescence images taken with both a standard and a modified camera body reveal that the image taken with the modified camera has 20 times higher intensity in the red channel, a signal nearly absent in the image taken with the standard camera body (Fig. 2). The signal in the green channel in the modified camera was also 1.4 higher than in the standard camera (green and red channel values obtained from averaging over the entire image). The modification expands the spectral range of the red channel up to 1200 nm, the long wavelength limit of the sensor (LifePixel Inc., personal communication). As the modified camera has increased sensitivity and spectral range making it superior to standard fluorescence imaging systems, it is used for constructing FluorIS and in all further experiments and results.
Spectral Intensity Measurements
To determine if FluorIS can provide quantitative information about the concentration of fluorophores, we conducted an experiment where we imaged fluorescence of 35 tagged corals from three species (Orbicella franksi- previously known as Montastrea franksi, Siderastrea siderea, Stephanocoenia michelinii) in the field at Bocas Del Toro, Panama. For the purposes of this study we restricted the measurements to corals that only expressed GFP and chlorophyll-a fluorescence. During the same dive, fluorescence was measured on each individual colony using an underwater spectrometer34 at three locations on each coral, marked with colored tacks (examples in Fig. 3a).
The absolute intensity of the fluorescence emission depends on the intensity of the excitation. We imaged a wide area underwater, and excited the fluorescence with wide-angle non-uniform strobes, creating a non-uniform illumination pattern across the FOV. In addition, water attenuates light with distance and therefore the illumination pattern depends on the 3D structure of the corals, that varied from image to image (image formation model is detailed in Supplementary Note 1). Thus, the intensity of the incident lighting was unknown and difficult to calibrate. Nevertheless, the ratio between the green emission (corresponding to GFP) and the red emission (corresponding to chlorophyll-a) in the camera measurements at each pixel does not depend on the intensity non-uniformity, as this non-uniformity cancels out when looking at ratios. This ratio might change as the fluorescence emission propagates from the coral to the camera as water attenuates the red long wavelengths stronger than the shorter green ones. Assuming imaging at constant distances (as with our framer) and coral size small relatively to the framer distance, this change is constant and does not harm correlation, but it should be taken into account in other cases. We termed this ratio GFP/CHL and compared it between FluorIS images and spectrometer measurements. Red and green image channel values were averaged across ellipses adjacent to the tack location. Spectrometer measurements were averaged across the spectral range corresponding to the fluorophore emission (Supplementary Fig. 3).
For example, values from a fluorescence image of a tagged Orbicella franksi were compared to the corresponding spectrometer measurements (Fig. 3a). At one of the locations (marked with a red tack), the spectrometer measurement revealed a lower GFP fluorescence intensity (500–630 nm) relative to the 685 nm peak at the other two locations. The same intensity difference is also demonstrated in the image values. As the camera captured a larger field of view than the spectrometer, it enabled comparison of fluorescence changes across an entire coral colony and even between different colonies (Fig. 3b). The correlation of the spectrometer and FluorIS GFP/CHL ratio measurements (n = 105) is significant (r = 0.898, p < 0.001, Spearman rank correlation coefficient, Fig. 3c). Thus the use of FluorIS for generating quantitative ratios for fluorescence quantification was validated.
Daytime Fluorescence Imaging
Here we show that a daytime fluorescence image can be obtained by subtracting the pixel values of an image taken with the excitation strobes off (Iambient) from an image taken during daytime with the excitation strobes on (Iday) resulting in the desired signal (Fstrobes), i.e., the fluorescence excited by the blue strobes. We term this method ambient light subtraction: Fstrobes = Iday – Iambient, and show that the resulting daytime fluorescence image approximately matches a fluorescence image of the same corals taken during nighttime (Fig. 4). In practice, the two images Iday and Iambient should be acquired with the same camera settings (ISO, aperture, shutter speed, focus), position and with minimal delay, so they can be aligned with minimal motion between images, and to avoid changes in ambient illumination such as clouds and wave caustics. Our camera was mounted on a framer to minimize movement between frames. Compared to nighttime GFP/CHL ratios, the accuracy of the daytime GFP/CHL ratio was 85% for the left coral and 90% for the coral on the right (Fig. 4). These daytime images were taken with an exposure time of 1/250s, the fastest available flash synchronization speed, to allow the minimal amount of sunlight in the image. In order to imitate what happens when Iambient is stronger relatively to Fstrobes we increased the exposure time in steps. The exposure time could be increased up to 1/80s yielding similar accuracies, but when the exposure time was increased to 1/50s the accuracies dropped significantly, suggesting that fluorescence values relative to ambient light dropped below noise levels in this case.
The variance of noise in images increases linearly with the image signal because of photon noise, and that becomes a limiting factor when an image contains a large additive signal over the signal of interest38. This is a concern for using the ambient light subtraction method, because the fluorescence signal is usually much weaker than the reflectance image. Thus, the accuracy of this method depends on Frelative = Fstrobes/Iambient, the relative fluorescence intensity. After subtracting Iambient, the remaining signal Fstrobes has to be above noise levels. The noise level in an image depends on the camera and the ISO setting. Define the minimum required signal-to-noise-ratio (SNR) as α, and ρ as the sensor gain. Following these definitions yields the condition for the minimum value of Frelative (derivation is in Supplementary Note 2):
Note that by definition39 ρ decreases with the ISO, so for higher ISO values, the minimum value of Frelative increases, as expected.
We calculated Frelative for two ISO values based on measured noise levels in the modified camera (Supplementary Fig. 4). In well-exposed images (where the maximum image values are close to the maximum possible intensity), the fluorescence signal can be 50 times smaller than ambient light contribution for ISO 160, and 25 times lower for ISO 640 and still yield a meaningful signal. This novel derivation underpins our ability to estimate a valid fluorescence signal when the recovered signal is greater than the sensor noise threshold.
As we observed, the quality of daytime fluorescence imaging increases as Frelative increases. Even in very clear water, only 13% of the red light (600 nm) is transmitted from the surface to a depth of 10 m due to strong absorption of long wavelengths by water40. This gives an inherent advantage to using long wavelength chlorophyll-a fluorescence information, because due to the water absorption, Frelative for chlorophyll-a fluorescence is high. Moreover, FluorIS integrates over the entire emission spectrum of chlorophyll-a (photosystem I and II), and thus is highly sensitive for chlorophyll-a fluorescence. In our experience performing daytime imaging deeper than 5 m, the red channel of Iday was mostly the result of chlorophyll-a fluorescence rather than reflectance, even without using the ambient light subtraction method (Figs. 4, 5b, 6), since water absorption attenuates ambient red light. Thus, deeper than 5 m the fluorescence signal has high contrast over the background in the red channel, even in a single fluorescence image without ambient light subtraction. This is useful for automatic annotation of live coral tissue in images, as described in the following section.
Fluorescence for Automated Annotation
Fluorescence images can provide high contrast between living benthic coral substrates that contain fluorescent pigments and non-living substrates. This differentiation is useful because live corals are often similar in color to their background (brown sand, sediments or dead coral substrate), making them difficult to differentiate with standard color photography41. With the improved contrast in fluorescence images, segmentation of living tissue areas from dead ones can be achieved in less time, making fluorescent images potentially superior to standard reflectance images for rapid automated quantification of live coral area. To test this possibility, semi-automated segmentation was done on reflectance and fluorescence images taken during both daytime and nighttime, using publicly available software42 that requires a few input strokes from the user to indicate foreground and background (Fig. 5). Fluorescence images required less user assistance resulting in an 80% reduction in the time required for the reflectance images to achieve similar segmentation contours (Fig. 5). On reflectance images, the automated segmentation was often confused between corals and their background. This source of error was greatly reduced using fluorescence images.
The Fluorescent Imaging System presented in this paper greatly facilitates wide field-of-view, high sensitivity in situ fluorescence imaging of coral ecosystems during night and day using off-the-shelf components. We demonstrate that FluorIS enables simultaneous imaging of two of the primary fluorescent spectral bands of interest for studies of scleractinian corals, (GFP and chlorophyll-a) in a single exposure, and over a larger area than previous methods, creating the potential for application across larger time and spatial scales. Thus, FluorIS potentially enables expanded physiological and ecological research applications utilizing in vivo fluorescence, addressing pressing issues in coral physiology, ecology, and conservation.
The strong correlation between relative intensity measurements of the green and red channels (GFP/CHL) obtained from raw FluorIS data and co-located data from a fluorescence spectrometer validates the ability of FluorIS to produce quantifiable fluorescence intensity data in specific bands (Fig. 3). Zawada and Jaffe22 have shown that relative fluorescence levels of green and orange bands can serve as predictors for bleaching and Mazel et al.27 used three bands centered at 520, 580, and 685 nm as an aid for automatic classification27. In both of these reports22,27 custom underwater multispectral cameras were built that were complex and relatively expensive. Here we demonstrate for two out of the three bands used in Mazel et al.27 that similar information on relative intensities can be extracted from a single wide-band image with our relatively low cost and easy to operate system. Therefore for some applications, FluorIS can be used instead of a spectrometer, saving considerable time and expense, enabling comparisons across whole images, and allowing for large-scale coverage.
The role of FPs in corals is still not entirely clear. Our system will potentially add a new perspective over lab experiments about this question by enabling large-scale in situ surveys of FP spatial abundance and biodiversity. In addition, it will enable in situ studies of the diel, seasonal and annual changes of FP expression, the correlation of this signal to coral resilience, pathology, and recovery in the field. Previous studies have already demonstrated changes in FP expression in response to light and temperature stress, coral growth, wound regeneration and innate immune responses12,13,22,43. Thus, the onset of risk for coral bleaching in the field might be observable using our system before any visible bleaching or pigment changes occur.
Having a camera system that can capture accurate fluorescence information during the day simplifies the logistics of conducting fluorescent surveys in the field. We showed that the fluorescence signal can be recovered from daytime images by subtracting a frame that was taken with the excitation strobes off, greatly facilitating large-scale daytime fluorescence surveys. The accuracy of the recovered fluorescence increases with the SNR of Fstrobes (strobe-induced fluorescence), that increases with Frelative (ratio of strobe-induced fluorescence to ambient light). In low Frelative values, the fluorescence signal is distinguishable and can be used for applications that do not require accurate fluorescence intensity, such as coral segmentation; with higher Frelative values, the signal itself can be also recorded quantitatively. Although the ambient light subtraction method can potentially be applied on most consumer cameras, FluorIS has a few advantages over standard cameras due to higher bit depth, lower noise for each ISO level, and increased sensitivity enabling the use of lower ISO levels for the same intensity. Specifically, the increased long-wavelength sensitivity gives FluorIS a major advantage over other unmodified high quality cameras.
During daytime there is an advantage to imaging long-wavelength fluorescence, which is easy with FluorIS, since these wavelengths are heavily attenuated in seawater and therefore the contribution from ambient light is minimal. In our experiments, the red channel of Iday (the single daytime fluorescence image mixed with ambient light) contained mostly fluorescence from photosynthetic pigments, as ambient red light is relatively weak deeper than ~5 m. Although the red channel still contains a small amount of reflected ambient light, it still yields high contrast for live organisms over the non-living substrates such as dead corals, sand and bare rocks, even without ambient light subtraction. In addition, having a weak signal of ambient reflectance provides context to the image in contrast to nighttime images where the background is black. In deeper mesophotic reefs the green ambient light becomes weaker, and we expect to get higher contrast for the GFP fluorescence in single daytime images as well. Thus, even without ambient light subtraction, single images using FluorIS provide notable advantages for automated image annotation and segmentation, as applied to both physiological and ecological research.
Coral image annotation is currently a bottleneck44 for photographic coral ecology surveys45,46, especially with increased use of autonomous underwater vehicles (AUVs) for coral reef surveys47. Recent automated analysis methods41,48 have shown potential to increase the speed of point-based processing of images from large-scale surveys. Nevertheless, as they are based mostly on shape analysis, these automated methods have difficulties in distinguishing between live and dead corals that retain the coral skeletal structure41. As non-living substrates generally do not fluoresce, fluorescence can overcome this issue for automatic recognition. GFP fluorescence has been used for better detection of corals (mainly recruits and juveniles) by divers and in images29,30,31,32. However, FP fluorescence alone might not be enough for this purpose since FP expression varies across coral species and individuals. Some corals do not express FPs, and among the FPs that are expressed, different excitation wavelengths are required, i.e., imaging should be done with different light source spectra. For example, Baird et al.30 point to a Porites spp. recruit that was not visible in the fluorescence image. In addition, the contrast of GFP over ambient illumination in daytime is low. Nevertheless, all live hermatypic corals have chlorophyll-a in their zooxanthellae, and thus fluoresce in the long red wavelengths that can be readily imaged with FluorIS. We showed that chlorophyll-a fluorescence is readily visible in daytime FluorIS fluorescence images below ~5 m depth, even when it is mixed with ambient reflectance, and we showed improved performance of user-assisted automatic segmentation of corals using FluorIS images. Algae growing on dead substrates also have chlorophyll-a fluorescence, but it is generally weaker and has a different texture, as demonstrated in Fig. 5b. Therefore, we believe that FluorIS is particularly well suited for improving automated analysis of coral reef surveying photos.
One of the most important future applications in this context is surveying field abundance of juvenile corals, as reproduction is critical for ecosystem recovery, and has been shown to be negatively affected in some reef ecosystems under future climate change conditions49. However, photographic surveys of juvenile corals are rarely done as it is difficult and time consuming to locate the juveniles in a reflectance image because their texture and color are very cryptic. The intensity of chlorophyll-a fluorescence in an image is not a sufficient indicator, as algae fluoresce as well. However, our preliminary work (Fig. 6) suggests that chlorophyll-a fluorescence images taken with FluorIS can serve as a better input for human annotators or an automatic algorithm looking for juveniles, because their strong chlorophyll-a fluorescence results in high contrast versus the background in the red channel of FluorIS. Note that the fluorescence itself is not enough to identify a juvenile vs. algae, and the candidate fluorescing organisms will need to be further sorted out according to shape and texture, that appear different in the images we acquired. Further development has the potential to allow for large scale, rapid surveys of coral juveniles which would be a major advance to coral reef science.
For future work, we plan several improvements to increase the functionality of our system: automated acquisition of daytime pairs (strobes on and off for the same scene), calibration of intensity for more quantitative measurements, deployment on underwater vehicles, custom light sources with additional excitation wavelengths, and integrated reflectance and fluorescence imaging. Thus we believe that wide field-of-view fluorescence imaging of corals can be integrated into rapid, automated, and synoptic coral reef surveys by humans and by autonomous underwater vehicles, and it could contribute to improved understanding of these vital and iconic ecosystems.
A fluorescence imaging system is composed of three main components: camera, emission filter, and an excitation source (Fig. 1c). Here we describe our considerations for choice of these components, and the specific components used for FluorIS. Similar systems can be built from other components complying with the spectral and sensitivity considerations described below.
Using a Consumer Camera for Chlorophyll-a Fluorescence
The relative spectral responses of the Canon 5D Mark-II we used (www.astrosurf.com/buil/50d/test.htm) are depicted in Supplementary Fig. 2a (blue, green and red curves). The sensitivity in the long visible wavelengths is very weak. Specifically it is ~5% at 685 nm, the peak of chlorophyll-a fluorescence emission. Even though CMOS and CCD sensors are sensitive to long wavelengths, most consumer cameras include an infrared (IR) filter on top of the color filters. Thus the measured color sensitivities (Supplementary Fig. 2a) are the product of the color filter sensitivities and the IR filter transmission. The standard IR filter cuts out wavelengths above ~650 nm (www.astrodon.com/products/filters/astrodon_inside_dslr), minimizing sensitivity to the desired chlorophyll-a fluorescence. Therefore, we modified the camera by removing the IR filter and replacing it with a filter that transmits the entire spectrum. The modification was done by LifePixel inc. (www.lifepixel.com), using their “full spectrum” option. There are several additional labs that offer this service.
This modification results in a camera with an increased sensitivity to the long wavelengths. Note that although the sensor sensitivities depicted in Supplementary Fig. 2a are specific to the camera we used, the characteristics of the IR filter are similar in many consumer cameras. There are reports regarding lenses that perform poorly with a modified camera due to IR reflections. We have not encountered problems with the lenses we used, but if using other lenses, it is advisable to check with reports such as the one in dpanswers.com/content/irphoto_lenses.php.
Our image analysis assumes that the camera response is linear, a common case for raw format images. The system's linearity was verified by imaging an Xrite ColorChecker chart at six exposures. Then the mean intensity of each of the three color channels in each of the 24 squares in the chart was calculated. This yields 72 functions of the color channel intensity as a function of the exposure. The R2 coefficient for these 72 functions ranged between 0.9967 and 1.0, where 1.0 indicates ideal linearity. Note that in order to maintain camera linearity, the raw images need to be converted in a mode that preserves the linearity. Such modes are usually available in the options of raw conversion software. We used Dcraw open source code (www.cybercom.net/~dcoffin/dcraw) and Matlab.
Developing Proper Illumination
A typical Xenon strobe emits IR wavelengths that are not fully attenuated by the blue NightSea filter (Supplementary Fig. 2b). This is not a problem with most standard cameras that have an IR block filter, but images taken with the modified camera under the illumination of a Xenon strobe filtered with just the Nightsea filter include significant IR reflectance component.
There are several solutions to this problem. These IR wavelengths can be filtered at the camera, by using a filter with a steep cutoff above 720 nm (peak of chlorophyll-a photosystem I). Such high precision filters have narrow acceptance angles and thus are not available for SLR lenses. There are commercial filters that are designed to be mounted over the camera sensor (e.g., Astrodon filter), but we chose not to use them because of some reports that they reduce image quality. Instead, we filter the infrared wavelengths at the strobe, where a steep cutoff is not needed, as the green and red wavelengths are already attenuated by the NightSea filter. To block the IR wavelengths we used a Schott glass BG39, in addition to the blue NightSea filter. Its transmission curve is depicted in Supplementary Fig. 2b. This filter attenuates the undesired long wavelengths above 744 nm by more than 99%, with very small (3%) attenuation in the wavelengths that are needed from the strobe. The standard Nightsea strobe mount is slightly modified to accommodate this extra filter, that is at the same diameter as the blue filter (87 mm). In addition, black electrical tape was placed around the strobes and the filters to block any unfiltered light leaking from the mounts. Spectral leakage with this system was tested by imaging various non-fluorescent targets with camera settings typically used in the field, and was found to be negligible (e.g., below noise levels). Note that theoretically adding an IR block filter is not needed when using blue LED strobes.
The intensity required from the strobes depends on the distance of objects from the strobes, the size of the imaged area, the camera sensitivity and the fluorescence efficiency of each fluorophore (image formation model in Supplementary Note 1). In general, the fluorescence efficiency is weak, and the most powerful strobes available are recommended for use. We used off-the-shelf Xenon strobes: for reflectance imaging we used Ikelite DS-161 (guide number 24) strobes (one or two strobes); for fluorescence imaging we used two Sea&Sea YS-250 (guide number 32) strobes during nighttime and two additional Inon Z-240 (guide number 24) strobes during daytime. These two models are the strongest commercial underwater strobes currently available, and also have a fast recharge rate. For daytime fluorescence we used 4 strobes positioned in the 4 corners around the camera. The strobes were attached with two strobe arms each, to enable placement as close as possible to the corals, while illuminating the entire field of view. This yielded well illuminated images with camera settings of f/8, and ISO 640. Note that target lights, that are sometimes used to focus when imaging at night also need to be filtered by the excitation filters to avoid contamination of white light, unless they turn off automatically when the strobes are triggered, or when using short exposures time such that the contribution from the target light is very weak relatively to the fluorescence.
Spectral Intensity Measurements
To compare quantitative information measured by the camera to spectrometer measurements, we conducted an experiment in Bocas Del Toro, Panama, where a total of 35 corals from three different species (Orbicella franksi, Siderastrea siderea, Stephanocoenia michelinii) with FluorIS were imaged at night in depths ranging from 1 m to 10 m. To validate the data from FluorIS, we measured the fluorescence intensity with a custom made underwater spectrometer34 (NightSea, benthic spectrofluorometer, with an Ocean Optics USB2000 spectrometer) during the same dives when images were acquired. This spectrometer measures an emission area of approximately 12 cm2, whereas an image with our framer covered an area of 3500 cm2. The spectrometer measurements were therefore done at three locations on each coral, each marked with colored tacks indicating the measured position in the image. The tacks had a line marked on them, that indicated the direction next to the tack where the spectrometer measurement was taken (Fig. 3a).
In a linear camera, the image intensities are related to spectrometer measurements (Supplementary Note 1, equation 10). To compare between the sensors, we manually marked on each image the approximate area next to the tack that was measured by the spectrometer (example of annotated ellipses in Fig. 3a). In the images, we calculated the average pixel values within the ellipse in each color channel (R & G). We compare these values to the integral of the intensities measured by the spectrometer for the relevant wavelength range that corresponds to the fluorescence emission detected by the camera:
where Ispectrometer (λ) is the intensity measured by the spectrometer at a wavelength λ. Integration ranges are ΛG = [520, 630] nm and ΛR = [630, 800] nm. The integration for the GFP emission starts at 520 nm as the yellow barrier filter attenuates shorter wavelengths. Since the peak of the GFP emission in the spectrometer measurements is outside the transmission of the yellow barrier filter (Supplementary Fig.2), the camera only measures the long wavelength side of the GFP fluorescence. Analysis of our spectrometer measurements revealed that the integral over the long wavelength shoulder of the GFP emission is strongly correlated to the peak intensity (r = 0.992, p < 0.001, Spearman rank correlation coefficient). Thus the integral of the GFP emission over the spectral region longer than the barrier filter is highly indicative of the peak intensity (Supplementary Fig.3).
Fluorescence Imaging During Daytime
To have registered images we used a framer and took subsequent images while manually turning the strobes on and off. A user of our system reported experimenting with leaving the strobes on, and taking subsequent images with no delay. In most cases the short delay between the frames did not allow for strobe recharge, voiding the need to turn the strobes off.
For segmentation experiments we used the method from Gulshan et al. (2010)42 (code available online in www.robots.ox.ac.uk/~vgg/software/iseg/). This method performs interactive segmentation, where the user indicates foreground areas and background areas using a few strokes. The algorithm then expands the strokes to find the contour separating the foreground and background. The user can refine the segmentation by adding strokes. To compare segmentation results between reflectance and fluorescence images, we used the same minimal set of user strokes.
The Spearman rank correlation coefficient was calculated to assess the correlation presented in Fig. 3. This non-parametric test was used because the data were not normally distributed, as determined from p < 0.001 using both the Kolmogorov-Smirnov goodness-of-fit against normal reference distributions and the Shapiro-Wilk normality test. The Spearman alpha value was 0.05 and the test was two tailed. The PASW Statistics 17 package was used for these analyses (Rel. 17.0.3. Sep 2009. Chicago: SPSS Inc.).
The work was supported by NSF grant ATM-0941760. Tali Treibitz was an Awardee of the Weizmann Institute of Science – National Postdoctoral Award Program for Advancing Women in Science. We thank Charles Mazel and Dimitri D. Deheyn for useful discussions; Adi Khen and Tsung-Han Lin for image annotations and programming; Lee Peterson from Marine Camera for imaging advice and support; Dalal Al-Abdulrazzak and Noah Ben-Aderet for field work; the Birch aquarium at Scripps for aquaria support; the UC Gump station, Moorea LTER project, and Peter Edmunds and Vinny Moriarty for collaboration in French Polynesia; and the Smithsonian Tropical research Institute for Panama facilities.
Wide Field-of-View Fluorescence Imaging of Coral Reefs Supplementary Material
This work is licensed under a Creative Commons Attribution-NonCommercial-NoDerivs 4.0 International License. The images or other third party material in this article are included in the article's Creative Commons license, unless indicated otherwise in the credit line; if the material is not included under the Creative Commons license, users will need to obtain permission from the license holder in order to reproduce the material. To view a copy of this license, visit http://creativecommons.org/licenses/by-nc-nd/4.0/