# A deep learning-enabled portable imaging flow cytometer for cost-effective, high-throughput, and label-free analysis of natural water samples

## Abstract

We report a deep learning-enabled field-portable and cost-effective imaging flow cytometer that automatically captures phase-contrast color images of the contents of a continuously flowing water sample at a throughput of 100 mL/h. The device is based on partially coherent lens-free holographic microscopy and acquires the diffraction patterns of flowing micro-objects inside a microfluidic channel. These holographic diffraction patterns are reconstructed in real time using a deep learning-based phase-recovery and image-reconstruction method to produce a color image of each micro-object without the use of external labeling. Motion blur is eliminated by simultaneously illuminating the sample with red, green, and blue light-emitting diodes that are pulsed. Operated by a laptop computer, this portable device measures 15.5 cm × 15 cm × 12.5 cm, weighs 1 kg, and compared to standard imaging flow cytometers, it provides extreme reductions of cost, size and weight while also providing a high volumetric throughput over a large object size range. We demonstrated the capabilities of this device by measuring ocean samples at the Los Angeles coastline and obtaining images of its micro- and nanoplankton composition. Furthermore, we measured the concentration of a potentially toxic alga (Pseudo-nitzschia) in six public beaches in Los Angeles and achieved good agreement with measurements conducted by the California Department of Public Health. The cost-effectiveness, compactness, and simplicity of this computational platform might lead to the creation of a network of imaging flow cytometers for large-scale and continuous monitoring of the ocean microbiome, including its plankton composition.

## Introduction

Plankton form the base of the oceanic food chain, and thus they are important components of the whole marine ecosystem. Phytoplankton are responsible for approximately half of the global photoautotrophic primary production1,2. High-resolution mapping of the composition of phytoplankton over extended periods is very important and but rather challenging because the concentration and composition of species rapidly change as a function of space and time3. Furthermore, the factors governing these changes are not fully understood4, and phytoplankton population dynamics are chaotic5. The changes in the seasonal bloom cycle can also have major environmental6 and economic effects7. The vast majority of phytoplankton species are not harmful, but some species produce neurotoxins that can enter the food chain, accumulate, and poison fish, mammals, and ultimately humans. Notable examples include Karenia brevis, which produces brevetoxin and causes neurotoxic shellfish poisoning8; Alexandrium fundyense, which generates saxitoxin and causes paralytic shellfish poisoning; Dinophysis acuminata, which produces okadaic acid and results in diarrhetic shellfish poisoning9; and Pseudo-nitzschia, which produces domoic acid and is responsible for amnesiac shellfish poisoning, potentially even leading to death10,11. Currently, monitoring of the concentrations of these species in coastal regions, including in California (USA), is usually performed by manual sample collection from coastal waters using plankton nets, followed by transportation of the sample to a central laboratory for light microscopy-based analysis12, which is very tedious, slow, and expensive and requires several manual steps performed by professionals.

As an alternative to light microscopy-based analysis, flow cytometry has been used to analyze phytoplankton samples for more than 35 years13. The technique relies on using a sheath flow to confine the plankton sample to the focal point of an illuminating laser beam and measuring the forward and side scattering intensities of each individual object/particle inside the sample volume. To aid classification, flow cytometry is usually coupled with a fluorescence readout to detect the autofluorescence of chlorophyll, phycocyanin, and phycoerythrin found in algae and cyanobacteria. Several field-portable devices based on flow cytometry have been successfully used for analyzing nano- and pico-phytoplankton distributions in natural water samples14,15,16. However, taxonomic identification based solely on scattering and fluorescence data is usually not feasible in flow cytometry, and thus these devices are coupled with additional microscopic image analysis17 or need to be enhanced with some form of imaging18,19. Consequently, imaging flow cytometry has become a widely used technique20 in which a microscope objective is used to image the sample (e.g., algae) within a fluidic flow. The image capture is triggered by a fluorescence detector, and thus objects with detectable autofluorescence are imaged. Some of the widely utilized and commercially available imaging flow cytometers include the Flowcam21 (Fluid Imaging Technologies), Imaging Flowcytobot22 (McLane Research Laboratories), and CytoSense23 (Cytobouy b.v.). Although these systems are able to perform imaging of the plankton in a flow, they still have some important limitations. The use of a microscope objective lens provides a strong trade-off between the image resolution and volumetric throughput of these systems; therefore, to obtain high-quality images, the measured sample volume is limited to a few milliliters per hour (e.g., 3–15 mL/h). Using lower-magnification objective lenses can scale up this low throughput by approximately tenfold at the expense of image quality. In addition, the shallow depth-of-field of the microscope objective necessitates hydrodynamic focusing of the liquid sample into a few-µm-thick layer using a stable sheath flow. This also restricts the size of the objects that can be imaged (e.g., to <150 µm) as well as the flow velocity and throughput of the system, thus requiring the use of additional expensive techniques such as acoustic focusing22. As a result of these factors, currently existing imaging flow cytometers used in environmental microbiology are fairly bulky (weighing, e.g., 9–30 kg) and costly (>$40,000–$100,000), limiting their widespread use.

Holographic imaging of plankton samples provides a label-free alternative to these existing fluorescence-based approaches; in fact, its use in environmental microbiology started more than 40 years ago using photographic films24 and subsequently continued via digital cameras and reconstruction techniques25. Holography provides a volumetric imaging technique that uses coherent or partially coherent light to record the interference intensity pattern of an object26. This hologram can subsequently be reconstructed to digitally bring the object into focus. The hologram contains information on the complex refractive index distribution of the object, and consequently, not only the absorption but also the phase distribution of the sample can be retrieved. There are several implementations of digital holography for imaging a fluidic flow. We can classify these digital holographic microscopy systems in terms of the presence of an external reference wave (in-line27 or off-axis28), magnification of the imaged volume, and utilization of a lens29 or spherical wavefront30 for illumination. Off-axis systems can directly retrieve the phase information from the captured hologram; however, their space-bandwidth product and image quality are generally worse than those of in-line systems26. Commercially available holographic imaging flow cytometer systems also exist, such as the LISST-Holo2. This platform is a monochrome system (i.e., does not provide color information) and offers relatively poor image quality compared to traditional imaging flow cytometers. The throughput and spatial resolution are coupled in this device, and therefore it can achieve high-throughput volumetric imaging at the cost of limited resolution (~25 µm), which makes it useful for detecting and identifying only larger organisms. Higher-resolution and better image quality systems using microscope objectives in the optical path have also been described in the literature28,29. However, the use of microscope objective lenses not only makes these systems more expensive but also limits the achievable field-of-view (FOV) and depth-of-field, thereby drastically reducing the throughput of the system to, e.g., ~0.8 mL/h31.

## Materials and methods

### Optical system

Our imaging flow cytometer uses a color image sensor with a pixel size of 1.4 µm (Basler aca4600-10uc). The housing of the camera is removed, and the circuit is rearranged to allow the sample holder to be placed in direct contact with the protective cover glass of the image sensor (see Fig. 1). Illumination of the holographic microscope is provided by using the red, green, and blue emitters from a light-emitting diode (LED) (Ledengin LZ4-04MDPB). The spatial and temporal coherence of the emitted light from the LEDs is increased to achieve the maximum resolution allowed by the sensor pixel size. The spatial coherence is adjusted by using a convex mirror (Edmund Optics #64-061) to increase the light path. The LED light is also spectrally filtered by two triple-bandpass optical filters (Edmund Optics #87–246, Chroma Inc. 69015m) to increase the temporal coherence of the illumination. The placement of the optical components is designed to tune the bandpass of the spectral filter angle to better match the emission maximum of the LEDs. Increasing the spatial and temporal coherence of the LEDs also decreases the intensity reaching the image sensor. In addition, the short exposure time required to avoid the motion blur when imaging objects in a fast flow makes it necessary for our configuration to utilize a linear sensor gain of 2. The additional noise generated from the gain is sufficiently low as to not interfere with the image-reconstruction process.

### Microfluidic channel and flow design

A microfluidic channel (Ibidi µ-Slide I) with an internal height of 0.8 mm is placed on the top of the image sensor, secured using a three-dimensional (3D)-printed holder, and connected to a peristaltic pump (Instech p625). The size of the active area of the image sensor is slightly smaller than the width of the channel (4.6 mm vs. 5 mm), and the channel is positioned so that the sensor measures the center of the liquid flow. We calculated the flow profile inside the channel (see Fig. 6) by solving the Navier–Stokes equation for noncompressible liquids assuming a nonslip boundary condition. The results show that the image sensor measures ~98% of the total volume passing through the microfluidic channel. The flow profile is a two-dimensional paraboloid, with the maximum flow speed located at the center of the microfluidic channel and measuring approximately 1.66 times higher than the mean velocity of the liquid (see Fig. 6). To acquire sharp, in-focus images of the objects in the continuously flowing liquid, we operate the image sensor in the global reset release mode and illuminated the sample by flash pulses, where the length of an illuminating pulse is adjusted to not allow an object traveling at the maximum speed inside the channel to shift by more than the width of a single sensor pixel. For a flowrate of 100 mL/h, this corresponds to a pulse length of 120 µs.

### Pulsed illumination, power, and control circuit

Because shortening the illumination time also constrains the available photon budget, we maximize the brightness of our LEDs by operating them at currents ranging from 2.2 to 5 A depending on their color. The currents are set for each LED emitter to create similar brightness levels at our image sensor, ensuring that we adequately light the sample at each color, a requirement for obtaining color images. The green LED spectrum is inherently wider than the red and blue counterparts, and thus the spectral filters will reduce its intensity the most. Therefore, we operate the green LED at the experimentally determined maximum possible current of 5 A. The red and blue LEDs require a current of ~2.2 A to match the intensity of the green LED on the image sensor to correct the white balance. We designed a circuit to control the necessary components of the device. The circuit is powered by either a 5-V wall-mounted power supply or a cellphone charger battery pack. The circuit fulfills four major roles: providing power to the peristaltic pump, charging the capacitors for providing power to the LEDs, synchronizing the LEDs to the camera and creating stable, short, high current pulses, and, finally, providing an interface for remote control by a laptop using an Inter-Integrated-Circuit (i2c) interface for setting various parameters. The peristaltic pump is powered by a high-efficiency step-up DC-DC converter at 16 V (TPS61086, Texas instruments), and its speed is controlled by a potentiometer via i2c components (TPL0401B, Texas Instruments). The charge for the high-current pulses is stored in three 0.1-F capacitors, which are charged using a capacitor charger controller (LT3750, Linear Technologies) to 12 V. The capacitor charge is initiated by the image sensor flash window trigger signal, which is active during the frame capture, and its length can be controlled by the camera software driver. The charger controller acquires an “on” state and keeps charging the capacitors until the preset voltage level of 12 V is reached. During the short illumination pulses, the voltage on the capacitors decreases only slightly, and they are immediately recharged as each frame capture resets the charge cycle, thereby allowing continuous operation. The LEDs are synchronized and their constant-current operation is ensured by a triple-output LED driver controller (LT3797, Linear Technologies). The controller uses the same flash window signal from the image sensor to turn on the LEDs for the exposure duration set by the software. The current of each LED is controlled between 0 and 12.5 A using digital i2c potentiometers (TPL0401A, Texas Instruments) and is kept constant for the subsequent pulses by the circuit, thus maintaining the same illumination intensity for each holographic frame. During startup, it takes ~3–4 frames for the circuit to stabilize at a constant light level. To avoid having multiple devices with the same address on the i2c line, we included an address translator (LTC4317, Linear Technologies) to interface with the potentiometers controlling the red and blue LEDs. To control the circuit, the laptop communicates with an Arduino microcontroller (TinyDuino from Tinycircuits), which is used as an interface for i2c communications only. During the initial startup, the circuit consumes ~1 A for the first ~8 s until the capacitors are fully charged. During its operation, the circuit consumes, on average, ~370 mA with a pump speed setting of 100 mL/h and a frame rate of 3 frames per second. This yields an average power consumption of less than 2 W. In addition, the image sensor’s typical power consumption is ~2.8 W according to the manufacturer’s data.

### Object detection and deep learning-based hologram reconstruction

For automatic detection and holographic reconstruction of the target objects found in the continuously flowing water sample (see Fig. 7), the static objects found in the raw full FOV image (e.g., dust particles in the flow channel) need to be eliminated first. This is achieved by calculating a time-averaged image of the preceding ~20 images containing only the static objects and subtracting it from the present raw hologram. To ensure appropriate reconstruction quality, the mean of this subtracted image is added back uniformly to the current frame. This yields a background-subtracted full FOV image in which only the holograms of the objects newly introduced by the flow are present. These objects are automatically detected and segmented from the full FOV for individual processing (see Supplementary Figure S2). The full FOV background-subtracted hologram is first Gaussian-filtered and converted into a binary image by hard-thresholding with its statistical values (mean + 1.5 × standard deviation), which isolates the peaks of the holographic signatures created by the objects included in the FOV. The binary contours with an area of a few pixels are removed to reduce misdetection events due to sensor noise. A closing operation is performed in the generated binary image to create a continuous patch for each object. The resulting binary contours represent the shapes and locations of the objects appearing in the FOV, and their morphological information is used to filter each contour by certain desired criteria (e.g., major axis). The center coordinate of the filtered contour is used to segment its corresponding hologram. We should emphasize that not only is it feasible to extract all objects in the FOV but it is also possible to prioritize the segmentation of the objects of interest for a specific goal by our approach. Thus, we can better utilize the computational resources of the laptop and maintain real-time processing for denser samples. After segmentation, the Bayer-patterned holograms are separated into three mono-color (i.e., red, green, and blue) holograms corresponding to the illumination wavelengths. To fully utilize the spatial resolution of the optical system, the orientation of the Bayer-patterned green pixels is rotated by 45° to regularize their sampling grid40. Concurrently, the red and blue mono-color holograms are upsampled by a factor of two, and a 45° rotation is applied to these upsampled holograms. These processes are jointly called “Resampling” in Fig. 7. Holographic autofocusing using the Tamura coefficient of the complex gradient41,42 is performed for each segmented object using only a single mono-color hologram to accurately estimate the distance of the respective object from the image sensor plane. At this point, we have 3D localized each object within the flow (per FOV). The coordinates of each detected object are then used in conjunction with the estimated flow profile from our calculations, and the location of each object is predicted at the next frame. If an object is found at the predicted coordinates, it is flagged to be removed from the total count and processing workflow to avoid reconstructing and counting the same object multiple times. At this point, the image preprocessing step (shown in cyan in Fig. 7) is complete.

The next step is the high-resolution color reconstruction (shown in pink in Fig. 7). We maximize the resolution of the reconstruction by further upsampling the holograms by a factor of four. Each color channel is then propagated to the obtained reconstruction distance by an angular spectrum-based wave-propagation algorithm26 and thus brought into focus. The slight incidence angle difference between the red, green, and blue emitters is corrected by modifying the propagation kernel accordingly43. To evaluate the resolution of the imaging flow cytometer system for the objects located inside our microfluidic channel, we replaced the flow channel with a 1951 Air Force test chart (see Supplementary Figure S3). Due to the partially coherent nature of our illumination, the resolution depends on the object–sensor distance; thus, we measured it by placing the test chart at various heights above the sensor. The width of the smallest resolved line varied between 1.55 µm and 1.95 µm depending on the height of the object, with 1.55 µm corresponding to the smallest resolvable feature for most flowing objects imaged by the imaging flow cytometer during its regular operation.

These raw reconstructions, however, are contaminated by self-interference and twin-image noise, which are characteristic of in-line digital holographic imaging systems due to the loss of the phase information of the hologram at the sensor plane. To achieve accurate image reconstruction without these artifacts, a deep learning-based digital holographic phase recovery method38,44 was employed using a convolutional neural network (see Fig. 7 and Supplementary Figure S4) pretrained with various phase-recovered reconstructions of water-borne micro-objects captured with our imaging flow cytometer. This method enables automated and accurate acquisition of the spectral morphology of an object without sacrificing the high-throughput operation of the holographic imaging cytometer, which otherwise would be very challenging as other existing phase-recovery methods require static repetitive measurements43,45,46,47,48 and/or time-consuming iterative calculations43,45,46,47,48,49,50, which would not work for flowing objects. For the visualization of transparent objects, such as plankton, we computed the color phase-contrast image based on the complex-valued reconstructions of the red, green, and blue channels, which assist in accurately resolving the fine features and internal structures of various water-borne microorganisms with high color contrast (see, e.g., Figs. 2, 3, and 7). The phase-contrast image was synthesized by (1) estimating the background field from the mean amplitude and phase of the refocused complex field, (2) calculating the object field by subtracting the background field from the refocused field, (3) shifting the phase of the background field by π/2, (4) adding the phase-shifted background field to the object field, and (5) taking the magnitude of the recalculated total field.

### Graphical user interface

We developed a GUI to operate the device. Through this GUI, all relevant measurement parameters can be specified, such as the liquid flow speed, the driving currents, the incidence angles for the red, green, and blue LEDs, the flash pulse duration, and the camera sensor gain. The GUI gives a real-time, full FOV reconstructed image at the center of the channel, thus allowing visual inspection during flow with and without background subtraction, and displays the total number of detected objects in the current frame. The GUI is also capable of visualizing up to 12 segmented, autofocused, and reconstructed objects in real time. The user can specify whether to digitally save any combination of the raw, background-subtracted holograms or reconstructed images. The GUI can also be run in demo mode to analyze previously captured image datasets without the presence of the imaging flow cytometer.

### Sample preparation and analysis

We followed the sampling protocol recommended by CDPH (USA) to obtain our ocean samples. We used a plankton net with a diameter of 25 cm and a mesh size of 20 µm and performed vertical tows with a total length of 15 m (5 × 3 m) from the end of the pier at each sampling location where a pier was present (Malibu, Santa Monica, Venice, Manhattan, and Redondo; California, USA). There was no pier at Point Dume; thus, we performed a horizontal tow from the shoreline. The plankton net condensed the micro- and nanoplankton found in the ocean into a sample volume of ~250 mL; i.e., in our case, a condensation ratio of ~3000×. We extracted 1 mL of the condensed sample, rediluted it with 50 mL of filtered ocean water, and imaged its contents using our imaging flow cytometer. The remaining samples were sent to CDPH for subsequent analysis (used for comparison purposes). During our field tests, we used the same plankton net but only performed one vertical tow from a depth of 1.5 m at each measurement. A 1-mL aliquot of the obtained sample was rediluted with 20 mL of filtered ocean water. To conserve the battery power of the controlling laptop, ~12 mL of this sample was imaged on-site. The imaging flow cytometer automatically detected and saved the reconstructed images of all detected plankton and provided the user real-time feedback on the total plankton count detected. Specific counting of Pseudo-nitzschia was performed manually by scanning through the dataset of the saved images and visually identifying Pseudo-nitzschia.

## References

1. 1.

Field, C. B., Behrenfeld, M. J., Randerson, J. T. & Falkowski, P. Primary production of the biosphere: integrating terrestrial and oceanic components. Science 281, 237–240 (1998).

2. 2.

Behrenfeld, M. J. et al. Biospheric primary production during an ENSO transition. Science 291, 2594–2597 (2001).

3. 3.

Smetacek, V. & Cloern, J. E. On phytoplankton trends. Science 319, 1346–1348 (2008).

4. 4.

Cloern, J. E., Jassby, A. D., Thompson, J. K. & Hieb, K. A. A cold phase of the East Pacific triggers new phytoplankton blooms in San Francisco Bay. Proc. Natl Acad. Sci. USA 104, 18561–18565 (2007).

5. 5.

Benincà, E. et al. Chaos in a long-term experiment with a plankton community. Nature 451, 822–825 (2008).

6. 6.

Campisi, J. & Ahmed, S. A toxic algae bloom in Florida is slaughtering marine life by the masses. https://www.cnn.com/2018/08/02/us/sea-turtles-florida-red-tide-wxc-trnd/index.html.

7. 7.

Qu, M. Z. et al. Algal blooms: proactive strategy. Science 346, 175–176 (2014).

8. 8.

Flewelling, L. J. et al. Brevetoxicosis: red tides and marine mammal mortalities. Nature 435, 755–756 (2005).

9. 9.

Gobler, C. J. et al. Ocean warming since 1982 has expanded the niche of toxic algal blooms in the North Atlantic and North Pacific oceans. Proc. Natl Acad. Sci. USA 114, 4975–4980 (2017).

10. 10.

Maldonado, M. T., Hughes, M. P., Rue, E. L. & Wells, M. L. The effect of Fe and Cu on growth and domoic acid production by Pseudo-nitzschia multiseries and Pseudo-nitzschia australis. Limnol. Oceanogr. 47, 515–526 (2002).

11. 11.

Lim, H. C. et al. Morphology and molecular characterization of Pseudo-nitzschia (Bacillariophyceae) from Malaysian Borneo, including the new species Pseudo-nitzschia circumpora Sp. Nov. J. Phycol. 48, 1232–1247 (2012).

12. 12.

Phytoplankton monitoring program. https://www.cdph.ca.gov/Programs/CEH/DFDCS/Pages/FDBPrograms/FoodSafetyProgram/PhytoplanktonMonitoringProgram.aspx.

13. 13.

Trask, B. J. & van den Engh, G. J. Elgershuizen JHBW. Analysis of phytoplankton by flow cytometry. Cytometry 2, 258–264 (1982).

14. 14.

Olson, R. J., Shalapyonok, A. & Sosik, H. M. An automated submersible flow cytometer for analyzing pico- and nanophytoplankton: FlowCytobot. Deep Sea Res. Part I Oceanogr. Res. Pap. 50, 301–315 (2003).

15. 15.

Pomati, F., Jokela, J., Simona, M., Veronesi, M. & Ibelings, B. W. An automated platform for phytoplankton ecology and aquatic ecosystem monitoring. Environ. Sci. Technol. 45, 9658–9665 (2011).

16. 16.

Gérikas Ribeiro, C., Marie, D., Lopes dos Santos, A., Pereira Brandini, F. & Vaulot, D. Estimating microbial populations by flow cytometry: comparison between instruments. Limnol. Oceanogr. Methods 14, 750–758 (2016).

17. 17.

Leroux, R. et al. Combining laser diffraction, flow cytometry and optical microscopy to characterize a nanophytoplankton bloom in the Northwestern Mediterranean. Prog. Oceanogr. 163, 248–259 (2018).

18. 18.

Olson, R. J. & Sosik, H. M. A submersible imaging-in-flow instrument to analyze nano-and microplankton: imaging FlowCytobot. Limnol. Oceanogr. Methods 5, 195–203 (2007).

19. 19.

Thyssen, M. et al. High-resolution analysis of a North Sea phytoplankton community structure based on in situ flow cytometry observations and potential implication for remote sensing. Biogeosciences 12, 4051–4066 (2015).

20. 20.

Dashkova, V., Malashenkov, D., Poulton, N., Vorobjev, I. & Barteneva, N. S. Imaging flow cytometry for phytoplankton analysis. Methods 112, 188–200 (2017).

21. 21.

Poulton, N. J. in Imaging Flow Cytometry (eds Barteneva, N. S. & Vorobjev, I. A.) 237–247 (Humana Press, New York, NY, 2016). https://doi.org/10.1007/978-1-4939-3302-0_17

22. 22.

Olson, R. J., Shalapyonok, A., Kalb, D. J., Graves, S. W. & Sosik, H. M. Imaging FlowCytobot modified for high throughput by in-line acoustic focusing of sample particles. Limnol. Oceanogr. Methods 15, 867–874 (2017).

23. 23.

Taucher, J. et al. Influence of ocean acidification on plankton community structure during a winter-to-summer succession: an imaging approach indicates that copepods can benefit from elevated CO2 via indirect food web effects. PLoS ONE 12, e0169737 (2017).

24. 24.

Heflinger, L. O., Stewart, G. L. & Booth, C. R. Holographic motion pictures of microscopic plankton. Appl. Opt. 17, 951–954 (1978).

25. 25.

Xu, W. B., Jericho, M. H., Meinertzhagen, I. A. & Kreuzer, H. J. Digital in-line holography for biological applications. Proc. Natl Acad. Sci. USA 98, 11301–11305 (2001).

26. 26.

Göröcs, Z. & Ozcan, A. On-chip biomedical imaging. IEEE Rev. Biomed. Eng. 6, 29–46 (2013).

27. 27.

Pfitsch, D. W. et al. Development of a free-drifting submersible digital holographic imaging system. In Proc. of OCEANS 2005 MTS/IEEE 690–696 (IEEE, Washington, DC, USA, 2005), https://doi.org/10.1109/OCEANS.2005.1639833.

28. 28.

Zetsche, E. M. et al. Imaging-in-flow: digital holographic microscopy as a novel tool to detect and classify nanoplanktonic organisms. Limnol. Oceanogr. Methods 12, 757–775 (2014).

29. 29.

Kiss, M. Z. et al. Special multicolor illumination and numerical tilt correction in volumetric digital holographic microscopy. Opt. Express 22, 7559–7573 (2014).

30. 30.

Bochdansky, A. B., Jericho, M. H. & Herndl, G. J. Development and deployment of a point-source digital inline holographic microscope for the study of plankton and particles to a depth of 6000 m. Limnol. Oceanogr. Methods 11, 28–40 (2013).

31. 31.

Yourassowsky, C. & Dubois, F. High throughput holographic imaging-in-flow for the analysis of a wide plankton size range. Opt. Express 22, 6661–6673 (2014).

32. 32.

Kamykowski, D. & Zentara, S. J. The diurnal vertical migration of motile phytoplankton through temperature gradients1. Limnol. Oceanogr. 22, 148–151 (1977).

33. 33.

Villarino, M. L. et al. Evidence of in situ diel vertical migration of a red-tide microplankton species in Ría de Vigo (NW Spain). Mar. Biol. 123, 607–617 (1995).

34. 34.

Suárez, L. V. et al. Thin layers of Pseudo-nitzschia spp. and the fate of Dinophysis acuminata during an upwelling-downwelling cycle in a Galician RÃa. Limnol. Oceanogr. 53, 1816–1834 (2008).

35. 35.

Sullivan, J. M., Donaghay, P. L. & Rines, J. E. B. Coastal thin layer dynamics: consequences to biology and optics. Cont. Shelf Res 30, 50–65 (2010).

36. 36.

Bishara, W., Su, T. W., Coskun, A. F. & Ozcan, A. Lensfree on-chip microscopy over a wide field-of-view using pixel super-resolution. Opt. Express 18, 11181–11191 (2010).

37. 37.

Bishara, W., Zhu, H. Y. & Ozcan, A. Holographic opto-fluidic microscopy. Opt. Express 18, 27499–27510 (2010).

38. 38.

Wu, Y. C. et al. Extended depth-of-field in holographic image reconstruction using deep learning based auto-focusing and phase-recovery. ArXiv: 1803.08138, 2018.

39. 39.

OpenROV underwater drones, an open source submersible platform. https://www.openrov.com/products/openrov28/.

40. 40.

Isikman, S. O., Greenbaum, A., Luo, W., Coskun, A. F. & Ozcan, A. Giga-pixel lensfree holographic microscopy and tomography using color image sensors. PLoS ONE 7, e45044 (2012).

41. 41.

Zhang, Y. B., Wang, H. D., Wu, Y. C., Tamamitsu, M. & Ozcan, A. Edge sparsity criterion for robust holographic autofocusing. Opt. Lett. 42, 3824–3827 (2017).

42. 42.

Tamamitsu, M., Zhang, Y. B., Wang, H. D., Wu, Y. C. & Ozcan, A. Comparison of Gini index and Tamura coefficient for holographic autofocusing based on the edge sparsity of the complex optical wavefront. ArXiv: 1708.08055, 2017.

43. 43.

Luo, W., Zhang, Y. B., Göröcs, Z., Feizi, A. & Ozcan, A. Propagation phasor approach for holographic image reconstruction. Sci. Rep. 6, 22738 (2016).

44. 44.

Rivenson, Y., Zhang, Y. B., Günaydın, H., Teng, D. & Ozcan, A. Phase recovery and holographic image reconstruction using deep learning in neural networks. Light Sci. Appl. 7, 17141 (2018).

45. 45.

Luo, W., Zhang, Y. B., Feizi, A., Göröcs, Z. & Ozcan, A. Pixel super-resolution using wavelength scanning. Light Sci. Appl. 5, e16060 (2016).

46. 46.

Luo, W., Greenbaum, A., Zhang, Y. B. & Ozcan, A. Synthetic aperture-based on-chip microscopy. Light Sci. Appl. 4, e261 (2015).

47. 47.

Greenbaum, A. & Ozcan, A. Maskless imaging of dense samples using pixel super-resolution based multi-height lensfree on-chip microscopy. Opt. Express 20, 3129–3143 (2012).

48. 48.

Faulkner, H. M. L. & Rodenburg, J. M. Movable aperture lensless transmission microscopy: a novel phase retrieval algorithm. Phys. Rev. Lett. 93, 023903 (2004).

49. 49.

Bao, P., Zhang, F. C., Pedrini, G. & Osten, W. Phase retrieval using multiple illumination wavelengths. Opt. Lett. 33, 309–311 (2008).

50. 50.

Fienup, J. R. & Wackerman, C. C. Phase-retrieval stagnation problems and solutions. J. Opt. Soc. Am. A 3, 1897–1907 (1986).

## Acknowledgements

This research was funded by the Army Research Office (ARO, W56HZV-16-C-0122). The Ozcan Research Group at UCLA acknowledges the support of the NSF Engineering Research Center (ERC, PATHS-UP), the ARO Life Sciences Division, the National Science Foundation (NSF) CBET Division Biophotonics Program, an NSF Emerging Frontiers in Research and Innovation (EFRI) Award, an NSF INSPIRE Award, the NSF Partnerships for Innovation: Building Innovation Capacity (PFI:BIC) Program, the National Institutes of Health, the Howard Hughes Medical Institute (HHMI), the Vodafone Americas Foundation, the Mary Kay Foundation, and the Steven & Alexandra Cohen Foundation. The authors would also like to thank Vanessa Zubkousky-White at the California Department of Public Health Marine Biotoxin Monitoring Program for providing microscope comparison data and helpful discussions.

## Author information

Authors

### Contributions

Z.G. initiated the project; A.O. supervised the project; Z.G., M.T., S.R., Y.W., and H.C.K. planned and conducted experiments. Z.G., M.T., S.R., and Y.W. analyzed the data; Z.G., M.T., V.B., P.W., K.S., Y.W., and Y.R. developed the algorithms; Z.G., M.T., and K.Y. designed and fabricated the flow cytometer; Z.G., M.T., and A.O. wrote the manuscript, and all coauthors contributed to the manuscript.

### Corresponding author

Correspondence to Aydogan Ozcan.

## Ethics declarations

### Conflict of interest

A.O. and Z.G. have a pending patent application on the presented imaging flow cytometer.

### Article accepted preview

Accepted article preview online: 31 August 2018

## Rights and permissions

Reprints and Permissions

Gӧrӧcs, Z., Tamamitsu, M., Bianco, V. et al. A deep learning-enabled portable imaging flow cytometer for cost-effective, high-throughput, and label-free analysis of natural water samples. Light Sci Appl 7, 66 (2018). https://doi.org/10.1038/s41377-018-0067-0

• Revised:

• Accepted:

• Published:

• ### Review—Recent Progress in Portable Fluorescence Sensors

• Young-Ho Shin
• , M. Teresa Gutierrez-Wing
•  & Jin-Woo Choi

Journal of The Electrochemical Society (2021)

• ### Learning from lanthanide complexes: The development of dye-lanthanide nanoparticles and their biomedical applications

• Guochen Bao
• , Shihui Wen
• , Gungun Lin
• , Jingli Yuan
• , Jun Lin
• , Ka-Leung Wong
• , Jean-Claude G. Bünzli
•  & Dayong Jin

Coordination Chemistry Reviews (2021)

• ### Lab-on-a-chip technology for in situ combined observations in oceanography

• Tatsuhiro Fukuba
•  & Teruo Fujii

Lab on a Chip (2021)

• ### AI on a chip

• Akihiro Isozaki
• , Jeffrey Harmon
• , Yuqi Zhou
• , Shuai Li
• , Yuta Nakagawa
• , Mika Hayashi
• , Hideharu Mikami
• , Cheng Lei
•  & Keisuke Goda

Lab on a Chip (2020)

• ### Automated plankton classification from holographic imagery with deep convolutional neural networks

• , Lisa Nyman