Computers are powerful tools for carrying out tasks such as image classification or identification as well as or better than human experts. Conventional machine learning approaches are widely used for segmentation and phenotyping in fluorescence microscopy. These tools are now being largely outperformed by their deep-learning-based counterparts, some of which are available as user-friendly tools (PLoS Biol. 16, e2005970, 2018; Nat. Methods 15, 677–680, 2018; Nat. Methods https://doi.org/10.1038/s41592-018-0261-2, 2018).

Images of a planarian before (top) and after (bottom) content-aware image restoration. Credit: M. Weigert, T. Boothe, and F. Jug

But a perhaps more astonishing wave of developments has recently come about through the use of deep learning not for image analysis but for image transformation. In these cases, deep convolutional networks are trained to transform one type of image into another. For example, two studies have shown the power of deep learning for the creation of fluorescence micrographs of cells directly from bright-field or phase images, to facilitate multiplexed and longitudinal imaging (Cell 173, 792–803, 2018; Nat. Methods 15, 917–920, 2018). Researchers have also used deep learning to go from low signal-to-noise images to high-quality images, which opens the door to extended imaging of even very light-sensitive living organisms (Nat. Methods 15, 1090–1097; 2018).

Deep learning can similarly overcome obstacles associated with super-resolution microscopy. Two approaches, ANNA-PALM and DeepSTORM, were developed to improve the speed of localization microscopy, which is one of the major hurdles of the technique (Nat. Biotechnol. 36, 460–468, 2018; Optica 5, 458–464, 2018). Deep learning can also enable cross-modality imaging, where applications such as a shift from confocal images to stimulated-emission-depletion-microscopy-resolution images could democratize super-resolution imaging.

As with any method, the caveats associated with deep learning in such applications, such as the potential for artifacts, must be carefully considered and analyzed. Nevertheless, we think we have seen only the tip of the iceberg, and that deep learning stands to improve all aspects of imaging, from acquisition to analysis.