Nikon’s Denoise.ai functionality for NIS-Elements software was developed to target and remove image shot noise. All imagingmodalities capture images that contain a signal and a noise component. The shot noise component is a result of discreetlysampling and digitizing emitting photons from the sample to form images. Shot noise follows a Poisson distribution, and has routinely been calculated as the square-root of the input signal. Thus, as signal-level decreases, the contribution of shot noise to the signalto-noise ratio (SNR) increases dramatically.
Noise in confocal images is overwhelmingly a result of shot noise; this makes confocal images an ideal target for deep learning,as the source of noise is essentially constrained to one source. Nikon employed a convolutional neural network encoded withseveral thousand examples of acquired Nikon confocal data. The neural network assigned input images learnable weights, whichresulted in the network learning to make correlations and recognize patterns. The common pattern to all images was the Poisson shot noise component.
The network was then trained to recognize and remove the shot noise component from confocal datasets. It could then beenabled on a graphic processing unit (GPU) for fast computation, even allowing for live denoising. These features are enabled in the Denoise.ai plug-in for NIS-Elements software.
As a result, the network can estimate and detect noise contributions to input images, then remove the noise component foroutput data. Result data maintains the same intensity and structure as the original image data, having only the variances in intensity removed (Fig. 1). Removing the noise component greatly enhances the quality of output images, particularly in low-intensity data.
Figure 1 | Evaluation of result images Raw confocal and AI-denoised images (top), with an intensity profile line overlaid to compute the intensity of pixels along the line (bottom). The intensity profile shows that the denoised result (yellow line) maintains the original intensity, but removes the shot noise variance visible in the raw image (blue line). Image: microtubules, 60x.
The ability to remove shot noise from confocal data allows users more flexibility in acquisition (Fig. 2). Typically, in confocalimaging, averaging frames or increasing pixel dwell time is used to reduce shot noise contributions, but this results in a decrease in sampling frequency and an increase in the number of illumination passes. Both have detrimental effects on specimens that are sensitive to illumination. Longer illumination times, and more illumination passes, slow acquisition and increase the potential for bleaching. Oftentimes, these measures offer diminishing returns, as the more times a sample is imaged, the lower its SNR can be due to bleaching.
Figure 2 | Typical imaging results. Raw high-speed resonant scanning confocal image with 0.1 μs dwell time and no averaging, and the result of applying denoise.ai. Image: Drosophila sp. embryo, 20x.
Alternatively, denoising a raw image by using artificial intelligence (AI) generally results in an 8–16× improvement in image quality without acquiring additional image passes or slowing acquisition speed. This means acquisition frequency can also be increased and/or excitation power decreased when acquiring data. With an increase in sampling frequency, as well as a decrease in the number of acquisitions required to make images, the illumination time on the sample is greatly reduced and the ability to capture faster events is realized. Imaging 8–16× more rapidly or the ability to capture 8–16× more data points can be a huge asset in data collection.
Artificial intelligence has become commonly accepted in diagnostic imaging and is an increasingly popular tool for a number of applications. Its appeal over traditional mathematical approaches is both its speed and incredible accuracy. However, it is important to be able to validate the results of AI computations, and to utilize denoised results appropriately for computational analysis.
Since deep learning is employed to identify and remove shot noise, the input data should be of sufficient SNR such that the shotnoise component can truly be differentiated from the signal. If the SNR is sufficient, the variance in intensities following a Poisson shot noise pattern can be easily removed.
Nikon’s NIS-Elements also provides real-time tools to measure SNR in images to assist in assessing image quality. This guides users to prepare confocal imaging settings that result in images benefitting greatly from denoising. Users can also acquire ground-truth images (images acquired by traditional means to improve SNR, such as averaging, integration or increased pixel dwell time) to compare directly to denoised images to assess the successful removal of noise variance in image data.
Denoise.ai not only allows the ability to decrease illumination power and increase acquisition frequency for fast confocal imaging, but also improves images highly corrupted by shot noise. For fleeting events, it can mean the difference between not being able to image at all, or to successfully acquire data. Improvement in image quality can allow users to see structures more clearly, track objects over time, or assist in finding and focusing on rapidly moving objects.
As fluorescent probes improve and gene-editing techniques become more common, fluorescence detection will continue to be challenging, as labeling densities decrease and devices push for faster acquisition frequencies and lower illumination power. Utilizing artificial intelligence techniques for noise reduction can greatly assist in improving output data quality fordownstream analysis.