Rapid detection of neurons in widefield calcium imaging datasets after training with synthetic data

Widefield microscopy can provide optical access to multi-millimeter fields of view and thousands of neurons in mammalian brains at video rate. However, tissue scattering and background contamination results in signal deterioration, making the extraction of neuronal activity challenging, laborious and time consuming. Here we present our deep-learning-based widefield neuron finder (DeepWonder), which is trained by simulated functional recordings and effectively works on experimental data to achieve high-fidelity neuronal extraction. Equipped with systematic background contribution priors, DeepWonder conducts neuronal inference with an order-of-magnitude-faster speed and improved accuracy compared with alternative approaches. DeepWonder removes background contaminations and is computationally efficient. Specifically, DeepWonder accomplishes 50-fold signal-to-background ratio enhancement when processing terabytes-scale cortex-wide functional recordings, with over 14,000 neurons extracted in 17 h.

a. First row, standard deviation (STD) of 1000 simulated frames at 10 Hz by NAOMi1p, from 0.1 to 1.1 mW/mm 2 illumination power density. Second row, STD of the de-background images by RB-Net. Third row, correlation image of the virtual captures by CNMF-E.
b. First row, segmentation results of a by DeepWonder, from 0.1 to 1.1 mW/mm 2 illumination power. Blue masks represent corrected segments compared to ground truth, red masks represented missed segments, and brown masks represent wrong segments. Second row, segmentation results by CNMF-E. c. Red curve: structure similarity index (SSIM) of STD of raw captures from 0.1 to 1.1 mW/mm 2 , with STD of raw captures at 1.5 mW/mm 2 as the reference. Blue curve: the SSIM across the same power density range but by correlation image from CNMF-E. Green curve: the SSIM across the same power density range but by STD of RB-Net de-background recordings.

Supplementary Figure 10
Calibration of the joint 2p-widefield detection system. a. First row, 2p (left) and widefield (right) imaging of 1-µm diameter fluorescent beads. The white box isolates one of fluorescent beads and the intensity profile across the red dashed line is indicated in the zoom-in panel. Red dots indicate raw data and blue curves indicated Gaussian fitted data with MATLAB command fit. The full width at half maximum (FWHM) of each profile is marked on the bottom side, and relative peak intensity is marked on the right side. The second row and the third row are similar to the first row but defocus the sample by z = 5 µm and z = 10 µm, respectively. As defocus increases, the low-NA 2p PSF shows consistent tight focus and high intensity, while the PSF by the widefield spread fast. Note when the defocus reaches z = 10 µm, the defocus ring of beads in widefield becomes too large, and the FWHM thus is only measured at the central peak and has a smaller value.
b. Axial resolution in 1/e changes with excitation NA in simulation 3 . With calibration, we find that our system has a valid NA of 0.27.

Validation of DeepWonder functional signals with high-NA ground truth.
a. Standard deviation (STD) images of the raw movie by widefield (WD, left), DeepWonder processed movie (middle), and high-NA 2p recordings (right). Some neurons that are clear in the DeepWonder processed movie are invisible in high-NA 2p recordings. b. Neuron segmentation by DeepWonder processed movie compared to high-NA 2p recordings. Shallow blue circles indicate neurons that are found in both DeepWonder and high-NA data, and red circles indicate neurons that are only found in DeepWonder data.
c. Neuronal calcium activities from DeepWonder data (red) and corresponding high-NA functional ground truth (blue). Traces by DeepWonder are offset vertically for clarity. d. Distributions of temporal correlation between DeepWonder extracted neuronal traces and corresponding high-NA traces across 54 paired neurons. White circle: median. Vertical grey bar: interquartile range. Transparent disks: data points. Transparent violin-shaped areas: kernel density estimate of data distribution. e. Spike inference scores (precision, sensitivity, and F-score) achieved by OASIS 4 on DeepWonder traces compared to high-NA functional ground truth, across 54 paired neurons in a capture of 4500 frames. Central black mark: Median. Bottom and top edges: 25th and 75th percentiles. Whiskers extend to extreme points excluding outliers (1.5 times above or below the interquartile range). c. Standard deviation (STD) across temporal frames of DeepWonder processed movie (left) and corresponding segmentation (right). Blue masks represent corrected segments compared to ground truth segments, green masks represent missed segments, and red masks represent wrong segments. d. Neuronal activity traces corresponding to arrows in c are used for performance quantifications. Red traces for DeepWonder movie and blue traces for 2p movie, and DeepWonder traces are offset vertically for clarity. e. Temporal correlations of detected neurons with 2p functional ground truth from DeepWonder movie (0.82 ± 0.15, mean ± SD, blue) and raw movie (0.55 ± 0.18, mean ± SD, red), over n = 198 neurons from 6 recordings across 3 mice. Central black mark: Median. Bottom and top edges: 25th and 75th percentiles. Whiskers extend to extreme points excluding outliers (1.5 times above or below the interquartile range).
f. F1 scores of segmentation by DeepWonder are 0.86 ± 0.15 (mean ± SD) from 6 recordings over 3 mice. Box plot elements as in e. g. Schematics of an imaging condition where vessels and target neuronal population are in the same depth. Vessels are clear in z = 100 µm and neurons are clear in z = 100 µm.
h-j are the same as b-d but in new data (where vessels and target neuronal population are in the same depth, like in g).
k. Temporal correlations of detected neurons with 2p functional ground truth from DeepWonder movie (0.83 ± 0.09, mean ± SD, blue) and raw movie (0.64 ± 0.09, mean ± SD, red), over n = 121 neurons from 5 recordings across 3 mice. Box plot elements as in e.
l. F1 scores of segmentation by DeepWonder are 0.88 ± 0.15 (mean ± SD) from 5 recordings across 3 mice. Box plot elements as in e.