Establishment of a guided, in vivo, multi-channel, abdominal, tissue imaging approach

Novel tools in humane animal research should benefit the animal as well as the experimentally obtained data. Imaging technologies have proven to be versatile and also in accordance with the demands of the 3 R principle. However, most imaging technologies are either limited by the target organs, number of repetitive imaging sessions, or the maximal resolution. We present a technique-, which enables multicolor abdominal imaging on a tissue level. It is based on a small imaging fiber endoscope, which is guided by a second commercial endoscope. The imaging fiber endoscope allows the distinction of four different fluorescence channels. It has a size of less than 1 mm and can approximately resolve single cells. The imaging fiber was successfully tested on cells in vitro, excised organ tissue, and in mice in vivo. Combined with neural networks for image restauration, high quality images from various abdominal organs of interest were realized. The second endoscope ensured a precise placement of the imaging fiber in vivo. Our approach of guided tissue imaging in vivo, combined with neuronal networks for image restauration, permits the acquisition of fluorescence-microscope like images with minimal invasive surgery in vivo. Therefore, it is possible to extend our approach to repetitive imaging sessions. The cost below 30 thousand euros allows an establishment of this approach in various scenarios.


Training and reconstruction procedure for the application of CSBDeep
We included the python library provided by the CSBDeep [1] project in a custom python script.
The library includes a deep neural network structure called U-Net, primarily specialized in enhancing images from fluorescence microscopes. We trained this network structure on images of the pancreas, jejunum, and liver separately. For the pancreas, we used 140 gold-standard images from which we generated 420 training images. The network specialized on images from the liver was trained with 124 gold-standard images and 372 training images. The model for the jejunum was calculated by using 131 gold-standard images and 393 training images. All images were saved as TIFF files with a size of 512 x 512 pixels and processed as gray-scale images.
To generate training images, which resemble the image quality of the endoscope, we took confocal images of the different organs as gold-standard and modified them digitally to obtain the training images. We added different levels of blur and noise to the training images and varied the image intensity (see and Supplementary Table 1   Following we compared three different approaches for training of the neuronal network. First, training was performed on simply blurred images ("blurred" network approach). This was used in combination with final images, which underwent a gaussian blur to remove the fiber core structure before reconstruction. Second, training was performed with synthetic fiber cores to directly reconstruct fiber images ("synthetic fiber" network approach). We used the diameter of the fiber cores of the endoscope to add a layer of adjacent circles with the same diameter to the training images. This layer was blurred after addition, to merge the synthetic core structure with the training image. Third, we created an overlay image of the fiber acquired with brightfield illumination from our setup and the training images ("overlay fiber" network approach). All three approaches were analyzed and compared.
Supplementary Figure 3 Generation of training data exemplarily shown for a liver image. Initially, we used three different training approaches. The "blurred" approach was later only applied to unknown images, which were also blurred to remove the fiber cores. The "synthetic fiber" and overlay fiber" approach were used for direct reconstruction of fiber images.

Training of the neuronal networks and reconstruction of images
Each data set was split into a training set and a validation set. We used 80 % of the data as training set and 20 % as validation set, which is a common approach in deep learning [2].
Training accuracy was measured by the mean absolute error to the gold-standard image.
Training was performed on a NVIDIA GTX 1060 GPU and took 3h33m for the liver data, 2h for the jejunum data set and 2h13m for the pancreas data set ("blended fiber approach"). For the application of the trained models we used the CSBDeep plugin for Fiji.
As detailed above, initially, we removed the fiber core structure in the data set by applying a gaussian blur to the images ("blurred" network approach). The networks were then trained using these images. Before applying the trained network to the unknown images, we also blurred these images. However, the reconstructed images showed many artifacts (see Supplementary Figure 2). Blurring of the images does not lead to a complete loss of the core structure even though the core structure is not visible for the eye. Therefore, blurring the images beforehand is not recommendable, instead we tried to remove the core structure using the network.
We decided to keep the core structure and to increase the depth of the U-Net architecture by resembling the fact, that another geometric structure has to be learned by the network. The network trained on the images with an added "synthetic fiber" layer, converged during the training process (see Supplementary Figure 2) with a training time of 3h29m. Restoration on the training images worked well. But applying the network to unknown images showed, that the original fiber structure could not be removed completely. Especially areas with potentially damaged fiber cores showed artifacts. Additionally, the fluorescent background light is not removed, resulting in comparatively low contrast.
The network with the "blended fiber" approach showed the best result. In the training history with a training time of 3h33m, the generalization error lies just below the training error. Because the gap between training and generalization error stays the same with increasing capacity of the model, the model is still converging. The mean absolute error of the network is the highest for all approaches with a difference of roughly 0.01 to the other models. This is not surprising, because the training set is the most challenging. Using an image of the actual fiber in training resulted in the best reconstruction. Areas with damaged fiber cores showed little artifacts and the fiber core structure was removed completely in the relevant areas.