Randomness assisted in-line holography with deep learning

We propose and demonstrate a holographic imaging scheme exploiting random illuminations for recording hologram and then applying numerical reconstruction and twin image removal. We use an in-line holographic geometry to record the hologram in terms of the second-order correlation and apply the numerical approach to reconstruct the recorded hologram. This strategy helps to reconstruct high-quality quantitative images in comparison to the conventional holography where the hologram is recorded in the intensity rather than the second-order intensity correlation. The twin image issue of the in-line holographic scheme is resolved by an unsupervised deep learning based method using an auto-encoder scheme. Proposed learning technique leverages the main characteristic of autoencoders to perform blind single-shot hologram reconstruction, and this does not require a dataset of samples with available ground truth for training and can reconstruct the hologram solely from the captured sample. Experimental results are presented for two objects, and a comparison of the reconstruction quality is given between the conventional inline holography and the one obtained with the proposed technique.


Introduction
Digital holography(DH) has emerged as a powerful tool for recording and reconstructing the amplitude and phase information of the wave [1][2][3][4] .The ability of the DH to retrieve complex amplitude information has a wide range of applications in 3D displays 5 , microscopy 6 , biomedical imaging 7 , and many more.The DH provides a spatially resolved quantitative phase images and depth reconstructions.In-line, off-axis, and phase shifting are a few widely used schemes.In an off-axis holography, two coherent and angularly separated beams interfere to record the hologram information 8 .Since available digital detectors have a limited pixel pitch, angular separation between interfering beams introduce a limitation for the off-axis DH.Moreover presence of unmodulated term and conjugate limits utilization of the full bandwidth in an off-axis DH geometry.Phase shifting is another technique that uses multiple recordings of the same object with phase shifts in the reference wave 9 .Among several holographic techniques, in-line holography has a compact design, and a high space bandwidth product(SBP) 10,11 .In-line holography schemes can be designed by using a single path and obtained by the interference of diffracted and un-diffracted waves emerging from the object 11 .However, a bottleneck of in-line holography is the ubiquitous twin image problem.Various techniques have been developed to resolve this issue using optical and computational methods [12][13][14] .
A quality of reconstruction in the DH depends on the recording configurations.Due to digital recording and constraints on the detectors, improving resolution is an emerging interest in the DH.Resolution in a digital holographic setup is influenced by factors such as; the numerical aperture, detector pitch, and diffraction.In past, various techniques have been proposed to improve the resolution in the DH and some of these techniques are lowering the wavelength 15 , down-sampling the detector pitch 16 , increasing the effective numerical aperture 17,18 , expanding computational bandwidth 19 .Recently several new ideas have been proposed to enhance the SBP 6,20,21 .Structured light illumination has also been used to improve the image quality and resolution [22][23][24][25][26][27][28] .Zheng et al used structured illumination in different orientations combined with an iterative algorithm to enhance the spatial resolution in the DH 28 .Speckle field illumination has also been used in the DH for high-resolution imaging and in enlarging field of view 6,[29][30][31][32][33][34] .However, these speckle illumination methods need recording of several holograms with the random illumination patterns for proper cancellation of randomness.
On the other hand, use of the randomness rather than cancellation has shown significant potential in imaging such as ghost imaging, ghost diffraction microscopy 35,36 , resolution enhanced wide field imaging 21 and many more.Memory effect 37 within a speckle pattern offers a high-resolution imaging systems 38 .Correlations have been exploited in super-resolution optical fluctuation imaging (SOFI) with dynamics near field speckle patterns. 39.It has been demonstrated that higher-order (n) intensity arXiv:2303.17660v1[physics.optics]30 Mar 2023 correlations improve the resolution by a factor of √ n 40 and such higher-order correlations have been utilized for improving resolution beyond the diffraction limit 40,41 .A sub-Rayleigh imaging has been demonstrated using a second-order correlation measurement 42 .However, majority of these correlation techniques mainly deal with amplitude object, without any phase signature of the signal except a recent work 43 .In the reference 43 , a sub-Rayleigh dark-field imaging by speckle illumination is demonstrated and an autocorrelation image is presented for a binary phase object.
In this paper, we propose and demonstrate a new method to record an in-line hologram in intensity correlation rather than the intensity as in conventional holography.This is aimed to enhance the quality of reconstruction in the holography and demonstrate it by reconstruction of a complex-valued object in the inline DH.A common issue of twin images in the in-line holography is tackled with a deep learning method by utilizing an auto-encoder model.
We use a deep learning approach for a single-shot image reconstruction without a large training dataset 14 .The auto-encoder minimizes a well-defined objective function to reduce noise and remove twin image instead of suppressing it.Our results show that a neural network equipped with the proposed correlation methods provides an enhanced image quality of the complex-valued object.The proposed technique is experimentally verified and results are presented for two different cases; i.e. a conventional holographic recording and hologram recording by second-order correlation.A comparison of conventional holography and the proposed method highlight high-quality reconstruction in the new technique.A theoretical background and experimental demonstrations are discussed below.

Theory and Methodology
A conventional recording of an in-line hologram and its comparison with proposed technique is shown in Fig. 1.Diffracted and undiffracted beams from the object interfere at plane 1 and make an in-line hologram as a distribution of the intensity.This inline hologram is imaged and digitally recorded at plane 2 as shown in Fig. 1(a).An aperture is placed in the imaging system to control the numerical aperture and consequently analyze its impact on the image quality.An intensity of the in-line hologram at plane 1 is given by, where is the complex field at plane 1. E D (ρ) and E d (ρ) are diffracted and un-diffracted beams respectively and ρ is the spatial coordinate at plane 1.The optical field at plane 2 is represented as, where the digitally recorded hologram is I = |E H h| 2 , represents the convolution operator and r is the spatial coordinate at plane 2. h(r − ρ) represents the point spread function (PSF) of a diffraction-limited imaging system.The PSF for a diffraction-limited imaging lens is an airy disk and is represented as, where P(ρ 0 ) is the pupil function describing the effective entrance pupil, and d 1 and d 2 are the distances from the lens to the object plane and image plane, respectively, and d 1 =d 2 =2f.Eq. 2 represents a digital recording of the hologram which is numerically reconstructed to recover the complex-valued object from the DH.Now, consider a scatterer at the plane 1 which hides the direct recording of the hologram from the detector.A random scatterer in the path of the light scrambles the wavefront and generates a speckle pattern as shown in Fig. 1(b).A single realization of the field immediately after the scatterer is given by, where φ (ρ,t) is the random phase introduced by the scatterer and t represents time corresponding to different random patterns.The complex field at the recording plane 2 is represented as, The random intensity at plane 2 is Imaging of a hologram through a scatterer has been used in the past for looking around corners 44 and in the correlation holography 45 .These methods rely on recovering conventionally recorded hologram i.e, I(ρ) from the averaged random fields.
On the other hand, here we propose and experimentally demonstrate a new technique by recording the hologram in terms of the second-order correlation of intensity rather than intensity for high-quality holographic reconstruction.Following Eq. 6, intensity fluctuation is given as, where angular bracket ... represents ensemble average and I is mean intensity.We introduce and use correlation of intensity fluctuations to record the hologram and this correlation function is represented as, Eq. 8 represents our quantity of interest to record the in-line hologram.In comparison to the conventional hologram recording in I(r) , the proposed technique records hologram as g 2 (r, r).For the Gaussian random field, second-order intensity correlation can be expressed as a modulus square of field correlations as, where asterisk * represents the complex conjugate.The second-order field correlation is represented as, For a spatially incoherent source, the correlation at plane 1 is represented as, Substituting Eq. 11 in Eq. 10 leads to Therefore second-order intensity correlation is expressed as, For a uniform source, i.e, I(ρ) = 1 ;the second order intensity correlation transforms to, Equation 14 represents that the correlation of intensity fluctuations is proportional to the fourth power of the PSF of an imaging lens.A comparison of Eq. 2 and Eq. 14 shows that the recording quality is enhanced in the higher-order correlation due to a narrower size of the PSF as compared to the conventional case.This is quantitatively analyzed and experimentally tested in the coming section.Although the quality of recording the inline hologram is improved in the randomness-assisted approach, reconstruction of an inline hologram demands the removal of the twin images.This is tackled by using an auto-encoder deep learning method as explained in the coming section. .This SLM is having 1280×768 pixels with a pixel pitch of 20µm.An object is inserted in the incident light using the SLM and the light reflected from the SLM is folded by the BS to propagate toward the plane 1(without RGG).Interference of diffracted and undiffracted waves from the object makes an inline hologram at the plane 1 and this plane is imaged by lens L2 at the camera plane.Focal length of lens L2 is 100mm and a variable aperture A2 is placed to control the numerical aperture of the imaging lens as shown in Fig. 2. A CMOS camera with 1024×1280 pixels, and pixel size 5.4µm, records the in-line hologram.
In order to replace conventional hologram recording by recording in the correlation of intensity fluctuations, we insert a rotating ground glass(RGG) at plane 1 in the experimental system.An inline hologram at plane 1 is now scrambled due to the random phase introduced by the RGG.Therefore recorded pattern at the camera plane represents a speckle pattern for a particular position of the RGG and the pattern is represented as |E h| 2 .This random phase is common to both the diffracted and undiffracted beams at the plane 1, thus hologram information can be retrieved by averaging the random patterns in spite of randomness in the path of the light.The randomly scattered inline hologram is recorded using different orientations of the RGG at different times t.For experimental demonstrations of the proposed technique, we used 150 frames which are independently generated by the RGG.A series of random patterns with independent randomness are utilized to measure the correlation of intensity fluctuations and to record the inline hologram.In the randomness-assisted recording of an inline hologram, a correlation of intensity fluctuations is measured by estimating correlations of intensity fluctuations over varying random patterns.We considered averaging over 150 random patterns and is represented as This equation represents the correlation of the fluctuations of intensity patterns corresponding to 150 independent random patterns generated by the ground glass.Each shot of the recorded image has a noisy pattern for fixed orientation of RGG at a time t.We make use of averaging over a number of random patterns and g 2 (r, r) is estimated.This results in a hologram with reduced noise and an inline hologram appearing as a distribution of g 2 (r, r).Experiment is performed for two different objects, a Greek letter β and an English letter "V" .An inline hologram of letter β is shown in Fig. 3(a) recorded as a distribution of the intensity I(r) and recording of an inline hologram of the same object as a distribution of second order intensity correlation

Deep learning for numerical reconstruction of inline hologram
In the past, different methods have been proposed to tackle reconstruction of the in-line hologram, including the phase retrieval algorithms [46][47][48][49][50][51] .Accurate recovery of the phase can lead to strong suppression of the twin image 52,53 .We use a deep learning approach for reconstructing the in-line hologram using an encoder-decoder architecture 14 .This network maps a high dimensional input image, α, into a low dimensional latent code, β .The mapping from the input image to the latent code and the reconstruction of the output from the latent code are represented by the functions F encode (α) = β and F decode (β ) = α, respectively.For a supervised image restoration, the output image is compared to a ground truth image, and the error between them is penalized.This model can learn the uncorrupted and realistic features of the input image through an iterative process using an untrained network, allowing it to converge on a denoised output.To address the twin image issue in an inline hologram, a physical model is designed for training the auto-encoder.The auto-encoder, denoted as F auto−encoder (α, w), is trained to minimize the error between the captured in-line hologram H (I(r) or g 2 (r, r) ) and the reconstructed in-line hologram, which is the forward-propagated result of the reconstructed complex object wave to the plane 1 using F transmission .The weights w are randomly initialized at the start of training.The goal is to find the weights w that minimize the error between the captured in-line hologram and the reconstructed in-line hologram.The objective function can be written as: Where |.| 2 is L 2 norm.As depicted in Fig. 4, here we use an "Hourglass" type auto-encoder architecture.This encoder transfers the fixed network input into lower-dimensional space, encoding steps, and decoding steps are done by reconstructing the complex object from their representation by the latent variables.We use a wavelet transform and its inverse transform to provide downsampling (encoding) and upsampling (decoding), which substitute strided convolution, pooling, or interpolation 14 .The proposed method is shown step-by-step in Fig. 5.The in-line hologram obtained from Eq. 14 is first back-propagated to the object plane to determine the amplitude and phase distributions.These are then fed into the auto-encoder model.The network outputs the reconstructed amplitude and phase, which are then forward propagated to the in-line hologram plane using F transmission to get a reconstructed in-line hologram.The mean absolute error (MAE) loss between the original hologram and reconstructed hologram is calculated to update the model's weights."

Results and Discussion
We record inline holograms in two different experimental configurations as shown in Fig. 1.In the first case, an inline hologram is recorded in free space i.e, without a scatterer.The inline hologram at plane 1 is imaged by a lens L2 with a variable aperture A2in front of the lens L2.In the second case, when a scatterer is placed at plane 1, a speckle is generated due to the randomness and interference of randomly scattered waves.Light is scrambled into the speckle pattern without any direct resemblance with the hologram I(r).For randomness-assisted inline hologram recording, a series of random patterns are captured and these random patterns are digitally processed to obtain the second-order correlation of intensity.The inline hologram is back-propagated to the object plane by using an angular spectrum method.However,inline configuration of recording introduces a twin image in the reconstruction.Therefore a deep learning is applied to quantitatively reconstruct the inline hologram without a twin image.We employ the auto-encoder model to resolve this problem and this is implemented using the PyTorch framework and runs on a GPU workstation with an Nvidia tesla P100 graphics card.The Adam optimization algorithm is 5/10 Reconstruction results are presented for two different objects viz, β (size : 2.8mm × 1.48mm) and letter V (2.84mm × 2.64mm) as shown in Fig. 6 and Fig. 7 respectively in free space and randomness assisted geometry.Fig. 6 (a) and (b) are the amplitude and phase of letter β in the inline holography without scatterer.Fig.6(c) and (d) represent the retrieved amplitude and phase for the randomness-assisted hologram recording.Fig. 7(a) and (b) are the amplitude and phase for letter V in the inline holography without scatterer and Fig. 7(c) and (d) show reconstructed amplitude and phase distribution for the proposed randomness-assisted inline holography technique.Comparison of inline holography in free space and through randomness shows that reconstruction quality is significantly enhanced in the hologram recording with the second-order intensity correlation.Edges and frequency contents are enhanced in the holography with g 2 (r, r) in comparison to the holography with I(r).This is observed by accessing the spatial frequency contents of the reconstructed object for both inline holographic recording methods i.e, I(r) and g 2 (r, r).For a quantitative evaluation and comparison of the quality of reconstruction of both hologram recording methods, we also introduce two parameters, such as visibility and reconstruction efficiency 54 .Visibility gives a measure of the degree to which the signal reconstruction can be differentiated from the background noise.It is defined as the ratio of the signal region's average intensity level to the background region's average intensity level in the reconstructed image.On the other hand, reconstruction efficiency is the proportion of the signal region's measured power to the total of the signal and background regions' measured powers in the reconstructed image.Visibility values for letter β in the conventional and proposed scheme are 0.042 and 0.21, whereas respective reconstruction efficiency values are 0.55 and 0.885.For other object V, visibility and reconstruction efficiency values are 0.081 and 0.67 for conventional case.The improved Visibility and reconstruction efficiency values for object V in the proposed scheme are 0.55 and 0.92.

Conclusion
In conclusion, we have demonstrated a new method to record holograms in the second-order intensity correlation and the advantage of this recording appears in the quality of the reconstructed object from the hologram.In order to demonstrate the proposed scheme of recording in the holography, we recorded inline holograms in both conventional and randomness-assisted approaches and the results are compared.A twin image issue of the inline hologram is resolved with a deep learning method that uses an auto-encoder model.The quantitative features are retrieved and quality is compared in a conventional and the proposed scheme.The scheme is expected to be useful in digital holography and other imaging schemes.

Figure 1 .
Figure 1.A comparison of (a) conventional and (b) proposed technique

Figure 3 .
Figure 3. (a)Conventional inline hologram for β ;(b)Inline hologram in terms of second order correlation

Figure 4 .Figure 5 .
Figure 4.An auto-encoder network utilizing deep convolutional layers in an hourglass architecture