Spatial Multiplexing Technique for Improving Dynamic Range of Speckle Correlation based Optical Lever

Speckle correlation based optical levers (SC-OptLev) possess attractive characteristics suitable for sensing small changes in the angular orientations of surfaces. In this study, we propose and demonstrate a spatial multiplexing technique for improving the dynamic range of SC-OptLev. When the surface is in its initial position, a synthetic speckle intensity pattern, larger than the area of the image sensor is created by transversely shifting the image sensor and recording different sections of a larger speckle pattern. Then, the acquired images are stitched together by a computer program into one relatively large synthetic speckle pattern. Following the calibration stage, the synthetic speckle intensity pattern is used to sense changes in the surface’s angular orientation. The surface is monitored in real-time by recording part of the speckle pattern which lies within the sensor area. Next, the recorded speckle pattern is cross-correlated with the synthetic speckle pattern in the computer. The resulting shift of the correlation peak indicates the angular orientations of the reflective surface under test. This spatial-multiplexing technique enables sensing changes in the angular orientation of the surface beyond the limit imposed by the physical size of the image sensor.

the above equation reduces to d = 2 Lβ, providing a linear relationship between d and β. Therefore, the angular sensitivity of the measurement can be increased easily by increasing L. As a result, the conventional OptLev with a mirror and a sensor can be easily adapted to measure small variations with a high accuracy. For an image sensor with a pixel size of Δ and consisting of m × m pixels, the physical size of the image sensor is Δ·m along the vertical and horizontal directions. The dynamic range of measurement of β is R = ± 0.5tan −1 (Δ·m/2 L), which is the maximum displacement detectable by the image sensor of breadth Δ·m. The minimum angle which can be sensed by the OptLev when the detected signal is shifted by a full pixel is β min = (Δ/2L). Since the sensitivity is defined as S = 1/β min , the dynamic range can be expressed as a function of the sensitivity as R ≈ ± m/ (2 S). From this last equation, it is clear that when R increases, S decreases and vice versa, indicating the trade-off between the sensitivity and dynamic range.
In 1976, a tilt measurement technique using speckle correlation was introduced by Gregory 15 . This idea began to gain attention in the next few years [16][17][18][19][20][21] . A real-time surface inspection technique based on speckle correlation was proposed and demonstrated with an optical matched filter in 1992 by Hinsch et al [.22 . Several factors made speckle correlation more attractive compared to the interference techniques that had been used for sensing tilt and deformation [23][24][25][26] till then. For example, speckle correlation required fewer optical elements, used an interferenceless optical configuration and gave rise to the possibility of real-time monitoring 22 . Alternatively, structured light techniques with spatial calibration were developed to measure tilts 27 . Currently, the speckle correlation technique is used as a sensing tool for various applications such as measurement of random processes in rough surfaces 28,29 , quantification of the corrosion process 30 , surface slope, deformation and motion measurements [31][32][33][34][35] , and sub-micrometer displacement measurement 36 .
Considering the disadvantages of interferometry pertaining to the need for two beam interference and vibration isolation, speckle correlation based OptLev (SC-OptLev) is chosen in this study. The SC-OptLev is compared against the conventional OptLev (C-OptLev) 13,14 with the help of Fig. 2. Light from a coherent source illuminates a diffuser plate and the light scattered by the diffuser plate is incident on a plane mirror. The mirror is oriented with respect to the optical axis and deflects the scattered light such that the speckle pattern is recorded by the image sensor. When the mirror is tilted, the incident scattered light acquires a linear phase according to the change in the angular orientation of the mirror, propagates in a different direction and reaches the image sensor at a different lateral location. The scattered light from the diffuser is incident on the mirror and the complex amplitude after the mirror can be expressed as A(x,y)·exp[jΦ(x,y)], where A(x,y)∈ [0,1] and Φ(x,y) ∈ [0,2π]. The light reflected from the mirror reaches the image sensor after propagating a distance of L with a complex amplitude given by A(x,y)·exp[jΦ(x,y)]ΨQ(1/L), where Q is the quadratic phase function given as Q(a) = exp[iπaλ −1 (x 2 + y2)] and '⊗' is a 2D convolution operator. The complex amplitude of light when the mirror undergoes an angular variation of β along the x direction is given as A(x,y)·exp[j{Φ(x,y) + 2πλ −1 sin(2β)x}] and the complex amplitude reaching the image sensor in this case is A(x,y)·exp[j{Φ(x,y) + 2π λ −1 sin(2β)x}]ΨQ(1/L). The intensity patterns recorded by the sensor before and after the angular variation of the mirror are I 0 = |A(x,y)·exp[jΦ(x,y)]ΨQ(1/L)|2 and I β = |A(x,y)·exp[j{Φ(x,y) + 2π λ −1 sin(2β)x}]ΨQ(1/L)|2 respectively. The shift of the pattern on the image sensor is calculated by comparing the location of the correlation peaks obtained from C 1 = I 0 ⊗I 0 and C 2 = I β ⊗I 0 , where '⊗' is a 2D correlation operator. If the two intensity patterns I 0 and I β are zero-padded to double their size, the dynamic range of the C-OptLev is doubled 37 . Optical configuration of a conventional OptLev. Light from a coherent source illuminates a mirror attached to observe the sample. The light reflected from the mirror is captured by the image sensor. Any variation in the angular orientation of the sample is measured from the shift of the optical beam on the image sensor. α is the initial angle of incidence of the beam on the mirror, whose tilt (β) is to be measured. θ′ is the new angle of incidence after the mirror tilts. The displacement of the beam on the sensor due to the tilt is d.
SC-OptLev has various advantages compared to C-OptLev as the correlation process can extract not only tilt but also most of the surface deformations [28][29][30][31][32][33] . Even when the comparison is focused only on the measurement of angular variation of an sample, SC-OptLev exhibits a dynamic range twice as much as that of a C-OptLev 37 . Moreover, since the light from a coherent source such as a He-Ne laser has a beam diameter of few millimeters, it is often necessary to use a lens to focus the light to a smaller diameter on the sensor and therefore a C-OptLev is susceptible to lens aberrations. On the other hand, SC-OptLev requires simple components such as a diffuser plate, or an inexpensive thin scatterer. Under the assumption of space invariance, the cross-correlation used for the tilt measurement cancels out any aberrations induced by the optical components, since the same aberrations are expected in both recorded speckle intensity patterns. Furthermore, the width of the correlation peak can be engineered by signal processing techniques, and by that the measurement performances can be improved 38 . Therefore, it is advantageous to use an SC-OptLev compared to a C-OptLev for various applications. The accuracy of the two methods depends upon how narrow a focal spot or a correlation peak can be obtained for C-OptLev and SC-OptLev, respectively, and improving the accuracy is out of the scope of the current study.
However, SC-OptLev like C-OptLev suffers from the trade-off between the dynamic range and sensitivity, although SC-OptLev has different limitations compared to C-OptLev in these two parameters. Moreover, in some optical configurations the width of the reconstructed correlation peak is dependent upon the scattering degree of the diffuser plate and the various distances of the optical setup [39][40][41] . There are other problems associated with an SC-OptLev such as the presence of background noise due to the correlation of speckle intensity patterns 42,43 . In this manuscript, we propose a spatial multiplexing technique to improve the dynamic range beyond the range of ref. 37 , without significantly affecting the sensitivity of the SC-OptLev. The presence of background noise due to the cross-correlation of speckle intensity patterns is also minimized herein. Note that it is not possible to monitor the angular variation in real-time since in SC-OptLev the observed signal is only a movement of a speckle pattern, unlike the movement of a focused spot in the case of C-OptLev. Various solutions are provided in ref. 37 to address the real-time problem and to develop a SC-OptLev that is as simple as C-OptLev, but with a double the dynamic range.

Methodology
The optical configuration of the spatial multiplexing SC-OptLev (SM-SC-OptLev) is shown in Fig. 3. As mentioned above, light from a coherent source is incident on a diffuser plate, scattered and deflected by a surface whose angular tilt is to be monitored. The light deflected from the surface is recorded by an image sensor located at a distance of L from the tested surface. The features of the speckle pattern on the sensor are dependent upon the scattering degree of the diffuser plate, its size and the distance L 37 . If D is the size of the image sensor along the x and y directions, the maximum angular variation of the surface that can be detected using C-OptLev is β 1-max = ± 0.5tan −1 (D/2 L). However, under the assumption that the scattered light of the diffuser plate covers at least the entire image sensor, or more, in the initial state of no surface tilt, the maximum detectable angular variation in the case of SC-OptLev is β 2 -max = ±0.5tan −1 (D/L) 37 . Even when the speckle pattern is much larger www.nature.com/scientificreports www.nature.com/scientificreports/ than the sensor area, the maximum detection angle is still limited by the sensor area. In this section, a spatial multiplexing technique SM-SC-OptLev is proposed to overcome this limit.
The proposed technique involves generation of a synthetic speckle intensity pattern, created by shifting the image sensor to different lateral locations and recording the speckle pattern at each location. Following the recording, the synthetic pattern is synthesized by a computational stitching procedure. If (x 0 , y 0 ) is the center of the image sensor and I 0 (x 0 , y 0 ) is the speckle intensity pattern when the mirror is in its initial position, the part of the speckle intensity pattern recorded by the image sensor is given as I 0 (x 0 ,y 0 )Rect[(x 0 ,y 0 )/D]. The synthetic speckle intensity pattern I s0 , with a size of p times (odd) that of the image sensor, is expressed as When the mirror is tilted by an angle of β, the speckle intensity pattern shifts across the image sensor by a distance d = L·tan(2β) and the shifted intensity pattern I β is recorded by the image sensor. I β is cross-correlated with the larger synthetic intensity pattern stitched computationally according to Eq. (1) from the p recorded speckle patterns. The maximum tilt angle of the mirror along the x and y directions is± 0.5 tan −1 (pD/2L), where p = 1,3,5… The synthetic speckle intensity pattern stored in the computer is used as the reference pattern for the on-going tilt measurements. The speckle intensity pattern I β (x 0 , y 0 ,t)Rect[(x 0 , y 0 )/D] recorded in real-time by the image sensor is zero-padded to match the size of the synthetic speckle pattern and is denoted by I β '(x 0 ,y 0 ,t). The location of the correlation spot is calculated by a 2D cross-correlation between I s0 and I β ' . However, previous studies have shown that a correlation with the matched filter is not the ideal filter to obtain sharpest correlation peak due to the generated background noise. Alternatively, a phase-only filter is often implemented to obtain the sharp correlation peak I R with less background noise as given in Eq. (2) 37,43-45 , , where  and  −1 are the Fourier transform and inverse Fourier transform operators, respectively. Recently, a non-linear adaptive correlation was implemented to reconstruct images with a background noise lesser than that of the phase-only filter 46,47 . The reconstructed image I R with a non-linear cross-correlation is given by where the values of o and b are tuned to minimize the background noise. In [46,47], entropy was used to find the values of (o,b) that would reconstruct an object with the minimum background noise.

experiments
The spatial multiplexing technique is experimentally demonstrated using a set-up similar to Fig. 3. A He-Ne laser with a wavelength λ = 632.8 nm is used to illuminate a diffuser (HO-DF-25-X, Holmarc, X = 22 µm), where X is the grade of the diffuser. The light scattered by the diffuser is incident on a plane mirror which is oriented at an angle of 45° with respect to the optical axis and deflects the incident light by an angle of 90° with respect to the optical axis. The light deflected from the mirror was captured by an image sensor (Thorlabs Camera DCC1240M, 1024 × 768 pixels, pixel size: 4.65 µm) located at a distance of 7 cm from the mirror. The distance between the diffuser and mirror was 3 cm. The size of the image sensor is 4.76 × 3.57 mm. For demonstration purposes and to avoid mechanical movements, only the central part of the camera, roughly 1/25 th of the total area i.e., 0.95 × 0.71 mm, is used for recording the speckle intensity pattern. By doing so, the dynamic range is reduced to 1/5 th of the initial value of the full image sensor. In the above optical configuration, for a C-OptLev, the dynamic range is decreased from β x-max = ± 0.97° and β y-max = ± 0.73° to β x-max = ± 0.19° and β y-max = ± 0.15°. On the other hand, for an SC-OptLev, the dynamic range is decreased from β x-max = ± 1.95° and β y-max = ± 1.46° to β x-max = ± 0.38° and β y-max = ± 0.3°. In this experiment, SM-SC-OptLev is implemented to sense variation in angular orientation of about five times higher than the limit imposed by the physical size of the image sensor. In the first step, a synthetic speckle intensity pattern is recorded when the mirror is at an angle of 45° with respect to the optical axis by shifting the sensing area of 0.95 × 0.71 mm to different lateral locations on the camera plane, such that the entire 4.76 × 3.57 mm area of the speckle intensity pattern is recorded. One of the features of using SC-OptLev is that it is not possible to know when the angular orientation reaches the limit since on the sensor, one can see only a motion of a speckle intensity pattern. On the other hand, in C-OptLev, the angular orientation of the mirror reaches its limit when the spot disappears from the sensor. To overcome this problem associated with SC-OptLev, an open source based image acquisition procedure is proposed to convert the motion of the speckle pattern into the motion of a correlation peak in real-time 37 . Real-time monitoring of the correlation peak is also important in SM-SC-OptLev, in order to shift the location of the camera accurately while recording the synthetic speckle intensity pattern. A peak detection mechanism and real-time display of the angular orientation have been reported 37 .
The mirror was tilted manually to different orientations and the speckle intensity pattern with the size of 0.95 × 0.71 mm was recorded and zero-padded in the computer to the size of 4.76 × 3.57 mm. As verified from previous studies 37 , a non-linear correlation is a more effective method compared to correlation with existing matched and phase-only filters. First, a non-linear correlation was executed between the synthetic speckle intensity pattern and the pattern recorded when the angular orientation of the mirror was varied along the , C(m,n) is the correlation distribution, and (m,n) are the indexes of the correlation matrix 46 . The variation in the angular orientation of the mirror was determined from the distances of the correlation peak occurred between the autocorrelation peak (x 0 = 512, y 0 = 384) of C 1 and the cross-correlation peak (x β = 645, y β = 244) of C 2 .
The speckle intensity patterns recorded for different angular orientations of the mirror and the cross-correlation results using non-linear correlation are shown in Fig. 5. It is seen from cases 5-7 that with the SM-SC-OptLev, it is possible to detect an angular variation beyond the limit imposed by the physical dimension of the image sensor. The pixel location of the autocorrelation peak of the synthetic speckle intensity pattern is (512,384). The pixel locations of the cross-correlation peaks for the different angular variations of Fig. 5 are given in Table 1. In the cases 1-4, the angular orientation of the mirror is within the limit of the image sensor while in the cases 5-7, the angular orientation of the mirror is greater than at least twice the size of the image sensor. In cases 2 and 3, the angular orientation of the mirror was varied only along the x and y directions respectively, while in case 4, the angular orientation of the mirror was varied along both x and y directions. In cases 5 and 6, the angular orientation of the mirror was again varied only along x and only y directions, respectively, but this time beyond the physical limits of the image sensor. Finally, in case 7, the angular orientation of the mirror was varied along both x and y directions beyond the physical limits of the image sensor. The correlation results of cases 5-7 for SM-SC-OptLev are indicative of the possibilities of sensing the angular orientation beyond the limits of C-OptLev as well as SC-OptLev.
From the experimental results of Fig. 5, it is seen that when the variation in the angular orientation is larger, there is an increase in the width of the correlation peak and decrease in the SNR. The experiment was repeated for different angular orientations of the mirror and the corresponding correlation peaks are plotted in Fig. 6. The SNR is defined as 1/(Average background noise) and is plotted for the different angular orientation of the mirror. The SNR shown in Fig. 7 is normalized using the highest SNR obtained for the zero variation of the angular orientation (autocorrelation). From Figs 6  www.nature.com/scientificreports www.nature.com/scientificreports/ width of the correlation peak increases and the SNR decreases. However, the expected reduction in the sensitivity due to wider correlation peaks is by far lower than the sensitivity reduction obtained from the inverse relation of S ∝ R −1 mentioned in the introduction.  www.nature.com/scientificreports www.nature.com/scientificreports/ conclusion There are two important parameters namely dynamic range and sensitivity in the monitoring the angular orientation of the sample under observation. We have proposed and demonstrated a spatial multiplexing technique for improving one of the performances of SC-OptLev -the dynamic range. In the spatial multiplexing technique, a synthetic speckle intensity pattern much larger than the size of the image sensor is generated and used as a reference pattern. It must be emphasized that this synthetic speckle intensity pattern is recorded only once and can be used to sense the variations of the angular orientation of the sample any number of times over any period of time. In this study, an optical configuration is constructed to exhibit a dynamic range enhancement of 4.6 times that of   a C-OptLev. In theory, there is no limit on the number of times the dynamic range can be increased. However, as much as the dynamic range is increased, it is expected that the sensitivity and the SNR will be reduced. With an increase in the tilt angle, at some point, the correlation peak's intensity becomes lower than the background noise. This transition point was taken as the dynamic range of the filters. Different filtering techniques have been studied in the past and it was concluded that non-linear correlation is the best candidate for reducing the noise 37 . In the SM-SC-OptLev, a non-linear filter is used to achieve maximum suppression of the background noise. The advantage of SM-SC-OptLev is that the different areas of a larger speckle pattern are recorded only once followed by a computational stitching procedure. For real-time monitoring of the variations in the angular orientation, only one camera shot is required. We believe that the above advantage arises from the fact that a synthetic intensity pattern is used. The only disadvantage of SM-SC-OptLev is the one-time longer recording and processing time when the synthetic speckle intensity pattern is generated. Assistive optical and computational technologies using low-cost web camera and open source software have been developed for implementing an SC-OptLev for real-time monitoring of the variations in the angular orientation of the mirror.
In conclusion, the principle of spatial multiplexing has been implemented to improve the dynamic range without significantly reducing the sensitivity of an SC-OptLev. In other words, the existing trade-off between the dynamic range and sensitivity is relaxed such that the dynamic range has much larger ceiling and much smaller floor with a minor effect on the sensitivity. It must be stressed that the above technologies are not limited to sense only angles but can be used as powerful tools to sense deformations of various surfaces.