Measurement precision enhancement of surface plasmon resonance based angular scanning detection using deep learning

Angular scanning-based surface plasmon resonance measurement has been utilized in label-free sensing applications. However, the measurement accuracy and precision of the surface plasmon resonance measurements rely on an accurate measurement of the plasmonic angle. Several methods have been proposed and reported in the literature to measure the plasmonic angle, including polynomial curve fitting, image processing, and image averaging. For intensity detection, the precision limit of the SPR is around 10–5 RIU to 10–6 RIU. Here, we propose a deep learning-based method to locate the plasmonic angle to enhance plasmonic angle detection without needing sophisticated post-processing, optical instrumentation, and polynomial curve fitting methods. The proposed deep learning has been developed based on a simple convolutional neural network architecture and trained using simulated reflectance spectra with shot noise and speckle noise added to generalize the training dataset. The proposed network has been validated in an experimental setup measuring air and nitrogen gas refractive indices at different concentrations. The measurement precision recovered from the experimental reflectance images is 4.23 × 10–6 RIU for the proposed artificial intelligence-based method compared to 7.03 × 10–6 RIU for the cubic polynomial curve fitting and 5.59 × 10–6 RIU for 2-dimensional contour fitting using Horner's method.

www.nature.com/scientificreports/ employed to train the neural network. First, the simulated reflectance spectra with added shot-noise and speckle noise were prepared to mimic images from the experimental setup generalizing the simulated dataset for the CNN training. The proposed trained network was later evaluated in an experimental setup measuring refractive index change in real-time to quantify the measurement precision enhancement compared to conventional methods, including one-dimension and two-dimensional curve fitting methods. To the best of the authors' knowledge, the proposed CNN method to enhance the accuracy in SPR measurement has never been reported before in the literature.

Materials and methods
Surface plasmon resonance optical alignment. An SPR angular interrogation-based Kretschmann configuration was aligned to demonstrate the capability of the proposed CNN method compared with cubic polynomial curve fitting and Horner's method. The setup consisted of two main parts: a gas flow-control system for varying the refractive index of the SPR sensing region and an optical system, as depicted in Fig. 1.
The gas flow-control system equipped with pressure gauges was employed to control nitrogen gas pressure in the sensing region (gas chamber), providing an independent measurement to cross-validate and approximate the refractive indices of the gas inside the chamber. The gas system consisted of a nitrogen gas tank, a gas regulator for pressure control, an inlet valve, an 18 × 18 × 18 mm 3 plastic gas chamber, and an outlet valve. The optical detection system captured the reflectance spectrum using a CMOS camera (MT9P006, Huatengvision). The camera was monochromatic with a bit-depth of 8 bits, 2592 pixels × 1944 pixels, pixels size 2.2 μm × 2.2 μm, the camera well depth of 4796 electrons per pixel, and quantum efficiency of 61% at the incident wavelength λ of 650 nm. The plasmonic layer consisted of a 45 nm plasmonic gold with a 2 nm chromium adhesion layer on a standard BK7 microscope coverslip (No. 0, Sigma-Aldrich) prepared by electron beam sputtering. The gold plasmonic sensor was mounted on an 80 mm diameter semicylindrical glass prism (80 mm prism, Scitrader) using a microscope matching oil (MOIL-30, Olympus). For SPR excitation, the system consisted of a 7 mW laser diode 650 nm (L650P007, Thorlabs), a half waveplate (WPH10E-633, Thorlabs); an achromatic plano-convex doublet lens with a diameter of 12.7 mm and focal length of 20 mm (LA1074-A-N-BK7, www.nature.com/scientificreports/ Thorlabs) and an achromatic plano-convex doublet lens with a diameter of 38.1 mm and focal length of 150 mm (LA1388-A-N-BK7, Thorlabs) for beam expansion. The magnified beam then passed through an iris diaphragm (D255, Thorlabs), reducing the beam diameter to 20 mm. The beam is then focused using a 1-inch plano-convex cylindrical lens with a focal length of 100 mm (LJ1567L2-A-N-BK7). Finally, a plano-convex cylindrical lens expands the reflected beam with a focal length of 200 nm (LJ1309L1-A-N-BK7). The coverage incident angle of the sensor can be calculated from the optical system to 5.73 × 10 -4 degrees per pixel or 0.00001 in sinθ 0 unit, similar to the angular resolution employed in the literature 30 . The 5.73 × 10 -4 degrees per pixel or 0.00001 in sinθ 0 unit were calculated from the demagnified camera sensor size of 5.7 mm using two cylindrical lenses, providing a demagnification factor of 2 when projecting back to the f100 cylindrical lens, as shown in Fig. 1. The demagnified camera pixel can cover a specific range of incident angles, which can be calculated using numerical aperture (NA) relationship sin(tan −1 (D/2f)) corresponding to the NA of 0.014, where D is the camera size. The 2NA can then be divided by the total number of pixels of 2592 pixels along with the angular space leading to the angular resolution of 0.00001 in sinθ 0 . SPR active research groups have widely adopted the camera-based SPR dip measurement for various applications 31,32 , although measurement accuracy and precision can be improved by employing a linear diode array 33 . Here, the paper aims to demonstrate that the measurement precision can be improved by increasing the efficacy of data utilization using artificial intelligence compared to the other conventional methods with no need for additional optical instrument modification.
Experimental procedure. The experimental procedure consisted of (1) opening the inlet and outlet valves for airflow through the gas chamber for SPR detection at the air refractive index stage, (2) flowing the nitrogen gas to replace the air in the gas chamber, (3) closing the outlet valve for keeping the nitrogen gas of 5 pressure levels of 5 psi, 40 psi, 110 psi, 130 psi, and 150 psi, respectively, (4) closing the inlet-controlled valve when the pressure raised to the stabilized pressure. The plasmonic dip was then measured as a video file capturing at an 8-fps camera framerate. At the stable pressure levels, 100 video frames for each pressure level were employed as a testing dataset to evaluate the performance of three plasmonic angle measurement methods. The experiment was carried out when the vibration isolation system was switched off and deflated the optical table. Note that the gas pressure range is the maximum pressure that the plastic chamber can withstand before the gas leaks out, and the range is within the detection requirements for gas sensing reported in the literature 34,35 . Calibration procedure and method to recover sample's refractive index. The plasmonic angles recovered for the air case from each method were calibrated to the same initial value of sinθ sp of 0.6783, corresponding to the air refractive index of 1.000276 36 . The sinθ sp of 0.6783 was obtained from the theoretical value computed using Fresnel equations and the transfer matrix approach for the 45 nm gold and the 2 nm chromium layer on a BK7 glass substrate and the incident wavelength of 650 nm computed using the chromium and gold refractive indices reported by Johnson and Christy 37 . Then the change in the plasmonic angle Δsinθ sp due to the nitrogen pressures can be then determined based on the number of pixels that the plasmonic dip moves compared to the air case.
Nitrogen gas pressures. The gas refractive index can be related to its concentration, proportional to the gas pressure. The experimental setup was in a condition of a constant temperature and volume. A gas refractive index can be estimated as gas pressure based on the empirical equation and Boyle's relationship. For nitrogen and air compound in a constant volume chamber, the gas pressure and its corresponding gas refractive index were extracted from the experimental results reported in Wong et al. 38 and fitted using a linear function as expressed in Eq. (1). Note that the coefficient of determination R 2 of 0.9999 and the root mean square error (RMSE) of 1.0087 × 10 −6 RIU.
where, n s is the gas refractive index, and P is the tank pressure in the Pa unit.
Data analysis procedure. The video recorded file from the experiment described in the earlier section was analyzed using the following three methods: Cubic polynomial curve fitting. Figure 2 shows the process flow to determine the plasmonic angle from a recorded camera frame. First, the line-scan reflectance was prepared by averaging all the rows in the camera frame and applying a cubic polynomial curve fitting 39 through the minimum reflectance for the averaged SPR dip. It will be shown later in the "Results" section that the accuracy of the curve fitting method depends on the number of data points included in the polynomial curve fitting.
Horner's curve fitting method. Horner's curve fitting method is a well-known algorithm for 2-dimensional polynomial surface fitting. This method was employed to calculate a fitting surface of plasmonic dip image data, as depicted in Fig. 3 and expressed in Eq. (2). First, a line-scan reflectance was evaluated from the curve fitting contour by considering all the rows. Then, the 2D function with polynomial coefficients allowed us to accurately obtain the sinθ sp for each row using function differentiation; these sinθ sp values were averaged to determine the sinθ sp for a camera frame.
It is crucial to point out that the plasmonic dips are usually asymmetrical because of the loss and coupling mechanism 40 . Several asymmetrical SPR dip curve fitting techniques claim the performance enhancement, such www.nature.com/scientificreports/ as the sigmoid-asymmetric fitting algorithm 18 and poles-zeros-based fitting function 41 . Horner's curve fitting nature allows the cross-polynomial terms between x and y to form an asymmetric fitting, as depicted in Fig. 3. As a result, the method should provide better data usage of the asymmetrical fitting and noise averaging than the symmetrical one-dimensional curve fitting explained earlier.
where, p n are the polynomial coefficients. n is the number of orders of x, which is the sinθ 0 . m is the number of orders of y, which is the row number of the camera frame, and in this study, m is 1. In other words, linear response along the camera rows.
CNN based method. The CNN predicted the plasmonic angle position in the pixel, which later converted to the plasmonic angle sinθ sp . Table 1 shows the proposed neural network architecture for the image regression problem in this study. The network consisted of an input image size of 70 pixels × 2592 pixels and four additional levels. Each level consisted of a 9 × 9 × 32 convolutional layer, a batch normalization layer, and a rectified linear unit (ReLU) layer. The last layer was a 1 × 1 fully connected layer and a mean-absolute-error regression layer to estimate the pixel number of the minimum reflectance with four additional decimal points, which was equiva-  www.nature.com/scientificreports/ lent to 0.00000001 in sinθ 0 unit and 4.20 × 10 -8 RIU. Note that the minimum RIU corresponded to the current state-of-the-art RIU measurement precision of metasurfaces 42,43 and phase-sensitive SPR measurement 44,45 .
The training options consisted of adaptive momentum estimation (Adam) optimizer 46 , batch size of 64 images, and 0.0001 initial learning rate. The CNN training was under a single graphic processor unit (GPU) NVIDIA TITAN RTX with 16 GB RAM. Note that the 64 images were the maximum number of images under the 16 GB RAM environment. The effect of the number batch size will be discussed in the "Results" section.
The network was trained with 1000 epochs to ensure that the root-mean-square validation error (RMSE) reached the convergence of RMSE of 10 -4 pixels, which was the same level as the decimal precision.
It is essential to point out that the 70 input rows were much lower than the 1944 rows in the camera frame. It will be shown later that this is not due to the memory limitation in the GPU, but the higher number of rows can degrade the accuracy.
Training dataset preparation for CNN training. The training dataset was simulated camera frames computed using the Fresnel equations and the transfer matrix method 47 . The simulation process is as described and shown in Fig. 4. Firstly, each camera frame had 70 pixels in a row and the same columns as raw data of 2592 pixels. The control parameters for simulated reflectance consisted of a 45 nm gold thickness-based sensor with a 650 nm wavelength source mimicking the experimental setup. The simulation then varied the refractive index in the sensing region to form 64 camera frames with 8-bit precision covering the sample refractive index of 1.00-1.05 with the RIU step size of 0.00078 RIU increment.
The shot and speckle noises were added to the simulated images to generalize and mimic the experimental responses; experimental data obtained in an SPR experiment is usually shot noise dominant due to low optical intensity at the plasmonic dip 30 . The shot noise was randomly added with the noise amplitude proportional to R p , where R p is the reflectance simulated using Fresnel equations and the transfer matrix approach, as depicted in Fig. 5a and followed by adding a random speckle-noise pattern, as shown in Fig. 5b. The random speckle-noise patterns were generated using the random Gaussian process method proposed by Byun et al. 48 . The process can be summarized as (1) generating a digital randomized array of 0 and 1 for 70 pixels × 2592 pixels. (2) The digital randomized array was then convoluted to a Gaussian distribution function with a mean of 0 pixels and a standard deviation of 100 pixels to generate a speckle pattern, which was (3) then normalized the speckle pattern to 0 to 1 range and added to the shot-noise added images. These simulated camera frames were then applied in the CNN training.
Testing dataset for the CNN based method. The testing dataset was the experimental data obtained from the experimental procedure explained in the "Experimental procedure" section. The raw data from the camera was recorded in a video format that captured a range of refractive index changes from air to different nitrogen pressures of 5 psi, 40 psi, 110 psi, 130 psi, and 150 psi. Each camera frame was then separated into subframes with an identical size of 70 pixels × 2592 pixels with no overlapping rows, as depicted in Fig. 6. Thus, there were 27 subframes from each video frame, and the bottom 54 rows were excluded from the CNN analysis. The 27 sub-     www.nature.com/scientificreports/ Measurement standard deviation ( σ ) represents the lowest quantity that the sensor can measure, which is of sin sp values recovered from 100 recovered sin sp values for each gas pressure and the three measurement methods.

Results and discussion
The experiment was calibrated at the initial air stage at 23 °C standard room temperature and pressure; Fig. 7a shows the corresponding SPR angular response. The N 2 pressure was then increased to the five pressure levels of 5 psi, 40 psi, 110 psi, 130 psi, and 150 psi; the SPR angular responses for each pressure level were shown in Fig. 7b-f, respectively. Note that the equivalent reflectance spectra shown on the right side of Fig. 7a-f were simulated the Fresnel equation and the transfer matrix approach explained earlier with the sample refractive indices of 1.000276 for the refractive index of air 31 and 1.000373 1.001035, 1.002360, 1.002738 and 1.003117 for the other five N 2 pressure levels calculated using Eq. (1).
The first 100 frames for the air case were analyzed using the three methods working out the corresponding plasmonic angles from the three methods and calibrating them to the sinθ sp of 0.6783 for the theoretical air refractive index 1.000276 36 . Cubic polynomial method. For the curve fitting method, the number of data points around the minimum reflectance dip plays a crucial role in the accuracy of the plasmonic dip. The number of data points included in the cubic polynomial fitting varied from 41 to 401 pixels to demonstrate the point. The minimum value of this fitted curve is an approximated SPR dip position in pixels unit.
The first camera frame of the experimental data for the air case shown in Fig. 7a was averaged to a 1D line scan, as shown in Fig. 8a. Then the different amount of data points around the minimum dip was included in cubic spline polynomial curve fitting, as shown in Fig. 8b. Next, the corresponding sinθ sp for each data point widths were evaluated and stored. Finally, the same analysis was applied to the other 99 air case images and the σ of the 100 recovered sinθ sp values for each data point width, as shown in Fig. 8c. For example, the number of data points of 153 had the lowest σ of 5.52 × 10 -6 in sinθ sp , corresponding to 3.16 × 10 -4 degrees and 7.03 × 10 -6 RIU. The number of data points included in the curve fitting also affect the sensitivity; here, we applied the same data processing to the data obtained for the pressure level of 150 psi and determined the averaged sinθ sp for the same data point widths. The Δsinθ sp values between the two cases for the analyzed data points are shown in Fig. 8d. The higher number of data points degraded the sensitivity performance for the polynomial curve fitting method. The average sinθ sp and σ values for all the pressure levels recovered using the polynomial curve fitting are summarized in Table 2 compared to the other methods.
Horner's method. The same experimental test dataset was analyzed using Horner's method for surface fitting. First, the images were cropped to 1944 pixels × 1944 pixels, where the 648 rightmost columns were discarded from the computation to center the plasmonic dip since they did not contain reflectance dip information, www.nature.com/scientificreports/ as shown in Fig. 7a. There were two polynomial degrees in Eq. (2): the parameters n for the x-axis (sinθ 0 ) and m for the y-axis (camera's row). The m parameter was fixed at 1, mimicking the consistent plasmonic dip linescans in all the rows. The n parameter was then varied from the 3rd to the 9th polynomial order to determine the effects of the polynomial on the recovered measurement values by evaluating σ for the 100 frames of the air case and Δsinθ sp between the air, and 150 psi cases, as shown in Fig. 9a,b. The lowest σ of 4.39 × 10 -6 in sinθ sp , corresponding to 2.34 × 10 -4 degrees and 5.59 × 10 -6 RIU, was at the n of 6 orders, which was lower than the cubic polynomial fitting. Thus, Horner's method can utilize the image data more effectively using the surface fitting, considering the relationship between rows and columns. Horner's method based on sixth-order polynomial was then employed to analyze the other pressure levels and summarized in Table 2, comparing to the other measurement methods.

CNN based method.
Multiple networks with varying input row numbers were trained for 50 rows to 100 rows optimizing for the lowest σ value. All the networks and training functions were the same except the number of input rows. The network with 100 rows was the last configuration to be trained under the GPU before exceeding the GPU memory capacity. Figure 10a shows the σ values for the 100 experimental frames from the  www.nature.com/scientificreports/ air backing experiment. The network output was in pixels, later converted to sinθ sp by the calibration process described in the "Materials and methods" section. Figure 10b shows the Δsinθ sp between the air and 150 psi cases analyzed using the networks. The number of input rows did affect the sensitivity much when it was less than 100 rows; moreover, the higher row numbers did not produce a higher measurement accuracy. It is possibly due to the relative size of noise artifacts in the experimental results. The lowest σ of 3.32 × 10 -6 in sinθ sp , corresponding to 1.90 × 10 -4 degrees and 4.23 × 10 -6 RIU, was at the input row number of 70. Like the first two SPR dip recovery methods, all the other gas concentrations were then analyzed using the CNN-based network with 70 pixels × 2592 pixels input image size. The results in Fig. 10 were analyzed using the batch size of 64 images. Figure 11 shows the effect of networks trained using different batch sizes ranging from 20 to 64 images. The trained networks were then applied to analyze the 100 experimental frames for the air case to evaluate the standard deviation and the mean value. Note that the 64 images were the maximum limit for the computer memory consisting of 64 images of 8-bit grey-scale images with the size of 70 × 2592 pixels. Therefore, a larger batch size can provide higher accuracy for the same number of epochs of 1000. In contrast, the smaller batch size needs a higher epoch. In other words, it requires a longer training time to achieve the same accuracy. Therefore the batch size of 64 images was applied in the analysis for the following section.  www.nature.com/scientificreports/ Performance comparison of the three methods. Table 2 summarizes the recovered mean values and standard derivations of all the three mentioned plasmonic angle recovery methods in sinθ sp . The σ values of the three methods indicated that the cubic spline had the worse performance with an averaged σ of 5.59 × 10 -6 ; followed by Horner's curve fitting method with an averaged σ of 4.51 × 10 -6 , and the best-performed method was the CNN with its average σ of 3.71 × 10 -6 . Therefore, the CNN can improve the σ value by 66% and 32% compared to the cubic polynomial curve fitting and Horner's method. The stability in the σ measurement tells us about the reproducibility of experiments. The three methods had a similar reproducibility performance regardless of the measurement methods and the gas concentrations. Figure 12 shows the sensorgram for the air case and the 5 N 2 pressures and summarized in Table 2 for 100 frames of stabilized responses for each case. Student t tests were then applied to statistically analyze and compare the recovered sinθ sp values for the three plasmonic angle measurement methods. For the cubic polynomial curve fitting and Horner's method, the t and the p values were 0.1872 and 0.8517, respectively. The cubic polynomial curve fitting and the CNN method had the t value of 0.3990 and the p-value of 0.6903. In addition, Horner's method and the CNN method had the t value of 0.2271 and the p-value of 0.8206. The three measurement methods were not significantly different, which indicated that the sinθ sp values recovered from the three methods agreed with each other. The statistical analysis shows that CNN has the narrowest distribution, followed by Horner's method and the cubic polynomial curve fitting.
The sinθ sp in Table 2 can then be converted to the corresponding refractive index change compared to the theoretical values calculated from Fresnel equations and the transfer matrix approach, as reported in Table 3. For precision and sensitivity demanding applications, including single-molecule detection, DNA and virus detection, scientists and engineers would need to increase the measurement precision of their sensor as much as possible. In addition, it is challenging to enhance the measurement performance, especially when the system operates near its measurement precision limit 49 . Here we demonstrate one of the ways to use the measurement data and the artificial intelligence to learn and relate the measurement noise to cancel out the noise and improve the measurement accuracy with no need for additional instrumentation and sophisticated equipment for any existing SPR setup. Only the software upgrade is needed, and also the software upgrade and the CNN training can be done as a background operating when the SPR system is in use. Table 3 shows the recovered refractive index inside the sensing chamber, and the σ values in the RIU unit, and the expected refractive index calculated using the pressure reading from the second pressure gauge and Eq. (1) to validate the recovered refractive indices. Again, the pressure gauge reading and the SPR measurements agree well.
The current state-of-the-art measurement precision for angular scanning SPR ranges from 10 -5 RIU 50,51 to 10 -6 RIU 52 for angular interrogation measurements. Therefore, the σ value of the CNN agrees with the reported measurement precision values in the literature. In addition, several optical techniques 53 have been employed to enhance the limit involving a more demanding optical instrumentation, for example, an optical interferometer 54 , a spatial light modulator-based illumination 55 , metasurfaces 56,57 , and nanowires 58 .
Here, we have demonstrated that the proposed CNN method can enhance and utilize the data by finding the relationship between pixels to cancel the noise out. In other words, deep learning provides more efficient data utilization than the other two conventional methods without generating any new information. This study's   www.nature.com/scientificreports/

Conclusion
The proposed deep learning for SPR measurement precision enhancement of surface plasmon resonance-based angular scanning detection has been developed. The convolutional neural network was utilized to approximate plasmonic angle position in pixel and later converted to corresponding plasmonic angle. The designed CNN network architecture was designed for this purpose. The simulated dataset was simulated reflectance spectra of the p-polarized incident wave at 650 nm wavelength, computed using the Fresnel equation and the transfer matrix for network training. The paper has also demonstrated that simulated data can be employed for deep learning neural network training; the trained network was later tested with experimental data. The experimental setup was developed, integrating an optical system and a gas flow-control system for experimental data collection measuring six refractive index levels controlled by gas pressure. The experimental data were analyzed using two conventional curve fitting methods compared to the proposed CNN network. The three measurement methods show similar sensitivity and reproducibility responses. Furthermore, the refractive indices recovered using the SPR measurements agreed well with the refractive indices converted from recorded gas pressures. The measurement standard deviations were 4.23 × 10 -6 RIU for the proposed CNN compared to 7.03 × 10 -6 RIU for the cubic polynomial curve fitting and 5.59 × 10 -6 RIU for Horner's method corresponding to 66% and 32% enhancement. The CNN operates by identifying the relationship between every pixel in the input frame of the CNN, leading to more efficient usage of existing data than the cubic polynomial curve fitting and Horner's method without creating an artificial profile or higher frequency components.