Hyperspectral imaging for dynamic thin film interferometry

Dynamic thin film interferometry is a technique used to non-invasively characterize the thickness of thin liquid films that are evolving in both space and time. Recovering the underlying thickness from the captured interferograms, unconditionally and automatically is still an open problem. Here we report a compact setup employing a snapshot hyperspectral camera and the related algorithms for the automated determination of thickness profiles of dynamic thin liquid films. The proposed technique is shown to recover film thickness profiles to within 100 nm of accuracy as compared to those profiles reconstructed through the manual color matching process. Subsequently, we discuss the characteristics and advantages of hyperspectral interferometry including the increased robustness against imaging noise as well as the ability to perform thickness reconstruction without considering the absolute light intensity information.

Hyperspectral imaging is a spectral imaging technique that combines spectroscopy and digital photography, thus yielding a spectrum at each pixel in the image of a scene. Due to the enhanced spectral resolution, hyperspectral imaging has been routinely used for many remote sensing applications such as monitoring agriculture and vegetation, and for detecting mineral and oil deposits [1][2][3] . Recently, hyperspectral imaging techniques have proliferated medical imaging 4 , including for cancer tissue detection 5,6 and ophthalmology 7,8 . Furthermore, hyperspectral imaging is increasingly being used for industrial machine vision applications such as for food and process quality control 9,10 . As hyperspectral imagers are rapidly becoming more robust, compact and economical, this technology is poised to significantly revolutionize machine vision in many different settings.
One such setting is dynamic thin film interferometry 11 . Thin film interferometry is a technique that employs light interference to measure thickness of films that are usually a few microns thick. Thin film interferometry has been used since the early 1950's to characterize static films 12,13 , and in the subsequent years the technique was adopted for studying dynamic thin liquid films that have a spatially and temporally evolving thickness profile 14 . This technique has since then become the conventional method for characterizing thin liquid films in research settings, primarily due to its simplicity and non-invasive nature. Some of the important areas that make use of this technique includes characterizing foam 15,16 and emulsion 17,18 stability, drop impacts 19 , tear films [20][21][22] and liquid film coatings 23 . Despite the popularity of the technique in research settings, dynamic thin liquid film interferometry has not proliferated industrial or commercial settings.
The primary obstacle that prevents the wide spread commercial use of interferometry for characterizing dynamic thin films is the difficulty in automatically analyzing the interferograms to recover the underlying film thicknesses. The automatic analysis is complicated by the transcendental phase-periodic governing equations that non-uniquely relates the pixel intensities in interferograms to film thicknesses. This inherent complexity is amplified by the uncertainty introduced by the unavoidable imaging and background noise. As a result, the unconditional determination of the thickness of spatiotemporally evolving thin films from interferograms has remained a challenging research problem since the inception of this technique.
Researchers have traditionally side stepped this problem broadly by the following approaches. One of the approaches involves manually identifying the film thickness at a region utilizing a reference colormap 11,24 . This approach is quite robust, however it is quite slow and suffers from human subjectivity. Another common approach utilizes fringe counting from a known absolute reference thickness in the film [25][26][27] . This approach is quite fast but is not robust and requires assumptions on the spatial structure of the film. Yet another approach involves pre-calibrating the pixel intensities in interferograms to film thicknesses utilizing liquid films of known thickness profiles 28,29 . This approach is quite fast and robust, however, this technique is restricted to scenarios where such calibrations are possible. www.nature.com/scientificreports/ In this manuscript we report an arrangement utilizing hyperspectral imaging for the unconditional and automated reconstruction of thickness profiles from thin liquid film interferograms. Approaches utilizing hyperspectral imaging (using the pushbroom like techniques) have previously been reported for characterizing static thin films 30,31 . However, these approaches are not suited for dynamic films, primarily due to the nature of image acquisition, and to the best of our knowledge have never been modified for use with dynamic films. Here, we describe a compact setup employing a snapshot hyperspectral imager and the related algorithms for automated determination of thickness profiles of thin films. In section "Methods", we detail the theory, the experimental setup and the developed algorithms. In section "Results and discussion", we establish the accuracy of this technique by comparing the absolute pixel-wise differences across the manually reconstructed and the automatically reconstructed thickness profiles. Finally, we conclude the paper by discussing the key advantages and unique characteristics of hyperspectral thin film interferometry.

Methods
theory. The theory of thin film interference was formalized in the early 19th century by Fresnel, and has since been discussed by many researchers in the context of measuring the thickness of thin films such as for bubbles 14 , tear films 32 , and for surface profiling 13 . Here we will briefly develop a formulation relevant for a hyperspectral camera.
Consider a beam of light having intensity I 0 ( ) incident on a thin liquid film of thickness d and refractive index n 2 . The film is bounded on top and bottom by media having refractive indices n 1 and n 3 respectively. Assuming normal incidence and non dispersive films, the reflected light intensity I(d, ) emanating from the thin film can be written as, Here is the wavelength of light, φ is the phase difference and 1 is the indicator function that captures the phase phase shift of π radians that occurs when light passes in to a medium with a higher refractive index. R 1 and R 2 are the power (intensity) reflectivity coefficients obtained from the Fresnel equations evaluated for normal incidence, and are given by, Finally, the intensity perceived by the ith channel of a pixel H in a hyperspectral camera as a function of the film thickness can be computed as, Here I r ( ) is the spectral response of filters in the system, 0 and f are the smallest and largest wavelengths within the global band pass filter transmission window, and S i ( ) is the spectral sensitivity of the i th channel of a pixel. See Supplementary Materials for details on the assumption of normal incidence and an uncertainty analysis related to the refractive index.
During an experiment (Fig. 1), a hyperspectral camera having h channels at every pixel will encode reflections from a thin film of thickness d as a h dimensional vector. Utilizing Eq.5, we can invert this h dimensional vector to recover the thickness of the thin film. In principle, when using a RGB camera this can be accomplished by first generating a color map that establishes a color to film thickness relationship (Fig. 1c), and subsequently using this map to assign thicknesses to colors in the interferogram. Unfortunately, due to the periodic nature of the cosine function, the generated color map for a RGB camera has nearly identical colors mapping to different thicknesses. Coupled with the imaging noise, the automated mapping of colors to interference patterns becomes infeasible (see Supplementary Materials). Hence in practice, a manual matching process is usually adopted, the details of which are available in Frostad et al. 11 . In the subsequent sections, we will detail the use of hyperspectral imaging for automatically reconstructing film thickness from interferograms utilizing a spectral map obtained from Eq. 5. experimental setup. The single bubble coalescence experiments used to validate the utility of hyperspectral imaging for thin film thickness measurements were performed using a modified Dynamic Fluid-Film Interferometer (DFI). The construction 11 and the utility 33-36 of the DFI has been previously discussed in a number of publications and in references therein.
For the current study, the DFI was modified to have a 16 channel snapshot HyperSpectral Imaging (HSI) camera (Model: MQ022HG-IM-SM4X4-VIS, Manufacturer: Ximea GmbH, Germany) having a maximum acquisition frame rate of 170 frames per second as its top camera (Fig. 1a). As the filter array inside the HSI camera has narrow spectral response bands ( Supplementary Fig. S4), the dichroic triband filter utilized with the top light (3) Scientific RepoRtS | (2020) 10:11378 | https://doi.org/10.1038/s41598-020-68433-0 www.nature.com/scientificreports/ for the same purpose (reducing the FWHM of spectral bands of a RGB camera) 11 was removed. The removal of the dichroic filter thus resulted in 120% increase in the luminous flux entering camera -improving the signal to noise ratio in the acquired hyperspectral interferograms. Further details of the setup including the make and model of the light are available in Frostad et al. 11 .
To benchmark the thin film measurement capability of the hyperspectral camera, single bubble experiments were also performed using RGB cameras (IDS UI 3060CP), commonly used for thin film interferometery 11,34 . image processing. To recover the film thickness from the hyperspectral image, the following steps were executed utilizing Matlab. Initially, the raw images from the snapshot HSI camera were sliced and spliced appropriately to reconstruct the hyperspectral cubes. Subsequently, background subtraction was performed on the cubes, followed by cropping, intensity correction and normalization. Background subtraction is accomplished by imaging the measurement region in the absence of the thin film and subtracting this background spectrum from the measured spectrum. Intensity correction when necessary (for flat field corrections) is accomplished by imaging a surface of known reflectivity, and using the result to compute the correction mask for each spectral band. Intensity normalization is accomplished by normalizing the hyperspectral interferogram by the largest intensity in the interferogram. Subsequently, a k-Nearest neighbour search utilizing the cosine distance metric is performed between each pixel in the resulting HSI cube and the theoretical spectral map (Fig. 2a) generated from Eq.5. The thicknesses obtained as the first nearest neighbour in the k-Nearest neighbour search is used to construct the initial estimate of the thickness profile.
Finally, a spatial optimization algorithm is utilized to correct for the any incorrectly assigned points. The algorithm basically enforces the C 0 spatial continuity in film thickness by replacing any incorrectly assigned thickness with an appropriate thickness from the k possible thickness values at that point. In practice, this is accomplished by identifying regions with unphysical gradients and utilizing the gradient descent technique to iteratively correct the thickness in these regions. All steps involved in the thickness reconstruction pipeline is summarized as a block diagram in Fig. 2b.

Results and discussion
Thickness reconstruction performance. The performance of the new camera system is established by reconstructing the dynamic film thickness of a bubble in a silicone oil mixture. The results before and after optimization are compared to the manually reconstructed thickness profiles 11 in Fig. 3. The reconstructed thickness profile before optimization (Fig. 3c) broadly resembles the manually reconstructed profile (Fig. 3b). However, due to noise and spectral mixing in the interference data, there are regions in the reconstructed profile that have physically unrealistic gradients. These artifacts are removed utilizing the optimization routines, and the resultant thickness profile after optimization (Fig. 3d) is seen be very similar to the manually reconstructed profile. www.nature.com/scientificreports/ The mean film thickness ( T(x, y)dxdy/ dxdy ), a very common metric used to report the thickness of thin films 11,34,37 , is very similar across the three cases with values of 358.9 nm, 344.4 nm and 347.9 nm for the manual, unoptimized and optimized thickness profiles respectively.
The absolute pixel-wise height difference between the manually reconstructed and the automatically reconstructed thickness profiles pre and post optimization are shown in Fig. 3e,f. Clearly, the reconstructions are very accurate with over 80% of the pixels differing by less than 100 nm for the unoptimized case, while over 90% of the pixels differ by less than 100 nm for the optimized case (see Supplementary Fig. S8 for more statistics). The remaining 10% of the regions have errors larger than 100 nm, and are most likely a result of inadequate background subtraction as well as due to the lack of pixel by pixel calibration information for the camera. As a reference, it is also worth noting that reconstructions using the 3 channel data available from a RGB camera has an unacceptable accuracy, with less than 20% of the pixels being classified within 100 nm ( Supplementary Fig. S3).
Finally, a convenient advantage of the automated reconstruction is the ease of analyzing time sequential data to obtain the temporal evolution of the film thicknesses. Supplementary Video V1 shows one such reconstruction of a temporally evolving film thickness profile of a bubble in a silicone oil mixture alongside its corresponding raw interferogram.

Robustness against noise.
Theoretically, the three channels in RGB interferograms are sufficient to disambiguate the underlying film thicknesses 28 . However in practice, the inherent noise in the acquired image data breaks this theoretical uniqueness between the RGB intensities and the film thickness; thus requiring optimization (even for static films) 38 and/or calibration 28 for thickness recovery. Unlike the RGB interferograms, hyperspectral interferograms are relatively more robust to noise.
To illustrate the robustness of hyperspectral interferograms against noise, we perform numerical experiments using a ramp thickness profile having a minimum thickness of 1 nm and a maximum thickness of 2,000 nm ( Supplementary Fig. S1). This thickness profile is mapped to corresponding color interferograms using the color maps of the tested cameras. Subsequently, Gaussian noise (see Supplementary Text) is added to the images. Inset in Fig. 4a shows one such RGB image before and after addition of the noise (see Supplementary Fig. S6 for a block diagram detailing the pipeline for generating the test images). Reconstruction algorithms (without optimization) are then used to predict the thickness (T) at each pixel in the noisy images. T can be compared to the ground truth thickness ( T g ) to quantify the reconstruction accuracy. We visualize the reconstruction accuracy in Fig. 4(a) by plotting the number of pixels having an error of less than 50 nm i 1(|T g i − T i | < 50) as a function of the number of channels in the camera.
From Fig. 4a we see that for an RGB camera only about 20% of the pixels are classified within acceptable error. However, for a HSI camera (utilizing all 16 channels) we are able to recover the thickness from 97% of the pixels within acceptable error; thus showing that HSI cameras are more robust to noise. For the tested HSI camera, we also obtain similar reconstruction accuracy using either the real or virtual filters (see Supplementary Figs. S4, S5 for the definitions and the spectral responses of the filters) on a HSI camera, thus suggesting any type of filter may be used in the reconstruction process. Further we also see that having an ideal light (uniform intensity across wavelengths) or ideal filters (responses given by hat functions) can enhance the reconstruction performance; with Finally, its also interesting to note that the results indicate that a three channel camera (RGB camera) with ideal filters has a performance comparable to a regular HSI camera utilizing all 16 channels. The increased robustness of HSI interferograms against noise is related to the higher dimensionality of the color co-ordinates that correspond to a given film thickness. As a result of the higher dimensionality, the pair wise Euclidean distances of HSI color co-ordinates are larger than the corresponding pair wise distances of color coordinates obtained from a RGB camera. This observation is quantified in Fig. 4b by plotting the cumulative pair wise distance distribution of the color co-ordinates (resolved at every nanometer) for a HSI and a RGB camera for two different film thickness ranges-[0, 2,000] and [0, 4,000] nm. From Fig. 4b, we clearly see that for a given thickness range, the curves corresponding to HSI cameras as compared to RGB cameras are shifted towards larger distances (r). Hence, as a consequence of the increased separation of HSI color co-ordinates, perturbations due to noise are less likely to result in a color co-ordinate becoming similar to another.
Absolute light intensity independent thickness reconstruction. Another advantage of using hyperspectral imaging is that the absolute intensity of light at any point in the interferogram may be neglected during thickness reconstruction. This can be shown to be true using two different arguments. Firstly, standard transformations (such as RGB to HSV) can isolate the intensity information into a single channel. Neglecting the intensity channel, still gives us (for the tested HSI camera) information from 15 channels to unambiguously (see Fig. 4) reconstruct the thickness. Since an RGB camera has only three channels, neglecting the intensity makes it impossible to unambiguously determine the thickness. Secondly, the cosine distance metric utilized in this paper to determine the film thicknesses completely ignores the intensity information. Despite ignoring the intensity information, we are able to recover the thickness profiles quite well (Fig. 3d), thus confirming that A direct consequence of this result is that the reconstruction techniques described in this paper are robust against the natural spatial variation of incident light intensity (vignetting) over the interferogram. Hence in addition to obviating the need for accurately obtaining the absolute light intensities, the corrections for vignetting may also be conveniently avoided. Note that flat field corrections may still be required if there are spatial variations in pixel sensitivities in a camera.
Characteristics of hyperspectral thin film interferometry. Hyperspectral imaging when applied to thin film interferometry has some interesting characteristics. Firstly, the number of spectral classes is higher than in traditional hyperspectral imaging used for remote sensing or medical imaging. The number of spectral classes in thin film interferometry goes as O(T max /T resolution ) , which is equal to 4,000 when trying to resolve film thickness of upto T max = 4,000 nm with a resolution of T resolution = 1 nm . Consequently, spectral matching routines used for thin film interferometry should have a high degree of specificity, and may also need optimization algorithms (as used in this paper) to completely reconstruct the thickness profile.
Secondly, the spectra vary gradually across spatially adjacent spectral classes. Unlike traditional hyperspectral imaging where there are no restrictions on spatially adjacent classes (and hence on the spectra), the spatial continuity of thin films restricts the spatially adjacent classes to correspond to contiguous film thicknesses. Hence, as a consequence of Fig. 2, there is a gradual variation of the spectrum across spatially adjacent classes. Consequently, spectral mixing (due to lack of sufficient camera resolution) does not pose difficulties during thickness reconstruction, thereby obviating any need for spectral unmixing techniques 39 .
Thirdly, as thin liquid films are dynamic, the spectral signatures change rapidly both in space and time. As a consequence, snapshot hyperspectral imaging (as opposed to techniques such as the pushbroom) is better suited for thin film interferometry. comparison to spectroscopic ellipsometry. We close the discussion by comparing the reported hyperspectral thin film interferometry technique, which in many ways is an improved form of spectroscopic reflectometry, to the commonly used spectroscopic ellipsometry. Ellipsometers utilize the change in the polarization state of light as it is reflected obliquely from a thin film to measure the film thickness. Ellipsometers are known for their sub-Angstrom precision, particularly when dealing with nanometric and sub-nanometric films 40 . Spectroscopic ellipsometers are a class of ellipsometers that employ a range of wavelengths to perform the measurement, resulting in an improved measurement range spanning a few Angstroms to tens of microns. In addition with the help of adequate models, they are also used to determine many optical and geometrical characteristics of thin films 41 .
The reported hyperspectral thin film interferometry technique currently has a measurement accuracy of about a 100 nm and can measure film that are between 100 nm and 4,000 nm. Despite having an inferior range  Figure 4. (a) Robustness against noise as a function of number and type filters in a camera. The reconstruction accuracy is seen to increase with the number of channels in the camera. Further, a hypothetical camera with perfect (hat function) filters is seen to require the least number of channels for a high fidelity thickness reconstruction. (b) The cumulative pair wise distance distributions for the theoretical colormap (RGB camera) and spectralmap (HSI camera) entries. Here N is the number of contiguous thickness values in the theoretical maps starting at 0 nm and separated by 1 nm. The spectral entries corresponding to a hyperspectral camera are separated by a larger distance (r) as compared to that of a RGB camera. www.nature.com/scientificreports/ and accuracy, hyperspectral interferometry complements spectroscopic ellipsometry by addressing the following two important limitations. Firstly, ellipsometers have a poor spatial resolution, often restricted to measuring the thickness at a single point. This makes ellipsometers unsuitable for characterizing dynamic liquid films that are evolving in both space and time. Secondly, measurements using ellipsometers require an oblique angle of incidence. This makes the imaging of thin films on curved geometries very challenging. The capability of the current hyperspectral technique to characterize dynamic liquid films over curved geometries makes this technique attractive to many technical fields including that of colloid and interface science where thin liquid films involving bubbles 34,35,37 , drops 17,18 and the tear film [20][21][22] are routinely studied.

conclusions
In this paper we reported a compact setup employing snapshot hyperspectral imaging and the related algorithms for the automated determination of thickness profiles of dynamic thin liquid films. We showed that we can reconstruct dynamic thin film profiles to within 100 nm of those reconstructed manually. As manually reconstructed profiles themselves are only accurate upto 50 nm, the automatic reconstructed profiles are a faithful representation of the ground truth. We also demonstrated through numerical experiments that hyperspectral interferometry has two key advantages, namely, the increased robustness against imaging noise and the ability to neglect the absolute light intensity information during thickness reconstruction. Future studies may be focused on improving both the hardware and software aspects presented in this work. Improvements in the spatial resolution, the sensor quantum efficiency, and the filter properties like transmissivity and FWHM of the hyperspectral imager will enhance the reconstruction performance of the system. Our results also suggest that improving the filters (especially the Q-factor) will have a significant impact on the robustness of the system. Finally improvements in image pre-processing (like background substraction) and in spectral matching algorithms (such as enhancing the specificity and robustness) will also aid in perfecting the proposed system.