Laser excited super resolution thermal imaging for nondestructive inspection of internal defects

A photothermal super resolution technique is proposed for an improved inspection of internal defects. To evaluate the potential of the laser-based thermographic technique, an additively manufactured stainless steel specimen with closely spaced internal cavities is used. Four different experimental configurations in transmission, reflection, stepwise and continuous scanning are investigated. The applied image post-processing method is based on compressed sensing and makes use of the block sparsity from multiple measurement events. This concerted approach of experimental measurement strategy and numerical optimization enables the resolution of internal defects and outperforms conventional thermographic inspection techniques.


A photothermal super resolution technique is proposed for an improved inspection of internal defects.
To evaluate the potential of the laser-based thermographic technique, an additively manufactured stainless steel specimen with closely spaced internal cavities is used. Four different experimental configurations in transmission, reflection, stepwise and continuous scanning are investigated. The applied image post-processing method is based on compressed sensing and makes use of the block sparsity from multiple measurement events. This concerted approach of experimental measurement strategy and numerical optimization enables the resolution of internal defects and outperforms conventional thermographic inspection techniques.
The nondestructive testing (NDT) of internal defects such as blowholes, inclusions or delaminations is of huge interest in industry. There are several ways to detect internal defects without destroying the specimen such as ultrasonic testing (UT) or radiographic testing (RT). UT is typically not contact-free and suffers from reconstruction accuracy if defects are not oriented perpendicular to the coupled ultrasound. In contrast, RT methods like computed tomography provide reliable and accurate results 1 and are contact-free, but end up to be costly, slow, complex and only suitable ex-situ. Unlike UT and RT, active thermographic testing (TT) represents a contactless, simple, less expensive and in-situ suitable alternative by measuring the infrared (IR) radiation intensity of the specimen with IR cameras 2 .
In active TT, light sources such as lasers can be used to generate heat in the specimen 3,4 . Compared to other light sources such as flash lamps, halogen lamps or LED, lasers do not exhibit spectral overlap with the IR camera 5 and can be tightly focused which helps to realize structured illumination (SI). Beside light sources, other energy sources, e.g. induction coils and ultrasonic transducers are possible as well for thermal NDT 6 .
The diffuse nature of heat propagation in the material causes a degradation in spatial resolution and therefore also in reconstruction accuracy 7 . To solve this problem, various measurement and thermal image processing strategies were applied such as pulsed-phase thermography 8 or lock-in thermography 9 , making use of the relative amplitude or phase change to a reference area. A relatively new method to circumvent spatial heat blurring is the introduction of virtual waves, which increases the signal-to-noise ratio (SNR) in the measured thermal images by transforming diffuse thermal waves into virtual propagating waves 10-12 . Apart from that, so-called optical super resolution (SR) imaging-serving as an alternative measurement strategy to enhance the spatial resolution-gained attention in fields of structured illumination microscopy 13,14 . These SR techniques rely on multiple measurements with a small position shift. The result is a spatial frequency mixing of the illuminated target pattern and the illumination pattern enabling an improvement in spatial resolution. While these optical SR techniques aim to enhance the optical diffraction limit of the imaging system, geometrical SR techniques aim to enhance the resolution of the digital imaging sensors. While the latter approach is known 15,16 and already implemented in commercial IR camera systems, a method to overcome the diffusion limit of TT by means of an optical SR analogue was out of reach so far.
Compressed sensing (CS) based algorithms can be used in post-processing which benefit from multiple measurements all referring to a reconstruction result that is sparse 17 . Since defects are sparse in space and CS algorithms rely on reconstructing a sparse data set from given measurements, CS is highly attractive and applicable in NDT scenarios as well. Thus, CS based algorithms based on sparsity regularization with ℓ 1 -minimization or Orthogonal Matching Pursuit (OMP) have been successfully employed to thermographic data improving the thermal image quality 18,19 . However, the application of a simple ℓ 1 -minimization or OMP would not benefit Novel photothermal SR approaches combine the SI measurement strategy and CS based processing algorithms, such as the iterative joint sparsity algorithm (IJOSP) to better separate two closely spaced defects 7,23-25 . So far, the suitability of these techniques has been investigated in transmission configuration with anomalies on the backside of the specimen. This paper focuses on the applicability of laser excited super resolution thermal imaging to an additively manufactured stainless steel sample with internal defects as test specimen.
The major contributions of this paper are listed as follows.
• Different SI experiments (laser step and continuous scan in reflection and transmission configuration) are shown to create suitable data to perform SR and to resolve all internal defects (so far in literature: SR suitable experiments only in transmission configuration examining sample surface anomalies). • The thermal image processing and the application of the model-based IJOSP algorithm is described in details and adopted for the investigation of a specimen with internal defects in transmission and reflection configuration (so far in literature: photothermal SR image processing description only available based on transmission setups analyzing sample surface anomalies). • The SR reconstruction results are compared qualitatively as well as quantitatively with conventional laser thermography reconstruction results based on homogeneous illumination. The comparison shows that the proposed SR techniques outperform conventional photothermal techniques resolving internal defects in steel with an at least four times better spatial resolution. • It is discussed which of the proposed SR techniques-based on different experimental setups-exhibits the most promising results in improving the spatial resolution for TT of metals with internal defects.

Methods
Experimental setup. In the following we focus on the measured data obtained by using the transmission and reflection setup shown in Fig. 1a,b, both performed in step scan and continuous scan measurements, respectively. An exemplary experimental thermographic setup is shown as a photograph in Fig. 2 which would correspond to the shown configuration in Fig. 1a. As shown in our previous work 25 , step scan means that we use laser pulses ( t pulse = 500 ms) to heat up the sample, wait until the sample is cooled down ( t cooling = 20 s), shift the position slightly with a distance of r = 0.2 mm and repeat for around 250 individual measurements. In all our measurements we have used a fiber-coupled high-power diode laser with a maximum output power of 530 W, a linear shaped spot ( 0.4 mm × 17 mm) and a wavelength of 940 ± 10 nm. Furthermore, we have used a mid-wave IR camera (Infratec IR9300, f cam = 100 Hz, full frame: 1280 × 1024 pixel, sensitive in 3-5 µ m wavelength range, NETD-noise equivalent temperature difference of ∼ 30 mK) which was triggered by a photodiode that recognizes when the laser is switched on. We have measured around 1000 frames from the beginning of the pulse for each measurement with a pixel resolution of r cam = 54 µm/pixel. This leads to a huge amount of data. In contrast, the continuous scanning method provides a smaller measured data set, since the specimen is scanned continuously while the IR camera is measuring without any intermediate cooling process. Thus, the size of the measured data is controlled by the chosen scanning velocity. The power of the laser has been adjusted in each measurement configuration so that we reach temperature differences of around T = 3-5 K for an assumed emissivity of ǫ = 1 (since only relative changes in temperature are of interest, this parameter was not considered critical).

Mathematical model for super resolution laser thermography.
Without restricting the generality of our approach, we simplify the 3dim problem to a 2dim one by using a set of linear defects and a linear laser. We calculate the mean over the vertically arranged pixels (see dimension y in Fig. 1) and end up with a problem formulation in the r-z-domain. In addition, the mean over 315 pixels provides a better SNR of �T/NETD · √ 315 ≈ 4 K/30 mK · 18 = 2400. www.nature.com/scientificreports/ To describe the measured temperature difference values T meas = T , we make use of a Green's function with rectangular coordinates as a solution for the heat diffusion equation for a line source and a plate with a finite thickness. In our previous studies 24, 25 , we investigated steel samples which have been blackened on the front side so that the measured temperature from the backside could be described for each measurement i = 1 . . . N meas by whereby represents the heat diffusion followed from illumination and x i = I i • a represents the element-wise (Hadamard) product of the illumination I i and absorption pattern a which represents the defects in space 7,24,25 . In contrast to our previous model, there is no analytical solution for the case of internal defects to be investigated now, i.e. the required now is unknown. Since we are not interested in an exact reconstruction of the internal geometry, but only in the best possible separation of closely adjacent internal defects, we can continue to use this approach as an approximation. This means, we pretend to have an exactly describable sample without defects by and allow the internal defects sparsely distributed in the r-z-domain to lead to a sparsely distributed contrast in the result T meas , which we interpret as defect position in the r-domain. Thus, can be described by 25,26 for step scanning step = , whereby we set z = 0 for reflection and z = L for transmission configuration, r stands for the position of the horizontally arranged pixels and t stands for the time. Thus, Eq. (2) describes a thermal point spread function that considers the convolution with the laser pulse length in step scanning by I t (convolution in time with the variable t). For continuous scanning cont we have to substitute I t := 1 since the laser is switched on continuously, r := r − v · t due to motion consideration and to add an integral over the previous time stamps 25,27 instead of having an integral due to the pulse length consideration as shown in Eq. (2). ρ stands for the mass density, c p for the specific heat, α for the thermal diffusivity, R for the thermal reflectance from the material to air, and L for the thickness of the specimen. We have identified the following values for the material parameters of our investigated additively manufactured stainless steel 316L 1.4404 sample: α = 4 × 10 −6 m 2 /s, ρ = 7990 kg/m 3 , c p = 500 J/kg/K and R = 1. Indeed, R = 1 was chosen for the sake of simplicity, but it is a very good approximation since R actually should be around 0.95. It should be noted that the lal-tetrahydropalmatineser line width is not taken into account in , which means that this quantity has to be considered in x 25 . Figure 3 shows step and cont in reflection and transmission configuration, respectively. To reduce the huge amount of data generated by performing step scanning measurements, we applied the maximum thermogram (MT) method 24 to the step scanning data which eliminates the time dimension from T meas,step ∈ R N r ×N t ×N meas by selecting the time stamp t = t MT where the maximum temperature amplitude is reached (c.f. Fig. 4). Effectively, this leads to T step ∈ R N r ×N meas and a vertical section through � step := � step (r, z = 0|L, t = t MT ) in Fig. 3a,b), represented by the dashed vertical lines. Consequently, we reformulate Eq. (1) and describe the measured data for step and continuous scanning by the following two equations: Since we are measuring the data, we have to describe the measured data by discrete values T i step/cont [k] = T i step/cont (k · �r cam ) with k = 1 . . . N r . It should be noted that the measured continuous scanning data indeed considers the time, but here we interpret each time stamp as a measurement i.

Inspection of internal defects using IJOSP.
The iterative joint sparsity (IJOSP) approach promotes joint sparse solutions and is particularly of interest if blind illumination is used 21,28 . In this work, the data is treated as if we had measured blindly, since we consider the position of the illumination in x and not in . This is a worst-case scenario, which is of practical nature, because we do not always know the exact position of the illumination. Therefore, the IJOSP approach empirically enables us to find a solution x ≈ x and to describe the defect pattern in space considering the following minimization problem: (4) min 2 and Tikhonov regularization �x� 2 2 . To find a good solution with x ≈ x , there are several ways to solve the minimization problem in Eq. (4). In this work we have used the optimization algorithm Block-FISTA (does not make use of Tikhonov regularization) and Block-Elastic-Net method 25,29,30 . Both optimization methods use an updating step size related factor L c within the gradient descent implementation which is richly described in literature as the Lipschitz constant for the gradient of the least squares error term shown in Eq. (4). The number of updates is described by the number of iterations N iter .

Quantitative evaluation of the final reconstruction result. The final reconstruction result x 1D as
shown in the flow chart in Fig. 4 is calculated by the sum over all measurements of x . Since SR experiments are performed with multiple measurements and small position shifts resulting in overlaps between the measurements, the normalized sum of all illuminations I i over the number of measurements equals an one-array over all pixels. This means that the sum over all measurements of x would approximately equal to a. In the following we work with a rec =x 1D / max x 1D = ix i / max ix i as final result to quantitatively evaluate and compare with the originally manufactured a of the specimen.
Thus, δT represents the contrast between defective and non-defective area investigating one specific defect pair. Further, δr denotes the distance between one peak in a rec to the real defect position in a referring to the full width half maximum (FWHM). Moreover, δT 1/2 ∈ [0, δT 1/2, max ] with δT 1/2, max = 1 due to the implemented normalization and δr 1/2/3/4 ∈ [0, δr 1/2/3/4, max ]. Assuming that r A1 and r A2 limit the investigated area according to Fig. 5, the maximum values can be calculated by: δr 1, max = r A2 − r D1 , δr 2, max = r A2 − r D2 , δr 3, max = r D3 − r A1 and δr 4, max = r D4 − r A1 . The reconstruction accuracies (ra) for δT and δr can then be given by: The overall reconstruction accuracy for an area A covering one defect pair is further determined by: ra A = 0.5 · (ra δT + ra δr ). Figure 4 shows the processing steps from the raw data averaged along y to the final 1dim reconstruction results of the investigated internal defects shown in Fig. 6. To obtain these final results (red curves), we have used the parameters listed in Table 1.

Results and discussion
Analyzing Fig. 6 clearly shows that we are able to resolve all internal defects in reflection and transmission configuration for the additively manufactured stainless steel specimen. Especially in the step scanning reflection case (see Fig. 6a) we obtain outstanding results being able to reconstruct each internal defect almost perfectly. Even with continuous scanning (Fig. 6c,d) we are able to resolve the internal defects, but not as accurate as in step scanning. Here, the most prominent peaks indicate the correct positions of the investigated internal defects, but sometimes some minor peaks occur as small artifacts as well. But of course we also have to take into account that we have to measure two to three orders of magnitude longer for the step scan (c.f. Table 1). Furthermore, it is noticeable that the red result curves in diagrams (a,b) exhibit broader peaks than the narrow peaks in (c,d). This comes from the fact that in (a,b) we have used the Block-Elastic-Net optimization within IJOSP approach that makes use of the Tikhonov regularization ( 2 > 0 , c.f. Table 1) which smoothens the signal in the spatial sparsity domain, whereas the Block-FISTA ( 2 = 0 ) algorithm is used in (c,d).
The results can be further examined quantitatively by using the reconstruction accuracy as defined in the Eq. (5). Table 2 shows all calculated values based on the resolution accuracy given by the specification of the used IR camera in terms of spatial resolution r cam = 54 µ m and temperature resolution T cam = 25mK.  www.nature.com/scientificreports/ Table 2 confirms the above mentioned statements based on the qualitative analysis. In addition, it can be seen that the use of SR results in reconstruction accuracies of over 90% whereas conventional methods fail completely (see ra A3 and ra A4 ). Even the reconstruction based on continuous scanning data, that is of particular interest for industrial applications, exhibits reconstruction accuracies of around 70-80%. Consequently, the quantitative analysis shows that very high reconstruction accuracies ( > 90% ) can be achieved with the proposed method only for laser step scan measurements. If an exact reconstruction is not necessary, but only defects should be detected, the continuous scanning could be sufficient as data basis, which has the advantage that these data can be generated much faster than with the step scan.

Conclusion and outlook
In this study we could show that we are able to resolve closely spaced internal defects inside an additively manufactured stainless steel specimen. We obtained accurate results in 1D reconstruction outperforming conventional thermographic methods realized by homogeneous illumination of the whole sample surface. The resulting Table 1. Chosen experimental and processing parameters to obtain the IJOSP results (red curves) in Fig. 6.  in (a, b) show additionally the conventional result obtained by simply applying the maximum thermogram method to the raw data of a measurement where the whole sample surface is illuminated-here we illuminated the whole sample surface with a laser square 60 × 60 mm 2 , t pulse = 2 s.
Scientific Reports | (2020) 10:22357 | https://doi.org/10.1038/s41598-020-77979-y www.nature.com/scientificreports/ diagrams after applying IJOSP show an at least four times better spatial resolution since we could easily separate defects with a defect depth to defect distance ratio up to 4:1 instead of 1:1 in conventional thermographic inspection techniques. The best results have been obtained in step scan reflection mode but the very fast continuous scanning mode does lead to convincing results as well. To obtain these outstanding results, we have used standard measurement technology, as found in many thermography laboratories, but upgraded with a SR measurement strategy and a CS-based post-processing. Thus, these studies encourage the use of SR laser thermography in metal industry for an accurate inspection of e.g. production samples with blowholes or other inclusions. However, the regularization parameters within the IJOSP approach have been chosen manually. Therefore, as an outlook, we are working on a deep neural network approach to figure out the optimal regularization parameter based on simulated and/or experimental training data.

Data availability
The data that support the findings of this study are available from the corresponding author upon reasonable request.
Received: 10 June 2020; Accepted: 17 November 2020 Table 2. Calculated reconstruction accuracies ra ∈ [0, 1] with ra = 0.25 · (ra A1 + ra A2 + ra A3 + ra A4 ) for each shown result in Fig. 6, A1 indicates the area of the defect pair with the largest distance to each other and A4 the defect pair area with the shortest distance to each other. Table cells with a dash inside mean that it was not possible to identify two defects. The defect geometries in the last column are speciefied by the distance between two defects within a defect pair, the width of one defect (all defects have the same width) and the depth of the defects (all defects have the same depth). A graphical visualization of the investigated defect geometries is shown in Fig. 1b).