Research on region selection super resolution restoration algorithm based on infrared micro-scanning optical imaging model

With spring up of infrared imaging related industry, infrared imaging technology has become mainstream development direction of intelligent photoelectrical detection due to its good concealment, wide detection range, high positioning accuracy, long distant penetration, light weight, little volume, low power dissipation and high solidity. However, the features of infrared dim-small target image such as less details and low SNR become bottleneck of infrared image application. How to enhance imaging effect of infrared dim-small target becomes research hotspot. Starting from the point of ‘restoration as foundation’, the theory and technology of infrared dim-small target super-resolution restoration by utilizing the theory and technology of super-resolution restoration are explored in this paper. This paper mainly focuses on the research of super-resolution restoration algorithm of infrared dim-small target based on infrared micro-scanning optical model. Aiming at solving super-resolution restoration problem of infrared dim-small target, the traditional super-resolution restoration algorithm is optimized and the improved algorithm is proposed. Meanwhile, infrared micro-scanning optical model is introduced to break theoretical limit of simple image processing algorithm. And the performance of infrared image super-resolution restoration is improved.


Reflective infrared micro-scanning optical model
The reflective infrared micro-scanning optical model designed in this paper can achieve controllable microscanning at any step length, thus providing low resolution images with corresponding frames for various super magnification imaging. The real time infrared images are obtained through the high speed vibration of high speed mirror. This model can obtain high speed and real time multi frame sub-pixel displacement images. The reflective infrared micro-scanning optical model is shown in Fig. 1. The diagrammatic sketch of reflective infrared micro-scanning optical model is shown in Fig. 2. The model mainly consists of four parts: infrared lens, micro-scanning controller, refrigeration infrared detector and computer. The high speed galvanometer is connected with micro-scanning controller, and the sub-pixel microscanning imaging on the XY plane can be realized by operating micro-scanning control software on the computer.
Aiming at solving the problem of error in the input source of traditional super resolution restoration algorithm, a reflective infrared micro-scanning optical model is proposed. The multi frame low resolution image sequences are generated by using this model. There are no time lap and spatial displacement of the multi frame low resolution image sequences. So the error in the input source of super resolution restoration is eliminated. Furthermore, the combination of infrared micro-scanning optical model and super resolution restoration breaks through the theoretical limit comparing with single imaging algorithm and the performance of super resolution restoration is greatly improved.

A POCS super resolution restoration algorithm based on region selection (RPOCS)
Considering real-time processing in photoelectric tracking system, this paper introduces threshold segmentation method to distinguish target region and background region. The selection of threshold directly affects integrity of target extraction and restoration effect. If the threshold value is higher, it is easy to classify target to background, so it is unable to extract target region. If the threshold value is lower, it is easy to classify background to target, so false alarm occurs. Traditional Otsu method determines single threshold value according to the maximum 4-7 .
The single threshold value is not accurate and the target region cannot be completely separated when the infrared background radiation is strong. Traditional Otsu method is not suitable for infrared dim-small target image. In this paper, an adaptive multi-threshold segmentation method is introduced. Supposing the infrared image is f (x, y) , and its gray level is {L 0 , L 1 , ..., L M } . The gray histogram of infrared image is shown as Eq. (1).
In Eq. (1) n L i is pixel number of gray value L i in the whole image, and N is total pixel number in the whole image.
For the convenience of derivation, we take m = 3 . For the three sections, the probabilities are shown as Eqs. (3), (4) and (5).
In Eq. (5), δ 3 = L M . According to the above probability distribution, the gray mean value µ of the whole image can be described as Eq. (6).
In conclusion, it can be solved shown as Eq. (11).
The energy function ω is defined as weighted Euclidean distance from three inner mean value µ(δ 1 ) , µ(δ 2 ) and µ(δ 3 ) to global mean value of the whole image, shown as Eq. (12).
(1) www.nature.com/scientificreports/ The threshold values δ 1 and δ 2 should be divided as far as possible, that is, maximize the energy function shown as Eq. (13).
Double threshold segmentation is divided into two processes. Firstly threshold value δ 1 is used to roughly segment the image. Some target regions will be submerged by background if background radiation is strong. Secondly threshold value δ 2 is used to further segment the image. Some target regions will be separated from background region.
For images containing infrared dim-small targets, there is no obvious peak phenomenon in histogram distribution when the contrast between background region and target region is not large. Double threshold segmentation can achieve better segmentation results.
In this paper the process of super resolution restoration for the low resolution image sequences {Lp 1 , Lp 2 , ..., Lp n } , firstly the threshold values δ 1 and δ 2 are determined, secondly the threshold value δ 1 is used to segment shown as Eq. (14).
In Eq. (14), Lp i ′ is low resolution image sequence that segmented with threshold value δ 1 . Then threshold value δ 2 is used to further segment. Some surplus target regions will be separated shown as Eq. (15).
In Eq. (15), Lp i ′ ′ (x, y) is low resolution image sequence that finally segmented with the threshold value δ 2 . After threshold segmentation, the remaining white region S w i (i = 1, 2, ..., n) is considered to be target region. Then target region is overlying shown as Eq. (16).
The white region S w calculated by Eq. (16) contains all target regions in the whole image sequence. In optoelectronic tracking system, we only concern the target region. So, only S w region needs super resolution restoration. Secondly for low resolution image sequences {Lp s 1 , Lp s 2 , ..., Lp s n } , we take one frame Lp s r for bilinear interpolation as reference frame Hp s r . The other frames need image registration and iterative projection correction with reference frame.
Finally we gain high resolution image Hp s of target region. Hp r is obtained by bilinear interpolation of Lp r . Hp s is overlay to Hp r . The high resolution image Hp is gained. The background region of Hp is obtained by bilinear interpolation. The target region of Hp is obtained by POCS 8 .
In conclusion, the steps of RPOCS are described as follows.
Step1: The threshold is determined according to Eq. (13). The threshold segmentation is applied to the whole low resolution image sequence according to Eqs. (14) and(15). The target region of each image S w i is gained.
Step2: All S w i are combined according to Eq. (16). All target regions S w are gained. The minimum circumscribed rectangle min Rect w,h top,left is calculated. Step3: The sub image sequences of target region are extracted to be new low resolution image sequences {Lp s 1 , Lp s 2 , ..., Lp s n }. Step4: A frame of high resolution image is constructed by bilinear interpolation.
Step5: The motion parameters between low resolution image sequences are calculated.
Step6: The low resolution image is mapped onto high resolution grid according to projection operator. Then gray is corrected.
Step7: The exit condition of iteration is judged. Yes, execute Step 8, otherwise, return Step 6.
Step8: Hp s is high resolution region of target region. Hp r is large scale high resolution image constructed by bilinear interpolation. Hp is final high resolution image.
The flow chart of RPOCS is shown in Fig. 3.

Experimental evaluations of RPOCS based on reflective infrared micro-scanning optical model
According to infrared micro-scanning optical model built in Fig. 2, the simulated infrared camera uses a resolution of 320 × 240 for experimental verification. Background is sky. The resolution of plane image in sky background is 320 × 240. The plane images in sky background are shown in Fig. 4. The restoration resolution of plane image in sky background is 640 × 480. The effect of two restoration algorithms for plane image in sky background is shown in Fig. 5. www.nature.com/scientificreports/ From Fig. 5a, it can be seen that traditional POCS restoration algorithm has strong noise in both background and target region. It shows that traditional POCS algorithm has poor anti noise ability, and eve noise amplification occurs locally. At the same time, the outline of target is also fuzzy. The restoration effect of POCS is not ideal.
From Fig. 5b, it can be seen that RPOCS restoration algorithm also has large noise in background. This is because RPOCS only restores target region and bilinear interpolation is used to background region.
POCS and RPOCS are compared by SSIM_NCCDFT 9 (Structural SIMilarity Normalized Cross Correlation of DFT) and processing time. The SSIM_NCCDFT and processing time of plane image in sky background are shown in Table 1.
From Table 1, it can be seen RPOCS has better performance than traditional POCS by using SSIM_NCCDFT for evaluation. From the overall effect, RPOCS has better performance than traditional POCS. From the processing time, it can be seen RPOCS is further faster than traditional POCS. This is because RPOCS only restores target region and traditional POCS restores whole image. The processing time of RPOCS is 1% lower than traditional POCS. The traditional POCS is poor on noise suppression and edge detail retention. The processing time is longer. Therefore it is inapplicable for photoelectric tracking system. RPOCS uses bilinear interpolation in background and only restores target region. So the processing time is faster. So it is suitable for photoelectric tracking system.

Conclusions
Aiming at solving the problem of error in the input source of traditional super resolution restoration algorithm, a reflective infrared micro-scanning optical model is proposed. The multi frame low resolution image sequences are generated by using this model. There are no time lap and spatial displacement of the multi frame low resolution image sequences. So the error in the input source of super resolution restoration is eliminated. Furthermore, the combination of infrared micro-scanning optical model and super resolution restoration breaks through the theoretical limit comparing with single imaging algorithm and the performance of super resolution restoration is greatly improved.
For the long iteration of POCS and the shortcomings of incapability to meet real-time detecting of optical detection system, a POCS super resolution restoration algorithm based on region selection is proposed. The target region is the key point that we focus on in the optical detection system, while this region contains only very small number of pixels. Therefore, we use threshold segmentation and combination to acquire the union of all targets regions. Then we execute super resolution restoration only in the union of all targets regions. In this www.nature.com/scientificreports/ way we decrease huge computation of background restoration and greatly reduce the operation time to achieve real-time or near real-time. So this super resolution restoration algorithm can be applied in the practical infrared image processing system.