Abstract
Conventional singlepixel cameras recover images only from the data recorded in one arm of the digital micromirror device, with the light reflected to the other direction not to be collected. Actually, the sampling in these two reflection orientations is correlated with each other, in view of which we propose a sampling concept of complementary compressive imaging, for the first time to our knowledge. We use this method in a telescopic system and acquire images of a target at about 2.0 km range with 20 cm resolution, with the variance of the noise decreasing by half. The influence of the sampling rate and the integration time of photomultiplier tubes on the image quality is also investigated experimentally. It is evident that this technique has advantages of large field of view over a long distance, highresolution, high imaging speed, highquality imaging capabilities, and needs fewer measurements in total than any singlearm sampling, thus can be used to improve the performance of all compressive imaging schemes and opens up possibilities for new applications in the remotesensing area.
Introduction
Compressed sensing (CS)^{1,2,3} is an amazing sampling paradigm that goes against the common wisdom in data acquisition followed by compression. CS theory asserts that one can reconstruct the signals known a priori to be sparse or compressive in some basis from far fewer measurements than traditional approaches. What is remarkable is that it allows a sensor to very efficiently acquire just the important information in a signal without trying to measure fulllength signal, so it saves the storage and facilitates the transmission. The general robustness of CS is socalled restricted isometry property (RIP)^{4}. In a nutshell, in order to satisfy RIP, we should find sensing matrices with the property that column vectors taken from arbitrary subsets are approximately orthogonal. It is found that random matrices with independent and identically distributed entries, e.g., Gaussian or ±1 binary entries, are largely incoherent with any fixed basis^{5}. In truth, the random sampling theorem is a very effective sensing mechanism and has triggered the many related CS applications. The singlepixel camera^{6,7,8} proposed by R. G. Baraniuk et al. is a good example, which uses a digital micromirror device (DMD) to perform light intensity modulation, and a bucket (singlepixel) detector without spatial resolution to measure merely the total intensity impinging on it. To our knowledge, this system is very similar to computational ghost imaging^{9} proposed by J. H. Shapiro. This singlepixel camera scheme, in particular, has gained considerable attention in the research community because of its highly undersampled measurements, high luminous flux and the fact that it only needs one pixel to acquire a twodimensional image without raster scanning. Recently, this scheme has been widely used in many areas, including optical encryption^{10,11}, adaptive computational ghost imaging^{12,13}, timeofflight range acquisition^{14}, and fluorescence microscopy for biological and hyperspectral imaging^{15}. However, for the practical implementation of a CSbased image acquisition system, it is essential to determine which kind of measurement matrix should be used since the patterns on the DMD constitute a binary matrix taking on the two values 0 or 1. Generally, a measurement matrix can be generated by randomly drawing from a fast transform such as a Walsh, Hadamard, or noiselet transform^{7}. And then it will be converted to a binary matrix. Take a Hadamard transform for example, V. Studer et al.^{15} use a Hadamard pattern h where each entry is either −1 or +1, and make a shifted and rescaled version of h via ϕ = (h + 1)/2. Actually, the binary matrix doesn't necessarily obey the Bernoulli distribution. R. Berinde and P. Indyk^{16} presented sequential sparse matching pursuit, which depends on a scattered binary matrix with the number of 1 much smaller than that of 0. This method not only saves the storage but also gets a better image quality than traditional Bernoulli distributed measurement matrix consisting of 0 or 1. Nevertheless, in the actual optical system, it sacrifices the flux and will fail to work under specific circumstances where the luminous flux is low. These methods all use only a single arm of micromirror reflection orientation. Furthermore, to our knowledge, the binary matrix taking on the two values 0 or 1 is not the best choice for compressive imaging.
Recently, the analyzing work of N. D. Hardy and J. H. Shapiro^{17} have indicated that computational reflective ghost imaging is feasible for remote sensing. As compressive imaging technique can be transplanted to ghost imaging, C. Q. Zhao et al.^{18} have extended pseudothermal light ghost imaging to the area of remoteimaging and proposed a highresolution lidar system based on ghost imaging. The field of view (FOV) of the receiving system as well as the emitting system at 900 m range is about 2 m and 1 m, respectively. G. A. Howland et al.^{19} have also experimentally demonstrated CS laser radar for 3D imaging in the laboratory with objects at depths ranging from 0.3 to 2.8 m from the device. All these experimental setups can only detect targets with active pulsed laser illumination, acquiring relatively small FOV over either a long or short distance.
In this paper, we demonstrated a sampling approach for the practical telescopic system named complementary compressive imaging (CCI), based on the traditional singlepixel camera but making full use of ±12 degrees double arm detection. Highresolution largeFOV imaging has been experimentally acquired at a distance about 2.0 km range.
Results
Experimental setup
The schematic diagram of the experimental apparatus is given in Fig. 1. Our optical setup is based on a Cassegrain telescopic unit and a standard singlepixel camera. The object is Tsinghua Tongfang logo on the lightemitting diode (LED) billboard of size 2.6 m × 2.6 m each Chinese character, located on the roof of their working building about 2 kilometers away from our test site. The photons emitting from the target are received by a Cassegrain telescope of aperture 120 mm and focal length 4104 mm, which can be focused by moving the secondary curved mirror of diameter 26 mm. Its maximum FOV on the target plane is of diameter 20 m at 10.0 km range, so it covers large FOV over a long distance, which is very useful for remote huge target imaging. Then the light is imaged onto a DMD. The DMD is a 1024 × 768 array of addressable micromirrors of size 13.68 μm × 13.68 μm each, which rotates about a hinge and can be shifted between two positions oriented at +12 degrees (1) or −12 degrees (0) with respect to the surface of the DMD. Therefore, the light can be reflected to two directions depending on the pattern encoded on the DMD. In Fig. 1, we set the left reflection orientation for +12 degrees, the right for −12 degrees. In our experimental apparatus, the highest frame frequency of the DMD is 32,552 Hz. Two photomultiplier tubes (PMTs) (of the same type Hamamatsu H746820) are used as the bucket (singlepixel) detectors for ultraweak light detection, which can be operated at up to 450 Hz. Although our technique is limited by the readout frequency of PMTs, it is still reasonably fast. These PMTs are located in the left and right reflection directions to record the total intensity of two orientations in form of current signals, respectively. In this system, some neutral density filters inserted in front of PMTs attenuate the light intensity in order to avoid PMTs to be lightsaturated. To assure perfect focus and make the image match the size of the screen, before starting the sampling, we insert an optical unit consisting of a beam splitter (BS) and a chargecoupled device (CCD) camera in front of DMD, about 50 mm away. After all the measurements have been done, a fine image can be reconstructed by CCI technique.
Experimental results
Figure 2 illustrates the process that creates complementary compressive images. In the experiment, the images of the target often fill the entire screen of DMD, but it will take a long time and huge memory storage for CS to recover such a large image. To simplify the computation but not weaken the total flux, for a 768 × 768 pixels image in the center of the DMD, we set the resolution of the image to be 12 × 12 pixel size, which is also the minimum size of each speckle (white or black). Then we encode M binary speckle patterns (actually 768 × 768 pixels for each pattern but can be easily converted to 64 × 64 pixels) on the DMD, and will obtain a final 64 × 64 pixels image. By reshaping each pattern into a row vector (1 × 4096), the measurement matrix A (M × 4096) is obtained after all the measurements having been performed. Meanwhile, the values recorded by the PMTs constitute a column vector b (M × 1). According to the measurement matrix and recorded values, we can retrieve the image by CS algorithm. As shown in Fig. 2, images in the first row are recovered from the data in the left reflection orientation; the ones in the next row reconstructed from the data in the right. Now we calculate the difference vector of these two data vectors, then divide each element in the difference vector by 2, and the result will be treated as new measured values. In the meantime, we set . After that we acquire the complementary compressive images as shown in the third row of Fig. 2. For comparison, we took a telescopic image of the target by a CCD.
Now we define the sampling rate as the ratio between the number of measurements and the length of the image signal. To demonstrate the effects of the sampling rate on traditional singlepixel cameras and CCI, the experimental reconstruction results for the same target in the identical detection conditions are given in the different columns of Fig. 2. We can see that as the sampling rate increases, the image quality of these methods all becomes better. Although there exists atmospheric turbulence, CCI can still reconstruct the images with sharper edges than those recovered from any single arm. In this experiment, only the center 768 × 768 pixels of the DMD are effective for imaging, but the light from all the 1024 × 768 pixels of the DMD is collected. During each flip time interval of the frames, in order to reduce the influence of the ineffective pixels, the two 768 × 128 pixels margins of the DMD are firstly all set to zeros for the left arm measurements, then all reversed to ones for the right arm measurements, but with the pattern in the center effective region of DMD unchanged. By measuring the left arm and the right arm alternately, we can ensure that each frame pair of pixel size 768 × 768 for the both arms of the system is complementary. Additionally, the noise in CCI may be incurred by the stroboscopic LED with bright and dark change or the observational error before and after reversing micromirrors. These factors may also lead to the inconsistent distribution of the noise in both arms, which may be manifested in the mean or variance or both. In the future, if the fullscreen micromirrors are all used, adopting the synchronous measurements will easily weaken the influence of these factors.
The width of slits in the red circles is 20 cm. We draw a threedimensional map of the image recovered at the singlearm sampling rate of 14.6% (29.2% in total) by setting pixel coordinates as xy axis and gray values as z axis, in which the slits can be distinguished well while the other two narrower slits can hardly be seen. Thus highresolution is also demonstrated, about 2.0 km range with 20 cm resolution. A further experiment for imaging a target at a distance about 10 km and quantitative analysis will be presented in a future paper. In order to get a better image quality in CCI, we have tried methods of increasing the integration time of PMTs, which defines how long the PMTs acquire the input light intensity signal during each flip time interval of the micromirrors. As the integration time of PMT increases, the signal will be enlarged, while the noise will be averaged, so the image quality ought to become relatively satisfying. In view of this idea, we performed an experiment to check the validity of this theory, and the results are given in Fig. 3. We can see that the quality is roughly proportional to the integration of PMTs. It is interesting to note that there are some oscillations due to the atmospheric turbulence. Actually, from the CCD, we will see the images shaking all the time, which may also partly because of the wind.
Performance evaluation
To get a more quantitative measure of the reconstructed image quality, we calculated the mean square error (MSE) of recovered image as compared to the original image T_{o} of pixel size p × q. Naturally, the smaller the MSE value is, the better the quality of the image recovered. We created a head phantom image^{20} as the original image, which is normalized to a range of 0–1, and did some numerical simulation experiments. The random matrix used here also consisted of two values 0 or 1. Figures 4(a)–(b) show the images, the details of which are all degraded, reconstructed from the data in the left and right reflection directions with 30% sampling rate, respectively. Figure 4(c) gives the result with better visibility restored from a subtracted and rescaled version of these two measured vectors. Actually, the number of measurements has increased to double its original one. In order to compare these three sets of data more intuitively, we treat the sampling rate in the singlearm as the horizontal axis of Figs. 4(d)–(e). As shown in Fig. 4(d), it is clearly visible that the MSE saturates at a pretty high level when using just one arm, while the result is orders of magnitude better when using both arms. This is a general result, whether there is additional random noise added to the imaging process or not. Furthermore, the advantage will be highlighted under conditions that there exists independent and identically distributed noise in both arms. From the curves, we can say that the total number of measurements in CCI is much smaller than that in singlepixel sampling methods in order to get the same MSE value, although CCI needs to measure the signals in both reflection arms of the DMD. Besides, it is evident that as the sampling rate increases, the computation time of CCI gradually becomes shorter than that of either one of these two reflection orientations, as shown in Fig. 4(e).
Discussion
Considering gaps between adjacent micromirrors and gaps that appear when micromirrors flip, we calculate that the proportion of the area of 768 × 1024 micromirrors to that of total reflection plane of the 0.7 in. DMD is , and the proportion of the projected area of all the micromirrors when they are positioned in one of two states to the area of all the micromirrors is cos(12°) × 100% = 97.815%, as shown in the right part of Fig. 5, thus the proportion of the area of all the gaps to that of the entire reflection plane of the DMD is about (1 − 0.96986 × 0.97815) × 100% = 5.133%. Namely, the area of all the gaps (the shaded regions as shown in Fig. 5) is a constant no matter what kind of pattern encoded on the DMD. As we know, these gaps show the metal surface under the micromirrors. Besides, there is a mirror support post with a little light transmittance in the center of each micromirror. The metal surface together with the axis will incur some random noise. Actually, the metal surface is a combination of mirrorlike and diffuse reflections. The specular reflections of the metal surface will not affect the detection, and can be ignored. In the latter, it bounces off the light in many directions. Thus, the PMT in each detection arm receives only a very small part of the light reflected diffusely from the metal surface, which is as well associated with each pattern encoded on the DMD to a large extent and is approximately independent identically distributed. In our experiment, as the analogtodigital (A/D) conversion of PMTs is in 12 bits precision, the saturation value is 4096. To assess the impact of this kind of noise, we set the pattern to be all zero, and record the value reading from the PMT in the left reflection arm, which is turned out to be zero. If we change the pattern to be all one, the other arm shows the same result. It can be seen that the influence of this noise on the measured value is less than 1, thus can be neglected here. However, if the singlepixel detector is replaced by a Geigermode avalanche photodiode (APD) or a PMT with higher precision, this kind of noise is worth consideration. Fortunately, complementary measurements can make up for the loss of the image quality caused by this kind of noise.
Since CCI regards a subtracted and rescaled vector of these two measured data from different detection arms as the final input values, denoted as b, it not only averages the variance of the noise, but also makes the ratio δ of the variance to the mean of measured sequence b larger, that is equivalent to enlarging the useful signal partially but subtracting the background signal. And this is exactly the reason for its good image quality. We also found that R. Berinde's sparse binary matrix has a mean which tends to be zero, and actually amplifies the variance of b as well.
The setup presented here can be considered as a twopixel camera (with complementary measurements) or as a single pixel camera scheme with an altered binary measurement matrix using ±0.5 instead of 0 and 1 as the binary values. It may be of interest to discuss the relation to nonbinary imaging matrices. Take a grey scale of n levels for example, the dynamical range of the measurement matrix is 0 ~ n. If the measurement matrix for the left arm is A_{1}, then its complementary matrix for the right arm should be n − A_{1}. Following the same strategy, we can obtain a distributed altered measurement matrix . Therefore, our complementary sampling protocol can be easily extended to nonbinary measurement matrices, with similar or even better effect. Moreover, a grey scale of n levels can be realized by adopting binary pulse width modulation technique (that is, the nonbinary imaging matrices encoded on the DMD have a binary relationship to different display periods of pixels) or using different devices instead of DMDs.
In summary, our results represent the first complementary compressive imaging of a target at about 2.0 km range with 20 cm resolution, realizing large FOV imaging over a long distance. Unlike conventional singlepixel cameras which use measured data from only one detection arm, our scheme makes full use of both reflection directions, and recalculates a new observed vector by dividing the difference of the two arms data by two. By solving the mathematical model of CCI, we acquire highresolution images with the variance of the independent and identically distributed noise decreasing by half. The use of CCI for the telescopic system is a leap forward for improving the image quality of traditional singlepixel cameras where the binary patterns encoded on the DMD generally consist of the two values 0 or 1, opening up possibilities for new applications. Moreover, our protocol can be also easily extended to nonbinary measurement matrices, which may be of great interest. We have experimentally investigated the influence of the sampling rate in the singlearm and the integration time of PMTs on the image quality. It is evident that CCI for the telescopic system has the advantages of long remotesensing distance, high imaging speed and highresolution imaging capabilities. The challenge arises by the fact that, in practice, the influence of the atmospheric turbulence on the image quality seems to be unavoidable and will lead to the degradation of the image quality, especially when the distance of freespace light propagation increases. As a result, an ongoing research is to find some ways to reduce the impact of atmosphere turbulence to some extent.
Methods
Compressed sensing
Generally speaking, many natural signals, such as images, have sparse representations when expressed in a certain basis. Assume we have a twodimensional image I of pixel size p × q, with can be stretched for a column , where N = p × q. We can expand x in an orthonormal basis (e.g., discrete cosine transform basis, Fourier transform basis, wavelet basis) Ψ = [ψ_{1},ψ_{2},…ψ_{N}] as follow: , where is the coefficient sequence of x. Considering that is the coefficient vector with all but the largest K(≤N) elements set to zero, and this vector can be called Ksparse. By taking about O(K·log(N/K)) random projections, one has enough information to recover the signal x with accuracy.
The problem of reconstructing a signal x can be described by: where (M < N) is a random measurement matrix, which satisfies RIP, is the observation of x via some linear measurements, and e is a stochastic noise term.
For image restoration, recent research has confirmed that the use of total variation (TV) regularization instead of the l_{1} term in CS problems makes the recovered image quality sharper by preserving the edges or boundaries more accurately, which is essential to characterize images. Thus, TV regularization is very competent for singlepixel cameras as the core reconstruction solver. A solver (named TVAL3^{21}), which adopts alternating minimization scheme, is a good example for TV minimization. C. B. Li has already validated with numerical results that TVAL3 is more efficient and robust in comparison to other TV solvers (such as NESTA, NESTA_UP, TwIST, l_{1}Magic, SOCP) and most l_{1} minimization algorithms (e.g., FPC, YALL1) and accepts a vast range of measurement matrices (unlike RecPF^{22}, which is restricted to partial Fourier measurements only). The core TV norm minimization of TVAL3 can be written as: where D_{i}x denotes the discrete gradient vector of x at position i, and μ > 0 is the penalty parameter.
Mathematical model of CCI
The mathematic model of complementary compressive imaging can be described as: where A_{2} = 1_{M}_{ × N} − A_{1} (1_{M}_{ × N} denotes an array of all ones), e_{1} together with e_{2} corresponds to the noise on the different sides respectively. Then we can have the following formula: If we set , , and , we will obtain the same equation as Eq. (2), but obtaining ±0.5 binary measurement matrix instead of the binary matrix consisting of 0 and 1, with the variance of the independent and identically distributed noise decreasing by half. It solves the problem of the singlepixel camera in obtaining highquality images.
References
 1.
Candès, E. J., Romberg, J. & Tao, T. Robust uncertainty principles: exact signal reconstruction from highly incomplete frequency information. IEEE Trans. Inf. Theory 52, 489–509 (2006).
 2.
Donoho, D. Compressed sensing. IEEE Trans. Inf. Theory 52, 1289–1306 (2006).
 3.
Candès, E. J. Compressive sampling. Proc. Int. Cong. Math: invited lectures, Madrid, Spain. 10.4171/0223/69 (2006, Aug. 22–30).
 4.
Candès, E. J. The restricted isometry property and its implications for compressed sensing. C. R. Math. 346, 589–592 (2008).
 5.
Romberg, J. Imaging via compressive sampling. IEEE Signal Process. Mag. 25, 14–20 (2008).
 6.
Takhar, D. et al. A new compressive imaging camera architecture using opticaldomain compression. Proc. SPIE: Computational Imaging IV, San Jose, CA, USA. 10.1117/12.659602 (2006, Feb. 2).
 7.
Duarte, M. F. et al. Singlepixel imaging via compressive sampling. IEEE Signal Process. Mag. 25, 83–91 (2008).
 8.
Chan, W. L. et al. A singlepixel terahertz imaging system based on compressed sensing. Appl. Phys. Lett. 93, 121105 (2008).
 9.
Shapiro, J. H. Computational ghost imaging. Phys. Rev. A 78, 061802 (2008).
 10.
Li, S., Yao, X. R., Yu, W. K., Wu, L. A. & Zhai, G. J. Highspeed secure key distribution over an optical network based on computational correlation imaging. Opt. Lett. 38, 2144–2146 (2013).
 11.
Yu, W. K. et al. Protocol based on compressed sensing for highspeed authentication and cryptographic key distribution over a multiparty optical network. Appl. Opt. 52, 7882–7888 (2013).
 12.
Aßmann, M. & Bayer, M. Compressive adaptive computational ghost imaging. Sci. Rep. 3, 1545 (2013).
 13.
Yu, W. K. et al. Adaptive compressive ghost imaging based on wavelet trees and sparse representation. Opt. Express 22, 7133–7144 (2014).
 14.
Kirmani, A., Colaço, A., Wong, F. N. C. & Goyal, V. K. Exploiting sparsity in timeofflight range acquisition using a single timeresolved sensor. Opt. Express 19, 21485–21507 (2011).
 15.
Studer, V. et al. Compressive fluorescence microscopy for biological and hyperspectral imaging. Proc. Natl. Acad. Sci. U. S. A. 109, E1679–E1687 (2012).
 16.
Berinde, R. & Indyk, P. Sequential sparse matching pursuit. Proc. 47th Annu. Allerton Conf. Commun. Control Comput. Monticello, IL. 10.1109/ALLERTON.2009.5394834 (2009, Sept. 30–Oct. 2).
 17.
Hardy, N. D. & Shapiro, J. H. Reflective ghost imaging through turbulence. Phys. Rev. A 84, 063824 (2011).
 18.
Zhao, C. Q. et al. Ghost imaging lidar via sparsity constraints. Appl. Phys. Lett. 101, 141123 (2012).
 19.
Howland, G. A., Dixon, P. B. & Howell, J. C. Photoncounting compressive sensing laser radar for 3D imaging. Appl. Opt. 50, 5917–5920 (2011).
 20.
Jain, A. K. Fundamentals of Digital Image Processing [438–439] (Englewood Cliffs, NJ, Prentice Hall, 1989).
 21.
Li, C. B. An efficient algorithm for total variation regularization with applications to the single pixel camera and compressive sensing. (Master's thesis, Rice University, 2010).
 22.
Yang, J., Zhang, Y. & Yin, W. A fast alternating direction method for TVL1L2 signal reconstruction from partial Fourier data. IEEE J. Sel. Top. Signal Process. 4, 288–297 (2010).
Acknowledgements
The authors warmly acknowledge Wendong Xu and Chengqiang Zhao for providing the optical and machine design and supervising the experiments. We further thank Bin Dai for helpful contributions to the electronic design. This work was supported by the National Key Scientific Instrument and Equipment Development Project of China (Grant No. 2013YQ030595), the National High Technology Research and Development Program of China (Grant No. 2013AA122902), and the National Natural Science Foundation of China (Grant No. 11275024).
Author information
Affiliations
Key Laboratory of Electronics and Information Technology for Space System, Center for Space Science and Applied Research, Chinese Academy of Sciences, Beijing 100190, China
 WenKai Yu
 , XueFeng Liu
 , XuRi Yao
 , Chao Wang
 & GuangJie Zhai
University of Chinese Academy of Sciences, Beijing 100049, China
 WenKai Yu
 & XuRi Yao
College of Electronic and Information Engineering, Nanjing University of Aeronautics and Astronautics, Nanjing 210016, China
 Yun Zhai
Authors
Search for WenKai Yu in:
Search for XueFeng Liu in:
Search for XuRi Yao in:
Search for Chao Wang in:
Search for Yun Zhai in:
Search for GuangJie Zhai in:
Contributions
W.K.Y. conceived the idea and provided the leading contribution to the experiments, collected and analyzed the data and wrote the manuscript. X.F.L. participated in the building of the experiments. X.R.Y. assisted on the theoretical analysis and discussion. C.W. designed the hardware. Y.Z. took part in the noise analysis. The whole work was supervised by G.J.Z. All authors contributed to the scientific discussion and revision of the article.
Competing interests
The authors declare no competing financial interests.
Corresponding authors
Correspondence to WenKai Yu or GuangJie Zhai.
Rights and permissions
This work is licensed under a Creative Commons AttributionNonCommercialNoDerivs 4.0 International License. The images or other third party material in this article are included in the article's Creative Commons license, unless indicated otherwise in the credit line; if the material is not included under the Creative Commons license, users will need to obtain permission from the license holder in order to reproduce the material. To view a copy of this license, visit http://creativecommons.org/licenses/byncnd/4.0/
To obtain permission to reuse content from this article visit RightsLink.
About this article
Further reading

Singlepixel imaging with sampling distributed over simplex vertices
Optics Letters (2019)

Cryptographic key distribution over a public network via variancebased watermarking in compressive measurements
Applied Optics (2019)

Compressive ghost imaging in the presence of environmental noise
Optics Communications (2019)

Super SubNyquist SinglePixel Imaging by Means of CakeCutting Hadamard Basis Sort
Sensors (2019)

High throughput dualwavelength temperature distribution imaging via compressive imaging
Optics Communications (2018)
Comments
By submitting a comment you agree to abide by our Terms and Community Guidelines. If you find something abusive or that does not comply with our terms or guidelines please flag it as inappropriate.