Quantum face recognition protocol with ghost imaging

Face recognition is one of the most ubiquitous examples of pattern recognition in machine learning, with numerous applications in security, access control, and law enforcement, among many others. Pattern recognition with classical algorithms requires significant computational resources, especially when dealing with high-resolution images in an extensive database. Quantum algorithms have been shown to improve the efficiency and speed of many computational tasks, and as such, they could also potentially improve the complexity of the face recognition process. Here, we propose a quantum machine learning algorithm for pattern recognition based on quantum principal component analysis, and quantum independent component analysis. A novel quantum algorithm for finding dissimilarity in the faces based on the computation of trace and determinant of a matrix (image) is also proposed. The overall complexity of our pattern recognition algorithm is \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$O(N\,\log N)$$\end{document}O(NlogN)—N is the image dimension. As an input to these pattern recognition algorithms, we consider experimental images obtained from quantum imaging techniques with correlated photons, e.g. “interaction-free” imaging or “ghost” imaging. Interfacing these imaging techniques with our quantum pattern recognition processor provides input images that possess a better signal-to-noise ratio, lower exposures, and higher resolution, thus speeding up the machine learning process further. Our fully quantum pattern recognition system with quantum algorithm and quantum inputs promises a much-improved image acquisition and identification system with potential applications extending beyond face recognition, e.g., in medical imaging for diagnosing sensitive tissues or biology for protein identification.


INTRODUCTION
In any intelligent image processing system, there are essentially two main steps: the acquisition of the image and the recognition of the desired patterns.Image acquisition for any pattern recognition method can be performed in multiple ways.For instance, classical sources (incoherent light from thermal radiation or a coherent beam from a laser) or quantum sources (entangled photons obtained from down conversion or squeezed light) can be used to obtain the images.Classical bright field imaging techniques employing the former sources, have the disadvantage of high probe illumination requirement, especially while imaging sensitive samples.Additionally, they are also plagued by the shot noise inherent in the intensities, and the background noise from the environment.Quantum techniques such as quantum illumination, or ghost imaging or even interaction-free imaging, alleviates the problems of background noise, and the probe illumination by utilizing quantum correlations between photon pairs [1].Furthermore, quantum sub-shot noise imaging [2] and super resolution techniques [3] enhance the noise sensitivity and resolution in any images beyond the classical limits.
As a second important step, pattern recognition in the acquired images is a prominent feature of any intelligent imaging system.Face recognition [4] is one of the branches of pattern recognition, with numerous applications such as face ID verification, passport checks, entrance control, computer access control, criminal investigations, crowd surveillance, and witness face reconstruction [5], among several others.For face recognition, several classical machine learning algorithms exist [6], generally requiring huge computational resources especially when faced with the problem of identification from a large database.Quantum machine learning algorithms employing quantum features such as superposition and entanglement [7][8][9][10][11][12][13][14][15] promise enhancements in terms of the computing resources and the speed compared to the classical counterparts.Several experimental researches have been done to implement these algorithms [16][17][18][19][20][21].In this article, we present a quantum algorithm for face recognition as one of the potential applications of quantum algorithms in machine learning.
The problem of identification of faces from any images generally constitutes different steps (shown in Figure 1): creating a database of faces consisting of training and test images, feature extraction using principal component analysis (PCA), linear discriminant analysis (LDA) or independent component analysis (ICA), feature matching using dissimilarity measures, and recognition [22].PCA extracts the eigenstates (or eigenfaces) of the covariance matrix of the images FIG. 1. Flowchart of the quantum algorithm for face recognition.The quantum algorithm is proposed to be performed in a quantum processor, which we call it quantum pattern recognition processor (QPRP).First the image is converted into matrix form, on which feature extraction algorithms such as quantum principal component analysis (QPCA) or quantum independent component analysis (QICA) are applied.QPCA extracts the eigenstates (or eigenfaces) of the covariance matrix of the images in the database.The eigenfaces include information like average face, gender (male, female), face direction, brightness, shadows, etc. QICA extracts the independent elements such as eyes, eyebrows, mouth, nose, etc. in a face.The complexity of this stage is O(log N ) -N is the dimension of th image.Then, the given faces are compared with the faces in the database by using dissimilarity measure based on the log determinant divergence, and the best match among the faces in the database is identified.
in the database, including information like average face, gender (male or female), face direction, brightness, shadows, etc. ICA, however, extracts the independent elements such as eyes, eyebrows, mouth, nose, etc. in a face.Quantum algorithms which provide speedup for PCA and ICA have already been proposed [7,23].Here, we focus on three main steps: (1) Quantum Principle Component Analysis (QPCA) [7], (2) Quantum Independent Component Analysis (QICA) [23], and (3) Dissimilarity measures (i.e., face matching), to develop a quantum algorithm for face recognition.In what follows, we present a quantum algorithm for dissimilarity measures for face matching with speedup.This is based on a quantum algorithm to compute the log determinant divergence using both the determinant and the trace of a matrix.Our algorithm combined with the inputs obtained from quantum imaging techniques provides a fully intelligent pattern identification system, with the joint benefit of the low-dose and higher resolution of quantum imaging methods, and the speedup and efficiency of the quantum algorithms.Figure 1 shows the flowchart of the quantum algorithm for the pattern identification.

QUANTUM FACE RECOGNITION
Classical algorithms are unable to process quantum data directly.During the conversion of the quantum states (qu-dits) to classical data (bits), most of the information is lost in the measurement process, due to the "collapse" of the wavefunction.Although techniques such as quantum state tomography implemented on unlimited ensemble of the states can be used to fully reconstruct the quantum states from classical projections, these processes are generally complex and expensive.Therefore, the optimal input to our quantum algorithms, would be the quantum states directly obtained from quantum processes, for example, quantum imaging methods, or from a quantum memory, without performing a strong measurement on the wavefunction.Photonic quantum memories [24], allowing storage and on-demand retrieval of quantum states of light, is one of the key components for the realization of quantum optical pattern-recognition technology.Quantum memories essentially form a quantum database for the matching stage in the recognition process.With the state-of-art quantum memories, the possibility of storing hundreds of spatial modes has already been shown in experimental studies using atomic-cold gases [25,26].Furthermore, using solid-state atomic memories, it is possible to simultaneously store hundreds of photonic quantum states in distinct temporal modes, thus allowing us to store patterns scanned at separate times [27,28].In addition, optically accessible spin-states of certain atomic systems can reach several hours of coherence time [29].A very recent experimental demonstration reports one-hour memory lifetime for light storage, showing the feasibility of long-lived photonic quantum memory devices [30].Atomic memory approaches have also been shown to reach high retrieval efficiencies up to 92% [31] and high fidelities above 99% [32].However, an implementation with all of the aforementioned properties still remains as a challenge in developing a practical quantum database memory.
Quantum techniques such as quantum ghost imaging [33], quantum lithography [34], or quantum sensing [35], when appropriately interfaced with photonic quantum processors, for example an array of optical fibers connected to an integrated quantum photonic circuit, can also act as inputs to our algorithms (see Figure 2).Here for the case of our face recognition algorithm, we assume that the input images are acquired by quantum ghost imaging [33].Ghost imaging exploits the spatial correlations between photon pairs generated through a nonlinear process called spontaneous parametric down-conversion (SPDC).Since the images are obtained by triggering the shutter in order to capture only the "coincident" photon pairs, the level of background noise is significantly reduced, along with a reduction in probe illumination.In a variation of this technique using non degenerate photon pairs, the image detection and sample interaction can happen at different wavelengths, which can be useful when imaging sensitive tissues when limited in detection technologies [36].Combining quantum detection techniques such as interactionfree measurement with ghost imaging, the illumination level required for the same levels of Signal to Noise ratio (SNR) in images [37] is further reduced significantly.Figure 3 shows some of the images of human faces obtained in a quantum ghost imaging setup, where spatially correlated photon pairs (namely signal and idler), are generated by pumping a BiBO crystal with pump photons.Phase holograms placed in a Spatial Light Modulator, a liquid crystal device, created by superimposing the human faces with a diffraction grating acts as an object for the signal photon, while the idler photon passes to the Intensified Charged Coupled Devices (ICCD) camera via a delay line.The images are obtained by triggering the ICCD shutter with the signal photons detected through a Single Pho-ton Avalanche Diode (SPAD) detector -see Supplementary Information (SI) for the detail of the experimental setup.
A. Quantum principal component analysis (QPCA) We have now the input images either retrieved from a quantum memory or directly as outputs from a quantum imaging setup.The pattern recognition processor applies Quantum Principal Component Analysis (QPCA) [7,38] to extract the principal eigenvectors of the covariance matrix C X , formed by the set of the training images.
Let us consider a set of N -dimensional training images (or faces), {|x (1) , . . ., |x (M ) }. Here, |x (i) is the i-th training image, which is given by, q are the components, and |ψ are the basis kets.The covariance matrix C X can be formed as a sum over M training faces [38], The next step is to exponentiate the covariance matrix C X , so that we can use the Quantum Phase Estimation (QPE) subroutine for finding the eigenvectors and eigenvalues.It has been shown that the exponentiation of the covariance matrix, i.e., e −iC X t , can be performed in O(log N ) time [7].
In QPCA algorithm, for the phase estimation subroutine, we apply the operator U = e −iC X t on C X [38].The action of U on one of the states |x (i) in C X is: where |φ j 's are the eigenvectors of C X , and where λ (j) c 's are the corresponding estimated eigenvalues of C X with precision n [7,38].
In order to obtain the principal eigenfaces (the eigenvectors of the covariance matrix with larger eigenvalues), we define a score s (ij) , which is the projection of an eigenvector |φ (j) on a training vector |x (i) , where φ n are the components of the eigenvector |φ j .The eigenvectors corresponding to the r highest scores are the principal components (or eigenfaces).Each face can be expanded in terms of the r eigenfaces (principal components) but with different weights ω j s as follows The "mean image" is the eigenface corresponding to the largest eigenvalue of C X .The QPCA algorithm is efficient for the case r N [38].

B. Quantum independent component analysis (QICA)
In classical machine learning, Independent Component Analysis (ICA) is performed to decompose an observed signal into a linear combination of unknown independent signals [22].Similar to the PCA, the ICA finds a new basis to represent the data, however with a different goal.We assume that there is a data set of faces s ∈ R d that is a collection of d independent elements in the face such as nose, eye, eyebrow, mouth, etc.Each image observed through a camera can be expressed as x = F • s, where F is a mixing matrix of the independent face elements.Repeated observation gives us a dataset x as {x (i) , . . ., x (M ) }, and ICA estimates the independent sources s (i) that had generated the face.We let W = F −1 which is the unmixing matrix and solve the linear systems of equations s (i) = W x (i) for estimating the independent elements of the face.We should note here that s (i) is a d-dimensional vector and s (i) j is the data of element j.Similarly, x (i) is an d-dimensional vector, and x (i) j is the observed (or recorded) element j by camera.The ICA can be exponentially speedup via a quantum algorithm for sparse matrices, with the Harrow-Hassidim-Lloyd (HHL) algorithm [23], which is used to solve linear systems of equations optimally with O(log N ).For comparison, classically it takes a time O(N 3 ) to be solved via the Gauss elimination, and approximately O(N √ κ) via iterative methods [23] for a sparse matrix of size N × N , with κ being the ratio between the greatest and the smallest eigenvalue.

C. Pattern matching: Comparing Faces
As important details of a face are obtained either by using QPCA or QICA, each face is represented in the form of a sparse matrix in which non-important elements are set to zero.The last and important step of the algorithm is comparing the face patterns to recognize the target face.Pattern matching algorithms investigate exact matches in the input with pre-existing patterns in the database.In fact, the problem here is comparing matrices with each other.The evaluation of matching between matrices (or face patterns) can be done by using "dissimilarity" [39] measures that calculate the "distance" between the matrices.The lower the values of the dissimilarity/distance measures, more similar the matrices, with the fully matched matrices having a zero distance.One such distance measure used to compare two matrices X and Y is called the "Log-determinant divergence" [39,40] defined as, where N is the dimension of the matrices.When D = 0, the matrices X and Y are completely matched, and higher the distance value the more different are the matrices.The least value among the all distance values identifies the best match and consequently recognizes the face.As it is seen in the distance formula, it is a benefit to be able to calculate the trace and the determinants of matrices with speedup to expedite the distance calculation.In the following, we propose quantum algorithms for computation of the determinant and the trace of a sparse matrix.

Quantum Computation of Sparse Matrix Determinants and Trace:
To obtain a measure of dissimilarity between two matrices we need to calculate the determinant and the trace of the sparse matrix A = X • Y −1 .First we calculate Y −1 using the HHL algorithm [23] and obtain A by multiplying it with X.We then apply the Quantum Phase Estimation (QPE) subroutine, which consists of a quantum Fourier transform (QFT) followed by a controlled Unitary (CU) operation, with U = e −iA t , and a inverse quantum Fourier transform.We then apply a controlled Rotation operation followed by the inverse Quantum Phase Estimation (QPE) subroutine.At the end we have a multiplication operator Π which finally gives us the product of the eigenvalues -the algorithm steps are explained in more detail in the Supplementary Information.The running time of the algorithm up to the third step, i.e. applying the controlled-U operator, is O(log N (s 2 κ 2 / )) [23], where s is the sparsity, κ is the ratio of largest eigenvalue to the smallest eigenvalue of A, and is the acceptable error.Additionally, the multiplication operation in the last step can be performed in time O(log N ) and the algorithm should run N times.Therefore, the overall complexity of the algorithm is O N log N (1 + s 2 κ 2 / ) , which is much faster than the classical ones (see Table I).In order to compute the trace of the matrix A, an adder quantum algorithm [44] can speedup the computation.The adder operation between two diagonal elements is mainly based on the quantum Fourier transform (QFT), i.e. |Φ(a) N |k and the inverse QFT, i.e., QFT −1 |Φ(a) = |a .By continuation of this method sequentially for the all diagonal elements, one can obtain the trace of the matrix.The detail of the adder algorithm and the quantum circuit for the computation of trace is discussed in the Supplementary Information (Figure S2 shows the corresponding quantum circuit).The whole process which is based on QFT and QFT −1 has a complexity of O(log N ).QPCA and QICA both have logarithmic complexities, i.e., O(log N ).For the calculation of the log determinant divergence, the computation of trace has a complexity of O(log N ), while the determinant has complexity of O(N log N )).Hence, the overall complexity of the whole algorithm is O(N log N ).Table II shows a summary of estimated complexities along with the complexity of the general quantum face recognition algorithm.

CONCLUSION
In summary, we propose a new concept of a quantum protocol for 2D face recognition, combining the benefits of quantum imaging in image acquisition with the speedup from the quantum machine learning algorithms.In this concept, we consider images to be obtained via a ghost imaging protocol either as inputs to the quantum memories or as a hardware encoding of quantum information for the photonic pattern recognition processor.Feeding the "images" directly from a quantum protocol also eliminates the need for the conversion of classical data to quantum inputs for the processor saving valuable computational resources.The quantum pattern recognition processor then runs an algorithm composed of three main subroutines: (1) quantum principal components analysis (QPCA), (2) quantum independent component analysis (QICA), and (3) quantum dissimilarity measures for comparing faces.For the QPCA and QICA, we propose slight modifications in the existing algorithms, whereas for finding the dissimilarity measure, we propose a novel algorithm for obtaining the distance between two matrices based upon a metric called log-determinant divergence.Our algorithm obtains the determinant and the trace of the two matrices in O(N log N ) time -N is the dimension of the matrix.Complexity analysis shows that all of the three parts have speedup as compared to their classical counterparts, with the overall complexity given by O(N log N ).Our conceptual protocol provides a framework for an intelligent and fully quantum image recognition system with quantum inputs and a quantum machine learning processor.The joint benefits of the quantum image acquisition and quantum machine learning promises exciting technological developments in the field of image recognition systems.Author contributions: V.S. and D.P. contributed equally to this work.V.S. and E.K. developed the idea and consulted it with D.P., S.B., M.R., and E.S. to design the protocol.V.S., M.G., and M.Ab.developed the algorithm, D.P. performed the experiments under the supervision of E.K.All authors contributed to discussions.V.S., D.P., and E.K. wrote the manuscript, and V.S., S.B., and E.K. revised the final version.
Competing interests: The authors declare that they have no competing interests.
Data and materials availability: All data needed to evaluate the conclusions in the paper are present in the paper and/or the Supplementary Materials.Additional data related to this paper may be requested from the authors.

FIG. 2 .
FIG. 2. Intelligent pattern recognition in quantum imaging.Data from quantum imaging methods such as (a) Interaction Free Imaging and (b) Ghost Imaging act as an input to (c) Quantum Pattern Recognition Processor (QPRP).The latter, i.e., QPRP, applies quantum machine learning to find the patterns in the database.

FIG. 3 .
FIG. 3. Face recognition in ghost images.(a) Images of the original human faces (top) and the corresponding experimental ghost images (bottom) obtained in a ghost imaging setup.A femtosecond laser is used to generate spatially entangled photon pairs.One of the photons illuminates a spatial light modulator, which imprints different images onto the photon, and can act as a trigger for the other photon that was detected by an intensified CCD camera.(b) Quantum Independent Component Analysis (QICA), and Quantum Principal Component Analysis (QPCA), of the faces to detect the independent components, and principal features in the faces.(c) Dissimilarity measure between the ghost images with the images in the database for their identification.
Funding: D. P., M. R. and E. K. acknowledge the support of Ontario's Early Researcher Award (ERA), Canada Research Chairs (CRC), and the European Union's Horizon 2020 Research and Innovation Programme (Q-SORT), grant number 766970.V. S. is grateful for the financial support by the Spanish State Research Agency through BCAM Severo Ochoa excellence accreditation SEV-2017-0718 and BERC 2018-2021 program.S.B. acknowledges funding by the Natural Sciences and Engineering Research Council of Canada (NSERC) through its Discovery Grant, funding and advisory support provided by Alberta Innovates through the Accelerating Innovations into CarE (AICE) -Concepts Program, and support from Alberta Innovates and NSERC through Advance Grant.

TABLE I .
A Comparison of complexities between the classical approaches and our quantum approach, current work (CW), for the computation of determinant.

TABLE II .
Summary of estimated complexities in quantum face recognition algorithm.