Abstract
Pixelation occurs in many imaging systems and limits the spatial resolution of the acquired images. This effect is notably present in quantum imaging experiments with correlated photons in which the number of pixels used to detect coincidences is often limited by the sensor technology or the acquisition speed. Here, we introduce a pixel superresolution technique based on measuring the full spatiallyresolved joint probability distribution (JPD) of spatiallyentangled photons. Without shifting optical elements or using prior information, our technique increases the pixel resolution of the imaging system by a factor two and enables retrieval of spatial information lost due to undersampling. We demonstrate its use in various quantum imaging protocols using photon pairs, including quantum illumination, entanglementenabled quantum holography, and in a fullfield version of N00Nstate quantum holography. The JPD pixel superresolution technique can benefit any fullfield imaging system limited by the sensor spatial resolution, including all already established and future photoncorrelationbased quantum imaging schemes, bringing these techniques closer to realworld applications.
Similar content being viewed by others
Introduction
The acquisition of a highresolution image over a large field of view is essential in most optical imaging applications. In this respect, the widespread development of digital cameras made of millions of pixels has strongly contributed to create imaging systems with large spacebandwidth products. In classical imaging, it is therefore mainly the imaging systems with strong spatial constraints that suffer from pixelation and undersampling, such as lensless onchip microscopes^{1,2}. However, this effect is very present in quantum imaging schemes where it severely hinders the progress of these techniques towards practical applications.
Quantum imaging systems harness quantum properties of light and their interaction with the environment to go beyond the limit of classical imaging or to implement unique imaging modalities^{3}. Of the many approaches, imaging schemes based on entangled photon pairs are the most common and are among the most promising. Proofofprinciple demonstrations range from improving optical resolution^{4,5} and imaging sensitivity^{6,7,8,9} to the creation of new imaging protocols, such as ghost imaging^{10,11}, quantum illumination^{12,13,14,15} and quantum holography^{16,17}. Contrary to classical imaging, photoncorrelationbased imaging systems operate by measuring photon coincidences between many spatial positions of the image plane in parallel (except from inducedcoherence imaging approaches^{18,19,20}). In practice, this process is much more delicate than forming an intensity image by photon accumulation and therefore requires specific photodetection devices. Although such a task was originally performed with rasterscanning singlephoton detectors^{10}, today most implementations use singlephoton sensitive cameras such as electron multiplied charge coupled devices (EMCCD)^{21,22}, intensified complementary metaloxidesemiconductor (iCMOS)^{23} and singlephoton avalanche diode (SPAD) cameras^{24}.
EMCCD cameras are the most widely used devices for imaging photon correlations thanks to their high quantum efficiency, low noise and high pixel resolution. However, these cameras suffer from a very low frame rate (~100 fps) due to their electronic amplification process operating in series and therefore require very long acquisition times (~10 h) to reconstruct correlation images^{13,14,17,25}. Using a smaller sensor area or a binning technique can reduce the acquisition time, but at the cost of a loss in pixel resolution. By using an image intensifier, iCMOS cameras do not use such slow electronic amplification processes and can therefore reach higher frame rates (~1000 fps). However, so far these cameras have only enabled correlation images with relatively small number of pixels (and still several hours of acquisition), mostly because of their low detection efficiency and higher noise level^{23,26}. Finally, SPAD cameras are an emerging technology that can detect single photons at low noise while operating at very high frame rate (up to 800 kfps) despite their low quantum efficiency (~20%) and typically low resolution (~1000 pixels).
It is clear from the above that nearly all sensor technologies currently used in quantum imaging experiments suffer from a poor pixel resolution, either directly when the technology does not have cameras with enough pixels or indirectly when the experiment can only operate in a reasonable time with a small number of pixels. In these systems, objects are therefore often undersampled, resulting in a loss of spatial information and the creation of artefacts in the retrieved images. Here, we demonstrate a quantum image processing technique based on entangled photon pairs that increases the pixel resolution by a factor two. We experimentally demonstrate it in three common photonpairbased imaging schemes: two quantum illumination protocols using (i) a nearfield illumination configuration with an EMCCD camera^{13} and a (ii) farfield configuration with a SPAD camera^{27}, and (iii) an entanglementenabled holography system^{17}. In addition, we use our JPD pixel superresolution method in (iv) a fullfield version of N00Nstate quantum holography, a scheme that has only been demonstrated so far using a scanning approach^{6}. Note that we refer to our technique as ’pixel superresolution’ to avoid confusion with the term ’superresolution’ describing imaging techniques capable of overcoming the classical diffraction limit.
RESULTS
Experimental demonstration
Figure 1a describes the experimental setup used to demonstrate the principle of our technique for the widely used case of p = 2 spatially entangled photons. The spatially entangled photon pairs are produced by typeI SPDC in a thin βbarium Borate (BBO) crystal and illuminate an object, t, using a nearfield illumination configuration (i.e. the output surface of the crystal is imaged onto the object). The biphoton correlation width in the camera plane is estimated to be σ ≈ 13 μm. The object is imaged onto an EMCCD camera with a pixel pitch of Δ = 32 μm. In our experiment, t is a horizontal squaremodulation amplitude grating. The recorded intensity image in Fig. 1b shows a region of the object composed of 10 grating periods imaged with 25 rows of pixels. It is clear that the image suffers from the effect of aliasing, due to an harmonic above the Nyquist frequency of the detector array, leading to a lowfrequency Moire modulation with a period of approximately 5 pixels. If we are able to double the sampling frequency so as to superresolve the image, it is expected that we will be able to image all the harmonics and hence remove this Moire pattern.
We also measure the spatially resolved JPD of the photon pairs Γ_{ijkl} by identifying photon coincidences between any arbitrary pair of pixels (i, j) and (k, l) centred at spatial positions \(\overrightarrow{{r}_{1}}=({x}_{i},{y}_{j})\) and \(\overrightarrow{{r}_{2}}=({x}_{k},{y}_{l})\) using the method described in^{28}. The information contained in the JPD can be used for various purposes. For example, it was used in pioneering works with EMCCD cameras to estimate position and momentum correlation widths of entangled photons pairs in direct imaging of the EinsteinPodolskyRosen paradox^{21,22}. In the experiment here, the diagonal component of the JPD, Γ_{ijij} reconstructs an image of the object i.e. Γ_{ijij} = ∣t(x_{i}, y_{j})∣^{4}. Such a diagonal image is the quantity that is conventionally measured and used in all photonpairbased imaging schemes using a nearfield illumination configuration^{5,13,25,29}. As shown in Fig. 1c, measuring the diagonal image in our experiment does not however improve the image quality i.e. the Moire pattern is still present.
There is another way to retrieve an image of the object without using the diagonal component of the JPD. For that, one can project the JPD along its sumcoordinate axis, achieved by summing Γ:
where N_{Y} × N_{X} is the number of pixels of the illuminated region of the camera sensor, P^{+} is defined as the sumcoordinate projection of the JPD and (i^{+}, j^{+}) are sumcoordinate pixel indexes. Such a projection retrieves an image of the object sampled over four times the number of pixels of the sensor: \({P}_{{i}^{+}{j}^{+}}^{+}= t({x}_{i},{y}_{j}){ }^{4}\) for even pixel indices i.e. i^{+} = 2i and j^{+} = 2j, with x_{i} = iΔ and x_{j} = jΔ; \({P}_{{i}^{+}{j}^{+}}^{+}= t({x}_{i+1/2},{y}_{j+1/2}){ }^{4}\) for odd pixel indices i.e. i^{+} = 2i + 1 and j^{+} = 2j + 1, with x_{i+1/2} = x_{i} + Δ/2 and y_{j} + Δ/2. Pixel resolution is therefore increased by a factor 2 (see section JPD pixel superresolution principle for more details). Figure 1d shows the resulting sumcoordinate projection of the JPD measured in the experiment shown in Fig. 1a. Thanks to pixel superresolution, we observe that the spurious low frequency Moire modulation has been removed and the 10 grating periods are now clearly visible. As a comparison, such a highresolution image is very similar to a conventional intensity image acquired using a camera with half the pixel pitch i.e. 16 μm (Fig. 1e).
The frequency analysis of these different images is shown in Fig. 1f and provides more quantitative information about our approach. In particular, we observe the absence of a frequency peak at 0.2 in the sumcoordinate image spectrum (dashed blue), while it is present in both intensity (solid red) and JPDdiagonal (dashed black) image spectra. It is instead substituted by a peak at 0.8 that is the true frequency component of the object (harmonics), as confirmed by its presence in the spectrum of the intensity image acquired using the highresolution, 16 μmpixelpitch camera (dashed red). Removal of the aliased frequency component corresponds also to the disappearance of the Moire pattern in the real space. This confirms that our approach achieves pixel superresolution and retrieves information that was lost due to undersampling. Additional measurements provided in supplementary document section 3 are acquired using a camera of even lower resolution (48 μmpixelpitch) and show that JPD pixel superresolution can also recovers the fundamental spectral component (main peak) even when this is absent in both the intensity and diagonal images. JPD pixel superresolution is also confirmed by simulations detailed in supplementary document section 2.8.
Figure 1g shows system modulation transfer functions (MTF) calculated using the slantededge technique^{30} with different imaging modalities. The MTF obtained using the JPD sumcoordinate projection (dashed blue) is approximately 1.7 times broader than those acquired using intensity (solid red) and JPD diagonal (dashed black) images, almost matching the MTF retrieved by intensity measurement with the highresolution 16 μmpixelpitch camera (dashed red). This shows that JPD pixel superresolution not only doubles the Nyquist frequency (1/(2Δ) → 1/Δ), but also broadens the system MTF which results in less attenuation of higher frequencies. The broadening of the MTF is explained by the fact that the effective size of a ’pixel’ during a JPD measurement (i.e. the size of the surface over which the coincidences are integrated) is on average smaller than the real size of the pixels. This shows that images retrieved by JPD pixel superresolution are similar to conventional intensity images obtained with a camera that has four times more pixels (higher Nyquist frequency) but that also has smaller pixels (broader MTF) (more details about MTF measurements are provided in Methods and in supplementary document section 4.)
Interestingly, results in Fig. 1 also lead to another conclusion that, contrary to a common belief in the field, conventional imaging with photon pairs (i.e. using the JPD diagonal) does not always improve image quality compared to classical intensity imaging. For example here, the spurious Moire effect in the diagonal image (Fig. 1c) is more intense than in the direct intensity image (Fig. 1b), which is also confirmed by a higher intensity of the 0.2 frequency peak in the diagonal image spectra (Fig. 1f).
JPD pixel superresolution principle
We gain further insight into the underlying principle of JPD pixel superresolution from inspection of the JPD of a spatially entangled twophoton state: this is a 4dimensional object containing much richer information than a conventional 2D intensity image. The JPD contains correlation information not only between photons detected at the same pixel, but also correlation information about photons detected between nearestneighbour pixels.
Figure 2 illustrates this concept in the onedimensional case. In Fig. 2a, photon pairs with a correlation width σ illuminate a onedimensional object t(x) imaged onto an array of pixels with pitch Δ and pixel gap δ (i.e. the width of nonactive areas between neighbouring pixels). When measuring the JPD, there are two main contributions: (i) The first contribution originates from pairs of photons detected at the same pixel (blue rays) and form the JPD diagonal elements, Γ_{kk}. Because these pairs crossed the object around the same positions as the pixels i.e. x_{k}, the resulting image (Fig. 2b) provides a sampling of the object Γ_{kk} ~ ∣t(x_{k})∣^{4} similar to that performed by a conventional intensity measurement. (ii) The second contribution originates from photon pairs detected by nearestneighbour pixels and forms the JPD offdiagonal elements Γ_{kk+1}. Because these photons crossed the object around positions located between the pixels i.e. x_{k+1/2}, the resulting image (Fig. 2c) provides a sampling of the object Γ_{kk+1} ~ ∣t(x_{k+1/2})∣^{4} similar to an intensity measurement performed with a sensor shifted by Δ/2 in the transverse direction (see Methods for derivations of Γ_{kk} and Γ_{kk+1}).
Finally, projecting the JPD along the sumcoordinate (Eq. 1) retrieves a highresolution image (Fig. 2d) by interlacing diagonal and offdiagonal elements. To understand this recombination, one can expand Eq. 1 in the one dimensional case for even and odd pixels:
where N is the number of pixel of the sensor. In theory, when operating under the constraint σ < Δ, correlations between nonneighbouring pixels are nearly zeros i.e. Γ_{ij} ≈ 0 if ∣i − j∣ > 2. The sum terms in Eqs. 2 and 3 then become negligible compared to Γ_{kk} and Γ_{kk+1}. It leads to \({P}_{{i}^{+} = 2k}^{+}={{{\Gamma }}}_{kk}\) and \({P}_{{i}^{+} = 2k+1}^{+}=2{{{\Gamma }}}_{kk+1}\), providing the superresolved image. In practice, however, experimentally measured correlation values are noisy. All the noise then adds up in the sums in Eqs. 2 and 3, ultimately producing a noise with greater variance that dominates the diagonal and offdiagonal elements in the final image. To solve this issue, the JPD is filtered before performing the projection to remove the weakest correlation values i.e. all terms except Γ_{kk} and Γ_{kk+1} are set to zero. In doing so, we also remove the noise associated with these values, that do not add up in the sums, and significantly improve the quality of the final image (see Methods and Fig. 5 for more details).
In addition, it should be noted that JPD diagonal values (i.e. Γ_{ijij} in the two dimensional case) are not measured directly with singlephoton cameras, as most of these devices are not photonnumber resolving. To circumvent this limitation, diagonal values are estimated from correlations between either vertical (Γ_{i(j±1)ij}) or horizontal (Γ_{(i±1)jij}) directneighbouring pixels. This has the practical consequence of restricting the superresolution effect to one dimension in the complementary axis. In the general case, an image superresolved in two dimensions can still be obtained by combining two images superresolved in each direction. In the specific case of EMCCD cameras, this is however not possible because of charge smearing^{5}. This effect compromises horizontal correlations values and truly restricts the superresolution effect to one dimension along the vertical spatial axis (see Methods). These limitations are lifted when operating in a farfield illumination configuration^{14,17,27} (see section Application to quantum holography) and in quantum imaging schemes using two distinct cameras^{16,31}.
The previous analysis also shows that two experimental conditions must be verified to achieve pixel superresolution. First, the sensor fillfactor must be large enough to allow photon coincidence detection between neighbouring pixels i.e. δ < σ. Second, the spatial resolution of the imaging system must be limited by the pixel size. This is true if the higher spatial frequency component of the optical field in the object plane is both smaller than the imaging system spatial frequency cutoff and larger than the sensor Nyquist frequency 1/(2Δ). In practice, one must verify at least that σ < Δ. Otherwise, a JPD sumcoordinate projection can still be retrieved, but will not contain more information than a conventional intensity measurement. In the following, we apply our approach to real quantum imaging experiments in which the condition δ < σ < Δ is true. We note that this condition is straightforward to satisfy. Indeed, σ < Δ is the starting requirement for any form of pixel superresolution to actually make sense–without this condition, there will not even be any aliasing effects in the first place. The other condition δ < σ, is always satisfied unless the pixel fill factor is extremely low.
Applications to quantum illumination
We demonstrate our technique on two different experimental schemes based on twophoton quantum illumination protocols described in^{13} and^{27}. In the first protocol, an amplitude object t_{1} is illuminated by photon pairs and imaged onto an EMCCD camera using a similar experimental setup than this shown in Fig. 1a. Simultaneously, another object t_{2} is illuminated by a classical source and also imaged on the camera (see Methods and supplementary document section 7.1 for more details about the experimental setup). The intensity image, Fig. 3a, shows a superposition of both quantum and classical images. The goal of such protocol is to segment the quantum object and therefore retrieve an image showing only object t_{1} illuminated by photon pairs, effectively removing any classical objects or noise. Conventionally, this is achieved by measuring the diagonal image shown in Fig. 3b. Using the JPD pixel superresolution processing method, we can now perform such a task and simultaneously retrieve a pixel superresolved image of t_{1} (Fig. 3c), in which we observe the clear removal of the Moire effect due to aliasing.
In addition, we reproduced the same experiment using a higher resolution camera i.e. 16 μmpixelpitch (Fig. 3d–f). In this case, we observe that JPD pixel superresolution antialiasing is mainly visible at the edges and corners of the object, in particular with the attenuation of the socalled staircase effects. Thus, even in imaging situations where aliasing does not produce artifacts as clear as Moire effect, the JPD pixel superresolution approach still provides clear benefits in removing it and improving the overall image quality.
We also apply our technique in another quantum illumination protocol demonstrated in^{27}. This protocol performs the same task as in Fig. 3 i.e. distilling the quantum image from the classical, but uses a farfield illumination configuration, operates in reflection and detects photons with a SPAD camera (see supplementary document section 7.2 for results and experimental arrangement).
Application to quantum holography
We now demonstrate JPD pixel superresolution on two different experimental quantum holography schemes. The first approach was demonstrated in^{17} and its experimental configuration is described in Fig. 4a. Pairs of photons entangled in space and polarisation illuminate an SLM and a birefringent phase object t_{6} positioned in each half of an optical plane using a farfield illumination configuration. The object and SLM are then both imaged onto two different parts of an EMCCD camera.
In such a farfield configuration, the information of the JPD is now concentrated around the antidiagonal (i.e. Γ_{ij −i −j}) because photon pairs are spatially anticorrelated in the object plane^{32}. This antidiagonal is the quantity that is conventionally measured and used in all photonpairbased imaging schemes that use a farfield illumination configuration^{14,17,27}. In this case, the JPD pixel superresolution technique must be adapted by using the minuscoordinate projection P^{−} of the filtered JPD in place of the sumcoordinate projection P^{+} to retrieve the highresolution image (see Methods).
The object considered here is a section of a bird feather, shown in Fig. 4b. The SLM is used to compensate for optical aberrations and implement a four phaseshifting holographic process by displaying uniform phase patterns with values {0, π/2, π, 3π/2} (see Methods). In the original protocol, four different images are obtained for each step of the process by measuring the antidiagonal component Γ_{ij −i −j} of the JPD. These images are then combined numerically to reconstruct the spatial phase of the birefringent object (Fig. 4c). Using the new JPD pixel superresolution approach, the four images are now obtained from the minuscoordinate projection P^{−} of the JPD and recombined to retrieve a phase image with improved spatial resolution (Fig. 4d). We do not observe the clear removal of aliasing artefacts in the superresolved image as in Figs. 1d and 3c due to the relatively smooth shape of the bird feather that is mostly composed of low frequencies below the Nyquist limit. Resolution improvements are therefore mainly located at the edges and corners, that visually translates into an overall improvement of the image quality.
The second approach is a fullfield version of a N00Nstate holographic scheme based on photon pairs (N = 2). N00N states are known for providing phase measurements with Ntimes better sensitivity than classical holography^{7}. Our results show that a fullfield version of N00Nstate holography with photon pairs not only preserves the twofold sensitivity enhancement, but also provides a better pixel resolution, an advantage that could only be matched by a scanning approach^{6} that would also increase the acquisition time by a factor of four (see supplementary document section 6 for experimental layout and results).
Discussion
We have introduced a pixel superresolution technique based on the measurement of a spatially resolved JPD of spatially entangled photons. This approach retrieves spatial information about an object that is lost due to pixelation, without shifting optical elements or relying on prior knowledge. We demonstrated that this JPD pixel superresolution approach can improve the spatial resolution in already established quantum illumination and entanglementenabled holography schemes, as well as in a fullfield version of N00Nstate quantum holography, using different types of illumination (nearfield and farfield) and different singlephotonsensitive cameras (EMCCD and SPAD). Our approach has the advantage that it can be used immediately in quantum imaging schemes based on photon pairs, and even in some cases by only reprocessing already acquired data. In addition, our approach can also be implemented in any classical imaging system limited by pixelation, after substituting the classical source by a source of correlated photons with similar properties. Indeed, contrary to classical pixel superresolution techniques, such as shiftandadd approaches^{33}, wavelength scanning^{34} and machinelearningbased algorithms^{35}, the JPD pixel superresolution approach has the advantage that it does not require displacing optical elements in the system or having prior knowledge about the object being imaged. Although we experimentally demonstrated this technique for the case of p = 2 (photon pair) entanglement, we anticipate that our approach could be generalised for all p to further increase the pixel resolution. Photon pair sources are without doubt the current experimental choice in any given lab but recent efforts have shown promising progress towards direct generation of spatially entangled threephoton states^{36}. We also underline that although the schemes shown here used spatially entangled photons, strictly speaking it is not entanglement but only spatial correlations that are used to generate the JPD. This opens the intriguing prospect for future work to investigate the potential of classical sources of light, e.g. thermal light, to achieve similar pixel superresolution as shown here but with ready access to p > 2 JPDs.
Methods
Experimental layouts
Experiment in Fig. 1a
BBO crystal has 0.5 × 5 × 5 mm and is cut for type I SPDC at 405 nm with a half opening angle of 3 degrees (Newlight Photonics). It is slightly rotated around horizontal axis to ensure nearcollinear phase matching of photons at the output (i.e. ring collapsed into a disk). The pump is a continuouswave laser at 405 nm (Coherent OBISLX) with an output power of approximately 200 mW and a beam diameter of 0.8 ± 0.1 mm. A 650 nmcutoff longpass filter is used to block pump photons after the crystals, together with a bandpass filter centred at 810 ± 5 nm. The camera is an EMCCD (Andor Ixon Ultra 897) that operates at −60 ^{∘}C, with a horizontal pixel shift readout rate of 17 MHz, a vertical pixel shift every 0.3 μs, a vertical clock amplitude voltage of +4V above the factory setting and an amplification gain set to 1000. The camera sensor has a total 512 × 512 pixels with 16 μm pixel pitch and nearly unity fill factor. In Fig. 1b–d, the camera is operated with a pixel pitch of Δ = 32 μm by using a 2 × 2 binning. In Fig. 1e, the camera is operated with a pixel pitch of 16 μm. Exposure time is set to 2 ms. The camera speed is about 100 frames per second (fps) using a region of interest of 100 × 100 pixels. The twolens imaging system f_{1}−f_{2} in Fig. 1a is represented by two lenses for clarity, but is composed of a series of six lenses with focal lengths 45, 75, 50, 150, 100, 150 mm. The first and the last lens are positioned at focal distances from the crystal and the object, respectively, and the distance between two lenses in a row equals the sum of their focal lengths. Similarly, the second twolens imaging system f_{3} − f_{4} in Fig. 1a is composed of a series of four consecutive lenses with focal lengths 150, 50, 75, 100 mm arranged as in the previous case. The imaging system magnification is 3.3. The photon correlation width in the camera plane is estimated as σ ≈ 13 μm.
Experiment used to acquire images in Fig. 3
The experimental setup is the same as this shown in Fig. 1a, with some changes in the lenses used and the addition of an external arm to superimposed the classical image. It is shown in Fig. 15a of the supplementary document. The output surface of the crystal is imaged onto an object t_{1} using a twolenses imaging system with focal lengths f_{5} = 35 mm and f_{6} = 75 mm. The object is then imaged onto the camera using a singlelens imaging system composed of one lens with focal length f_{7} = 50 mm positioned at a distance of 100mm from the object and the camera. Another object t_{2} is inserted and illuminated by a spatially filtered lightemitting diode (LED) and spectrally filtered at 810 ± 5 nm. Images of both objects are superimposed on the camera using a beam splitter. t_{1} and t_{2} are negative and positive amplitude USAF resolution charts, respectively. They are shown in Figs. 15b and c of the supplementary document. Exposure time is set to 6 ms. The imaging system magnification 2.1. The biphoton correlation width in the camera plane is estimated as σ ≈ 8 μm. 6.10^{6} frames were acquired to retrieve intensity image and JPD in approximately 20 h of acquisition. The same setup was used in^{13}. More details in section 7 of the supplementary document.
Experiment in Fig. 4a
The paired set of stacked BBO crystals have dimensions of 0.5 × 5 × 5 mm each and are cut for type I SPDC at 405 nm. They are optically contacted with one crystal rotated by 90 degrees about the axis normal to the incidence face. Both crystals are slightly rotated around horizontal and vertical axis to ensure nearcollinear phase matching of photons at the output (i.e. rings collapsed into disks). The pump laser and camera are the same than in Fig. 1a. A 650 nmcutoff longpass filter is used to block pump photons after the crystals, together with a bandpass filter centred at 810 ± 5 nm. The SLM is a phase only modulator (Holoeye Pluto2NIR015) with 1920 × 1080 pixels and a 8 μm pixel pitch. For clarity, it is represented in transmission in Fig. 4a, but was operated in reflection. Exposure time is set to 3 ms. The camera speed is about 40 frame per second using a region of interest of 200 × 200 pixels. The singlelens Fourier imaging system f_{11} is composed of a series of three lenses of focal lengths 45, 125, 150 mm. The first and last lenses are positioned at focal distance from the crystal and the objectSLM plane, respectively, and the distance between each pair of lenses equals the sum of their focal lengths. The twolens imaging system f_{12} − f_{13} is in reality composed by a series of four lenses with focal lengths 150, 75, 75, 100 mm. The first and the last lens are positioned at focal distances from respectively the SLMobject and the camera, respectively, and the distance between two lenses in a row equals the sum of their focal lengths. The imaging system effective focal length is 36 mm. The photon correlation width in the camera plane is estimated as σ ≈ 9 μm. 2.5.10^{6} frames were acquired to retrieve intensity image and JPD in each case in approximately 17 h of acquisition. The same setup was used in^{17}. More details in section 5 of the supplementary document.
JPD measurement with a camera
Γ_{ijkl}, where (i, j) and (k, l) are two arbitrary pair of pixels centred at positions \(\overrightarrow{{r}_{1}}=({x}_{i},{y}_{j})\) and \(\overrightarrow{{r}_{2}}=({x}_{k},{y}_{l})\), is measured by acquiring a set of M + 1 frames \({\{{I}^{(l)}\}}_{l\in [[1,M+1]]}\) using a fixed exposure time and then processing them using the formula:
In all the results of our work, M was on the order of 10^{6}–10^{7} frames. However, it is essential to note that not all the JPD values can be directly measured using this process. When using an EMCCD camera, (i) correlation values at the same pixel, i.e. Γ_{ijij}, cannot be directly measured because Eq. 4 is only valid for different pixels (i, j) ≠ (k, l)^{28}, and (ii) and correlation values between vertical pixels, i.e. Γ_{iji (j±q)} (where q is an integer that defines the position of a pixel above or bellow (i, j)), cannot be measured because of the presence of charge smearing effects. To circumvent this issue, these values are interpolated from neighbouring correlation values of the JPD i.e. [Γ_{ij (i+1) j} + Γ_{ij (i−1) j}]/2 → Γ_{ijij} and [Γ_{ij (i+1) (j±q)} + Γ_{ij (i−1) (j±q)}]/2 → Γ_{iji (j±q)}, as detailed in^{5}. As a result, the gain in resolution along the xaxis in the experiments using nearfield imaging configurations (Figs. 1 and 3, and Fig. 15 and 10 of the supplementary document) is not optimal. However, it is important to note that the gain in pixel resolution along the yaxis is not affected by this interpolation technique. Therefore, the spectral analyses performed in Fig. 1 are also not impacted because the gridobjects used are horizontal (no spectral component on the xaxis) and all the resulting spectral curves are obtained by summing along the xaxis. In addition, this interpolation also does not affect experiments using farfield illumination configurations in Fig. 4a and Fig. 16 of the supplementary document because the JPD diagonals do not contain any relevant imaging information (that is in the JPD antidiagonals). More details are provided in^{28} and in section 1 of the supplementary document.
JPD pixel superresolution in farfield illumination configuration
When an object is illuminated by photonpairs using a farfield illumination configuration (i.e. the crystal is Fourierimaged on the object), the JPD pixel superresolution technique must be adapted and the sumcoordinate projection P^{+} cannot be used to retrieve the highresolution image. First, instead of the diagonal, information about the object is retrieved by displaying the antidiagonal component of the JPD i.e. Γ_{ij −i −j} ≈ ∣t(x_{i}, y_{i})∣^{2}∣t(−x_{i}, −y_{i})∣^{2}. Γ_{ij −i −j} is the quantity that is conventionally measured and used in all photonpairsbased imaging schemes using a farfield illumination configuration^{14,17,27}. Second, instead of using the sumcoordinate projection, a superresolved image of the object is retrieved by integrating the JPD along its minuscoordinate axis:
where N_{Y} × N_{X} is the number of pixels of the illuminated region of the camera sensor, P^{−} is defined as the minuscoordinate projection of the JPD and (i^{−}, j^{−}) defines the sumcoordinate pixel to which a spatial variable \(\overrightarrow{{r}^{}}=({x}_{{i}^{}},{y}_{{j}^{}})\) is associated. Each value \({P}_{{i}^{}{j}^{}}^{}\) is obtained by adding all the values Γ_{ijkl} located on an antidiagonal of the JPD defined by i − k = i^{−} and j − l = j^{−} (i.e. \(\overrightarrow{{r}_{1}}\overrightarrow{{r}_{2}}=\overrightarrow{{r}^{}}\)). In theory, calculating the minuscoordinate projection of the JPD can therefore achieve pixel superresolution and potentially retrieved lost spatial information of undersampled objects. However, it is important to note that the JPD antidiagonal Γ_{ij−i−j} and minuscoordinate projection P^{−} images are not directly proportional to ∣t(x_{i}, y_{i})∣ as in the nearfield configuration, but to ∣t(x_{i}, y_{i})∣∣t(−x_{i}, −y_{i})∣, which does not always enable to retrieve ∣t∣ without ambiguity. In works using a farfield illumination configuration, this problem is solved by illuminating t with only half of the photon pairs beam (i.e. t(x, y ≤ 0) = 1). The object then appears twice in the retrieved image (object and its symmetric), but no information is lost.
JPD filtering
Figure 5a shows the sumcoordinate projection P^{+} calculated using equation (1) from the unfiltered JPD measured in Fig. 3. This image is very noisy and does not reveal the object. To solve this issue, a filtering method is applied and consists in setting to 0 all values of the JPD that have a negligible weight (i.e. values close to zero) so that their associated noise does not contribute when performing the sum. When using a nearfield illumination configuration (Figs. 1 and 3, and Figs. 10 and 15 of the supplementary document), filtering is applied by setting all JPD values to zeros except from those of the main JPD diagonal Γ_{ijij} and of the eight other diagonals directly above and bellow i.e. Γ_{ij (i±1) j}, Γ_{ij i (j±1)} and Γ_{ij (i±1) (j±1)}. When using a farfield illumination configuration (Figs. 4a and 16 of the supplementary document), filtering is applied by setting all JPD values to zeros except from those of the main JPD antidiagonal Γ_{ij −i −j} and those of the eight other antidiagonals directly above and bellow i.e. Γ_{ij (−i±1) −j}, Γ_{ij −i (−j±1)} and Γ_{ij (−i±1) (−j±1)}. diagonal Γ_{ijij} and of the eight other diagonals directly above and bellow i.e. Γ_{ij (i±1) j}, Γ_{ij i (j±1)} and Γ_{ij (i±1) (j±1)}.
Normalisation
Figure 5b shows the sumcoordinate projection P^{+} directly calculated from the filtered JPD measured in the experiment in Fig. 3 before normalisation. We observe that this image has an artefact taking the form of inhomogeneous horizontal and vertical stripes. This artefact is very similar to this commonly observed in shiftandadd superresolution techniques^{33} and is often refereed as a ’motion error artefact’. In our case, it originates from the difference in the effective detection areas of photons pairs when they are detected by the same pixel (diagonal elements) or by neighbouring pixel (offdiagonal elements), as illustrated in Fig. 2. Using an analogy with the shiftandadd technique, our problem is equivalent to a situation in which the different shifted lowresolution images were measured using cameras with different pixel width during the first step of the process. Then, when these lowresolution images are recombined into a highresolution one (second step of shiftandadd), the artifact appears in the resulting image because some pixels are less intense than others. In practice, the simplest way to remove this artifact is to normalize each lowresolution image by its total average intensity so that neighboring pixels in the highresolution image are at the same level. We use such a normalization approach in our work by dividing all values of the nonzero diagonals in the filtered JPD by their spatial average value i.e. \({{{\Gamma }}}_{ij(i+{i}^{})(j+{j}^{})}\to {{{\Gamma }}}_{ij(i+{i}^{})(j+{j}^{})}/{\sum }_{i,j}{{{\Gamma }}}_{ij(i+{i}^{})(j+{j}^{})}\), where (i^{−}, j^{−}) identifies a specific JPD diagonal. After normalization, Fig. 3f is obtained. In the case of farfield illumination, the same normalisation is applied to the values of the nonzero antidiagonals in the filtered JPD i.e. \({{{\Gamma }}}_{ij(i+{i}^{+})(j+{j}^{+})}\to {{{\Gamma }}}_{ij(i+{i}^{+})(j+{j}^{+})}/{\sum }_{i,j}{{{\Gamma }}}_{ij(i+{i}^{+})(j+{j}^{+})}\), where (i^{+}, j^{+}), where (i^{+}, j^{+}) identifies a specific JPD antidiagonal.
In some cases the artefact is reduced but still visible in the resulting image even after normalisation. The persistence of this artefact is due the fact that the difference in the effective integration areas between diagonal and offdiagonal elements is too large to be accurately corrected by simple sum normalization. For example, in the experiment shown in Fig. 10a of the supplementary document, the poor quality of the SPAD camera sensor, in particular its very low fillfactor (10.5%), is probably at the origin of the remaining artefact. To further reduce this artefact, one could use more complex normalisation algorithms, such as L_{1} or L_{2} norms minimisation approaches^{37} and kernel regression^{38}, that are commonly used in shiftandadd approaches.
Slantededge method
MTF measurements using the slantededge approach were performed with a razor blade titled by approximatively 100 mrad and positioned in the object plane of the experimental configuration shown in Fig. 1, followed by a standard method described in^{30}. Broadening of the curves is estimated by comparing spatial frequency values where MTF is 50% of its low frequency value (i.e. criteria MTF50). More details are provided in section 4 of supplementary document.
Estimation of the photon correlation width σ
Nearfield illumination configuration
The value of the correlation width in the image plane σ is obtained by calculating the positioncorrelation width at the output of the crystal using the formula \(\sqrt{\alpha L{\lambda }_{p}/(2\pi )}\) (L is the crystal thickness, λ_{p} is the pump beam wavelength and α is a parameter described in^{39}) and multiplying it by the magnification of the imaging system.
Farfield configuration
The value of σ is obtained by calculating the angularcorrelation width of photons at the output of the crystal using the formula λ_{p}/(2ω)^{39}, (ω is the pump beam waist) and multiplying it by the effective focal length of the imaging system.
In our work, values of σ are estimated using the theory and not with the experimental techniques described in^{21,22} because these approaches fail at providing an accurate result precisely when the correlation width is smaller than the pixel width. In addition, note also that these width values are not strict bounds but correspond to standard deviation widths when modelling spatial correlations with a Gaussian model^{40}. More details are provided in section 2.4 of the supplementary document.
Analytical derivation of Γ_{kk} and Γ_{kk+1}
We consider an unidimensional object t(x) illuminated by photon pairs using a nearfield illumination configuration. Photon pairs are characterised by a twophoton wavefunction Ψ_{t}(x_{1}, x_{2}) in the object plane. JPD values Γ_{kl} are measured using an array of pixels with pitch Δ and gap δ. Assuming that the imaging system is not limited by diffraction but only by the sensor pixel resolution, its point spread function can be approximate by a Dirac delta function and Γ_{kl} can be formally written as:
where we assumed unity magnification between object and camera planes. A graphical representation of this integral is shown in Fig. 5c. For clarity, we only represented an array of three pixels. The bivariate function ∣t(x_{1})t(x_{2})∣^{2} is represented in green and overlaps with an grid of squares of size Δ and spacing δ. Each square represents an integration area associated to a specific JPD value. For example, the central square corresponds to the integration area of Γ_{kk} i.e. \([{x}_{k}\frac{{{\Delta }}\delta }{2},{x}_{k}+\frac{{{\Delta }}\delta }{2}]\times [{x}_{k}\frac{{{\Delta }}\delta }{2},{x}_{k}+\frac{{{\Delta }}\delta }{2}]\). In addition, the bivariate function ∣Ψ_{t}(x_{1}, x_{2})∣^{2} is represented by two dashed black lines. These two lines delimit the most intense part of the function, which corresponds to a diagonal band of width σ using a double Gaussian model^{40}.
We seek to calculate the JPD values Γ_{kk} and Γ_{kk+1}. Graphically, these values are located at the intersection between the grid, the green area and the surface inside the dashed lines. They are represented in blue and red, respectively. For small widths σ < Δ, it is clear in Fig. 5c that the blue and red integration areas are tightening around the positions (x_{k}, x_{k}) and (x_{k+1/2}, x_{k+1/2}) positions, which results in Γ_{kk} ~ ∣t(x_{k})∣^{4} and Γ_{kk+1} ~ ∣t(x_{k+1/2})∣^{4}. More formally, one can also apply a change of variable and perform a firstorder Taylor expansion in Eq. 6 to reach the same results:
where \({S}_{0}\approx \sigma /2[1+2\sqrt{2}({{\Delta }}\sigma /\sqrt{2})]\) and S_{1} ≈ σ^{2}/2. Full calculations are provided in section 2.5 of the supplementary document. Note also that the difference between integration areas S_{1} ≠ S_{2} makes the normalization step after calculating the JPD projection necessary (see Methods section Normalization).
Data availability
The data generated in this study have been deposited in a database under accession code https://doi.org/10.5525/gla.researchdata.1269.
References
Bishara, W., Su, T.W., Coskun, A. F. & Ozcan, A. Lensfree onchip microscopy over a wide fieldofview using pixel superresolution. Opt. Express 18, 11181–11191 (2010).
Bishara, W. et al. Holographic pixel superresolution in portable lensless onchip microscopy using a fiberoptic array. Lab a Chip 11, 1276–1279 (2011).
Moreau, P.A., Toninelli, E., Gregory, T. & Padgett, M. J. Imaging with quantum states of light. Nat. Rev. Phys. 1, 367–380 (2019).
Boto, A. N. et al. Quantum interferometric optical lithography: exploiting entanglement to beat the diffraction limit. Phys. Rev. Lett. 85, 2733–2736 (2000).
Reichert, M., Defienne, H. & Fleischer, J. W. Massively parallel coincidence counting of highdimensional entangled states. Sci. Rep. 8, 7925 (2018).
Ono, T., Okamoto, R. & Takeuchi, S. An entanglementenhanced microscope. Nat. Commun. 4, 2426 (2013).
Israel, Y., Rosen, S. & Silberberg, Y. Supersensitive polarization microscopy using NOON states of light. Phys. Rev. Lett. 112, 103604 (2014).
Brida, G., Genovese, M. & Berchera, I. R. Experimental realization of subshotnoise quantum imaging. Nat. Photonics 4, 227–230 (2010).
Camphausen, R. et al. A quantumenhanced widefield phase imager. Sci. Adv. 7, eabj2155 (2021).
Pittman, T. B., Shih, Y. H., Strekalov, D. V. & Sergienko, A. V. Optical imaging by means of twophoton quantum entanglement. Phys. Rev. A 52, R3429–R3432 (1995).
Aspden, R. S., Tasca, D. S., Boyd, R. W. & Padgett, M. J. EPRbased ghost imaging using a singlephotonsensitive camera. N. J. Phys. 15, 073032 (2013).
Lopaeva, E. D. et al. Experimental realization of quantum illumination. Phys. Rev. Lett. 110, 153603 (2013).
Defienne, H., Reichert, M., Fleischer, J. W. & Faccio, D. Quantum image distillation. Sci. Adv. 5, eaax0307 (2019).
Gregory, T., Moreau, P.A., Toninelli, E. & Padgett, M. J. Imaging through noise with quantum illumination. Sci. Adv. 6, eaay2652 (2020).
Gregory, T., Moreau, P.A., Mekhail, S., Wolley, O. & Padgett, M. J. Noise rejection through an improved quantum illumination protocol. Sci. Rep. 11, 21841 (2021).
Devaux, F., Mosset, A., Bassignot, F. & Lantz, E. Quantum holography with biphotons of high Schmidt number. Phys. Rev. A 99, 033854 (2019).
Defienne, H., Ndagano, B., Lyons, A. & Faccio, D. Polarization entanglementenabled quantum holography. Nat. Phys. 17, 591–597 (2021).
Lemos, G. B. et al. Quantum imaging with undetected photons. Nature 512, 409–412 (2014).
Kviatkovsky, I., Chrzanowski, H. M., Avery, E. G., Bartolomaeus, H. & Ramelow, S. Microscopy with undetected photons in the midinfrared. Sci. Adv. 6, eabd0264 (2020).
Vanselow, A. et al. Frequencydomain optical coherence tomography with undetected midinfrared photons. Optica 7, 1729–1736 (2020).
Moreau, P.A., MouginSisini, J., Devaux, F. & Lantz, E. Realization of the purely spatial EinsteinPodolskyRosen paradox in fullfield images of spontaneous parametric downconversion. Phys. Rev. A 86, 010101 (2012).
Edgar, M. P. et al. Imaging highdimensional spatial entanglement with a camera. Nat. Commun. 3, 984 (2012).
Chrapkiewicz, R., Wasilewski, W. & Banaszek, K. Highfidelity spatially resolved multiphoton counting for quantum imaging applications. Opt. Lett. 39, 5090–5093 (2014).
Unternährer, M., Bessire, B., Gasparini, L., Stoppa, D. & Stefanov, A. Coincidence detection of spatially correlated photon pairs with a monolithic timeresolving detector array. Opt. Express 24, 28829–28841 (2016).
Toninelli, E. et al. Resolutionenhanced quantum imaging by centroid estimation of biphotons. Optica 6, 347–353 (2019).
Chrapkiewicz, R., Jachura, M., Banaszek, K. & Wasilewski, W. Hologram of a single photon. Nat. Photonics 10, 576–579 (2016).
Defienne, H., Zhao, J., Charbon, E. & Faccio, D. Fullfield quantum imaging with a singlephoton avalanche diode camera. Phys. Rev. A 103, 042608 (2021).
Defienne, H., Reichert, M. & Fleischer, J. W. General model of photonpair detection with an image sensor. Phys. Rev. Lett. 120, 203604 (2018).
Reichert, M., Defienne, H., Sun, X. & Fleischer, J. W. Biphoton transmission through nonunitary objects. J. Opt. 19, 044004 (2017).
Burns, P. D. et al. Slantededge mtf for digital camera and scanner analysis. In Is and Ts Pics Conference, 135–138 (Society for Imaging Science & Technology, 2000).
Devaux, F., Mosset, A., Moreau, P.A. & Lantz, E. Imaging spatiotemporal HongOuMandel interference of Biphoton states of extremely high Schmidt number. Phys. Rev. X 10, 031031 (2020). Publisher: APS.
Schneeloch, J. & Howell, J. C. Introduction to the transverse spatial correlations in spontaneous parametric downconversion through the biphoton birth zone. J. Opt. 18, 053501 (2016).
Farsiu, S., Robinson, M., Elad, M. & Milanfar, P. Fast and robust multiframe super resolution. IEEE Trans. Image Process. 13, 1327–1344 (2004).
Luo, W., Zhang, Y., Feizi, A., Göröcs, Z. & Ozcan, A. Pixel superresolution using wavelength scanning. Light.: Sci. Appl. 5, e16060–e16060 (2016).
Shi, W. et al. RealTime Single Image and Video SuperResolution Using an Efficient SubPixel Convolutional Neural Network. 1874–1883 (2016). https://www.cvfoundation.org/openaccess/content_cvpr_2016/html/Shi_RealTime_Single_Image_CVPR_2016_paper.html.
Borshchevskaya, N. A., Katamadze, K. G., Kulik, S. P. & Fedorov, M. V. Threephoton generation by means of thirdorder spontaneous parametric downconversion in bulk crystals. Laser Phys. Lett. 12, 115404 (2015).
Farsiu, S., Elad, M. & Milanfar, P. A practical approach to superresolution. In Visual Communications and Image Processing 2006, vol. 6077, 607703 (International Society for Optics and Photonics, 2006). https://www.spiedigitallibrary.org/conferenceproceedingsofspie/6077/607703/Apracticalapproachtosuperresolution/10.1117/12.644391.short.
Takeda, H., Farsiu, S. & Milanfar, P. Kernel regression for image processing and reconstruction. IEEE Trans. Image Process. 16, 349–366 (2007).
Chan, K. W., Torres, J. P. & Eberly, J. H. Transverse entanglement migration in Hilbert space. Phys. Rev. A 75, 050101 (2007).
Fedorov, M. V., Mikhailova, Y. M. & Volkov, P. A. Gaussian modelling and Schmidt modes of SPDC biphoton states. J. Phys. B: At., Mol. Optical Phys. 42, 175503 (2009).
Acknowledgements
D.F. acknowledges financial support from the Royal Academy of Engineering Chair in Emerging Technology, UK Engineering and Physical Sciences Research Council (grants EP/T00097X/1 and EP/R030081/1) and from the European Union’s Horizon 2020 research and innovation programme under grant agreement No 801060. H.D. acknowledges funding from the European Union’s Horizon 2020 research and innovation programme under the Marie SkłodowskaCurie grant agreement No. 840958. JZ has received funding from the European Union’s Horizon 2020 research and innovation programme under the Marie SkłodowskaCurie Grant Agreement No. 754354.
Author information
Authors and Affiliations
Contributions
H.D. conceived the original idea, designed and performed experiments, analysed the data and prepared the manuscript. P.C. performed the slantededge experiment. P.C. and H.D. performed the simulations. M.R. and J.F. contributed to developing the quantum illumination protocol with the EMCCD camera. J.Z. and E.C. contributed to developing the quantum illumination protocol with the SPAD camera. A.L. and B.N. contributed to developing the entanglementenabled quantum holography protocol. A.R.H. participated to the data analysis. All authors contributed to the manuscript. D.F. supervised the project.
Corresponding authors
Ethics declarations
Competing interests
The authors declare that there are no conflicts of interest related to this article. For the sake of transparency, the authors would like to disclose that E.C. holds the position of Chief Scientific Officer of Fastree3D, a company which is active in LIDAR and consumer electronics, and that he is cofounder of Pi Imaging Technology. Both companies have not been involved with the paper drafting, and at the time of writing have no commercial interests related to this article.
Peer review
Peer review information
Nature Communications thanks Eric Lantz and the other anonymous reviewers for their contribution to the peer review of this work.
Additional information
Publisher’s note Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
Supplementary information
Rights and permissions
Open Access This article is licensed under a Creative Commons Attribution 4.0 International License, which permits use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons license, and indicate if changes were made. The images or other third party material in this article are included in the article’s Creative Commons license, unless indicated otherwise in a credit line to the material. If material is not included in the article’s Creative Commons license and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder. To view a copy of this license, visit http://creativecommons.org/licenses/by/4.0/.
About this article
Cite this article
Defienne, H., Cameron, P., Ndagano, B. et al. Pixel superresolution with spatially entangled photons. Nat Commun 13, 3566 (2022). https://doi.org/10.1038/s41467022310526
Received:
Accepted:
Published:
DOI: https://doi.org/10.1038/s41467022310526
This article is cited by

A comparison between the measurement of quantum spatial correlations using qCMOS photonnumber resolving and electron multiplying CCD camera technologies
Scientific Reports (2024)

Quantum hyperentangled system with multiple qubits based on spontaneous parametric downconversion and birefringence effect
Optical and Quantum Electronics (2024)

Interferometric imaging of amplitude and phase of spatial biphoton states
Nature Photonics (2023)

Quantum microscopy of cells at the Heisenberg limit
Nature Communications (2023)

Quantum imaging with a photon counting camera
Scientific Reports (2022)
Comments
By submitting a comment you agree to abide by our Terms and Community Guidelines. If you find something abusive or that does not comply with our terms or guidelines please flag it as inappropriate.