Skip to main content

Thank you for visiting nature.com. You are using a browser version with limited support for CSS. To obtain the best experience, we recommend you use a more up to date browser (or turn off compatibility mode in Internet Explorer). In the meantime, to ensure continued support, we are displaying the site without styles and JavaScript.

Single-layer spatial analog meta-processor for imaging processing

Abstract

Computational meta-optics brings a twist on the accelerating hardware with the benefits of ultrafast speed, ultra-low power consumption, and parallel information processing in versatile applications. Recent advent of metasurfaces have enabled the full manipulation of electromagnetic waves within subwavelength scales, promising the multifunctional, high-throughput, compact and flat optical processors. In this trend, metasurfaces with nonlocality or multi-layer structures are proposed to perform analog optical computations based on Green’s function or Fourier transform, intrinsically constrained by limited operations or large footprints/volume. Here, we showcase a Fourier-based metaprocessor to impart customized highly flexible transfer functions for analog computing upon our single-layer Huygens’ metasurface. Basic mathematical operations, including differentiation and cross-correlation, are performed by directly modulating complex wavefronts in spatial Fourier domain, facilitating edge detection and pattern recognition of various image processing. Our work substantiates an ultracompact and powerful kernel processor, which could find important applications for optical analog computing and image processing.

Introduction

High-speed and high-efficiency computation is central in many modern technologies. The digital computation based on electron flows in microelectronic circuits relies on analog-to-digital conversions, which generally suffers from high energy consumption, low operation speed, and systematic complexity1, making it challenging especially for massive data processing. Instead, analog computing directly performs mathematical operations in parallel and thus avoids converting massive data to discretized bits. Traditional analog computers take various forms of mechanical, electronic, or hybrid devices and have large size and slow responses2,3,4, hindering their readiness of ultrafast computations. Recently, all-optical analog computing provides an alternative computational platform at the speed of light with low power consumption, and could be integrated into photonic chips with thin and planar profiles as well as enhanced compactness5, which is thus widely explored as the next-generation computation tools.

Fundamentally, analog optical computation can be implemented in temporal or spatial domain. Time-domain analog calculations process pulse signals from limited input ports6,7. In stark contrast, spatial analog computing modulates incident wavefronts in real space, enabling massive and high-throughput parallel operations for required signal-processing tasks such as spatial differentiation8,9,10, integration11 and solving equations12,13. Conventional physical architectures of spatial domain analog computers leverage upon the phase accumulation with stacked or series of optical elements14,15, making the whole system bulky and lossy. Nevertheless, metasurface has been captivated as a popular notion and a promising candidate for highly efficient, compact and ultrathin analog processors16,17,18.

For instance, an inverse-designed computational meta-structure is proposed to solve linear integral equations19. Therein, the analog operator coupled to waveguides can directly modulate the wave (Fig. 1a), solving Fredholm integral equation in a recursive system. Nevertheless, the feedback mechanism induces the time delay and intrinsic discretization errors in signal sampling, which may also impose demanding nanofabrication requirements in higher frequencies. To directly manipulate analog input waves, Fourier systems with engineered transfer functions are employed to attain the prescribed mathematical kernel. Generically, using the classical optical 4f system, two pieces of graded refractive index lenses are required to perform direct and inverse spatial Fourier transform, while a metasurface in Fourier plane can modulate the spatial frequency (Fig. 1b)2,20,21,22. Such method offers the real-time wave-based processing mechanism but still demands bulky systems in practical implementations, especially when the overall space occupied by the 4f system is counted.

Fig. 1: Conceptual sketch of representative analog metaprocessors.
figure 1

a Solving integral equation by a closed-loop network consisting of the metamaterial kernel, directional couplers and in/out waveguides. b Spatial analog computing based on the 4f Fourier optics system. (GRIN+ indicates a conventional GRIN, operates as a Fourier transformer. GRIN− has the inverse functionality of the GRIN+, which acts as an inverse Fourier transformer). c The proposed single-layer meta-processor in this work for shrinking the bulky Fourier-based designs. d Comparison of the proposed differentiator to the recently proposed designs5,24,26,31,32,33,34.

On the other hand, exploiting Green’s function (GF) kernel to directly operate on the angular scattering spectrum of impinging waves can bypass those challenges, since it avoids the subblocks of Fourier transform. The successful proposals along this paradigm include, for example, optical differentiation with multi-layered slabs23, a nonlocal metasurface with Fano response24, the Kretschmann configuration for polaritons25, photonic crystal slab26,27 and air-glass interface for spin Hall effect28. However, the scattering spectrum of the meta-atoms for those designs is angle dependent, which necessarily decreases incident-angle range and operation diversity23 (see Supplementary Note 1 for more details). Hence, we propose a single-layer metasurface-based analog processor (Fig. 1c). Such proposal with restrictions on the fixed input and output focal length overcomes the existing issues of compactness and maneuverability of traditional bulky Fourier optical devices and shows high flexibility compared to the GF approach (Fig. 1d).

Our analog optical processor is based on single-layer Huygens’ metasurface (Fig. 2a, b). By introducing specific phase factor into the complex wavefront profile on the interface, the single-layer meta-structure herein directly tailors the transverse wavevector exerted in spatial Fourier spectrum and hence imposes the customized transfer function for analog image processing, which can essentially compress the typical 4f optical system to the 2f structure (Fig. 2c). For this purpose, we use a single-layer Huygens’ metasurface, which can facilitate the complete control of wavefronts and amplitudes via judiciously tailored electric and magnetic dipoles29,30,31, to realize the desired transfer function required by the versatile implementable kernels of mathematical operations. As the proof of concept, we demonstrated differentiation and cross-correlation for the edge detection and object recognition, in an analog and compact manner. Compared with reported state-of-the-art edge detectors5,24,27,32,33,34,35, our proposed system demonstrates a relatively superior balance between the NA and device dimension (Fig. 1d), thanks to the on-demand electromagnetic control via Huygens’ metasurfaces. Our work could enable the real-time and high-throughput parallel computing tasks, overcoming the existing integration issues of traditional bulky Fourier optical devices while performing diverse mathematical operations in contrast to GF kernels. Hence, our proposed miniaturized meta-processor revels high-performance computing and can be readily generalized for tremendous tasks in analog imaging processing36,37 and computations such as equation solvers19,38, edge detection of patterns27,35, optical memory39, machine learning40,41,42 and others.

Fig. 2: Schematic diagram of analog computational system utilizing single-layer Huygens’ metasurface.
figure 2

a Derivative operation and b cross-correlation operation performed on the input image. The inset shows the photograph of the star in the badge of HIT, which is defined as the reference image here. c The experiment set-up with a zoomed-in view of the fabricated Huygens’ metasurface (see Supplementary Note 9 for photographs of the fabricated Huygens’ metasurfaces).

Results

Design of Huygens’ metasurface processor

We firstly discuss the principle of Huygens’ metasurfaces as the image processor. As shown in Fig. 2a, the input image, Huygens’ metasurface, and the output plane are positioned at \(z=-{f}_{1}\), \(z=0\) and \(z={f}_{2}\) respectively. To avoid forward and inverse Fourier transform subblocks in bulky 4f systems, electromagnetic responses of Huygens’ metasurface image processor should be elaborately designed. Under the paraxial approximation and in Fresnel regime, the wave in output plane (dubbed as E1), regarding the impinging image (E0) through the Huygens’ metasurface (Emeta), can be expressed:

$${E}_{1}({x}_{1},{y}_{1})= -\frac{{k}^{2}}{4{\pi }^{2}{f}_{1}{f}_{2}}\exp [ik({f}_{1}+{f}_{2})]\\ \mathop{\iint}\limits _{{\Sigma }_{meta}}dx^{\prime} dy^{\prime} \left\langle \mathop{\iint }\limits_{{\Sigma }_{0}}d{x}_{0}d{y}_{0}{E}_{0}({x}_{0},{y}_{0})\exp \left\{\frac{ik}{2{f}_{1}}\left[{(x^{\prime} -{x}_{0})}^{2}+{(y^{\prime} -{y}_{0})}^{2}\right]\right\}\right\rangle \\ {E}_{meta}({x}_{0},{y}_{0})\exp \left\{\frac{ik}{2{f}_{2}}\left[{({x}_{1}-x^{\prime} )}^{2}+{({y}_{1}-y^{\prime} )}^{2}\right]\right\}$$
(1)

To directly tailor the spatial Fourier spectrum of the input signal, the aperture function on the Huygens’ metasurface is defined as:

$${E}_{meta}(x^{\prime} ,y^{\prime} )=\exp \left[-\frac{ik}{2f}\big({x^{\prime} }^{2}+{y^{\prime} }^{2}\big)\right]{E}_{H}(x^{\prime} ,y^{\prime} )$$
(2)

where \(\frac{1}{f}=\frac{1}{{f}_{1}}+\frac{1}{{f}_{2}}\) (see Supplementary Note 2 for more details). Here, the phase factor \(\exp [-\frac{ik}{2f}({x^{\prime} }^{2}+{y^{\prime} }^{2})]\) is introduced to suppress the undesired phase terms for the construction of linear convolution relationship at the output plane. Hence, EH denotes the transfer function associated with desired mathematical operations in spatial Fourier domain, which plays central roles to modulate spatial spectrum responses. For the specific wavefront impinged on our Huygens’ metasurface, the integral in Eq. (1) can be re-evaluated as:

$${E}_{1}({x}_{1},{y}_{1})= \left\{\frac{{k}^{2}}{2\pi {f}_{1}{f}_{2}}\exp [ik({f}_{1}+{f}_{2})]\exp \left[\frac{ik}{2{f}_{2}}({{x}_{1}}^{2}+{{y}_{1}}^{2})\right]\right\}\\ \times\left\{{E}_{0}\left(-\frac{{f}_{1}}{{f}_{2}}{x}_{1},-\frac{{f}_{1}}{{f}_{2}}{y}_{1}\right)\phi ({x}_{1},{y}_{1})\right\} \odot F\{{E}_{H}(x^{\prime} ,y^{\prime} )\}[{k}_{x},{k}_{y}]$$
(3)

where the additional phase factor \(\phi ({x}_{1},{y}_{1})=\exp [\frac{ik{f}_{1}}{2{{f}_{2}}^{2}}({{x}_{1}}^{2}+{{y}_{1}}^{2})]\) and \({k}_{x}=\frac{k}{{f}_{2}}{x}_{1}\;{{{{{\rm{and}}}}}}\;{k}_{y}=\frac{k}{{f}_{2}}{y}_{1}\) are spatial frequencies (wavevectors) along x and y-axes, respectively. Besides, \(\odot\) represents two-dimensional convolution operation. Essentially, this equation shows that the output analog signal is the convolution of the inversed and rescaled input image and the spatial frequency-domain \({E}_{H}(x^{\prime} ,y^{\prime} )\) on Huygens’ metasurfaces. Hence, the following ouput images are rotated by π about the origin to compare with the input ones. Via applying the convolution theorem \(F\{f(x)\odot g(x)\}=F\{f(x)\}\times F\{g(x)\}\) to Eq. (3), the output image in Fourier spectrum can be obtained by multiplying input signals with repeated Fourier transforms of EH, as:

$$F\,\{{E}_{1}({x}_{1},{y}_{1})\} =\{{{{{\mathrm{..}}}}}.\}F\,\{{E}_{0}({x}_{1},{y}_{1})\phi ({x}_{1},{y}_{1})\}\times F\,\langle F\{{E}_{H}(x^{\prime} ,y^{\prime} )\}\rangle \\ =\{{{{{\mathrm{..}}}}}.\}F\,\{{E}_{0}({x}_{1},{y}_{1})\phi ({x}_{1},{y}_{1})\}\times {E}_{H}(-x^{\prime} ,-y^{\prime} )$$
(4)

Since the definition of transfer function, which is the ratio of the output signal to input signal in Fourier spectrum, is given as \({E}_{H}(-x^{\prime} ,-y^{\prime} )=\frac{F\{{E}_{1}({x}_{1},{y}_{1})\}}{F\{{E}_{0}({x}_{1},{y}_{1})\phi ({x}_{1},{y}_{1})\}}\), EH can model the spatial frequency response of the proposed metasurface-based processor quantitively and map the output image for each possible input. For maneuverability without loss of generality, the transfer function of first-order differential operator is \({E}_{H}(x)\propto jx\) and for the integrator, transfer function can be described as \({E}_{H}(x)\propto \frac{1}{jx}\). Overall, we can superimpose the specific phase factor related with the input and output focal length \(\exp [-\frac{ik}{2f}({x^{\prime} }^{2}+{y^{\prime} }^{2})]\) on the transfer function EH algorithmically. In this manner, the proposed single-layer Huygens’ metasurface can directly implement spatial frequencies for the target output signal, which avoids auxiliary optical elements.

To fulfill the appropriate transfer function, we use the Huygens’ metasurface here as the analog processor due to its extreme control of complex wavefronts across just one single-layer rather than multi-layer metasurface designs29. By incorporating orthogonal electric and magnetic dipoles with carefully tailored resonant amplitude into meta-atoms on the interface, Huygens’ surface introduces abrupt field discontinuities to generate prescribed wavefronts30 and enables both complete amplitude and phase manipulation with 100% transmission efficiency (see more details in Supplementary Note 3). Here, the metallic Huygens’ metasurfaces are designed in microwave regime, where the dissipative losses of metals are negligible. As displayed in Fig. 3a, our Huygens’ meta-atoms are composed by two split-ring resonators at both sides31. When excited by the incident wave carrying the image information, a magnetic dipole, aligned with incident magnetic field, is induced by the split-ring resonator on one side of substrate (marked in orange in Fig. 3b)43,44, while the electric dipole is on the other side, in a form of the electric-LC resonator with main surface current flowing along the incident electric field (marked in green in Fig. 3b)45. Importantly, the light-matter interaction strength as well as induced dipole moment can be further altered by optimizing geometrical parameters (le, he and lm, hm). Figure 3c provides the numerical result of 25 optimized Huygens’ meta-atoms using the commercial software CST Microwave Studio (see Supplementary Note 3 for more details). The proposed Huygens’ meta-atoms can achieve full-coverage modulation of transmission phase quantified into 8 levels and complete transmission amplitude control discretized into 4 levels at 10 GHz, as building blocks of the desired analog image processors. Particularly, when the transmission amplitude of meta-atom equals to 0, its phase shifts contribute no variations on the transmitted wavefront. Therefore, only one Huygens’ meta-atom is designed for transmission amplitude equivalent to 0 with transmission phase of −π. Moreover, the deviations between the simulated transmission coefficient and the predesigned value result from the minimum process tolerance, especially for meta-atoms with the transmission amplitude of 0.67 and 1, which are more sensitive to the offsets of overlaps between the magnetic and electric resonance. Significantly, the control of both amplitude and phase of monochromatic waves enables the diversity of mathematical operations. The single-layer meta-processor enables a large variety of mathematical operations, including spatial shifting, high-order derivative, spectral filtering and other Fourier transform algorithms. In the following, we will show two exemplary important applications of image detection.

Fig. 3: Design of Huygens’ metasurface processor.
figure 3

a Geometric structure of the proposed elementary Huygens’ meta-atom patterned on both sides of the dielectric substrate (\({\varepsilon }_{r}=3,\,\tan \delta =0.002\)). b Surface currents and equivalent electric circuits of the proposed meta-atom. The green line with arrows and orange arrows indicate the surface electric and magnetic current respectively. c Simulated transmission amplitude and phase of 25 extracted meta-atoms by modulating the geometrical parameters le, he and lm, hm at 10 GHz.

Edge detection operation

Edge detection is commonly explored to characterize subject boundaries of an image and, in a mathematic language, is reflected in a dramatic change of the derivatives of signals due to the sudden change of objects. The simplest yet powerful way for such implementation is to check the behavior of the first-order derivative operation, which is denoted (with respect to x-axis for example):

$${E}_{H}(x^{\prime} )\propto jx^{\prime} \iff F\{{E}_{H}\}[{k}_{x}]\propto \frac{d}{d{k}_{x}}\delta ({k}_{x})$$
(5)

According to Eqs. (3) and (5), the first-order differentiation of the relevant input image \({E}_{0}(-{x}_{1})\phi (-{x}_{1})\) in spatial domain maps the multiplication of \(jx^{\prime}\)in spatial Fourier spectrum. By modulating the amplitude and phase distributions of transmitted wavefronts according to Eq. (4), the absolute value of the output complex image given as:

$$|{E}_{1}({x}_{1},{y}_{1})| \propto \left|\frac{d\{{E}_{0}(-{x}_{1})ph(-{x}_{1})\}}{d{x}_{1}}\right|=\left|\frac{d\left\{{E}_{0}(-{x}_{1})\exp \left(\frac{ik}{2{f}_{2}}{{x}_{1}}^{2}\right)\right\}}{d{x}_{1}}\right|\\ =\sqrt{{\left[\frac{d{E}_{0}(-{x}_{1})}{d{x}_{1}}\right]}^{2}+{\left[\frac{k}{{z}_{1}}{x}_{1}{E}_{0}(-{x}_{1})\right]}^{2}}\approx \frac{d{E}_{0}(-{x}_{1})}{d{x}_{1}}$$
(6)

can be regarded as the processed results under first-order differentiation approximately. The derivation above demonstrates the spatial differentiator along x-direction. Similarly, the spatial differentiator along y-axis can be implemented by replacing variable x with y. Hence, the first-derivative operation along the orthogonal axes can be performed based on the proposed Huygens’ metasurfaces with complex wavefronts modulation, as shown in Fig. 4a, c. Figure 2c illustrates the image processing system in microwave regime. A feeding horn antenna launches a quasi-plane wave on the metal sheet with the predefined holes for the generation of the input image at z = −f1. Then, after modulating the wavefronts by Huygens’ metasurface at z = 0, the transmitted electric field in the output plane is measured at z = f2 with a near-field probe. As proof-of-concept experiment, the input image is set to be a specific rectangle with length l1 = 225 mm and width l2 = 305 mm under the focal-length conditions f1 = f2 = 100 mm. Based on Eqs. (2) and (5), the phase and amplitude profiles on Huygens’ metasurface are easily designed for edge detection in x and y-directions (see more details in Supplementary Note 4). Figure 4b, d displays the theoretical, numerical and measured electric field intensity profiles, which show an excellent agreement. One could clearly see strong signals at the edge location of images, verifying the 1D-edge detection capabilities of the proposed analog image processors based on Huygens’ metasurfaces.

Fig. 4: The first-order derivative operation.
figure 4

a Schematic diagram and b theoretical calculations (left), numerical simulations (middle) and experimental results (right) of the normalized electric field intensity distribution at the output focal length f2 = 100 mm and 10 GHz working frequency for 1D-edge detection along x-axis. c Schematic diagram and d theoretical calculations (left), numerical simulations (middle) and experimental results (right) of the normalized electric field intensity distribution at the output focal length f2 = 100 mm and 10 GHz working frequency for 1D-edge detection along y-axis. e Schematic diagram and f theoretical calculations (left), numerical simulations (middle) and experimental results (right) of normalized electric field intensity distribution at the output focal length f2 = 100 mm and 10 GHz working frequency for vertex detection. g Schematic diagram and h theoretical calculations (left), numerical simulations (middle) and experimental results (right) of normalized electric field intensity distribution at the output focal length f2 = 100 mm and 10 GHz working frequency for 2D-edge detection. i The input image of regular pentagon (left) with lp = 210 mm and output experimental results (right) of normalized electric field intensity distribution at the output focal length f2 = 100 mm and 10 GHz working frequency for 2D-edge detection. j The input image of regular hexagon (left) with lh = 175 mm and the output experimental results (right) of normalized electric field intensity distribution at the output focal length f2 = 100 mm and 10 GHz working frequency for 2D-edge detection.

Consequently, the 2D-edge and vertex detection, as shown in Fig. 4e, g, can be performed by further processing of the obtained orthogonal 1D-edge information. Firstly, based on the linearity of Fourier transform, the edge information on the x- and y-axes can be registered simultaneously by superimposing their corresponding transfer functions in spatial Fourier domain. Hence, 2D-edge detection \(\left(\left|\frac{d{E}_{0}({x}_{1})}{d{x}_{1}}+j\frac{d{E}_{0}({y}_{1})}{d{y}_{1}}\right|\right)\) can be achieved by the superposition operation on electric field profiles of Huygens’ metasurface along orthogonal axes, as:

$${E}_{H}(x^{\prime} ,y^{\prime} )={E}_{H}(x^{\prime} )+j{E}_{H}(y^{\prime} )\propto j(x^{\prime} +jy^{\prime} )$$
(7)

Secondly, by extracting the common parts of x- and y-directions edge data that coexist at the same positions, vertex detection \(\left(\left|\frac{d{E}_{0}({x}_{1})}{d{x}_{1}}\times \frac{d{E}_{0}({y}_{1})}{d{y}_{1}}\right|\right)\) can be implemented through the multiplication on 1D-edge detection Huygens’ metasurface profiles along x-axis and y-axis, as:

$${E}_{H}(x^{\prime} ,y^{\prime} )={E}_{H}(x^{\prime} )\times {E}_{H}(y^{\prime} )\propto -x^{\prime} y^{\prime}$$
(8)

Due to the direct modulation in spatial Fourier spectrum by the Huygens’ metasurface, the proposed processor can be optimized by increasing the number of meta-atoms for superior light-gathering ability and resolution (see more details in Supplementary Note 5). Hence, to detect the details of input rectangular image at 100 mm, the number of meta-atoms is set to be 100 × 140 to keep NA as high as 0.9. Due to the single-layer structure, the overall thickness of operator is ~λ/6, which ensures ultra-compactness and integration of our devices. As exhibited in Fig. 4f, h, the simulated and measured electric field intensity profiles are consistent with the theoretical results, demonstrating the 2D-edge and vertex detection functionalities of the proposed structures. Moreover, to validate the maneuverability of our proposed processor across versatile applications, the 2D-edge detection of more complex patterns, including regular pentagon and hexagon, are also implemented by increasing meta-atom number to 120 × 170. As illuminated in Fig. 4i, j, the 2D first-order derivative operations are completely performed on both images for edge information extraction. Nevertheless, in all experimental demonstrations, a minor deviation between the theoretical and measured figures can be seen, which is attributed to the limited dimensions of Huygens’ metasurfaces and wavefront discretization. The proposed processors could be further improved, and the output edge information can be more uniform via optimizing the phase- and amplitude- quantization (see Supplementary Note 6 for more details). In summary, our single-layer Huygens’ metasurface image processor can efficiently implement the edge and vertex detection techniques.

Cross-correlation operation

Note that edge detection essentially characterizes the sharp edges of objects. However, such method neglects other important details of the image, such as continuous areas of objects. Therefore, to provide global evaluations of the image, we resort to another image operation—correlation. In signal-processing, cross-correlation is a fundamental operation to measure the similarity of two images or data arrays:

$$f(x)\otimes g(x)=\int dx^{\prime} f(x^{\prime} )g^\ast (x^{\prime} -x)=f(x)\odot g^\ast (-x)$$
(9)

Here, \(\otimes\) represents the cross-correlation operation. Correspondingly, by the complex conjugate processing of \({E}_{H}(x)\), as\({E}_{H}^\ast (x)\iff F^\ast \{{E}_{H}\}[-{k}_{x}]\), the convolution relationship between the inversed input image and transfer function in Fourier domain in Eq. (3) can be transformed into cross-correlation operation:

$${E}_{1}({x}_{1},{y}_{1}) \propto {E}_{0}(-{x}_{1},-{y}_{1})\odot F^\ast \{{E}_{H}\}[-{k}_{x},-{k}_{y}]\\ ={E}_{0}(-{x}_{1},-{y}_{1})\otimes F\{{E}_{H}\}[{k}_{x},{k}_{y}]$$
(10)

In particular, the autocorrelation is defined as the cross-correlation of a function with itself. Clearly, the maximum of the autocorrelation is located at zero offset (i.e., no position shift), and its value denotes the signal energy, as \(f(0)\otimes f(0)=\int dx^{\prime} {|f(x^{\prime} )|}^{2}\). Besides, if g(x) is a shifted of f(x), since:

$$f(x)\otimes g(x)=f(x)\otimes f(x-t)=\int dx^{\prime} f(x^{\prime} ){f}^{\ast }(x^{\prime} -t+x)={R}_{ff}(x-t)$$
(11)

the peak of the cross-correlation operation will move at the same distance. According to this property, the cross-correlation operation can be utilized to look for the same features between the input image and the reference image (patterns to be detected) achieved by the predesigned aperture function on Huygens’ metasurface, which is of great potentials in various modern technologies including the auto-driving and face recognition.

To validate our metasurface-based analog computing processor, an input image is selected which contains the square #1 with size l11 = l12 = 60 mm and the square #2 with size l21 = l22 = 25 mm. Particularly, the square #1, part of the input signal, is set to be the reference image, described as the 2D rectangular function \(F\{{E}_{H}\}[{k}_{x}]=rect\left(\frac{{k}_{x}}{{L}_{x}}\right)rect\left(\frac{{k}_{y}}{{L}_{y}}\right)\). Hence, to build the reference image physically, the transfer function on Huygens’ metasurface is designed in the form as:

$${E}_{{{{{{\rm{H}}}}}}}^\ast (x^{\prime} ,y^{\prime} )={E}_{{{{{{\rm{H}}}}}}}(x^{\prime} ,y^{\prime} )\propto \left[{L}_{x}\frac{k}{{f}_{2}}{{{{{\rm{sinc}}}}}}\left({L}_{x}\frac{k}{{f}_{2}}x^{\prime} \right)\right]\times \left[{L}_{y}\frac{k}{{f}_{2}}{{{{{\rm{sinc}}}}}}\left({L}_{y}\frac{k}{{f}_{2}}y^{\prime} \right)\right]$$
(12)

where Lx = l11/2 = 30 mm, Ly = l12/2 = 30 mm and the input and output focal lengths f1 = f2 = 100 mm. By modulating the aperture function on Huygens’ metasurface based on Eq. (12), the peak-value position of the cross-correlation operation identifies the location of the reference image pattern consist in the input image. As shown in Fig. 5, the square #1 can be successfully detected in theoretical, simulated and measured results. Hence, specific patterns can be readily recognized from the analog input data by tailoring the wavefront profiles utilizing Huygens’ metasurfaces. To extend applications of the proposed computational metasurface, the input image is further designed to be one-dimensional sequence which contains two squares with width w1 = 70 mm. Then, three reference images which contains one, two or three rectangular patterns with the same width (named as ,,) are implemented in spatial Fourier spectrum respectively utilizing Huygens’ metasurfaces (see more details of transfer functions in Supplementary Note 7). As shown in Fig. 5e, the output intensity of electric field indicates the index of geometrical similarity and the location of peaks can identify the regions where input and reference images contain the same pattern features. Hence, our metasurface-based cross-correlators can also perform sequence alignment, offering opportunities for potential signal detection and even DNA sequencing.

Fig. 5: Cross-correlation operation.
figure 5

a The input image consisting of the square #1 with l11 = l12 = 60 mm and the square #2 with l21 = l22 = 25 mm at f1 = 100 mm. b The reference image of the square #1. c The phase profile (left) and the amplitude distribution (right) on the 60 × 80 Huygens’ metasurface as the physical block of the reference square #1. d Theoretical calculations (left), numerical simulations (middle) and experimental results (right) of the normalized electric field intensity distribution at the output focal length f2 = 100 mm and 10 GHz working frequency for pattern recognition of the square #1. e The cross-correlation operation between the input two-square sequence and the reference image containing one, two or three rectangular patterns with w1 = 70 mm (named as ,, respectively). Besides, The normalized intensity of output electric field along x-axis are demonstrated at focal length f1 = f2 = 600 mm and 10 GHz working frequency for sequence alignment.

The robustness of detection functions enabled by our Huygens’ metasurface for cross-correlation processing allows wide applications. Simplify searching a large database, the proposed single-layer correlator can directly map and locate the reference fragment with parallel process to decrease the power consumption and accelerate the speed of signal-processing, which are the major limitations of electrical processing to be tackled. The proposed scheme may benefit the radio-frequency sensing technologies, including locating, recognition and feature estimation. Also, the concept can be readily extended to terahertz and optical frequency ranges (see more details in Supplementary Note 8), which can find potential applications in pattern recognition, information retrieval, single particle analysis, electron tomography, and bioinformatics. Moreover, by adding tunable factors into design of Huygens’ meta-atoms, a single correlator can track and monitor versatile objects dynamically.

Discussion

In summary, we proposed a novel ultracompact analog processing system utilizing Huygens’ metasurface to overcome speed and energy limitations of digital devices. Based on Fourier optics, the output electric field intensity demonstrates the convolution of the input signal with the transfer function implemented via a Huygens’ metasurface with complex wavefronts modulation on the aperture. Significantly, the proposed single-layer processor can directly modulate the spatial Fourier spectrum without physical lenses for direct and inverse Fourier transform. To modulate the complex wavefront profiles on the interface, 25 Huygens’ meta-atoms are exploited for full and independent control of transmission amplitude and phase. By constructing the operator with proposed Huygens’ meta-atoms based on Fourier transform properties, differentiation and cross-correlation mathematical operations are performed in proof-of-concept experiments.

The analog processor uses single-layer Huygens’ metasurface to directly tailor the spatial Fourier frequencies, overcoming the volume defect and propagation loss of 4f Fourier optics system. Moreover, the proposed scheme presents a recipe of spatial arrangement of 25 static Huygens’ meta-atoms at the cost-effective computing load by GF method, allowing practical implementation of optical devices with predesigned and complex angular spectrums. Figure 1d showcases a comparative overview and the advantages of our proposed metasurface-based spatial analog processor have been unanimously depicted. Our results may pave toward high-speed, low-dimension, and high-throughput all-optical computing metasurfaces, empowering advanced technologies in analog-data processing, pattern detection and recognition, and even in bio-medical applications.

Methods

Experimental section

As demonstrated in Fig. 2c, the wavefronts generated from the illuminating horn antenna, propagate through the metal plane \(z=-{f}_{1}\) and later through the Huygens’ metasurface at \(z=0\), and finally ends at \(z={f}_{2}\) as the output profiles. The experimental equipment is established in an anechoic-type measurement chamber with the size of 310 × 310 × 650 mm3 bounded by absorbers. For the construction of the input image in microwave regime, a metal plane is perforated for the predesigned pattern apertures to allow incidence with specific shapes propagate through. The feeding horn antenna is placed 650 mm away from the metal plane sample to provide a quasi-planar wave illumination. The metal plane sample is surrounded by absorbing materials to eliminate diffraction on the borders. A fiber optic active antenna, operating as the field-sensing probe, is used to measure the transmitted electric field point by point by step of 2 mm. The feeding horn antenna and the probe are both connected to the vector network analyzer (Agilent 8722 ES) to measure the amplitude and phase of the transmission coefficient (component S21 of the scattering matrix).

Data availability

All data generated and analyzed are included in the paper and its Supplementary Information.

References

  1. Nejad, Z. F., Sounas, D. L., Alù, A. & Fleury, R. Analogue computing with metamaterials. Nat. Rev. Mater. 6, 207–225 (2020).

  2. Silva, A. et al. Performing mathematical operations with metamaterials. Science 343, 160–163 (2014).

    ADS  MathSciNet  CAS  MATH  Article  PubMed  Google Scholar 

  3. Clymer, A. B. The mechanical analog computers of hannibal ford and william newell. IEEE Ann. Hist. Comput. 15, 19–34 (1993).

    Article  Google Scholar 

  4. Solla Price, D. A history of calculating machines. IEEE Micro 4, 22–52 (1984).

    Article  Google Scholar 

  5. Zhu, T., Zhou, Y., Lou, Y., Ye, H. & Fan, S. Plasmonic computing of spatial differentiation. Nat. Commun. 8, 15391 (2017).

    ADS  CAS  Article  PubMed  PubMed Central  Google Scholar 

  6. Liu, F., Wang, T., Qiang, L., Ye, T. & Su, Y. Compact optical temporal differentiator based on silicon microring resonator. Opt. Express 16, 15880–15886 (2008).

    ADS  CAS  Article  PubMed  Google Scholar 

  7. Berger, N. K. et al. Temporal differentiation of optical signals using a phase-shifted fiber Bragg grating. Opt. Express 15, 371–381 (2007).

    ADS  Article  PubMed  Google Scholar 

  8. Abdollahramezani, S., Arik, K., Khavasi, A. & Kavehvash, Z. Analog computing using graphene-based metalines. Opt. Lett. 40, 5239–5242 (2015).

    ADS  CAS  Article  PubMed  Google Scholar 

  9. Chizari, A., Abdollahramezani, S., Jamali, M. V. & Salehi, J. A. Analog optical computing based on dielectric meta-reflect-array. Opt. Lett. 41, 3451–3454 (2016).

    ADS  Article  PubMed  Google Scholar 

  10. Youssefi, A., Nejad, Z. F., Abdollahramezani, S. & Khavasi, A. Analog computing by brewster effect. Opt. Lett. 41, 3467–3470 (2016).

    ADS  Article  PubMed  Google Scholar 

  11. Golovastikov, N. V., Bykov, D. A., Doskolovich, L. L. & Bezus, E. A. Spatial optical integrator based on phase-shifted bragg gratings. Opt. Commun. 338, 457–460 (2015).

    ADS  CAS  Article  Google Scholar 

  12. Abdollahramezani, S., Chizari, A., Dorche, A. E., Jamali, M. V. & Salehi, J. A. Dielectric metasurfaces solve differential and integro-differential equations. Opt. Lett. 42, 1197–1200 (2017).

    ADS  CAS  Article  PubMed  Google Scholar 

  13. Zhang, W., Qu, C. & Zhang, X. Solving constant-coefficient differential equations with dielectric metamaterials. J. Opt. 18, 075102 (2016).

    ADS  Article  Google Scholar 

  14. Luo, Y. et al. Computational imaging without a computer: seeing through random diffusers at the speed of light. eLight 2, 4 (2022).

    Article  Google Scholar 

  15. Wang, Z., Liu, J., Ding, X., Zhao, W. & Wu, Q. Three-dimensional microwave holography based on broadband huygens’ metasurface. Phys. Rev. Appl. 13, 014033 (2020).

    ADS  CAS  Article  Google Scholar 

  16. Zhou, J. et al. Metasurface enabled quantum edge detection. Sci. Adv. 6, eabc4385 (2020).

    ADS  CAS  Article  PubMed  PubMed Central  Google Scholar 

  17. Chen, K. et al. Directional Janus metasurface. Adv. Mater. 32, 1906352 (2020).

    CAS  Article  Google Scholar 

  18. Yu, N., Genevet, P., Kats, M. A. & Aieta, F. Light propagation with phase discontinuities: generalized laws of reflection and refraction. Science 334, 333–337 (2011).

    ADS  CAS  Article  PubMed  Google Scholar 

  19. Estakhri, N. M., Edwards, B. & Engheta, N. Inverse-designed metastructures that solve equations. Science 363, 1333–1338 (2019).

    ADS  MathSciNet  MATH  Article  CAS  Google Scholar 

  20. Pors, A., Nielsen, M. G. & Bozhevolnyi, S. I. Analog computing using reflective plasmonic metasurfaces. Nano Lett. 15, 791–797 (2015).

    ADS  CAS  Article  PubMed  Google Scholar 

  21. Huo, P., Zhang, C., Zhu, W., Liu, M. & Xu, T. Photonic spin-multiplexing metasurface for switchable spiral phase contrast imaging. Nano Lett. 20, 2791–2798 (2020).

    ADS  CAS  Article  PubMed  PubMed Central  Google Scholar 

  22. Zhou, J., Qian, H., Chen, C. F., Zhao, J. & Liu, Z. Optical edge detection based on high-efficiency dielectric metasurface. Proc. Natl Acad. Sci. USA 116, 11137–11140 (2019).

    ADS  CAS  Article  PubMed  PubMed Central  Google Scholar 

  23. Wu, W., Jiang, W., Yang, J., Gong, S. & Ma, Y. Multilayered analog optical differentiating device: performance analysis on structural parameters. Opt. Lett. 42, 5270–5273 (2017).

    ADS  CAS  Article  PubMed  Google Scholar 

  24. Kwon, H., Sounas, D., Cordaro, A., Polman, A. & Andrea, Alù Nonlocal metasurfaces for optical signal processing. Phys. Rev. Lett. 121, 173004 (2018).

    ADS  CAS  Article  PubMed  Google Scholar 

  25. Lee, D. et al. Hyperbolic metamaterials: fusing artificial structures to natural 2D materials. eLight 2, 1 (2022).

    Article  Google Scholar 

  26. Guo, C., Xiao, M., Minkov, M., Shi, Y. & Fan, S. Photonic crystal slab laplace operator for image differentiation. Optica 5, 251–256 (2018).

    ADS  Article  Google Scholar 

  27. Cordaro, A., Kwon, H., Sounas, D., Koenderink, A. F. & Polman, A. High-index dielectric metasurfaces performing mathematical operations. Nano Lett. 19, 8418–8423 (2019).

    ADS  CAS  Article  PubMed  PubMed Central  Google Scholar 

  28. Zhu, T. et al. Generalized spatial differentiation from the spin hall effect of light and its application in image processing of edge detection. Phys. Rev. Lett. 11, 034043 (2019).

    CAS  Google Scholar 

  29. Hu, G., Wang, M., Mazor, Y., Qiu, C. W. & Alù, A. Tailoring light with layered and Moiré metasurfaces. Trends Chem. 3, 342–358 (2021).

    Article  Google Scholar 

  30. Pfeiffer, C. & Grbic, A. Metamaterial huygens’ surfaces: tailoring wave fronts with reflectionless sheets. Phys. Rev. Lett. 110, 197401 (2013).

    ADS  Article  CAS  PubMed  Google Scholar 

  31. Zhu, B. O. & Feng, Y. Passive metasurface for reflectionless and arbitary control of electromagnetic wave transmission. IEEE Trans. Antennas Propag. 63, 5500–5511 (2015).

    ADS  MATH  Article  Google Scholar 

  32. Xue, W. & Miller, O. D. High-NA optical edge detection via optimized multilayer films. J. Opt. 23, 125004 (2021).

  33. Cordaro, A. et al. High-index dielectric metasurfaces performing mathematical operations. Nano Lett. 19, 8418–8423 (2019).

    ADS  CAS  Article  PubMed  PubMed Central  Google Scholar 

  34. Kwon, H., Cordaro, A., Sounas, D., Polman, A. & Alù, A. Dual-polarization analog 2D image processing with nonlocal metasurfaces. ACS Photonics 7, 1799–1805 (2020).

    CAS  Article  Google Scholar 

  35. Zhou, Y., Zheng, H., Kravchenko, I. I. & Valentine, J. Flat optics for image differentiation. Nat. Photon. 14, 1–8 (2020).

    Article  CAS  Google Scholar 

  36. Chang, X., Bian, L. & Zhang, J. Large-scale phase retrieval. eLight 1, 4 (2021).

    Article  Google Scholar 

  37. Beckers, G. L., Young, I. T. & Vliet, L. J. V. A nonlinear laplace operator as edge detector in noisy images. Comput. Vis. Graph. Image Process. 45, 167–195 (1989).

    Article  Google Scholar 

  38. Piggott, A. Y. et al. Inverse design and demonstration of a compact and broadband on-chip wavelength demultiplexer. Nat. Photon. 9, 374–377 (2015).

    ADS  CAS  Article  Google Scholar 

  39. Ferrera, M., Park, Y., Razzari, L., Little, B. E. & Azaa, J. On-chip cmos-compatible all-optical integrator. Nat. Commun. 1, 29 (2010).

    ADS  CAS  Article  PubMed  Google Scholar 

  40. Woods, D. & Naughton, T. J. Optical computing: photonic neural networks. Nat. Phys. 8, 257–259 (2012).

    CAS  Article  Google Scholar 

  41. Brunner, D., Soriano, M. C., Mirasso, C. R. & Fischer, I. Parallel photonic information processing at gigabyte per second data rates using transient states. Nat. Commun. 4, 1364 (2013).

    ADS  Article  CAS  PubMed  Google Scholar 

  42. Chen, Z. & Segev, M. Highlighting photonics: looking into the next decade. eLight 1, 2 (2021).

    Article  Google Scholar 

  43. Pendry, J. B., Holden, A. J., Robbins, D. J. & Stewart, W. J. Magnetism from conductors and enhanced nonlinear phenomena. IEEE Trans. Microw. Theory Tech. 47, 2075–2084 (1999).

    ADS  Article  Google Scholar 

  44. Ding, X. et al. Metasurface holographic image projection based on mathematical properties of Fourier transform. PhotoniX 1, 1–12 (2020).

    Article  Google Scholar 

  45. Schurig, D., Mock, J. J. & Smith, D. R. Electric-field-coupled resonators for negative permittivity metamaterials. Appl. Phys. Lett. 88, 041109 (2006).

    ADS  Article  CAS  Google Scholar 

Download references

Acknowledgements

X.D. acknowledges funding from National Key R&D Program of China (2021YFF0603500), Natural Science Foundation of Heilongjiang Province (YQ2021F004). G.H. acknowledges the support from A*STAR under its AME Young Individual Research Grants (YIRG, No. A2084c0172). H.L. acknowledges funding from National Natural Science Foundation of China under Grant No. 61805057. J.L. acknowledges funding from National Natural Science Foundation of China under Grant No. 5197050993. C.W.Q. acknowledges the support by AME Individual Research Grant (IRG) funded by A*STAR, Singapore (Grant No. A2083c0060). C.W.Q. is also supported by a grant (R-261-518-004-720 | A-0005947-16-00) from Advanced Research and Technology Innovation Centre (ARTIC).

Author information

Authors and Affiliations

Authors

Contributions

X.D., G.H., and C.W.Q. conceived the idea. Z.W. and X.W. conducted the numerical simulations and fabricated the samples. S.N.B. performed the measurements. Z.W., G.H., X.D., H.L., K.Z., and S.N.B. wrote the manuscript. Q.W., J.L., J.T., and C.W.Q. supervised the overall projects. All the authors analyzed the data and discussed the results. The authors read and approved the final manuscript.

Corresponding authors

Correspondence to Xumin Ding, Kuang Zhang, Haoyu Li, Shah Nawaz Burokur or Cheng-Wei Qiu.

Ethics declarations

Competing interests

The authors declare no competing interests.

Peer review

Peer review information

Nature Communications thanks Hailu Luo, Junsuk Rho and the other, anonymous, reviewer(s) for their contribution to the peer review of this work. Peer reviewer reports are available.

Additional information

Publisher’s note Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Supplementary information

Rights and permissions

Open Access This article is licensed under a Creative Commons Attribution 4.0 International License, which permits use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons license, and indicate if changes were made. The images or other third party material in this article are included in the article’s Creative Commons license, unless indicated otherwise in a credit line to the material. If material is not included in the article’s Creative Commons license and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder. To view a copy of this license, visit http://creativecommons.org/licenses/by/4.0/.

Reprints and Permissions

About this article

Verify currency and authenticity via CrossMark

Cite this article

Wang, Z., Hu, G., Wang, X. et al. Single-layer spatial analog meta-processor for imaging processing. Nat Commun 13, 2188 (2022). https://doi.org/10.1038/s41467-022-29732-4

Download citation

  • Received:

  • Accepted:

  • Published:

  • DOI: https://doi.org/10.1038/s41467-022-29732-4

Further reading

Comments

By submitting a comment you agree to abide by our Terms and Community Guidelines. If you find something abusive or that does not comply with our terms or guidelines please flag it as inappropriate.

Search

Quick links

Nature Briefing

Sign up for the Nature Briefing newsletter — what matters in science, free to your inbox daily.

Get the most important science stories of the day, free in your inbox. Sign up for Nature Briefing