Electromagnetic wave-based extreme deep learning with nonlinear time-Floquet entanglement

Wave-based analog signal processing holds the promise of extremely fast, on-the-fly, power-efficient data processing, occurring as a wave propagates through an artificially engineered medium. Yet, due to the fundamentally weak non-linearities of traditional electromagnetic materials, such analog processors have been so far largely confined to simple linear projections such as image edge detection or matrix multiplications. Complex neuromorphic computing tasks, which inherently require strong non-linearities, have so far remained out-of-reach of wave-based solutions, with a few attempts that implemented non-linearities on the digital front, or used weak and inflexible non-linear sensors, restraining the learning performance. Here, we tackle this issue by demonstrating the relevance of time-Floquet physics to induce a strong non-linear entanglement between signal inputs at different frequencies, enabling a power-efficient and versatile wave platform for analog extreme deep learning involving a single, uniformly modulated dielectric layer and a scattering medium. We prove the efficiency of the method for extreme learning machines and reservoir computing to solve a range of challenging learning tasks, from forecasting chaotic time series to the simultaneous classification of distinct datasets. Our results open the way for optical wave-based machine learning with high energy efficiency, speed and scalability.


Time-Floquet system
The transfer matrix equation relating the amplitudes of the fields on opposite sides of the time-Floquet system (See Supplementary Figure 1) can be expressed in multiplicative form in time-domain for each excitation frequency as whereΨ(ω k , t) is the time-varying transfer matrix. The transfer matrix equation can be taken into angular frequency domain by taking the Fourier transform of both sides as: Equation (2) implies that an input frequency ω k will be converted to a spectrum of output frequencies.
In the time-Floquet system, when the elements are varying in time with a periodic modulation having a modulation frequency of ω m , ε r = ε s + δ m cos(ω m t), the transfer matrix is also periodic and can be expanded into a Fourier series asΨ(t) = ∑ nΨ n (ω k )e inω m t . And the Fourier transform takes the following form:Ψ By choosing ω = ω q = ω k + qω m , ω ′ = ω p = ω k + pω m , q and p ∈ {..., -1, 0, +1, ...}, and substituting equations (3) into (2), we arrive at the following equations 1 : Now, let us consider adding a phase delay of φ to the sinusoidal modulation profile. Writing Equation (4)

2/11
Supplementary Figure 1. The schematic of a generic two-port time-Floquet system. Incident and reflected signals at ports 1 and 2 are represented by their time-varying complex amplitudes a 1,2 (t), and b 1,2 (t).

3/11 2 Time-Floquet layer with dynamical modulation depth
As an alternative approach to reach a highly nonlinear input-output mapping, we can entangle the modulation depth instead of the modulation phase with the input data (i.e., δ = f(ζ in ), where f is a linear function). In this case, the input information is encoded in the modulation depth of the Floquet layer in order to induce the required nonlinear entanglement, and no phase shifter is needed. Similar to the entanglement of modulation phase , the value of the modulation depth is directly determined by the value of the input data, which is fixed when the system is excited, automatically making the scattering process a highly non-linear function of the input, regardless of the input power. As an example, here, we show the efficiency of this approach by interpolating a highly nonlinear function (sinc(x)) similar to section 2.  • We used only 100 and 50 input and readout nodes, respectively.

Realistic physical platforms for the time-Floquet layer
In this section, we discuss realistic physical platforms for the time-Floquet layer in more details. Note that the modulation frequency is small compared to the operating frequencies f 1 and f 2 , since f m = |f 1 -f 2 |/2, and the modulation depth does not have to be large, as long as one can detect the Floquet harmonics above the noise level. This flexibility allows the proposed time-Floquet layer to be implemented in different frequency ranges.
At microwave frequencies, one can leverage a metasurface that incorporates a single temporally modulated capacitive layer backed by a dielectric layer. As a simple example, consider periodically arranged square patches (or other form of footprint patterns) with varactors soldered between the neighboring patches (see Supplementary Figure 3(a)). The time-varying modulation C(t) = C 0 (1-δ sin(ω m t + φ )) is introduced by applying a time-varying voltage to the varactors 5-10 .
In the terahertz and mid-infrared band, graphene is a good candidate to implement time-varying components due to its tunable electrical conductivity and compatibility with common micro-fabrication technologies. The sheet conductivity of graphene can be effectively modified via electrical bias 11, 12 (see Supplementary Figure 3(b)). We can model graphene as an infinitesimally thin sheet with surface impedance Z = 1/σ g , where σ g is the frequency-dependent complex conductivity of graphene. The surface conductivity of graphene including both intraband (σ intra ) and interband (σ inter ) transitions are governed 7/11 by the well-known Kubo formula 13 σ g (ω, τ, µ c , T) = σ intra (ω, τ, µ c , T) + σ inter (ω, τ, µ c , T) , σ inter (ω, τ, µ c , T) = -j e 2 4πh ln 2µ C -(ω-jτ -1 )h where e,h, and k B are constants corresponding to electron charge, the reduced Planck's constant, and the Boltzmann constant, respectively 13 . In the above equation, variables T, τ, and µ c correspond to the environmental temperature, relaxation time, and the chemical potential of the graphene, and ω is the radian angular frequency 13 . In the proposed structure (Supplementary Figure 3(b)), the unpatterned graphene layer is a lossy medium that can be modeled through a series RL circuit in the transmission line model.
If a temporally varying gate voltage is applied on the graphene sheet in Supplementary Figure 3(b), both its sheet resistance (R) and its inductance (L) will be harmonically modulated around their static values [15][16][17][18] .
In the optical domain, there are two main methods to achieve the needed time-varying responses. The first method is to apply a time-varying voltage on special materials such as indium tin oxide (ITO) as

8/11
an electro-optical material and utilize Al 2 O 3 /HfO 2 nanolaminates (HAOL) (see Supplementary Figure   3(c)). ITO is one of the most well-known transparent conducting oxides for realization of electro-optical modulators used in telecommunications 19 . The degenerate doping of ITO (ranging from 10 19 to 10 21 cm -3 ) can redshift its plasma frequency into infrared leading to a largely tunable optical response in infrared frequencies. Similar to graphene, here, by applying the time-varying voltage, dynamic phase modulation in reflection and transmission can be achieved 20,21 . Another approach leverages the changes in the phase shifts induced by the optical pump which enables fast temporal phase modulation. In this approach, two laser lines that are closely spaced in frequency results in a travelling-wave. Projecting this interference pattern on the metasurface/metamaterils imprints a travelling-wave phase profile onto the reflected wave 22 .