Skip to main content

Thank you for visiting nature.com. You are using a browser version with limited support for CSS. To obtain the best experience, we recommend you use a more up to date browser (or turn off compatibility mode in Internet Explorer). In the meantime, to ensure continued support, we are displaying the site without styles and JavaScript.

Arbitrary linear transformations for photons in the frequency synthetic dimension

Abstract

Arbitrary linear transformations are of crucial importance in a plethora of photonic applications spanning classical signal processing, communication systems, quantum information processing and machine learning. Here, we present a photonic architecture to achieve arbitrary linear transformations by harnessing the synthetic frequency dimension of photons. Our structure consists of dynamically modulated micro-ring resonators that implement tunable couplings between multiple frequency modes carried by a single waveguide. By inverse design of these short- and long-range couplings using automatic differentiation, we realize arbitrary scattering matrices in synthetic space between the input and output frequency modes with near-unity fidelity and favorable scaling. We show that the same physical structure can be reconfigured to implement a wide variety of manipulations including single-frequency conversion, nonreciprocal frequency translations, and unitary as well as non-unitary transformations. Our approach enables compact, scalable and reconfigurable integrated photonic architectures to achieve arbitrary linear transformations in both the classical and quantum domains using current state-of-the-art technology.

Introduction

Arbitrary linear transformations in photonics1,2,3 are of central importance for optical quantum computing4, classical signal processing and deep learning5,6,7,8,9,10. A variety of architectures are being actively studied to implement linear transformations for quantum computation and photonic neural networks, including those based on Mach–Zender interferometers (MZI)4,5, microring weight banks6,7,11, phase-change materials8,9, and diffractive metasurfaces10. All such approaches use path encoding of photons in real space. By contrast, implementing such linear transformations in the frequency space would open avenues beyond those possible with previously reported architectures, which are inherently time-invariant. For example, frequency-space transformations allow spectrotemporal shaping of light and generation of new frequencies, with wide-ranging applications in frequency metrology, spectroscopy, communication networks, classical signal processing12,13,14 and linear optical quantum information processing15,16,17,18,19,20,21,22,23,24. Nonlinear optics has traditionally been the workhorse for such spectrotemporal shaping, but the requirement of high-power fields and the difficulty of implementing arbitrary linear transformations motivates new architectures for manipulating states in the frequency domain. To that end, photonic synthetic dimensions offer an attractive solution to implement linear transformations in a single physical waveguide by harnessing the internal degrees of freedom of a photon25,26,27,28,29,30,31,32. Synthetic frequency dimensions in particular offer a small spatial footprint and inherent reconfigurability since multiple frequency modes can be addressed simultaneously, and the short- and long-range coupling29,30,33,34 between them can be controlled by applying an appropriate time-domain signal to a modulator.

Previous works have considered implementing photonic linear transformations using different frequency channels in parallel but without frequency conversions among them6,7,9,11 by demultiplexing the different frequencies into separate spatial channels. Additionally, optimized fast modulation has been used for tailoring single photon spectra from two-level quantum emitters35, or for quantum frequency conversion15 and linear optical quantum computation17,36, where the modulator is used as a generalized beam splitter in synthetic frequency dimensions. However, the design of an entire scattering matrix that implements an arbitrary N × N linear transformation in synthetic space, which is essential for many applications in quantum information processing and neural networks, has not yet been shown.

Here, we show that arbitrary linear transformations can be performed directly in the synthetic space spanned by the different frequency modes carried by a single physical waveguide. We use gradient-based inverse design to automate the process of designing the linear transformations, and demonstrate that a wide variety of transformations can be realized. As examples, we show single-frequency conversion, nonreciprocal frequency translations as well as general arbitrary unitary and non-unitary transformations, all achieved with high fidelities in a fully reconfigurable fashion.

Results

Theory

Consider a ring of radius R formed by a single mode waveguide with a refractive index n. The ring is coupled to an external waveguide of the same refractive index. Assuming sufficiently weak coupling between the ring and the external waveguide and neglecting group-velocity dispersion, the eigenmodes of the ring occur at frequencies ωm = ω0 + mΩR, where ω0 is the central frequency, m is an integer and ΩR = c/nR is the free spectral range (FSR) of the ring in angular frequency units, with c being the speed of light in vacuum. These eigenmodes take the form \({e}^{-i({m}_{0}+m)\phi }\), where m0 denotes the angular momentum of the 0th mode and ϕ is the azimuthal coordinate of the ring. Corresponding to these eigenmodes, we define \({a}_{m}(t){e}^{i{\omega }_{m}t}\) to be the amplitude of the mode centered at ωm, normalized such that am(t)2 corresponds to the photon number in the mth mode. Likewise, we define \({s}_{m}^{\pm }(t){e}^{i{\omega }_{m}t}\) to be the amplitudes of the modes of the external waveguide at the input and output ports, respectively, as shown in Fig. 1b. The coupling between the ring modes and waveguide modes at frequency ωm is described by an external coupling rate \({\gamma }_{m}^{e}\), while other losses occurring in the ring, such as absorption or bending loss, are captured by an internal decay rate \({\gamma }_{m}^{i}\). Lastly, we assume that the dielectric constant of the ring is modulated using an electro-optic modulator in the form \(\mathop{\sum }\nolimits_{l = 1}^{{N}_{{\rm{f}}}}\delta {\epsilon }_{l}(\phi )\cos (l{{{\Omega }}}_{{\rm{R}}}t+{\theta }_{l})\), where δϵl is the depth of the modulation and θl is the phase of the modulation at frequency lΩR. The angular dependence δϵl(ϕ) occurs due to the physical localization of the electro-optic modulator to a specific range of ϕ, as shown in Fig. 1. The dynamics of the coupled ring-waveguide system can be described by a coupled-mode theory (see Supplementary Note 1) given by:

$$-i{d}_{t}{a}_{m} =i\left({\gamma }_{m}^{e}+{\gamma }_{m}^{i}\right){a}_{m}+\sqrt{2{\gamma }_{m}^{e}}{s}_{m}^{+}\\ \quad + \mathop{\sum }\limits_{l=1}^{{N}_{{\rm{f}}}}\left({\kappa }_{l}{a}_{m-l}+{\kappa }_{-l}{a}_{m+l}\right),$$
(1)
$${s}_{m}^{-}={s}_{m}^{+}+i\sqrt{2{\gamma }_{m}^{e}}{a}_{m},$$
(2)

where

$${\kappa }_{\pm l}=-\frac{{\alpha }_{l}}{4{n}^{2}}{e}^{\mp i{\theta }_{l}}\mathop{\int}\nolimits_{\!\!\!\!0}^{2\pi }{e}^{\mp il\phi }\delta {\epsilon }_{l}(\phi )d\phi$$
(3)

is the modulation-induced coupling between the modes of the ring, with αl describing the radial and zenith-angle overlap of the eigenmodes of the ring with the electro-optic modulator (see Supplementary Note 1).

Fig. 1: Setup to implement arbitrary linear transformations in frequency space.
figure1

a An array of dynamically modulated rings (orange) coupled to an external waveguide. The green blocks represent electro-optic modulators (EOMs) and the black line is an external waveguide coupling to each of the rings. The output spectrum on the right is the result of the transformation implemented by the system on the input spectrum (left). b Detailed view of a single ring depicting the waveguide port inputs and outputs \({s}_{m}^{\pm }\). The smaller purple circles indicate auxiliary rings that couple selectively to modes am of the larger orange ring to implement frequency-dimension truncation to the ring spectrum. The time-periodic voltage profile applied to the EOM is a result of the inverse-design algorithm.

If the δϵl’s are real, i.e., only the real part of the refractive-index is modulated, then \({\kappa }_{l}^{* }={\kappa }_{-l}\). Therefore, the modulation conserves the total photon number summed across all frequency channels. Further, if \({\gamma }_{m}^{i}\) are negligible, then no photons are lost to absorption or radiation. Under these conditions, the setup of Eqs. (12) implements a unitary transformation between the fields \({s}_{m}^{+}\) at the input ports and the fields \({s}_{m}^{-}\) at the output ports. This unitary transformation can be obtained by first converting Eq. (1) to the frequency domain, resulting in

$${\bf{a}}={\left[{{\Delta }}\omega -i{{\Gamma }}-{\mathcal{K}}\right]}^{-1}\sqrt{2{{\Gamma }}}{{\bf{s}}}^{+},$$
(4)

where \({\bf{a}}={\{\ldots {a}_{-1},{a}_{0},{a}_{1},\ldots \}}^{t}\), \({{\bf{s}}}^{\pm }={\{\ldots {s}_{-1}^{\pm },{s}_{0}^{\pm },{s}_{1}^{\pm },\ldots \}}^{t}\), \({{\Gamma }}={\rm{diag}}(\ldots {\gamma }_{-1}^{e},{\gamma }_{0}^{e},{\gamma }_{1}^{e},\ldots )\), Δω is a constant detuning of the equally spaced frequencies of input comb s+ from the ring’s resonant frequencies, and \({{\mathcal{K}}}_{mm^{\prime} }\equiv {\kappa }_{m-m^{\prime} }\) as defined by Eq. (3). Then, from Eq. (2), we obtain \({{\bf{s}}}^{-}={\mathcal{M}}{{\bf{s}}}^{+}\), where

$${\mathcal{M}}=\left[{\mathcal{I}}+i\sqrt{2{{\Gamma }}}{\left[{{\Delta }}\omega -i{{\Gamma }}-{\mathcal{K}}\right]}^{-1}\sqrt{2{{\Gamma }}}\right].$$
(5)

A direct verification of the unitarity of \({\mathcal{M}}\) is included in Supplementary Note 2. In the idealized situation as described above, where the ring-waveguide system is assumed to be single-moded over a broad bandwidth and is free from group velocity dispersion, the matrix \({\mathcal{M}}\) is infinite-dimensional. In practice, the dimensionality of the scattering matrix can be controlled by introducing a “truncation” along the frequency dimension. Such a truncation can be implemented using one or more auxiliary rings coupled to the main ring (see Supplementary Note 3). The auxiliary rings couple to and perturb a few modes immediately outside the (2Nsb + 1) modes around the 0th mode, dispersively shifting and splitting them. These perturbed modes have frequencies such that the modulation tones of lΩR cannot couple these modes to the (2Nsb + 1) modes of interest. Therefore, the total number of modes under consideration in the coupled ring-waveguide system is 2Nsb + 1, and the scattering matrix defined in Eq. (5) is of size (2Nsb + 1) × (2Nsb + 1).

The main objective of our paper is to show that an arbitrary scattering matrix of size (2Nsb + 1) × (2Nsb + 1) can be created. To that end, we first note that the number of real degrees of freedom in the scattering matrix (Eq. (5)) of a single ring under modulation is equal to twice the number of distinct modulation tones, 2Nf, provided the modulation amplitudes δϵl and phases θl are independently controllable. Since the system is truncated to have 2Nsb + 1 frequencies, the largest harmonic of ΩR that will result in nonzero coupling between any two modes is 2Nsb, i.e., Nf ≤ 2Nsb. Since an arbitrary unitary matrix of size (2Nsb + 1) × (2Nsb + 1) has \({(2{N}_{{\rm{sb}}}+1)}^{2}\) real degrees of freedom whereas Nf≤2Nsb, we conclude that a single modulated ring is insufficient to approximate an arbitrary unitary matrix to a high degree of accuracy, even if all modulation tones up to 2NsbΩR are used. To overcome this problem, notice that products of unitary transformations are also unitary37. Therefore, as shown in Fig. 1a, instead of a single ring, we consider a sequence of Nr number of rings with each ring providing Nf complex degrees of freedom. Thus, if the total degrees of freedom in series of rings coupled to the waveguide, given by 2NfNr, exceeds \({(2{N}_{{\rm{sb}}}+1)}^{2}\), then the setup of Fig. 1a should be able to approximate an arbitrary unitary transformation to a high degree of accuracy.

Below, we optimize these 2NfNr degrees of freedom to enable physical approximation of arbitrary unitary and certain non-unitary transformations. For unitary transformations or parts thereof, we use as the objective function the fidelity, which measures the accuracy of an approximation V to a unitary transformation U:

$${\mathcal{F}}(U,V)=\frac{| \langle U,V\rangle | }{\sqrt{| | U| {| }_{{\rm{F}}}| | V| {| }_{{\rm{F}}}}},$$
(6)

where \(\langle U,V\rangle ={\sum }_{ij}{U}_{ij}^{* }{V}_{ij}\) is the element-wise inner product and \(| | U| {| }_{{\rm{F}}}=\sqrt{{\sum }_{ij}| {U}_{ij}{| }^{2}}\) is the Frobenius norm. The use of an absolute value in Eq. (6) allows for the tolerance of a single global phase, i.e., if \({\mathcal{F}}(U,V)=1\), then the transformation V achieved by the architecture is equal to UeiΦ for some phase Φ. To achieve a high fidelity for a given target matrix we use gradient-based inverse design to optimize the parameters of the modulated system. To enable such optimization, we implemented a numerical model of the unitary transformations defined by Eq. (5) in an automatic differentiation framework38. While explicitly defined adjoint variable methods have been widely used for photonic inverse design39, automatic differentiation is the generalization of the adjoint variable methods to arbitrary computational graphs. Automatic differentiation has recently been successfully applied to the inverse design of photonic band structures40 as well as photonic neural networks41, where explicit adjoint methods are challenging to implement. Here, automatic differentiation enables the efficient computation of the gradients of a scalar objective function with respect to complex control parameters, which in this case are the coupling constants κ±l as defined in Eq. (3). The advantage of using automatic differentiation is that one needs only to implement the computational model as described above, while the automatic differentiation framework manages the gradient computation through an efficient reverse-mode differentiation. Using the gradients from automatic differentiation, the Limited-memory Broyden–Fletcher–Goldfarb–Shanno (LBFGS) algorithm42 is used for optimization.

Implementation of linear transformations

For the results in this Section, we assume that the ring-waveguide system under consideration operates with Nsb = 2, i.e., 5 equally spaced lines followed by at least 4 perturbed lines on each side. The five relevant modes are indexed {−2, −1, 0, 1, 2}. For simplicity, we assume that all five ring modes couple to the waveguide with equal strength, i.e., \({\gamma }_{m}^{e}\equiv \gamma\) and \({\gamma }_{m}^{i}=0\,\forall m\). We also assume that the source frequencies in the waveguide are on resonance with the ring, i.e., Δω = 0 in Eq. (5). Examples of finite intrinsic loss (\({\gamma }_{m}^{i}\ne 0\)) and non-uniform detuning (Δω ≠ 0) are considered in Supplementary Notes 4 and 5. Note that the different source frequencies’ phases should not drift with respect to each other during the timescale of the transformation. To ensure such phase coherence between the different input frequency modes, the source could be a mode-locked laser or an electro-optic frequency comb with a tailored amplitude/phase spectrum to implement the input vector. Alternatively, active phase stabilization could be implemented to compensate for slow-timescale phase drifts. Under the assumptions made in this Section, the transformation in Eq. (5) is completely determined by the ratios κl/γ, where κl is controlled by the index perturbation amplitude δϵl and phase θl, as described by Eq. (3). Therefore, we optimize the amplitude and phase of κl (in units of γ) for Nr rings and Nf modulation tones per ring to implement a variety of transformations. Note that since we only optimize for the ratios κl/γ, our approach is robust to variations in γ during fabrication.

First, we consider the application of such ring-waveguide networks to implement high-fidelity frequency translation that is useful for frequency-domain beam-splitters or single-qubit gates. As an example, we show a design where an input signal in mode 0, after forward propagation through the network, results in a complete conversion to mode +2. Using our inverse-design framework, such a frequency translation corresponds to designing only one column of a unitary transformation and can be achieved with a fidelity exceeding 1 − 10−5 using just two rings and two modulation tones per ring, as shown in Fig. 2a, b. In Fig. 2c, we present the error function versus the number of iterations. The error function is defined as 1 − F+2, where F+2 is the normalized output photon flux in the mode +2. After a few iterations, almost all the photon flux is converted to frequency ω+2 at the output.

Fig. 2: Frequency translation.
figure2

A two-ring system with two modulation tones per ring (Nr = 2, Nf = 2) demonstrating a conversion efficiency exceeding 1 − 10−5 from mode 0 to +2. The bar plots show log-scale photon flux in each mode at a the input, and b the output. c The error function shown as a function of the number of iterations of the optimization algorithm to achieve the conversion efficiency of b. Modulation parameters are provided in Supplementary Note 6.

In addition to such high-fidelity frequency conversion implemented in forward propagation through the network, the transformations achieved in this architecture can be different in forward and reverse propagation due to the relative phase shift between the modulation tones across the different rings and the explicit time-varying nature of the dynamically modulated system43. This is in sharp contrast with MZI-based architectures, which are inherently reciprocal. As an example, we show in Fig. 3 that we can simultaneously realize with a fidelity exceeding 1 − 10−5 a frequency shift, say, 0 → 2, in forward propagation (Fig. 3a) and a different shift, say, 2 → 1, in reverse propagation (Fig. 3b) with three modulated rings.

Fig. 3: Nonreciprocal frequency conversion.
figure3

A three-ring system with three modulation tones per ring, demonstrating a a conversion efficiency exceeding 1 − 10−5 from mode 0 to mode +2 in forward propagation. The input and output field profiles are indicated by sL and sR, respectively, and b the complex-conjugated output profile, \({s}_{R}^{* }\), injected back into the output port results in a conversion efficiency exceeding 1 − 10−5 from mode +2 to +1 instead of mode 0 in backward propagation through the same system, indicating highly efficient nonreciprocal frequency shifts. Modulation parameters are provided in Supplementary Note 6.

Achieving frequency shifts using modulated rings, as shown in Figs. 2 and 3, requires designing only one and two columns of the 5 × 5 unitary matrix, respectively. On the other hand, if the number of modulation tones Nf and/or the number of rings Nr are increased, an arbitrary unitary transformation can be achieved with a high fidelity. As an example, we depict in Fig. 4a a 5 × 5 permutation matrix U, defined by U13 = U24 = U35 = U42 = U51 = 1, and zero otherwise. In Fig. 4b, we present the amplitudes of the matrix achieved using one ring and four modulation tones, resulting in a fidelity of 1 − 5.9 × 10−3. With four rings and four modulation tones, the fidelity is boosted to over 1 − 3.8 × 10−6, as shown by the amplitudes in Fig. 4c. In Fig. 4d, we tabulate as a function of Nr and Nf one minus the maximum fidelities obtained in approximating the 5 × 5 permutation matrix, showing that very high fidelities can be achieved using a wide variety of Nr and Nf combinations.

Fig. 4: Implementing elementwise amplitudes of a matrix.
figure4

a A 5 × 5 permutation matrix to be implemented by the ring-waveguide system. The amplitudes of the matrix elements are indicated along with a green colormap. Element-wise amplitudes of the optimized result using four modulation tones (Nf = 4) and b one ring (Nr = 1), achieving a fidelity of 1 − 5.9 × 10−3, and c four rings (Nr = 4), achieving a fidelity of 1 − 3.8 × 10−6. d One minus the maximum fidelities achieved by the inverse-design algorithm as a function of Nr and Nf. A value closer to zero indicates a better performance. Modulation parameters for c are provided in Supplementary Note 6.

In Fig. 4, we considered only the accuracy of the amplitudes achieved by our inverse-design approach. We now show that our architecture can also capture the phase of an arbitrary unitary transformation with a high fidelity. To demonstrate this, we consider a normalized 5 × 5 Vandermonde matrix, which is used to implement the discrete Fourier transform. This unitary transformation, defined by \({U}_{mn}={e}^{-2\pi imn/5}/\sqrt{5}\), has a constant amplitude across its matrix elements but significantly varying phase, as shown in Fig. 5a. With the use of one ring and four modulation tones, the inverse-design algorithm is able to achieve a fidelity of 0.8, with the corresponding phase profile shown in Fig. 5b up to a global phase of 0.0099π. As depicted in Fig. 5c, a significantly better performance is possible with the use of four rings and four modulation tones per ring, achieving a fidelity of 1 − 7.25 × 10−7 with a global phase of 0.596π. A map of one minus the maximum fidelities achieved by our inverse design approach as a function of the number of rings and modulation tones is shown in Fig. 5d.

Fig. 5: Implementing elementwise phases of a matrix.
figure5

a Element-wise phase as a fraction of π of the 5 × 5 Vandermonde matrix implementing the discrete Fourier transform. Element-wise phase achieved by the inverse-design algorithm for b Nr = 1 and Nf = 4, with a fidelity of 0.8 and global phase of 0.099π and c Nr = 4 and Nf = 4, with a fidelity of 1 − 7.25 × 10−7 and global phase 0.596π. d One minus the maximum fidelities achieved by the inverse-design algorithm as a function of Nr and Nf. A value closer to zero indicates a better performance. Modulation parameters for c are provided in Supplementary Note 6.

While unitary transformations are usually required for quantum information processing, matrices used in classical signal processing and in neural networks are in general non-unitary. The architecture presented thus far can also be used to implement non-unitary matrices with singular values less than or equal to one using one of two techniques. First, such non-unitary matrices can provably be embedded in larger unitary matrices44 using their singular value decomposition. Subsequently, the larger unitaries can be implemented using refractive index modulation as discussed thus far. As an example, we consider the following 3 × 3 non-unitary matrix that was randomly generated subject to the constraint that its largest singular value is equal to one:

$$M=\left(\begin{array}{lll}0.4993{e}^{i0.2483\pi }&0.3135{e}^{i0.3251\pi }&0.3150{e}^{i0.1519\pi }\\ 0.2580{e}^{i0.4129\pi }&0.2888{e}^{i0.1608\pi }&0.4420{e}^{i0.0492\pi }\\ 0.5277{e}^{i0.2319\pi }&0.2382{e}^{i0.1053\pi }&0.1992{e}^{i0.4087\pi }\end{array}\right).$$
(7)

The singular values of M are 1, 0.3755 and 0.1421, respectively. Since there are two singular values less than 1, M can be extended into a unitary matrix by adding two dimensions. The element-wise amplitude and phase corresponding to the extended 5 × 5 unitary matrix are shown in Fig. 6a, c, respectively. Using four rings (Nr = 4) and four modulation tones per ring (Nf = 4), our inverse-design algorithm achieves the extended unitary matrix with a fidelity exceeding 1 − 10−5, as shown in Fig. 6b, d. Notice that the phase of element (5, 4) is significantly different between Fig. 6c, d, but this is because the target amplitude for this element is zero. As an alternative approach, amplitude modulation, where the imaginary part of the refractive index is also modulated, can also be used to directly implement non-unitary matrices since the transformation of Eq. (5) is non-unitary under modulation of the imaginary part of the refractive index. Lastly, in order to implement matrices with singular values greater than 1, a gain element is necessary. For such matrices, a scaled version such that the singular values are below 1 can first be implemented using the methods outlined above, after which a uniform amplification for all frequency channels can rescale the matrix to its intended form.

Fig. 6: Implementing non-unitary transformations with unitary embedding.
figure6

The target non-unitary matrix, located in the upper-left 3 × 3 section of the matrix, is first extended to a unitary 5 × 5 target matrix. The element-wise a target amplitude, b achieved amplitude, c target phase, and d achieved phase as a fraction of π are shown. A near ideal implementation was achieved using Nr = 4 and Nf = 4 with a fidelity exceeding 1 − 10−5. Modulation parameters are provided in Supplementary Note 6.

Discussion

We have shown that combining the concepts of synthetic dimensions and inverse design enables the implementation of versatile linear transformations in photonics. A major advantage of using synthetic frequency dimensions for implementing an N × N linear transformation is that only O(N) photonic elements (modulators in our case) need to be electrically controlled. This is in contrast to real-space dimensions using path-encoding, such as MZI meshes or crossbar arrays, where the full O(N2) degrees of freedom need to be electrically controlled. Such control is nontrivial both from a scalability perspective as well as from a practical geometrical perspective of connecting N2 tunable elements (e.g. phase-shifters) to their driving electronics off-chip. The reduction in the number of individually controlled elements from O(N2) to O(N) in our scheme comes from the fact that the driving signal on each of the Nr EOMs can simultaneously address Nf frequency modes in the synthetic dimension.

Future work could leverage synthetic frequency dimensions for complicated quantum information protocols beyond single-qudit unitary transformations, such as realizing probabilistic entangling gates for linear optical quantum computing (LOQC)17,36. In particular, spectral LOQC using EOMs and pulse shapers has been shown to be universal for quantum computation17. However, pulse shapers involve demultiplexing the frequency modes into distinct spatial channels using gratings to apply mode-by-mode phase shifts, and limit the number of modes that can be accommodated within the modulator bandwidth due to a finite spectral resolution, thus reducing the benefit of using synthetic frequency dimensions. Such pulse shapers are also lossy and challenging to integrate on chip. Our architecture obviates the pulse shaper by exclusively using EOMs. The advent of ultralow-loss nanophotonic EOMs in lithium niobate45,46, as well as progress in silicon47,48 and aluminum nitride49 makes our architecture fully compatible with on-chip integration, since modulation at frequencies exceeding the ring’s FSR have been demonstrated14,47,50.

For applications in neural networks, the performance of our architecture in terms of the speed, compute density and energy consumption for multiply-and-accumulate (MAC) operations is important51. Assuming we need N modulation tones and N rings with FSR Δf = ΩR/2π to implement a matrix, we can input information encoded in the N frequencies and read out the matrix-vector product, which amounts to N2 MAC operations. Since we need a frequency-resolved measurement, the fastest readout bandwidth is Δf. We assume that the input data can be prepared at speed comparable to or faster than the readout speed. Then, the computational speed in MACs per second is given by

$$C={N}^{2}{{\Delta }}f.$$
(8)

The maximum number of channels is limited by the FSR and the modulation bandwidth. If we utilize the whole available bandwidth, B = NΔf, then the speed is

$$C=NB.$$
(9)

For a modulation bandwidth of 100 GHz and an FSR of 100 MHz (such that N = 1000), this yields a speed C = 1014 MACs per second or 100 TMAC per second, which is comparable with MZI meshes5,51,52. Although achieving such small FSRs on chip is challenging, recent progress in integrating low-loss delay lines on chip53,54 holds promise, since meter-scale delays were reported in an 8 mm2 footprint using spiral resonators, corresponding to an equivalent FSR of ~350 MHz53. These design techniques can be extended to lithium niobate rings with high modulation bandwidths14,46.

To optimize for computation density, i.e. MACs per second per unit area51, one can use a larger FSR Δf = 1 GHz, in a 1-mm2 footprint, and combine synthetic frequency dimensions within each 100-GHz modulation bandwidth with wavelength-division multiplexed channels separated by 100-GHz-wide stopbands, to parallelize several uncoupled MAC operations across the 5 THz telecommunications band, as has been done for crossbar arrays6,7,9,51. This leads to a compute density of ~10 TMAC s−1 mm−2, which is much better than MZI meshes and comparable with standard silicon microring crossbar arrays51, with the added advantage of only O(N) electronically controlled elements. We anticipate that future progress in modulation speed and power using high-confinement integrated photonic platforms will push these current estimates further, leading to experimental implementations of MAC operations using the architecture proposed here with improvements in complexity, speed, power and footprint.

Data availability

The data related to this study is available in the manuscript and the Supplementary Materials. Additional data is available from the authors upon reasonable request. Correspondence and requests for materials should be addressed to S.F.

References

  1. 1.

    Reck, M., Zeilinger, A., Bernstein, H. J. & Bertani, P. Experimental realization of any discrete unitary operator. Phys. Rev. Lett. 73, 58–61 (1994).

    ADS  CAS  PubMed  Article  Google Scholar 

  2. 2.

    Clements, W. R., Humphreys, P. C., Metcalf, B. J., Kolthammer, W. S. & Walmsley, I. A. Optimal design for universal multiport interferometers. Optica 3, 1460–1465 (2016).

    ADS  Article  Google Scholar 

  3. 3.

    Miller, D. A. Self-configuring universal linear optical component. Photonics Res. 1, 1–15 (2013).

    ADS  Article  Google Scholar 

  4. 4.

    Carolan, J. et al. Universal linear optics. Science 349, 711–716 (2015).

    MathSciNet  CAS  PubMed  MATH  Article  Google Scholar 

  5. 5.

    Shen, Y. et al. Deep learning with coherent nanophotonic circuits. Nat. Photonics 11, 441–446 (2017).

    ADS  CAS  Article  Google Scholar 

  6. 6.

    Tait, A. N., Nahmias, M. A., Shastri, B. J. & Prucnal, P. R. Broadcast and weight: an integrated network for scalable photonic spike processing. J. Lightwave Technol. 32, 4029–4041 (2014).

    Article  Google Scholar 

  7. 7.

    Tait, A. N. et al. Neuromorphic photonic networks using silicon photonic weight banks. Sci. Rep. 7, 1–10 (2017).

    CAS  Article  Google Scholar 

  8. 8.

    Feldmann, J., Youngblood, N., Wright, C., Bhaskaran, H. & Pernice, W. All-optical spiking neurosynaptic networks with self-learning capabilities. Nature 569, 208–214 (2019).

    ADS  CAS  PubMed  PubMed Central  Article  Google Scholar 

  9. 9.

    Feldmann, J. et al. Parallel convolutional processing using an integrated photonic tensor core. Nature 589, 52–58 (2021).

    ADS  CAS  PubMed  Article  Google Scholar 

  10. 10.

    Lin, X. et al. All-optical machine learning using diffractive deep neural networks. Science 361, 1004–1008 (2018).

    ADS  MathSciNet  CAS  PubMed  MATH  Article  Google Scholar 

  11. 11.

    Ohno, S., Toprasertpong, K., Takagi, S. & Takenaka, M. Si microring resonator crossbar arrays for deep learning accelerator. Jpn. J. Appl. Phys. 59, SGGE04 (2020).

    CAS  Article  Google Scholar 

  12. 12.

    Cundiff, S. T. & Weiner, A. M. Optical arbitrary waveform generation. Nat. Photonics 4, 760–766 (2010).

    ADS  CAS  Article  Google Scholar 

  13. 13.

    Supradeepa, V. et al. Comb-based radiofrequency photonic filters with rapid tunability and high selectivity. Nat. Photonics 6, 186–194 (2012).

    ADS  CAS  Article  Google Scholar 

  14. 14.

    Zhang, M. et al. Broadband electro-optic frequency comb generation in a lithium niobate microring resonator. Nature 568, 373–377 (2019).

    ADS  CAS  PubMed  Article  Google Scholar 

  15. 15.

    Lu, H.-H. et al. Electro-optic frequency beam splitters and tritters for high-fidelity photonic quantum information processing. Phys. Rev. Lett. 120, 030502 (2018).

    ADS  CAS  PubMed  Article  Google Scholar 

  16. 16.

    Menicucci, N. C., Flammia, S. T. & Pfister, O. One-way quantum computing in the optical frequency comb. Phys. Rev. Lett. 101, 130501 (2008).

    ADS  PubMed  Article  CAS  Google Scholar 

  17. 17.

    Lukens, J. M. & Lougovski, P. Frequency-encoded photonic qubits for scalable quantum information processing. Optica 4, 8–16 (2017).

    ADS  Article  Google Scholar 

  18. 18.

    Lu, H.-H., Weiner, A. M., Lougovski, P. & Lukens, J. M. Quantum information processing with frequency-comb qudits. IEEE Photonics Technol. Lett. 31, 1858–1861 (2019).

    ADS  CAS  Article  Google Scholar 

  19. 19.

    Joshi, C., Farsi, A., Clemmen, S., Ramelow, S. & Gaeta, A. L. Frequency multiplexing for quasi-deterministic heralded single-photon sources. Nat. Commun. 9, 847 (2018).

    ADS  PubMed  PubMed Central  Article  CAS  Google Scholar 

  20. 20.

    Roslund, J., de Araújo, R. M., Jiang, S., Fabre, C. & Treps, N. Wavelength-multiplexed quantum networks with ultrafast frequency combs. Nat. Photonics 8, 109–112 (2014).

    ADS  CAS  Article  Google Scholar 

  21. 21.

    Reimer, C. et al. High-dimensional one-way quantum processing implemented on d -level cluster states. Nat. Phys. 15, 148–153 (2019).

    CAS  Article  Google Scholar 

  22. 22.

    Joshi, C. et al. Frequency-domain quantum interference with correlated photons from an integrated microresonator. Phys. Rev. Lett. 124, 143601 (2020).

    ADS  CAS  PubMed  Article  Google Scholar 

  23. 23.

    Zhu, X. et al. Graph state engineering by phase modulation of the quantum optical frequency comb. http://arxiv.org/abs/1912.11215 (2019).

  24. 24.

    Hu, Y. et al. Reconfigurable electro-optic frequency shifter. arXiv preprint http://arxiv.org/abs/2005.09621 (2020).

  25. 25.

    Yuan, L., Lin, Q., Xiao, M. & Fan, S. Synthetic dimension in photonics. Optica 5, 1396–1405 (2018).

    ADS  CAS  Article  Google Scholar 

  26. 26.

    Ozawa, T. & Price, H. M. Topological quantum matter in synthetic dimensions. Nat. Rev. Phys. 1, 349–357 (2019).

    Article  Google Scholar 

  27. 27.

    Yuan, L., Shi, Y. & Fan, S. Photonic gauge potential in a system with a synthetic frequency dimension. Opt. Lett. 41, 741–744 (2016).

    ADS  PubMed  Article  Google Scholar 

  28. 28.

    Ozawa, T., Price, H. M., Goldman, N., Zilberberg, O. & Carusotto, I. Synthetic dimensions in integrated photonics: from optical isolation to four-dimensional quantum Hall physics. Phys. Rev. A 93, 043827 (2016).

    ADS  Article  Google Scholar 

  29. 29.

    Bell, B. A. et al. Spectral photonic lattices with complex long-range coupling. Optica 4, 1433–1436 (2017).

    ADS  CAS  Article  Google Scholar 

  30. 30.

    Wang, K. et al. Multidimensional synthetic chiral-tube lattices via nonlinear frequency conversion. Light. Sci. Appl. 9, 132 (2020).

    ADS  CAS  PubMed  PubMed Central  Article  Google Scholar 

  31. 31.

    Dutt, A. et al. A single photonic cavity with two independent physical synthetic dimensions. Science 367, 59–64 (2020).

    ADS  CAS  PubMed  Article  Google Scholar 

  32. 32.

    Qin, C. et al. Spectrum control through discrete frequency diffraction in the presence of photonic gauge potentials. Phys. Rev. Lett. 120, 133901 (2018).

    ADS  CAS  PubMed  Article  Google Scholar 

  33. 33.

    Dutt, A. et al. Experimental band structure spectroscopy along a synthetic dimension. Nat. Commun. 10, 3122 (2019).

    ADS  PubMed  PubMed Central  Article  CAS  Google Scholar 

  34. 34.

    Yuan, L., Xiao, M., Lin, Q. & Fan, S. Synthetic space with arbitrary dimensions in a few rings undergoing dynamic modulation. Phys. Rev. B 97, 104105 (2018).

    ADS  CAS  Article  Google Scholar 

  35. 35.

    Lukin, D. M. et al. Spectrally reconfigurable quantum emitters enabled by optimized fast modulation. npj Quantum Inf. 6, 1–9 (2020).

    ADS  Article  Google Scholar 

  36. 36.

    Lu, H.-H. et al. A controlled-NOT gate for frequency-bin qubits. npj Quantum Inf. 5, 1–8 (2019).

    ADS  Article  Google Scholar 

  37. 37.

    Huhtanen, M. & Perämäki, A. Factoring matrices into the product of circulant and diagonal matrices. J. Fourier Anal. Appl. 21, 1018–1033 (2015).

    MathSciNet  MATH  Article  Google Scholar 

  38. 38.

    Maclaurin, D., Duvenaud, D. & Adams, R. P. Autograd: Effortless gradients in numpy. In ICML 2015 AutoML Workshop, vol. 238 (ICML, 2015).

  39. 39.

    Molesky, S. et al. Outlook for inverse design in nanophotonics. Nat. Photonics 12, 659–670 (2018).

    ADS  CAS  Article  Google Scholar 

  40. 40.

    Minkov, M. et al. Inverse design of photonic crystals through automatic differentiation. ACS Photonics 7, 1729–1741 (2020).

    CAS  Article  Google Scholar 

  41. 41.

    Hughes, T. W., Williamson, I. A. D., Minkov, M. & Fan, S. Wave physics as an analog recurrent neural network. Sci. Adv. 5, eaay6946 (2019).

    ADS  PubMed  PubMed Central  Article  Google Scholar 

  42. 42.

    Byrd, R. H., Lu, P., Nocedal, J. & Zhu, C. A limited memory algorithm for bound constrained optimization. SIAM J. Sci. Comput. 16, 1190–1208 (1995).

    MathSciNet  MATH  Article  Google Scholar 

  43. 43.

    Fang, K., Yu, Z. & Fan, S. Photonic aharonov-bohm effect based on dynamic modulation. Phys. Rev. Lett. 108, 153901 (2012).

    ADS  PubMed  Article  CAS  Google Scholar 

  44. 44.

    Tischler, N., Rockstuhl, C. & Słowik, K. Quantum optical realization of arbitrary linear transformations allowing for loss and gain. Phys. Rev. X 8, 021017 (2018).

    CAS  Google Scholar 

  45. 45.

    Wang, C., Zhang, M., Stern, B., Lipson, M. & Lončar, M. Nanophotonic lithium niobate electro-optic modulators. Opt. Express 26, 1547–1555 (2018).

    ADS  PubMed  Article  Google Scholar 

  46. 46.

    Wang, C. et al. Integrated lithium niobate electro-optic modulators operating at CMOS-compatible voltages. Nature 562, 101–104 (2018).

    ADS  CAS  PubMed  Article  Google Scholar 

  47. 47.

    Tzuang, L. D., Soltani, M., Lee, Y. H. D. & Lipson, M. High RF carrier frequency modulation in silicon resonators by coupling adjacent free-spectral-range modes. Opt. Lett. 39, 1799–1802 (2014).

    ADS  CAS  PubMed  Article  Google Scholar 

  48. 48.

    Van Laer, R., Patel, R. N., McKenna, T. P., Witmer, J. D. & Safavi-Naeini, A. H. Electrical driving of X-band mechanical waves in a silicon photonic circuit. APL Photonics 3, 086102 (2018).

    ADS  Article  CAS  Google Scholar 

  49. 49.

    Tian, H. et al. Hybrid integrated photonics using bulk acoustic resonators. Nat. Commun. 11, 3073 (2020).

    ADS  CAS  PubMed  PubMed Central  Article  Google Scholar 

  50. 50.

    Hu, Y. et al. Realization of high-dimensional frequency crystals in electro-optic microcombs. Optica 7, 1189–1194 (2020).

    ADS  CAS  Article  Google Scholar 

  51. 51.

    Nahmias, M. A. et al. Photonic multiply-accumulate operations for neural networks. IEEE J. Sel. Top. Quantum Electron. 26, 1–18 (2019).

    Article  Google Scholar 

  52. 52.

    Williamson, I. A. D. et al. Reprogrammable electro-optic nonlinear activation functions for optical neural networks. IEEE J. Sel. Top. Quantum Electron. 26, 1–12 (2020).

    Article  Google Scholar 

  53. 53.

    Ji, X. et al. On-chip tunable photonic delay line. APL Photonics 4, 090803 (2019).

    ADS  Article  CAS  Google Scholar 

  54. 54.

    Ji, X. et al. Ultra-low-loss on-chip resonators with sub-milliwatt parametric oscillation threshold. Optica 4, 619–624 (2017).

    ADS  Article  Google Scholar 

Download references

Acknowledgements

This work is supported by the U.S. Air Force Office of Scientific Research (FA9550-17-1-0002, FA9550-18-1-0379). S.B. acknowledges the support of a Stanford Graduate Fellowship.

Author information

Affiliations

Authors

Contributions

S.B., M.M., and I.A.D.W. conceived the project. S.B., A.D., and M.M. performed the research. S.F. supervised the research. All authors analyzed the results and contributed to discussions. S.B., A.D. and S.F. wrote the manuscript.

Corresponding author

Correspondence to Shanhui Fan.

Ethics declarations

Competing interests

The authors declare no competing interests.

Additional information

Peer review information Nature Communications thanks the anonymous reviewer(s) for their contribution to the peer review of this work. Peer reviewer reports are available.

Publisher’s note Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Supplementary information

Rights and permissions

Open Access This article is licensed under a Creative Commons Attribution 4.0 International License, which permits use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons license, and indicate if changes were made. The images or other third party material in this article are included in the article’s Creative Commons license, unless indicated otherwise in a credit line to the material. If material is not included in the article’s Creative Commons license and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder. To view a copy of this license, visit http://creativecommons.org/licenses/by/4.0/.

Reprints and Permissions

About this article

Verify currency and authenticity via CrossMark

Cite this article

Buddhiraju, S., Dutt, A., Minkov, M. et al. Arbitrary linear transformations for photons in the frequency synthetic dimension. Nat Commun 12, 2401 (2021). https://doi.org/10.1038/s41467-021-22670-7

Download citation

Comments

By submitting a comment you agree to abide by our Terms and Community Guidelines. If you find something abusive or that does not comply with our terms or guidelines please flag it as inappropriate.

Search

Quick links

Nature Briefing

Sign up for the Nature Briefing newsletter — what matters in science, free to your inbox daily.

Get the most important science stories of the day, free in your inbox. Sign up for Nature Briefing