Abstract
Photonic systems for highperformance information processing have attracted renewed interest. Neuromorphic silicon photonics has the potential to integrate processing functions that vastly exceed the capabilities of electronics. We report first observations of a recurrent silicon photonic neural network, in which connections are configured by microring weight banks. A mathematical isomorphism between the silicon photonic circuit and a continuous neural network model is demonstrated through dynamical bifurcation analysis. Exploiting this isomorphism, a simulated 24node silicon photonic neural network is programmed using “neural compiler” to solve a differential system emulation task. A 294fold acceleration against a conventional benchmark is predicted. We also propose and derive power consumption analysis for modulatorclass neurons that, as opposed to laserclass neurons, are compatible with silicon photonic platforms. At increased scale, Neuromorphic silicon photonics could access new regimes of ultrafast information processing for radio, control, and scientific computing.
Introduction
Light forms the global backbone of information transmission yet is rarely used for information transformation. Digital optical logic faces fundamental physical challenges^{1}. Many analog approaches have been researched^{2,3,4}, but analog optical coprocessors have faced major economic challenges. Optical systems have never achieved competitive manufacturability, nor have they satisfied a sufficiently general processing demand better than digital electronic contemporaries. Incipient changes in the supply and demand for photonics have the potential to spark a resurgence in optical information processing.
A germinating silicon photonic integration industry promises to supply the manufacturing economomies normally reserved for microelectronics. While firmly rooted in demand for datacenter transceivers^{5}, the industrialization of photonics would impact other application areas^{6}. Industrial microfabrication ecosystems propel technology roadmapping^{7}, library standardization^{8, 9}, and broadened accessibility^{10}, all of which could open fundamentally new research directions into largescale photonic systems. Largescale beam steerers have been realized^{11}, and onchip communication networks have been envisioned^{12,13,14}; however, opportunities for scalable silicon photonic information processing systems remain largely unexplored.
Concurrently, photonic devices have found analog signal processing niches where electronics can no longer satisfy demands for bandwidth and reconfigurability. This situation is exemplified by radio frequency (RF) processing, in which frontends have come to be limited by RF electronics, analogtodigital converters (ADCs), and digital signal processors (DSP)^{15, 16}. In response, RF photonics has offered respective solutions for tunable RF filters^{17, 18}, ADC itself^{19}, and simple processing tasks that can be moved from DSP into the analog subsystem^{20,21,22}. RF photonic circuits that can be transcribed from fiber to silicon are likely to reap the economic benefits of silicon photonic integration. In a distinct vein, an unprecedented possibility for largescale photonic system integration could enable systems beyond what can be considered in fiber. If scalable information processing with analog photonics is to be considered, new standards relating physics to processing must be developed and verified.
Standardized concepts that link physics to computational models are required to define essential quantitative engineering tools, namely metrics, algorithms, and benchmarks. For example, a conventional gate has simultaneous meaning as an abstract logical operation and as an arrangement of electronic semiconductors and thereby acts as a conduit between device engineering and computational performance. In another case, neuromorphic electronics adopt unconventional standards defining spiking neural networks as eventbased packet networks^{23,24,25}. These architectures’ adherence to neural network models unlocks a wealth of metrics^{26}, algorithms^{27, 28}, tools^{29, 30}, and benchmarks^{31} developed specifically for neural networks. Likewise, scalable information processing with analog photonics would rely upon standards defining the relationship between photonic physics and a suitable processing model. Neural networks are among the most wellstudied models for information processing with distributed analog elements. The fact that distributed interconnection and analog dynamics are performance strongsuits of photonic physics motivates the study of neuromorphic photonics.
“Broadcastandweight”^{32} was proposed as a standard protocol for implementing neuromorphic processors using integrated photonic devices. Broadcastandweight is potentially compatible with mainstream silicon photonic device platforms, unlike freespace optical neural networks^{4, 33}. It is capable of implementing generalized reconfigurable and recurrent neural network models. In the broadcastandweight protocol, shown in Fig. 1(a), each neuron’s output is assigned a unique wavelength carrier that is wavelength division multiplexed (WDM) and broadcast. Incoming WDM signals are weighted by reconfigurable, continuousvalued filters called photonic weight banks and then summed by total power detection. This electrical weighted sum then modulates the corresponding WDM carrier through a nonlinear or dynamical electrooptic process. Previous work on microring (MRR) weight banks have established a correspondence between weighted addition operations and integrated photonic filters. In reference to the operation, MRR weight bank scalability^{34} and accuracy^{35} metrics can be defined, but MRR weight banks have not been demonstrated within a network.
In this manuscript, we demonstrate a broadcastandweight system configured by microring weight banks that is isomorphic to a continuoustime recurrent neural network (CTRNN) model. As opposed to “braininspired” and “neuromimetic”, “neuromorphic” is an unambiguous mathematical concept meaning that a physical system’s governing equations are isomorphic to those describing an abstract neural network model. Isomorphic dynamical systems share qualtitative changes in dynamics as a result of parameter variation. We adopt a strategy for proving neuromorphism experimentally by comparing these dynamical transitions (a.k.a. bifurcations) induced in an experimental device against those predicted of a CTRNN model. In particular, we observe singlenode bistability across a cusp bifurcation and twonode oscillation across a Hopf bifurcation. While oscillatory dynamics in optoelectronic devices have long been studied^{36, 37}, this work relies on configuring an analog photonic network that can be scaled to more nodes in principle. This implies that CTRNN metrics, simulators, algorithms, and benchmarks can be applied to larger neuromorphic silicon photonic systems. To illustrate the significance of this implication, we simulate a 24modulator silicon photonic CTRNN solving a differential equation problem. The system is programmed by appropriating an existing “neural compiler”^{29} and benchmarked against a conventional CPU solving the same problem, predicting an acceleration factor of 294×.
Results
The CTRNN model is described by a set of ordinary differential equations coupled through a weight matrix.
where s(t) are state variables with timeconstants τ, W is the recurrent weight matrix, y(t) are neuron outputs, w _{ in } are input weights, and u(t) is an external input. σ is a saturating transfer function associated with each neuron. Figure 1(b,c) shows the integrated, reconfigurable analog network and experimental setup. Signals u and y are physically represented as the power envelope of different optical carrier wavelengths. The weight elements of W and w _{ in } are implemented as transmission values through a network of reconfigurable MRR filters. The neuron transfer function, σ, is implemented by the sinusoidal electrooptic transfer function of a fiber MachZehnder modulator (MZM). The neuron state, s, is the electrical voltage applied to the MZM, whose timeconstant, τ, is determined by an electronic lowpass filter. We aim to establish a correspondence between experimental bifurcations induced by varying MRR weights and the modeled bifurcations^{38} derived in Supplementary Section 1.
Cusp Bifurcation
A cusp bifurcation characterizes the onset of bistability in a single node with selffeedback. To induce and observe a cusp, the feedback weight of node 1 is parameterized as w _{11} = W _{ F }. The neurosynaptic signal, s _{1}, is recorded as the feedback weight is swept through 500 points from 0.05–0.85, and the input is swept in the rising (blue surface) and falling (red surface) directions. The parameters τ, α, and κ in equation (S1.1) are fit to minimize root mean squared error between model and data surfaces. The best fit model has a cusp point at W _{ B } = 0.54. Figure 2(a) shows a modeled cusp surface described by equation (S1.1) and fit to data from Fig. 2(b). In Fig. 2(b), the data surfaces are interpolated at particular planes and projected onto 2D axes as red/blue points. The corresponding slices of the model surface are similarly projected as green lines. The u = 0 slice projected on the s − W _{ F } plane yields a pitchfork curve described by equation (S1.2). The W _{ F } = 0.08 slice projected on the u − s plane yields the bistable curve described by equation (S1.3). Finally, the s = 0 slice projected on the u − W _{ F } plane yields the cusp curve described by equation (S1.4).
The experimental reproduction of pitchfork, bistable, and cusp bifurcations is demonstrative of an isomorphism between the singlenode model and the device under test. An opening of an area between rising and falling data surfaces is characteristic of bistability. The transition boundary closely follows a cusp form. While the pitchfork and bistable slices reproduce the number and growth trends of fixed points, their fits have nonidealities. These nonidealities can be attributed to a hard saturation of the electrical transimpedance amplifier when the input voltage and feedback weight are high. Furthermore, the stability of cusp measurements serve as a control indicating the absence of timedelayed dynamics resulting from long fiber delays and causing spurious oscillations^{39}. Electrical lowpass filtering is used to eliminate these unmodeled dynamics in order to observe modeled bifurcations.
Hopf Bifurcation
Dynamical systems are capable of oscillating if there exists a closed orbit (a.k.a. limit cycle) in the state space, which must therefore exceed one dimension. The Hopf bifurcation occurs when a stable fixedpoint becomes unstable while giving rise to a stable limit cycle. Hopf bifurcations are further characterized by oscillations that approach zero amplitude and nonzero frequency near the bifurcation point^{38}. We induce a Hopf bifurcation by configuring the MRR weight matrix to have asymmetric, fixed offdiagonals and equal, parameterized diagonals.
Figure 3 compares the observed and predicted oscillation onset, amplitude, and frequency. Figure 3(a–c) show the time traces for below, near, and above the oscillation threshold. Above threshold, oscillation occurs in the range of 1–5 kHz, as limited by electronic lowpass filters and feedback delay. Figure 3(d) shows the result of a fine sweep of selffeedback weights in the 2node network, exhibiting the paraboloid shape of a Hopf bifurcation. The voltage of neuron 1 is plotted against that of neuron 2 with color corresponding to W _{ F } parameter. The peak oscillation amplitude for each weight is then projected onto the W _{ F } − y _{2} plane in black, and these amplitudes are fit using the model from equation (S1.8) (red). Bifurcation occurs at W _{ B } = 0.48 in the fit model. Figure 3(e) plots the oscillation frequency above the Hopf point. Data are discarded for W _{ B } < W _{ F } < 0.53 because the oscillations are erratic in the sensitive transition region. Frequency data are then fit with the model of equation (S1.10). The frequency axis is scaled so that 1.0 corresponds to the model frequency at region boundary, which is 4.81 kHz. The Hopf bifurcation only occurs in systems of more than one dimension, thus confirming the observation of a small integrated photonic neural network.
Significantly above the bifurcation point, experimental oscillation amplitude and frequency closely match model predictions, but discrepancies are apparent in the transition regime. Limit cycles with amplitudes comparable to noise amplitude can be destabilized by their proximity to the unstable fixed point at zero. This effect could explain the middle inset of Fig. 3, in which a small oscillation grows and then shrinks. Part of this discrepancy can be explained by weight inaccuracy due to interbank thermal crosstalk. The two MRR weight banks were calibrated independently accounting only for intrabank thermal crosstalk. As seen in Fig. 1(c), the physical distance between w _{12} (nominally −1) and w _{22} (nominally W _{ F }) is approximately 100 μm. While interbank crosstalk is not a major effect, w _{12} is very sensitive because weight −1 corresponds to onresonance, and the dynamics are especially sensitive to the weight values near the bifurcation point.
Emulation Benchmark
A dynamical isomorphism between a silicon photonic system and the CTRNN model of equations (1 and 2) implies that larger, faster neuromorphic silicon photonic systems could utilize algorithms and tools developed for generic CTRNNs. Here, we apply a “neural compiler” called the Neural Engineering Framework (NEF)^{40} to program a simulated photonic CTRNN to solve an ordinary differential equation (ODE). This simulation is benchmarked against a conventional central processing unit (CPU) solving the same task. The procedures for each approach are detailed in Methods. As opposed to implementationspecific metrics, benchmarks are taskoriented indicators suitable for comparing technologies that use disparate computing standards. Benchmarking approaches are therefore needed to evaluate the potentials of any unconventional processor in application domains currently served by conventional processors. The chosen benchmark problem consists of solving a wellknown ODE called the Lorenz attractor, described by a system of three coupled ODEs with no external inputs:
where x are the simulation state variables, and γ is a time scaling factor. When parameters, υ, β, and ρ, are set to (υ, β, ρ) = (6.5, 8/3, 28), the solutions of the attractor are chaotic. The photonic CTRNN and CPU solutions are compared in x phase space in Fig. 4(a,b) and the timedomain in Fig. 4(c,d). Figure 4(e) plots the physical modulator voltages, s, linear combinations of which represent simulation variables, x, as discussed in Methods. Because the two simulators are implemented differently, they cannot be compared based on equivalent metrics; however, the time scaling factor, γ, links physical realtime to a virtual simulation time basis, in which a direct comparison can be made.
Figure 4(c,d) plots photonic CTRNN and CPU solutions in the realtime bases, scaled to cover equal simulation intervals. The discretetime simulation is linked to physical realtime by the step calculation time, Δt = 24.5 ns, and its stability is limited by numerical divergence. We find that γ _{ CPU } ≥ Δt × 150 is sufficient for <1% divergence probability, resulting in γ _{ CPU } = 3.68 μs. The CTRNN simulation is linked to physical realtime by its feedback delay, t _{ fb } = 47.8 ps, and its stability is limited by timedelayed dynamics. We find that γ _{ Pho } ≥ t _{ fb } × 260 is sufficient to avoid spurious dynamics, resulting in γ _{ Pho } = 12.5 ns. The acceleration factor, γ _{ CPU }/γ _{ Pho }, is thus predicted to be 294×.
Implementing this network on a silicon photonic chip would require 24 laser wavelengths, 24 modulators, and 24 MRR weight banks, for a total of 576 MRR weights. The power used by the CTRNN would be dominated by static MRR tuning and pump lasers,as discussed in the Methods section. Minimum pump power is determined by the requirement that recurrently connected neurons are able to exhibit positive eigenvalues. Considering 24 lasers with a realistic wallplug efficiency of 5%, minimum total system power is expected to be 106 mW. The area used by the photonic CTRNN is split evenly between MRR weight banks (576 × 25 μm × 25 μm = 0.36 mm^{2}) and modulators^{41} (24 × 500 μm × 25 μm = 0.30 mm^{2}). The fundamental limits of these performance metrics are compared with other neuromorphic approaches below.
While the qualitative Lorenz behavior is reproduced by CTRNN and CPU implementations, the chaotic nature of the attractor presents a challenge for benchmarking emulation accuracy. Nonchaotic partial differential equations (PDEs) exist to serve as accuracy benchmarks^{42, 43}; however, most nontrivial ODEs are chaotic. One exception is work on central pattern generators (CPGs) that are used to shape oscillations for driving mechanical locomotion^{44}. CPGs have been implemented with analog ASICs^{45} and digital FPGAs^{46}. While work on CPG hardware has fallen in subkHz timescales, similar tasks could be developed for GHz timescales with possible application to adaptive RF waveform generation. Further work could develop CPGlike tasks to benchmark the accuracy of photonic CTRNN ODE emulators.
Accuracy can be assessed through metrics of weight accuracy. Previous work discussed the precision and accuracy to which MRR weight banks could be configured to a desired weight vector^{35}. Even in the presence of fabrication variation and thermal crosstalk, the dynamic weight accuracy was demonstrated to be 4.1 + 1(sign) bits and anticipated to increase with improved design. Since the weight is analog, its dynamic accuracy (range/command error) is not an integer; however, this metric corresponds to bit resolution in digital architectures. In the majority of modernday neuromorphic hardware, this resolution is selected to be 4 + 1(sign) bits^{24, 47, 48} as a tradeoff between hardware resources and functionality. Significant study has been devoted to the effect of limited weight resolution on neural network function^{49} and techniques for mitigating detrimental effects^{50}.
Neuromorphic photonic metrics
In addition to taskdriven benchmark analyses, we can perform componentlevel metric comparisions with other neuromorphic systems in terms of area, signal bandwidth, and synaptic operation (SOP) energy. The 24modulator photonic CTRNN used as an emulation benchmark was predicted to use a 4.4 mW/neuron using realistic pump lasers, and was limited to 1 GHz bandwidth to spoil spurious oscillations. This results in a computational efficiency of 180 fJ/SOP. The area of an MRR weight is approximately (l × w)/N ^{2} = 25 × 25 = 625 μm^{2}/synapse.
The coherent approach described by Shen et al.^{51} based on a matrix of MachZehnder interferometers (MZIs) would exhibit similar fundamental energy and speed limitations, given similar assumptions about detection bandwidth and laser efficiency. The power requirements of this approach were limited by nonlinear threshold activation, rather than signal cascadability. Supposing a laser efficiency of 5%, this was estimated to be around 20 mW/neuron. A 24neuron system limited to 1 GHz bandwidth would therefore achieve 830 fJ/SOP. While there is no onetoone correspondence between MZIs and synapses, there is still a quadratic area scaling relationship: (l × w)/N ^{2} = 200 × 100 = 20,000 μm^{2}/synapse, as limited by thermal phase shifter dimension.
The superconducting optoelectronic approach described by Shainline et al.^{52} would be optimized for scalability and efficiency, instead of speed. This can be attributed to the extreme sensitivity of cryogenic photodetectors, but this difference also limits signal bandwidth to 20 MHz. For a 700neuron interconnect, the wallplug efficiency is estimated to be around 20 fJ/SOP. Area was calculated to be (l × w)/N ^{2} = 1.4 × 15 = 21 μm^{2}/synapse.
A metric analysis can extend to include neuromorphic electronics, although metrics do not necessarily indicate signal processing merit. Electronic and photonic neuromorphics are designed to address complementary types of problems. Akopyan et al.^{24} demonstrated a chip containing 256 million synapses dissipating 65 mW. The signal bandwidth, determined by the effective tick, or timestep, is 1.0 kHz, and the chip area is 4.3 cm^{2}. This results in an effective 240 fJ/SOP and effective area of 6.0 μm^{2}/synapse. We note that TrueNorth is eventbased, meaning explicit computation does not occur for every effective SOP, but only when the input to a synapse is nonzero.
Discussion
We have demonstrated an isomorphism between a silicon photonic broadcastandweight system and a reconfigurable CTRNN model through observations of predicted bifurcations. In addition to proofofconcept, this repeatable method could be used to characterize the standard performance of single neurons and pairs of neurons within larger systems. Employing neuromorphic properties, we then illustrated a taskoriented programming approach and benchmark analysis. Similar analyses could assess the potentials of analog photonic processors against stateoftheart conventional processors in many application domains. Here, we discuss the implications of these results in the broader context of information processing with photonic devices.
This work constitutes the first investigation of photonic neurons implemented by modulators, an important step towards siliconcompatible neuromorphic photonics. Interest in integrated lasers with neuronlike spiking behavior has flourished over the past several years^{53, 54}. Experimental work has so far focused on isolated neurons^{55,56,57} and fixed, cascadable chains^{58, 59}. The shortage of research on networks of these lasers might be explained by the challenges of implementing lowloss, compact, and tunable filters in the active III/V platforms required for laser gain. In some cases where fanin is sensitive to input optical phase, it is also unclear how networking would occur without global laser synchronization. In contrast to lasers, MachZehnder, microring, and electroabsorption modulators are all siliconcompatible. Modulatorclass neurons are therefore a final step towards making complete broadcastandweight systems entirely compatible with silicon foundry platforms. While laserclass neurons with spiking dynamics present richer processing opportunities, modulatorclass neurons would still possess the formidable repertoire of CTRNN functions.
In parallel with work on individual laser neurons, recent research has also investigated systems with isomorphisms to neural network models. A fully integrated superconducting optoelectronic network was recently proposed^{52} to offer unmatched energy efficiency. While based on an exotic superconducting platform, this approach accomplishes fanin using incoherent optical power detection in a way compatible with the broadcastandweight protocol. A programmable nanophotonic processor was recently studied in the context of deep learning^{51}. Coherent optical interconnects exhibit a sensitivity to optical phase that must be resynchronized after each layer. In the demonstration, optical nonlinearity and phase regeneration were performed digitally. Analog solutions for counteracting signaldependent phase shifts induced by nonlinear materials^{60} have not yet been proposed. Recurrent neural networks have been investigated in fiber^{61}. While the current work employs fiber neurons, it is the first demonstration of a recurrent weight network that is integrated. Optical neural networks in freespace have also been investigated in the past^{4} and recently^{33}. Freespace systems occupy an extra dimension but can not necessarily use it for increased scalability. The volume between focal planes is used for diffractive evolution of the optical field and unused for network configuration. Spatial light modulators that configure the network are generally planar. Shainline et al. noted that integrated neuromorphic photonic systems could potentially be stacked to take advantage of a third dimension^{52}.
Reservoir computing techniques that take inspiration from certain brain properties (e.g. analog, distributed) have received substantial recent attention from the photonics community^{62,63,64,65}. Reservoir techniques rely on supervised learning to discern a desired behavior from a large number of complex dynamics, instead of relying on establishing an isomorphism to a model. Neuromorphic and reservoir approaches differ fundamentally and possess complementary advantages. Both derive a broad repertoire of behaviors (often referred to as complexity) from a large number of physical degreesoffreedom (e.g. optical intensities) coupled through interaction parameters (e.g. transmissions). Both offer means of selecting a specific, desired behavior from this repertoire using controllable parameters. In neuromorphic systems, network weights are both the interaction and controllable parameters, whereas, in reservoir computers, these two groups of parameters are separate. This distinction has two major implications. Firstly, the interaction parameters of a reservoir do not need to be observable or even repeatable from system to system. Reservoirs can thus derive complexity from physical processes that are difficult to model or reproduce, such as coupled amplifiers^{66}, coupled nonlinear MRRs^{67}, timedelayed dynamics in fibers^{64}, and fixed interferometric circuits^{63}. Furthermore, they do not require significant hardware to control the state of the reservoir. Neuromorphic hardware has a burden to correspond physical parameters (e.g. drive voltages) to model parameters (e.g. weights), as was shown in this paper. Secondly, reservoir computers can only be made to elicit a desired behavior through instancespecific supervised training, whereas neuromorphic computers can be programmed a priori using a known set of weights. Because neuromorphic behavior is determined only by controllable parameters, these parameters can be mapped directly between different system instances, different types of neuromorphic systems, and simulations. Neuromorphic hardware can leverage existing algorithms (e.g. NEF) and virtual training results. Particular behaviors, fully determined by the virtual/hardware weights, are guaranteed to occur. Photonic RCs can of course be simulated; however, they have no corresponding guarantee that a particular hardware instance will reproduce a simulated behavior or that training will be able to converge to this behavior.
At increased scale, neuromorphic silicon photonic systems could be applied to unaddressed computational areas in scientific computing and RF signal processing. A key benefit of neuromorphic engineering is that existing algorithms can be leveraged. A subset of CTRNNs, Hopfield networks^{68}, have been used extensively in mathematical programming and optimization problems^{27}. The ubiquity of PDE problems in scientific computing has motivated the development of analog electronic neural emulators^{42}. Further work could explore the use of NEF to emulate discrete space points of PDEs. Neural algorithms for CTRNNs have been developed for realtime RF signal processing, including spectral mining^{69}, spread spectrum channel estimation^{70}, and arrayed antenna control^{71}. There is insistent demand to implement these tasks at wider bandwidths using less power than possible with RF electronics. Additionally, methodologies developed for audio applications, such as noise mitigation^{28}, could conceivably be mapped to RF problems if implemented on ultrafast hardware. Unsupervised neuralinspired learning has been used with a single MRR weight bank for statistical analysis of multiple RF signals^{72}.
We have demonstrated a reconfigurable analog neural network in a silicon photonic integrated circuit using modulators as neuron elements. Networkmediated cusp and Hopf bifurcations were observed as a proofofconcept of an integrated broadcastandweight system^{32}. Simulations of a 24 modulator neuron network performing an emulation task estimated a 294× speedup over a verified CPU benchmark. Neural network abstractions are powerful tools for bridging the gap between physical dynamics and useful application, and silicon photonic manufacturing introduces opportunities for largescale photonic systems.
Methods
Experimental Setup
Samples shown in Fig. 1(b) were fabricated on silicononinsulator (SOI) wafers at the Washington Nanofabrication Facility through the SiEPIC Ebeam rapid prototyping group^{10}. Silicon thickness is 220 nm, and buried oxide (BOX) thickness is 3 μm. 500 nm wide WGs were patterned by Ebeam lithography and fully etched through to the BOX^{73}. After a cladding oxide (3 μm) is deposited, Ti/W and Al layers are deposited. Ohmic heating in Ti/W filaments causes thermooptic resonant wavelength shifts in the MRR weights. The sample is mounted on a temperature stabilized alignment stage and coupled to a 9fiber array using focusing subwavelength grating couplers^{74}. The reconfigurable analog network consists of 2 MRR weight banks each with four MRR weights with 10 μm radii.
Each MRR weight bank is calibrated using a multichannel protocol described in past work^{35, 75}: an offline measurement procedure is performed to identify models of thermooptic crosstalk and MRR filter edge transmission. During this calibration phase, electrical feedback connections are disabled and the set of wavelength channels carry a set of linearly seperable training signals. After calibration, the user can specify a desired weight matrix, and the control model calculates and applies the corresponding electrical currents.
Weighted network outputs are detected offchip, and the electrical weighted sums drive fiber MachZehnder modulators (MZMs). Detected signals are lowpass filtered at 10 kHz, represented by capacitor symbols in Fig. 1(c). Lowpass filtering is used to spoil timedelayed dynamics that arise when feedback delay is much greater than the state timeconstant^{37}. In this setup with onchip network and offchip modulator neurons, fiber delayed dynamics would interfere with CTRNN dynamical analysis^{39}. MZMs modulate distinct wavelengths λ _{1} = 1549.97 nm and λ _{2} = 1551.68 nm with neuron output signals y _{1}(t) and y _{2}(t), respectively. The MZM electrooptic transfer function serves as the nonlinear transfer function, y = σ(s), associated with the continuoustime neuron. A third wavelength, λ _{3} = 1553.46 nm, carries an external input signal, x(t), derived from a signal generator. Each laser diode source (ILX 7900B) outputs +13 dBm of power. All optical signals (u, y _{1}, and y _{2}) are wavelength multiplexed in an arrayed waveguide grating (AWG) and then coupled back into the onchip broadcast STAR consisting of splitting Yjunctions^{76} (Fig. 1(c)).
Photonic CTRNN Solver
Recently developed compilers, such as Neural ENGineering Objects (Nengo)^{77}, employ the Neural Engineering Framework (NEF)^{40} to arrange networks of neurons to represent values and functions without relying on training. While originally developed to evaluate theories of cognition, the NEF has been appropriated to solve engineering problems^{78} and has been used to program electronic neuromorphic hardware^{79}. Background on the NEF compilation procedure is provided in Supplementary Section 2. Simulation state variables, x, are encoded as linear combinations of real population states, s. Each neuron in a population has the same tuning curve shape, σ, but differ in gain g, input encoder vector e, and offset b. The inputoutput relation of neuron i – equivalent to equation (2), is thus s _{ i } = σ (g _{ i } e _{ i } · x + b _{ i }). In this formulation, arbitrary nonlinear functions of the simulation variables, f(x), are represented by linear combinations of the set of these tuning curves across the domain of values of x considered. Introducing recurrent connections in the population introduces the notion of state timederivatives, as in equation (1). By applying the decoder transform to both sides of equation (1) and using the arbitrary function mapping technique to find W, the neural population emulates an effective dynamical system of the form \(\dot{{\bf{x}}}={\bf{f}}({\bf{x}})\). Given equations (3) stated in this form, Nengo performs the steps necessary to represent the variables, functions, and complete ODE.
Modifications were made to the standard Nengo procedure. Firstly, we specify the tuning curve shape as the sinusoidal electrooptic transfer characteristic of a MZM. Secondly, to reduce the number of MZMs required, we choose encoders to be the vertices of a unitsquare {e} = [1, ±1, ±1], while they are typically chosen randomly. Thirdly, the MZM sinusoidal transfer function provides a natural relation to the Fourier basis. Gains are chosen to correspond to the first three Fourier frequencies of the domain: g ∈ s _{ π }/2 · {1, 2, 3}, where s _{ π } is the MZM halfperiod. Offsets were chosen to be b ∈ {0, s _{ π }/2}, corresponding to sine and cosine components of each gain frequency. The total number of modulator neurons is therefore #e · #g · #b = 4 · 3 · 2 = 24. Figure 4(e) shows the MZM states, s(t), of which simulation variables, x(t), are linear combinations. From this plot, it appears that some neurons are barely used. Thus, further optimizations of number of neurons could be made by pruning those neurons after compilation of the weight matrix.
The operational speed of this network would be limited by timeofflight feedback delay. In Fig. 1(a), the longest feedback path is via the drop port of the last (pink) MRR weight of the first (yellow) neuron’s bank. The path includes the perimeter of the square MRR weight matrix, plus a drop waveguide folded back along the bank. Supposing a minimum MRR pitch of 25 μm and MZM length of 500 μm, the feedback delay would then be (6 × 25 × 25 + 500) · n/c = 48 ps. We model this delayed feedback in the Nengo simulation and then adjust feedback strength to find the minimum stable simulation timescale. For γ _{ Pho }/t _{ fb } < 65, spurious timedelayed dynamics dominate. For γ _{ Pho }/t _{ fb } < 104, the butterfly phase diagram in Fig. 4(b) is not reproduced accurately. γ _{ Pho }/t _{ fb } ≥ 260 is chosen for robust reproduction of the expected dynamics.
Conventional CPU Solver
Conventional digital processors must use a discretetime approximation to simulate continuous ODEs, the simplest of which is Euler continuation:
where Δt is the time step interval. To estimate the realtime value of Δt, we develop and validate a simple CPU model. For each time step, the CPU must compute f(x[nΔt]) as defined in equation (3), resulting in 9 floatingpoint operations (FLOPs), and 12 cache reads of the operands. The Euler update in equation (4) constitutes one multiply, one addition, and one read/write for each state variable, resulting in 6 FLOPs and 6 cache accesses. Supposing a FLOP latency of 1 clock cycle, Level 1 (L1) cache latency of 4 cycles, and 2.6 GHz clock, this model predicts a time step of Δt = 33 ns. This model is empirically validated using an Intel Core i54288U. The machineoptimized program randomly initializes and loops through 10^{6} Euler steps of the Lorenz system, over 100 trials. CPU time was measured to be Δt = 24.5 ± 1.5 ns. The minimum stable simulation timescale is limited by divergent errors stemming from time discretization. We performed a series of 100 trials over 100 values of Δt/γ _{ CPU }, finding that <1% probability of divergence occurred for γ _{ CPU }/Δt ≥ 150.
Minimum Power Calculations
Static thermal power must be applied to each weight in order to track MRRs to the onresonance condition. Supposing a bank length set by an MRR pitch of 25 μm and count of 24, the MRR network would occupy a square with 600 μm sides. Within this length, resonances can be fabricated with repeatability within ±1.3 nm^{80}. Supposing a tuning efficiency of 0.25 nm/mW^{81}, it would take an average of 5.2 mW/weight to track resonance, for a static power dissipation of 3.0 W. On the other hand, if depletionbased tuning can be used, there would be negligible static power dissipation in the weights.
The laser bias power must be set such that a modulator neuron can drive downstream neurons with sufficient strength. A neuron fed back to itself should be able to elicit an equivalent or greater smallsignal response after one roundtrip. This condition is referred to as signal cascadability and can be stated as g ≥ 1, where g is roundtrip, smallsignal gain. If the cascadability condition is not met, all signals will eventually attenuate out with time. In other words, in a recurrent network, the real part of system eigenvalues would not be able to exceed zero. Roundtrip gain is expressed as
For a modulatorbased broadcastandweight system, this breaks down into receiver and modulator components. Assuming a voltagemode modulator, such as reversebiased MRR depletion modulator,
where V _{ π } is modulator πvoltage, P _{ pump } is modulator pump power, and R _{ PD } is detector responsivity. Because input power generates a photocurrent, yet a depletion modulator is voltagedriven, the receiver’s impedance, R _{ r }, determines the conversion and can be set externally. As R _{ r } increases, roundtrip gain also increases, but bandwidth decreases according to f = (2πR _{ r } C _{ mod })^{−1}, where C _{ mod } is PN junction capacitance of the modulator. By setting the cascadability condition: g = 1 and combining the above equations, we find that
The values of V _{ π }, C _{ mod }, and R _{ PD } on a typical silicon photonic foundry platform have been published^{41}. For an MRR depletion modulator, V _{ π } = 1.5 V, C _{ mod } = 35 fF. For a PD on the same platform, R _{ PD } = 0.97 A/W. This means that the minimum pump power for a given signal bandwidth is 2.2 × 10^{−13} W/Hz.
In this paper, we study a 24node CTRNN whose signal bandwidth is restricted to 1 GHz to avoid timedelay dynamics. This means that, for the cascadability condition to be met, modulator pumping must be at least 0.22 mW/neuron of optical power. Adding up 24 lasers and accounting for laser inefficiency, wallplug system power would be 106 mW.
References
Keyes, R. W. Optical logicin the light of computer technology. Optica Acta: International Journal of Optics 32, 525–535 (1985).
Reimann, O. A. & Kosonocky, W. F. Progress in optical computer research. IEEE Spectrum 2, 181–195 (1965).
McCormick, F. B. et al. Sixstage digital freespace optical switching network using symmetric selfelectroopticeffect devices. Appl. Opt. 32, 5153–5171 (1993).
Jutamulia, S. & Yu, F. Overview of hybrid optical neural networks. Optics & Laser Technology 28, 59–72 (1996).
Vlasov, Y. Silicon CMOSintegrated nanophotonics for computer and data communications beyond 100 G. IEEE Commun. Mag. 50, s67–s72 (2012).
Hochberg, M. et al. Silicon photonics: The next fabless semiconductor industry. IEEE SolidState Circuits Magazine 5, 48–58 (2013).
Thomson, D. et al. Roadmap on silicon photonics. Journal of Optics 18, 073003 (2016).
Lim, A.J. et al. Review of silicon photonics foundry efforts. IEEE J. Sel. Top. Quantum Electron. 20, 405–416 (2014).
Orcutt, J. S. et al. Open foundry platform for highperformance electronicphotonic integration. Opt. Express 20, 12222–12232 (2012).
Chrostowski, L. & Hochberg, M. Silicon Photonics Design: From Devices to Systems (Cambridge University Press, 2015).
Sun, J. et al. Largescale silicon photonic circuits for optical phased arrays. Selected Topics in Quantum Electronics, IEEE Journal of 20, 264–278 (2014).
Beausoleil, R. G. Largescale integrated photonics for highperformance interconnects. J. Emerg. Technol. Comput. Syst. 7, 6:1–6:54 (2011).
Le Beux, S. et al. Optical ring networkonchip (ORNoC): Architecture and design methodology. In Design, Automation Test in Europe Conference Exhibition (DATE), 2011, 1–6 (2011).
Narayana, V. K., Sun, S., Badawy, A.H. A., Sorger, V. J. & ElGhazawi, T. MorphoNoC: Exploring the Design Space of a Configurable Hybrid NoC using Nanophotonics. arXiv:1506.03264 (2017).
Capmany, J. et al. Microwave photonic signal processing. Journal of Lightwave Technology 31, 571–586 (2013).
Farsaei, A. et al. A review of wirelessphotonic systems: Design methodologies and topologies, constraints, challenges, and innovations in electronics and photonics. Optics Communications (2016).
Feng, N.N. et al. Thermallyefficient reconfigurable narrowband RFphotonic filter. Opt. Express 18, 24648–24653 (2010).
Zhuang, L., Roeloffzen, C. G. H., Hoekman, M., Boller, K.J. & Lowery, A. J. Programmable photonic signal processor chip for radiofrequency applications. Optica 2, 854–859 (2015).
Valley, G. C. Photonic analogtodigital converters. Opt. Express 15, 1955–1982 (2007).
Khan, M. H. et al. Ultrabroadbandwidth arbitrary radiofrequency waveform generation with a silicon photonic chipbased spectral shaper. Nature: Photonics 4, 117–122 (2010).
Chang, J., Meister, J. & Prucnal, P. R. Implementing a novel highly scalable adaptive photonic beamformer using “blind” guided accelerated random search. Journal of Lightwave Technology 32, 3623–3629 (2014).
Ferreira de Lima, T., Tait, A. N., Nahmias, M. A., Shastri, B. J. & Prucnal, P. R. Scalable wideband principal component analysis via microwave photonics. IEEE Photonics Journal 8, 1–9 (2016).
Merolla, P. A. et al. A million spikingneuron integrated circuit with a scalable communication network and interface. Science 345, 668–673 (2014).
Akopyan, F. et al. Truenorth: Design and tool flow of a 65 mW 1 million neuron programmable neurosynaptic chip. IEEE Trans. Comput. Aided Des. Integr. Circuits Syst. 34, 1537–1557 (2015).
Indiveri, G. & Liu, S. C. Memory and information processing in neuromorphic systems. Proceedings of the IEEE 103, 1379–1397 (2015).
Hasler, J. & Marr, H. B. Finding a roadmap to achieve large neuromorphic hardware systems. Front. Neurosci. 7 (2013).
Wen, U.P., Lan, K.M. & Shih, H.S. A review of Hopfield neural networks for solving mathematical programming problems. European Journal of Operational Research 198, 675–687 (2009).
Lee, T. & Theunissen, F. A single microphone noise reduction algorithm based on the detection and reconstruction of spectrotemporal features. Proceedings of the Royal Society of London A: Mathematical, Physical and Engineering Sciences 471 (2015).
Eliasmith, C. & Anderson, C. H. Neural engineering: Computation, representation, and dynamics in neurobiological systems (MIT Press, 2004).
Donnarumma, F., Prevete, R., de Giorgio, A., Montone, G. & Pezzulo, G. Learning programs is better than learning dynamics: A programmable neural network hierarchical architecture in a multitask scenario. Adaptive Behavior 24, 27–51 (2016).
Diamond, A., Nowotny, T. & Schmuker, M. Comparing neuromorphic solutions in action: implementing a bioinspired solution to a benchmark classification task on three parallelcomputing platforms. Frontiers in Neuroscience 9 (2016).
Tait, A. N., Nahmias, M. A., Shastri, B. J. & Prucnal, P. R. Broadcast and weight: An integrated network for scalable photonic spike processing. Journal of Lightwave Technology 32, 4029–4041 (2014).
Brunner, D. & Fischer, I. Reconfigurable semiconductor laser networks based on diffractive coupling. Optics letters 40, 3854–3857 (2015).
Tait, A. N. et al. Microring weight banks. IEEE Journal of Selected Topics in Quantum Electronics 22 (2016).
Tait, A. N., Ferreira de Lima, T., Nahmias, M. A., Shastri, B. J. & Prucnal, P. R. Multichannel control for microring weight banks. Opt. Express 24, 8895–8906 (2016).
Yamada, M. A theoretical analysis of selfsustained pulsation phenomena in narrowstripe semiconductor lasers. IEEE Journal of Quantum Electronics 29, 1330–1336 (1993).
Romeira, B. et al. Broadband chaotic signals and breather oscillations in an optoelectronic oscillator incorporating a microwave photonic filter. Lightwave Technology, Journal of 32, 3933–3942 (2014).
Beer, R. D. On the dynamics of small continuoustime recurrent neural networks. Adaptive Behavior 3, 469–509 (1995).
Zhou, E. et al. Silicon photonic weight bank control of integrated analog network dynamics. In Optical Interconnects Conference, 2016 IEEE, TuP9 (IEEE, 2016).
Stewart, T. C. & Eliasmith, C. Largescale synthesis of functional spiking neural circuits. Proceedings of the IEEE 102, 881–898 (2014).
Khanna, A. IMEC silicon photonics platform. In European Conference on Optical Communication (2015).
Roska, T. et al. Simulating nonlinear waves and partial differential equations via cnn. i. basic techniques. Circuits and Systems I: Fundamental Theory and Applications, IEEE Transactions on 42, 807–815 (1995).
Ratier, N. Analog computing of partial differential equations. In Sciences of Electronics, Technologies of Information and Telecommunications (SETIT), 2012 6th International Conference on, 275–282 (2012).
Vogelstein, R. J., Tenore, F. V. G., Guevremont, L., EtienneCummings, R. & Mushahwar, V. K. A silicon central pattern generator controls locomotion in vivo. IEEE Transactions on Biomedical Circuits and Systems 2, 212–222 (2008).
Arena, P., Fortuna, L., Frasca, M. & Patane, L. A cnnbased chip for robot locomotion control. IEEE Transactions on Circuits and Systems I: Regular Papers 52, 1862–1871 (2005).
BarronZambrano, J. H. & TorresHuitzil, C. {FPGA} implementation of a configurable neuromorphic cpgbased locomotion controller. Neural Networks 45, 50–61 Neuromorphic Engineering: From Neural Systems to BrainLike Engineered Systems (2013).
Friedmann, S., Fr’emaux, N., Schemmel, J., Gerstner, W. & Meier, K. Rewardbased learning under hardware constraints  using a RISC processor embedded in a neuromorphic substrate. Front. Neurosci. 7 (2013).
Benjamin, B. et al. Neurogrid: A mixedanalogdigital multichip system for largescale neural simulations. Proceedings of the IEEE 102, 699–716 (2014).
Pfeil, T. et al. Is a 4bit synaptic weight resolution enough? – constraints on enabling spiketiming dependent plasticity in neuromorphic hardware. Frontiers in Neuroscience 6, 90 (2012).
Binas, J., Neil, D., Indiveri, G., Liu, S.C. & Pfeiffer, M. Precise deep neural network computation on imprecise lowpower analog hardware. arXiv preprint arXiv:1606.07786 (2016).
Shen, Y. et al. Deep learning with coherent nanophotonic circuits. arXiv:1610.02365 (2016).
Shainline, J. M., Buckley, S. M., Mirin, R. P. & Sae Woo, N. Superconducting optoelectronic circuits for neuromorphic computing. arXiv preprint arXiv:1610.00053 (2016).
Nahmias, M. A., Shastri, B. J., Tait, A. N. & Prucnal, P. R. A leaky integrateandfire laser neuron for ultrafast cognitive computing. IEEE J. Sel. Top. Quantum Electron. 19, 1–12 (2013).
Prucnal, P. R., Shastri, B. J., Ferreira de Lima, T., Nahmias, M. A. & Tait, A. N. Recent progress in semiconductor excitable lasers for photonic spike processing. Adv. Opt. Photon. 8, 228–299 (2016).
Selmi, F. et al. Relative refractory period in an excitable semiconductor laser. Phys. Rev. Lett. 112, 183902 (2014).
Romeira, B., Av’o, R., Figueiredo, J. M. L., Barland, S. & Javaloyes, J. Regenerative memory in timedelayed neuromorphic photonic resonators. Scientific Reports 6, 19510 EP – (2016).
Nahmias, M. A. et al. An integrated analog O/E/O link for multichannel laser neurons. Applied Physics Letters 108 (2016).
Vaerenbergh, T. V. et al. Cascadable excitability in microrings. Opt. Express 20, 20292–20308 (2012).
Shastri, B. J. et al. Spike processing with a graphene excitable laser. Sci. Rep. 5, 19126 (2015).
Zhang, H. et al. Zscan measurement of the nonlinear refractive index of graphene. Opt. Lett. 37, 1856–1858 (2012).
Hill, M., Frietman, E. E. E., de Waardt, H., Khoe, G.D. & Dorren, H. All fiberoptic neural network using coupled soa based ring lasers. IEEE Trans. Neural Networks 13, 1504–1513 (2002).
Brunner, D., Soriano, M. C., Mirasso, C. R. & Fischer, I. Parallel photonic information processing at gigabyte per second data rates using transient states. Nat Commun 4, 1364 (2013).
Vandoorne, K. et al. Experimental demonstration of reservoir computing on a silicon photonics chip. Nat Commun 5 (2014).
Soriano, M. C., Brunner, D., EscalonaMor’an, M., Mirasso, C. R. & Fischer, I. Minimal approach to neuroinspired information processing. Frontiers in Computational Neuroscience 9, 68 (2015).
Duport, F., Smerieri, A., Akrout, A., Haelterman, M. & Massar, S. Fully analogue photonic reservoir computer. Scientific Reports 6, 22381 EP – (2016).
Vandoorne, K. et al. Toward optical signal processing using photonic reservoir computing. Opt. Express 16, 11182–11192 (2008).
Mesaritakis, C., Papataxiarhis, V. & Syvridis, D. Micro ring resonators as building blocks for an alloptical highspeed reservoircomputing bitpatternrecognition system. J. Opt. Soc. Am. B 30, 3048–3055 (2013).
Hopfield, J. J. & Tank, D. W. “Neural” computation of decisions in optimization problems. Biological Cybernetics 52, 141–152 (1985).
Tumuluru, V. K., Wang, P. & Niyato, D. A neural network based spectrum prediction scheme for cognitive radio. In Communications (ICC), 2010 IEEE International Conference on, 1–5 (2010).
Mitra, U. & Poor, H. V. Neural network techniques for adaptive multiuser demodulation. IEEE Journal on Selected Areas in Communications 12, 1460–1470 (1994).
Du, K.L., Lai, A., Cheng, K. & Swamy, M. Neural methods for antenna array signal processing: a review. Signal Processing 82, 547–561 (2002).
Tait, A. et al. Silicon microring weight banks for multivariate RF photonics. In CLEO: 2017 (IEEE, 2017 (accepted)).
Bojko, R. J. et al. Electron beam lithography writing strategies for low loss, high confinement silicon optical waveguides. J. Vac. Sci. Technol., B 29 (2011).
Wang, Y. et al. Focusing subwavelength grating couplers with low back reflections for rapid prototyping of silicon photonic circuits. Opt. Express 22, 20652–20662 (2014).
Tait, A., F de Lima, T., Nahmias, M., Shastri, B. & Prucnal, P. Continuous calibration of microring weights for analog optical networks. Photonics Technol. Lett. 28, 887–890 (2016).
Zhang, Y. et al. A compact and low loss Yjunction for submicron silicon waveguide. Opt. Express 21, 1310–1316 (2013).
Bekolay, T. et al. Nengo: a Python tool for building largescale functional brain models. Frontiers in Neuroinformatics 7, 48 (2013).
Friedl, K. E., Voelker, A. R., Peer, A. & Eliasmith, C. Humaninspired neurorobotic system for classifying surface textures by touch. IEEE Robotics and Automation Letters 1, 516–523 (2016).
Mundy, A., Knight, J., Stewart, T. & Furber, S. An efficient SpiNNaker implementation of the neural engineering framework. In Neural Networks (IJCNN), 2015 International Joint Conference on, 1–8 (2015).
Chrostowski, L. et al. Impact of fabrication nonuniformity on chipscale silicon photonic integrated circuits. In Optical Fiber Communication Conference, Th2A.37 (Optical Society of America, 2014).
Jayatilleka, H. et al. Wavelength tuning and stabilization of microringbased filters using silicon inresonator photoconductive heaters. Opt. Express 23, 25084–25097 (2015).
Acknowledgements
This work is supported by National Science Foundation (NSF) Enhancing Access to the Radio Spectrum (EARS) program (Award 1642991). Fabrication support was provided via the Natural Sciences and Engineering Research Council of Canada (NSERC) Silicon ElectronicPhotonic Integrated Circuits (SiEPIC) Program. Devices were fabricated by Richard Bojko at the University of Washington Washington Nanofabrication Facility, part of the NSF National Nanotechnology Infrastructure Network (NNIN).
Author information
Affiliations
Contributions
A.N.T. conceived of the methods and experiments. A.N.T., E.Z., and A.X.W. conducted the experiments and analyzed the results. T.FdL developed methods and simulations of the photonic CTRNN emulator. All authors reviewed the manuscript.
Corresponding author
Ethics declarations
Competing Interests
The authors declare that they have no competing interests.
Additional information
Publisher's note: Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
Electronic supplementary material
Rights and permissions
Open Access This article is licensed under a Creative Commons Attribution 4.0 International License, which permits use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons license, and indicate if changes were made. The images or other third party material in this article are included in the article’s Creative Commons license, unless indicated otherwise in a credit line to the material. If material is not included in the article’s Creative Commons license and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder. To view a copy of this license, visit http://creativecommons.org/licenses/by/4.0/.
About this article
Cite this article
Tait, A.N., de Lima, T.F., Zhou, E. et al. Neuromorphic photonic networks using silicon photonic weight banks. Sci Rep 7, 7430 (2017). https://doi.org/10.1038/s4159801707754z
Received:
Accepted:
Published:
DOI: https://doi.org/10.1038/s4159801707754z
Further reading

Ultrafast neuromorphic photonic image processing with a VCSEL neuron
Scientific Reports (2022)

A small microring array that performs large complexvalued matrixvector multiplication
Frontiers of Optoelectronics (2022)

Photonics enabled intelligence system to identify SARSCoV 2 mutations
Applied Microbiology and Biotechnology (2022)

Research progress in optical neural networks: theory, applications and developments
PhotoniX (2021)

Collective and synchronous dynamics of photonic spiking neurons
Nature Communications (2021)
Comments
By submitting a comment you agree to abide by our Terms and Community Guidelines. If you find something abusive or that does not comply with our terms or guidelines please flag it as inappropriate.