An optoacoustic field-programmable perceptron for recurrent neural networks

Recurrent neural networks (RNNs) can process contextual information such as time series signals and language. But their tracking of internal states is a limiting factor, motivating research on analog implementations in photonics. While photonic unidirectional feedforward neural networks (NNs) have demonstrated big leaps, bi-directional optical RNNs present a challenge: the need for a short-term memory that (i) programmable and coherently computes optical inputs, (ii) minimizes added noise, and (iii) allows scalability. Here, we experimentally demonstrate an optoacoustic recurrent operator (OREO) which meets (i, ii, iii). OREO contextualizes the information of an optical pulse sequence via acoustic waves. The acoustic waves link different optical pulses, capturing their information and using it to manipulate subsequent operations. OREO’s all-optical control on a pulse-by-pulse basis offers simple reconfigurability and is used to implement a recurrent drop-out and pattern recognition of 27 optical pulse patterns. Finally, we introduce OREO as bi-directional perceptron for new classes of optical NNs.


Introduction
Understanding the context of a situation is a powerful ability of the human brain, allowing it to predict possible outcomes and to make intelligent decisions.While humans can access the context of a situation via the short-term memory, machines struggle in contextualizing.Artificial neural networks, one of the most powerful computing architectures, face this problem as well.
To overcome this limitation, they can be equipped with recurrent feedback, allowing them to process current inputs based on previous ones.The so-called recurrent neural networks (RNNs) can contextualize, recognize, and predict sequences of information and are applied for numerous applications such as language processing tasks, and for video and image processing [1][2][3][4][5] .One of the simplest versions of a RNN is the Elman network 6 , which adds a recurrent operation to each neuron of its fully-connected network, analogous to the neuron's activation function.With this three-layer network, Elman was already able to understand simple grammatical structure.
Currently, the scientific community aims to transfer electronic neural networks into the optical domain.The resulting optical neural networks have attracted great interest due to their promises of high processing speed and broad bandwidth, and low dissipative losses [10][11][12] .Thus, they are considered to pave the way towards energy efficient and highly parallel optical circuits, enhancing the performance and capabilities of future artificial neural networks [13][14][15][16][17][18][19] .
Although the field of optical neural networks has made great progress in recent years, the field of recurrent optical neural networks is still very limited to concepts based on artificial reservoirs, such as free-space cavities 20 , delay systems 21,22 , and microring resonators 23 .These designs can face several challenging issues.Firstly, the usage of an artificial cavity, e.g. a ring resonator, can limit the scalability of those networks.Secondly, the cavity may not be frequency sensitive, preventing them from being applied for resource-efficient multi-frequency data processing.Finally, the cavity's recurrent weights cannot be varied rapidly, limiting the control of the recurrent process such as the implementation of recurrent dropout on single pulse level in order to regularize the network.
Here, we experimentally demonstrate an optoacoustic recurrent operator (OREO) based on stimulated Brillouin-Mandelstam scattering (SBS) that can unlock recurrent functionalities in existing optical neural network architectures (see Fig. 1 A).SBS is an interaction of optical waves with traveling acoustic waves which serve in our system as a latency component due to the slow acoustic velocity.OREO is therefore able to contextualize a time-encoded stream of information by using acoustic waves as a memory to remember previous operations (see Fig. 1B).
In contrast to previously reported approaches [20][21][22][23][24] , OREO controls its coherent recurrent operation completely optically on pulse level without the need of any artificial reservoir such as Figure 1: Schematic of the optoacoustic recurrent operator (OREO) and its proposed function in a recurrent NN.A -An example of a photonic recurrent network with N layers X n , which are connected by a matrix operation W n .The blue arrow indicates the recurrent nature of one neuron.B -The bi-directional perceptron contains an OREO and an activation function.OREO captures and links sequential information a D using a sound wave b, which is generated by SBS and controlled by an optical control pulse a C .The output of the acoustic recurrent neuron a ′ D is fed into the next layer of the neural network.C -Conceptional illustration of the SBS process with its interaction Hamiltonian H int .The sound wave b carries the information of the neuron's input and decays after the SBS process.D -G: Illustration of three recurrent operations performed by OREO.D shows the initial situation with three data-control pulse pairs separated by a deadtime dt.The data and control pulses are launched from opposite sides into a photonic crystal fiber (PCF).E shows the system after the SBS-interaction of a D,1 and a C,1 , which transfers energy from a D,1 to an acoustic wave b 1 .F shows the system after a second pulse pair has passed the PCF.The acoustic wave b 1 connects the interaction of a D,2 and a C,2 with the previous one, while the SBS process transfers information from a D,2 into b 2 .G highlights the acoustic link created by OREO between three optical pulse pairs.a ring resonator or a delay system.Hence, OREO does not rely on complicated manufacturing processes of microstructures.It functions in any optical waveguide, including on-chip devices, as it harvests the physical property of a sound wave [25][26][27] .In particular, with the announcement of the first on-chip EDFA 28 a fully integrated design is seemingly close.
We demonstrate OREO experimentally from different perspectives.Firstly, we show how OREO links different input states of subsequent optical pulses to each other via acoustic waves.
Secondly, we present how the all-optical control of OREO can be used to implement a recurrent dropout.Finally, we apply OREO as an acceptor 29 to predict up to 27 different patterns carried by a time series of input pulses.

Concept of an optoacoustic recurrent operator
The recurrent operation of OREO is based on the interaction of optical and acoustic waves through SBS, which is one of the most prominent third-order nonlinear effects and describes the coherent coupling of two optical waves, data and control, to an acoustic wave in a material.
The dynamic is illustrated in Figure 1C and follows from the Hamiltonian (1) [30][31][32] : using the optoacoustic coupling constant g 0 , the frequency relation between the optical fields ω D = ω C + Ω, and the wave packet operators a D , a C , b of the data, control and acoustic field, respectively.Similar to the clinking of a wine glass, the acoustic wave b persists beyond its excitation, decaying exponentially with time b(t) ∝ exp (−t/τ ac ), where τ ac ∝ Γ −1 ac is the acoustic lifetime, which depends on the properties of the used waveguide and is for a photonic crystal fiber (PCF) about τ ac ≈ 10 ns (see Fig. 1C).As a result, an acoustic wave b i can seed subsequent SBS processes j > i.Moreover, the acoustic builds up with each SBS process, which can be described as a superposition of all previous created acoustic waves b i with amplitude b 0,i , created at the time t i and carrying a phase φ i .Hence, the acoustic wave b N after N SBS interactions: yields the recurrence in the interaction Hamiltonian: Equation (3) shows furthermore that programming the field a C controls the acoustic feedback all-optically, enabling a pulse-by-pulse increase or suppression.For instance, setting a C,i = 0 corresponds to a recurrent dropout so that a D,i leaves the fiber unchanged.
We experimentally implement OREO in a telecom-fiber apparatus illustrated in Figure 1 D.
Here we launch several consecutive optical input data pulses a D,i and strong counter-propagating optical control pulses a C,i into a PCF.The optical data pulses are shifted up in frequency by Ω ≈ 10.6 GHz compared to the optical control pulses, which is close to the Brillouin frequency of the PCF.When a data and control pulse pair a D,1 and a C,1 meets inside the PCF, they induce SBS, depleting the data pulse and transferring its energy into the acoustic domain.Eventually, an acoustic wave b 1 is generated, which persists much longer than the optical interaction (see Figure 1 E).An optoacoustic recurrent operation is performed, when a subsequent data and control pulse pair (a D,2 and a C,2 ) reaches the acoustic wave b 1 before it has decayed.Hence, the deadtime dt until the second pulse pair arrives must be less then the acoustic lifetime.The previously generated acoustic wave connects to the subsequent SBS process between a D,2 and a C,2 and establishes a link of the second data pulse a D,2 to the first data pulse a D,1 .In addition, the second SBS process creates a second acoustic wave b 2 , carrying information of a D,2 .Now, the acoustic domain holds information of both data pulses a D,1 and a D,2 (see Figure 1 F).The discussed procedure could now be repeated also for a third pulse pair and, in general, as long as subsequent pules pairs arrive before the acoustic wave decayed completely (see Figure 1 G).

Experimental results
In the following, we study the acoustic link by sweeping the data pulse amplitude of either a D,1 or a D,2 , while keeping the corresponding subsequent pulses constant.For instance, if the input amplitude of a D,1 is varied, a D,2 and a D,3 are fixed in amplitude.The control pulses a C,i are kept constant over the entire study.For each amplitude step, we measure the pulse's area under curve (AuC) of the output pulses a ′ D,i .An AuC-measurement of a D,i without control pulses serves as reference.In total, three different acoustic links occur from this experimental configuration, namely, a D,1 → a ′ D,2 , a D,1 → a ′ D,3 , and a D,2 → a ′ D,3 (see Figure 2 A).In order to rule out drifting effects, we measure each amplitude twice in a random order and take the mean value afterwards.Furthermore, the amplitude sweep is performed for three different time delays dt = 2.5, 4.5, and 10 ns, as the acoustic link decays over time.For a deadtime of 2.5 ns, an increase in amplitude of a D,1 raises the output amplitude a ′ D,2 as shown by Figure 2 B. Because the degree of depletion is lower as for a single pulse interaction (SPI), e.g., a D,1 → a ′ D,3 , we conclude that the acoustic wave b 1 weakens the SBS process of a D,2 ↔ a C,2 .This finding can be explained with the different acoustic phases (see equation ( 2)), which can lead to constructive or destructive interference of the acoustic waves during the SBS process, The acoustic phase is introduced by detuning the frequency difference between data and control pulses slightly from the Brillouin frequency.The acoustic interference is also the reason for the decreasing behavior of the link a D,1 → a ′ D,3 , here, the acoustic wave b 1 enhances the SBS process of a D,2 ↔ a C,2 .The symbols + and − mark the constructive and destructive nature of the underlying acoustic interference in Figure 2 B, respectively.OREO achieves a maximal dynamic range (Max DR) of 33 %.For a deadtime of dt = 4.5 ns we observe a flip in the dynamic as all links switch their behavior from a constructive (+) to destructive (−) acoustic link, and vice versa (depicted in Figure 2 C).In addition, the overall level of depletion is larger in comparison to the SPI-case.For a deadtime of dt = 10 ns (equal to the acoustic lifetime), the dynamic range of the optical connection decreases further as we can see for the connection a D,1 → a ′ D,2 in Figure 2  D. Ultimately, the effect of the decaying acoustic wave becomes in particular visible for the interaction a D,1 → a ′ D,3 as a ′ D,3 remains constant over the entire sweep range of a D,1 (see Fig 2  D).Note that we marked vanished acoustic links with the •-symbol.
With this, we have shown that OREO connects the information carried by subsequent optical data pulses.The acoustic link is sensitive to the amplitude and deadtime of the involved optical data pulses.As the interaction is continuous, it can be used for digital and analogue recurrent tasks.Moreover, the acoustic interference observed with OREO ties in with previous studies based on continuous optical waves and our measurement extends the observation of acoustic interference into a pulsed context 33,34 .In the supplementary material, we study OREO numerically and experimentally in a highly nonlinear fiber (HNLF), using the framework presented in Reference 35 .With the HNLF we study the linear response of OREO, which occurs in the case that the frequency difference of data and control matches exactly the Brillouin frequency.
OREO controls the recurrent operation completely optically via the control pulses, enabling us to implement use case specific computations.For instance, in a pulse sequence consisting of three data pulses, one could skip the middle pulse a D,2 by dropping the second control pulse, which could be useful for regularization 36 .In order to demonstrate the recurrent dropout, we excluded the second control pulse a C,2 from the pulse train.Note, that the amplitudes of the other control pulses remain the same.In a next step, we vary the amplitude of data pulses a D,1 and a D,2 in upward and downward direction and check the impact on the subsequent data pulses (see Figure 2 E).Furthermore, we change the deadtime to investigate the influence of the acoustic interference on the interaction a D,1 → a ′ D,3 .OREO turns off the links between a D,1 → a ′ D,2 and a D,2 → a ′ D,3 as we can see in in Figure 2 F. As marked with the •-symbol, those two links show a constant behavior for the entire Figure 2: Observing OREO's optoacoustic linking and recurrent dropout capabilities.A -Schematic illustration of the amplitude sweep that investigates how different optical states are passed between the optical data pulses a D,i via an acoustic wave b.B to D -Experimental results of the amplitude sweep.While a D,i , i = 1, 2 is changed, its impact on the subsequent pulses a D,j , j > i is studied for different deadtimes dt.Each SBS process creates an acoustic wave b i , which interferes with pre-exisiting ones b k , k < i, eventually.We mark the links a D,i → a D,j with a +, −, and •, when they experienced an enhancement, a reduction and an annihilation of the SBS process, respectively.We added the depletion of a single pulse interaction (SPI) as reference.E -Schematic illustration of the pulse configuration used to study the OREO's feature to implement a recurrent dropout.F to H -Experimental results of OREO's recurrent dropout capabilities.amplitude sweep.Only the interaction a D,1 → a ′ D,3 is active as the control pulses a C,1 and a C,3 establish the required acoustic link.Note, that for the case of a D,2 → a ′ D,3 the interaction a D,3 ↔ a C,3 is influenced by the acoustic wave generated of the a D,1 ↔ a C,1 -interaction (a D,1 is constant).This link can also explain the lower degree of depletion of a D,1 → a ′ D,3 at dt = 4.5 ns (see Figure 2 G).Here, the a D,1 and a D,3 are already separated by 10 ns, which eliminated almost their acoustic link.At a deadtime of dt = 10 ns, the a ′ D,3 is completely disconnected from a D,1 and a D,2 as can be seen by the constant behavior of a ′ D,3 for both sweeps of a D,1 and a D,2 (see Figure 2 H).Besides, over all measurements, a ′ D,2 is below the reference level (a ′ D,2 < 1), e.g., for the interaction a D,1 → a ′ D,2 in Figure 2 F. The increased optical noise floor appears as soon as the EDFA is turned on and could lead to this intrinsic depletion.

Optical pattern recognition
From the beginning on, recurrent operators have been used to recognize patterns 6 .In the following section, we employ OREO as an acceptor 29 to recognize any pattern that can be created with two different data pulses a and b: aa, ab, ba & bb, where the b-pulse is half the amplitude of the a-pulse.Each pulse is launched with a matching control pulse a C,i into the PCF, where SBS is introduced.The deadtime between two consecutive pulses is 2.5 ns.As a measure of OREO's performance, we launch an evaluation pulse pair into the optical fiber and use the AuC of an output evaluation pulse (Eval) (see Figure 3 A).In total, we check all patterns 250-times in a random order and classify the resulting data set (70 % training, 30 %testing) with a RANDOM FOREST classifier 37 (RFC) implemented in SKLEARN-package 38 .Furthermore, we perform the described study twice, once with the SBS-process and once without to isolate OREO's effect.
When OREO is off, the RFC cannot distinguish the different patterns and shows the same accuracy as a random guess (see Figures 3 B).However with OREO, the RFC is capable of distinguishing the different patterns almost with an accuracy of almost a 100 % (see Figures 3

C).
Next, pushing OREO to the acoustic lifetime limit, we evaluate its performance for three different states encoded onto three pulses.The third state c is three quarters of the a state.In total, we test OREO to distinguish every possible permutation of a, b, and c, giving 27 different patterns.This time we launch a fourth data-control pair into the PCF, in order to evaluate OREO's memory (see Figure 3 D).Note that all four control pulses a C,i carry the same optical energy as in the three pulse configuration.We increased the sample size n per pattern from 250 to 500 in order to decrease statistical errors.Figure 3 E shows the corresponding confusion matrix.OREO functions as acceptor and generates distinguishable distributions for the 27 patterns.
The RFC achieves an accuracy of 45 %, exceeding the accuracy of a simple guess by 11-times.
Note that the classification task of the abc-case has 729 possible classification outcomes, and is 45 times more complex as the ab-case.In general, it is seven times more complex as an image classification task based on the MNIST dataset 39 with 10 × 10 degrees.The performance of OREO is currently limited by experimental precision, which is reduced by drifts of the optical pulses over the measurement period.Therefore, we perform a numerical analysis of OREO as an acceptor in the frequency matched case, in order to assess its potential performance.In this simulated experiment, OREO and the RFC achieve an accuracy of 92 %. Figure 3 F shows the corresponding confusion matrix.In the supplementary material, we describe the numerical analysis and check the impact of the pulse width, deadtime, acoustic lifetime, and experimental precision on OREO's pattern recognition performance.This analysis indicates that OREOs performance can even be pushed further to an accuracy of 97 %.

Discussion and future possibilities
The acoustic link employed by OREO enables the processing of time-encoded serial information within a PCF.Its capability to control the recurrent interaction all-optically, gives the concept unique features.The adjustable amplitudes of the control pulses allow OREO's behavior to be changed at the single pulse level, offering an all-optical degree of freedom to adjust its recurrent operation.Moreover, we have shown that it offers the possibility to exclude data pulses from the recurrent interaction.As a consequence, a single data pulse can propagate through OREO without experiencing any manipulation.This can be used to implement recurrent dropout as regularization for the RNN.
The coherent nature of the underlying SBS process offers OREO not only to compute amplitude information but also phase information.Eventually, OREO could compute quadrature amplitude modulated (QAM) data streams.Higher memory depths could be achieved with three different approaches.Firstly, a higher pulse density could be used to increase the number of operations that could be performed within the intrinsic acoustic lifetime.This could be achieved by decreasing the pulse width and the deadtime between the pulse pairs.For instance, with a pulse width of 100 ps and a deadtime of 100 ps (the minimal deadtime is dictated by the length of the waveguide), one could induce up to 50 recurrent interactions.Secondly, one could increase the acoustic lifetime to realize a deep recurrent link, for instance by using materials with longer acoustic lifetimes or operating at cryogenic temperatures.Thirdly, an optical refreshment of the acoustic waves could lead to an increase in memory depth 40 .Because the SBS process does not significantly change the optical control pulses, an optical recycling scheme could be applied to achieve high computational efficiencies.Computational efficiency is determined by the number of operations (OPS) that OREO can perform with one Joule of power.With an optical recycling scheme this value depends only on the deadtime between the pulse pairs, yielding an efficiency from up to ≈ 11 POPS J ; it could potentially increase the computational efficiency of the method described in Reference 22 by three orders of magnitude.A more detailed description of the computational efficiency can be found in the supplementary material.The information bandwidth of an optical signal can be significantly increased by employing different optical frequencies as independent information channels.This has been recently exploited by Sludds et al. 41 to implement an high-performance optical deep learning architecture for edge computing.OREO could be added to this scheme as SBS is highly frequency-selective 42 .This unique feature of the optoacoustic interaction could also be employed together with an optical multi-frequency matrix operator [43][44][45] to realize an multi-frequency recurrent neural network.

Conclusion
In conclusion, we have demonstrated the first optoacoustic recurrent operator (OREO), which connects the information carried by subsequent optical data pulses.Our work combines for the first time the field of traveling acoustic waves and artificial neural networks and paves the way towards SBS-enhanced computing platforms.This new fusion brings context to optical neural networks, but can also enable much more.Typical building-blocks of a neural network, such as nonlinear activation functions and other types of optoacoustic operators are within reach.Especially, the different time scales of optical and acoustic waves open up a whole new playground for the implementation of a variety of computing architectures.

Methods
To demonstrate OREO, we build the all-fiber setup shown in Figure 4.As a sample, we use a photonic crystal fiber (PCF) with a length of ≈ 40 cm, an average hole diameter of 1.44 µm, an average core diameter of 1.842 nm, a pitch of 1.756 µm, and d/Λ = 0.82.A continuous wave laser at 1550 nm is split into the data and control branch via a 50/50-splitter.An IQ-modulator shifts the data signal by Ω ≈ 10.6 GHz, which is close to the PCF's Brillouin frequency of Ω PCF ≈ 10.45 GHz.The data signal's spectrum is cleaned with a subsequent narrow bandpass filter and afterwards amplified by an Erbium-doped fiber amplifier (EDFA).An optical intensity modulator driven by an arbitrary waveform generator (AWG) generates the optical pulses and, thus, imprints the amplitude-encoded information.A single data pulse is 1 ns long and separated to an adjacent data pulse by a deadtime dt.The repetition rate of a pulse sequence is ≈ 1 MHz.
The pulses are guided to the PCF by an optical circulator and, afterwards, measured with a high-speed photodiode and a 16 GHz Oscilloscope.The optical power of the data pulse is about 1 mW.An additional narrow bandpass filter cleans the signal before detection.In the control branch, optical pulses are generated with the same pulse width and repetition rate as the data branch.Afterwards, the pulsed signal is amplified by an EDFA and filtered by a narrow bandpass filter before launched into a high-power EDFA.The amplified signal is filtered by a 1 nm-width bandpass filter and launched with an average power of about 126 mW into the SBS process.
Figure 4: Illustration of the setup used to demonstrate the recurrent optoacoustic operator.The bandwidth of the oscilloscope and photodiode are 16 GHz and 12 GHz, respectively.We introduce the Brillouin process inside photonic crystal fiber (PCF), which could be replaced with any other waveguide or an on-chip device.

Supplementary information: An optoacoustic field-programmable perceptron for recurrent neural networks
Steven Becker

S1 Temporal calibration
The temporal calibration ensures that only one data-control pulse pair interacts in the optical fiber.The isolation of a single stimulated Brillouin scattering (SBS) process allows us to rule out any parasitic interactions, e.g., a D,1 interacts with a C,2 .In addition, the acoustic wave b 1 exists only at a single location in the optical fiber.Note that for future applications of OREO, it might not be necessary to isolate the interactions that strictly.
The selected PCF has a length of 40 cm, corresponding to a pulse travel time of about 2 ns.Hence, by separating two pulses with a deadtime of 2.5 ns, one ensures single pulse interactions.Furthermore, the path lengths of the data and control branch differ in our setup due to the different devices in the corresponding path.Consequently, we compensate for the different path lengths by electrically delaying the control pulses with the arbitrary waveform generator (AWG) (see Figure 4 of the main text).
We sweep the temporal offset of the control pulses, while keeping the data offset constant.
In order to extract the optimal control offset, we measure the depletion of the data pulses a D,1 , a D,2 , and a D,3 for the case, where only one of the control pulses a C,i is active.This allows us to isolate the interaction between the different pairs.The blue solid curve in Figure 1 A represents the desired interaction a D,1 ↔ a C,1 .The dashed grey lines in the same plot represent the interaction of a C,1 either with a D,2 or with a D,3 .As discussed, those interactions are not of interest for the demonstration of OREO.we observe the desired interaction a D,i ↔ a C,i and no parasitic interaction.This region covers a time span of about 2 ns translating to the length of the fiber.Consequently outside this zone, the data pulse a D,i or the control pulse a C,i takes only partially part in the SBS-process reducing its efficiency.Based on the results presented in Figure 1, we chose 149 ns as offset for the control pulses.
Supplementary Fig. 1: Results of OREO's temporal calibration to ensure that only a single and a matched data-control pulse pair interacts in the optical fiber.A, B, and C study the interaction of a C,1 , a C,2 and a C,3 with the different data pulses, respectively.In each plot, the colored solid line marks the desired interaction, e.g., in A it is a D,1 ↔ a C,1 .Moreover, the gray dashed lines represent unwanted interactions, e.g. in A those are a D,2 ↔ a C,1 and a D,3 ↔ a C,1 .Possible offset values for the control pulses, where OREO performs only the desired isolated and matched interactions a D,i ↔ a C,i are covered by the green rectangular in each plot.

S2 Intrinsic correlation check
The all-optical control is one of OREO's unique features.In order to validate that the connection between the different data pulses is induced by the SBS-process alone, we check the correlation of the different data pulses.Therefore, we sweep the amplitude of data pulses a D,1 and a D,2 , while checking the impact on all subsequent data pulses.We perform this study twice with different settings.In the first sweep, the the control pulses are turned off and in a second sweep, the control pulses are on but temporally detunded by 351 ns to the optimal offset.If we sweep the amplitude of data pulse a D,1 , we can observe a small impact on the output pulses a ′ D,2 and a ′ D,3 as shown by Figure 2 A and C, respectively.The increase of the output amplitude a ′ D,2 for lower values of a D,1 (see Fig. 2 A) could be caused by the RF-amplifier which is placed between the arbitrary waveform generator (AWG) and the intensity modulator, which is used for the pulse generation.If the electrical amplitude of the data pulse a D,1 decreases the RF-amplifier has more energy to feed into the subsequent data pulse a D,2 .This could also explain, why the amplitude of a ′ D,3 decreases in the same time (see Fig. 2 C).The same effect can most probably be observed for the sweep data pulse a D,2 .Here, the output amplitude of the subsequent data pulse a ′ D,3 also increases for lower input values of a D,2 (see Fig. 2 D).In addition, we do not observe any effect of a D,2 on the output a ′ D,1 as illustrated in Figure 2 B, indeed, the data pulse remains almost constant.An overall drift of the system could explain, why the AuC value is slightly below 1.Moreover, if one compares the two measurements in Figure 2 B, one notices that the detuned control signal slightly depletes the data signal.Most probably this is could be an effect of the EDFA's amplified spontaneous emission (ASE).
In comparison to the results discussed in the main part, we can conclude that the measured intrinsic correlation are neglectable.However, one should keep in mind the data pulse depletion induced by the ASE of the EDFA, which might becomes stronger for higher pump powers of the EDFA.
Supplementary Fig. 2: Results for the intrinsic correlation check of OREO.In A and C we can see a small correlation between the input amplitude of data pulse a D,1 and the output amplitudes a ′ D,2 and a ′ D,3 , respectively.The reason for this dynamic could be the RF-amplifier located between the AWG and the intensity modulator.A similar dynamic between subsequent pulses occurs if we change the data pulse a D,2 and check its impact on the output a ′ D,3 (see D). Again the output amplitude increases for lower inputs.As expected, we can see no connection between the input of a D,2 and the output a ′ D,1 .However, we see that the EDFA, although detuned, reduces for all four cases the amplitude of the output data pulses.This effect might result from the EDFA's amplified spontaneous emission (ASE).Overall, the intrinsic correlation is significantly lower as the one induced via ORE and can, therefore, be neglected.

S3 Performance evaluation
This section asses the performance of OREO in terms of its speed and energy consumption.
The key parameters of OREOs performance are the deadtime dt between two pulse pairs, and the pulse width τ .The lower boundary of the deadtime can be estimated as dt > L n eff c 0 , where L is the waveguide's length, n eff is it effective refractive index, and c 0 is the speed of light.In total, the overall number of operations (OPS) which can be achieved in one second is: For a short fiber, one could operate OREO with a deadtime of dt = 500 ps and a pulse width The current approach of OREO requires about P pulse = 126 mW of optical power per control pulse a C,i in the PCF, translating to a pulse energy of E pulse = 126 nJ for the applied rep.
rate of 1 MHz.Hence, OREO has an energy efficiency of about η ≈ 8 MROPS J −1 .
A control pulse recycling scheme can enhance OREO's energy efficiency significantly.Such a scheme can be implemented because a control pulse is not depleted through the SBS process, but amplified.Hence, it offers the possibility to re-use it for subsequent computational steps.In addition, the experimental scheme presented in Figure 4 of the main text offers also an output for the control pulse.In the use case discussed above, a 10 cm delay line between the control input and output of OREO would already be enough for implementing a recycling scheme.
Under the assumption of an low loss recycling scheme, where the loss is compensated through the control pulse amplification of the SBS process, the control pulse recycling could increase the efficiency by nine orders of magnitude: For OREO with a long short-term memory that employs a deadtime of dt = 500 ps and a pulse width of τ = 250 ps the computational efficiency can be increased to η recy = 10.6 PROPS J −1 .
Note that a waveguide with a high optoacoustic gain could further increase the efficiency of OREO.

S4 All-optical control
The control pulses a C,i can be used to tune the strength of the optoacoustic interaction.Known from an optoacoustic memory 1,2 , the power of the control pulses dictates the degree of depletion and, therefore, the relative amount of acoustic that is created.Hence, the control pulses can be used to optimize the interaction for the desired need or to modify the light-sound interaction on pulse-by-pulse level.
In the following, we study the all-optical control with three data-control pairs (deadtime 2.5 ns), where we vary the amplitude either from control pulse a C,1 or a C,2 , while keeping the remaining control constant Fig. 3 results for four amplitudes scales, namely, full (α Ci = 1), three quarters (α Ci = 0.75), half (α Ci = 0.5), and quarter (α Ci = 0.25) are shown in Figure 3.As the acoustic control amplitude dictates the degree of depletion of the data pulse, we can use it to isolate the impact of certain acoustic contributions.For example, the amplitude sweep of control pulse a C,1 (see Fig. 3 B to E) displays the impact of the acoustic interference when one studies the interaction a D,1 → a ′ D,2 .In the process of reducing the control pulse amplitude, one can see the transition from an inhibition to annihilation to amplification of the interaction a D,2 ↔ a C,2 in the plots B, C, and D, respectively.An equivalent dynamic can be observed for the acoustic link a D,1 → a ′ D,3 , which is influenced by the acoustic interference of b 1 and b 2 .Here, the acoustic link first amplifies the interaction of a D,3 ↔ a C,3 , then annihilates it and, finally, inhabits it as one can see in the plots B, D, and E, respectively.Next, the acoustic wave b 1 serves the acoustic link a D,2 → a ′ D,3 as an amplifier because the degree of depletion of a D,3 shrinks for higher control pulse attenuations α C1 (see Fig. 3 B to E). Next, the amplitude sweep of control pulse a C,2 (see Fig. 3 F to I) reveals information about the influence of the acoustic wave b 2 .This interaction inhibits the acoustic link between a D,1 → a ′ D,3 because the depletion of a ′ D,3 increases with lower amplitudes of a C,2 (see Fig. 3 F).In addition, the acoustic link a D,2 → a ′ D,3 shows also the effect of the acoustic interference between b 1 and b 2 : for α C1 = 1, the SBS-process is inhibited; for α C1 = 0.75 it is annihilated; and for α C1 = 0.5 it is amplified (see Fig. 3 F to H, respectively).
Overall, the control pulses a C,i serve OREO as an additional degree of freedom to manipulate the recurrent operation on pulse-by-pulse basis.Supplementary Fig. 3: Study of OREO's capability to control the recurrent operation completely optically on pulse-by-pulse basis.The pulse pairs are seperated by a deadtime of dt = 2.5 ns.A -Schematic illustration of the control pulse sweep that investigates the impact of different control pulse amplitudes on the recurrent interaction.B to E -Experimental results of reducing the amplitude of control pulse a C,1 .The relative change in control amplitude is given by α C1 .This measurement isolates the impact of the acoustic wave b 1 on subsequent SBS processes.F to I -Experimental results of reducing the amplitude of control pulse a C,2 .This measurement isolates the impact of the acoustic wave b 2 .

S5 OREO with a highly nonlinear fiber
In the following, we demonstrate OREO in a ≈ 43 cm long highly nonlinear fiber (HNLF).
Analogous to the PCF measurement, we launch three consecutive data-control pulse pairs into the fiber and vary the amplitude of a D,1 and a D,2 (see Figure 4 A).In contrast to the PCF study, we investigate OREO's dynamic without a frequency detuning, i.e., the frequency difference between data and control pulses matches the Brillouin frequency.Hence, we drop the influence of the acoustic phase.We study the dynamic again for different deadtimes.The recurrent dynamic is less complex as the one of the PCF due to the lack of acoustic interference.As we can see from Figure 4 B, each SBS process is enhanced by the previous ones.As a result the output amplitude of a ′ D,2 is higher as the one of a ′ D,3 for all interactions.In contrast to the PCF, this dynamic does not change for different deadtimes (see from Figure 4 C).The overall higher output amplitudes a ′ D, i for the different acoustic links (a D,i → a ′ D, j) can be explained with the decay of the acoustic wave.Although the link a D,1 → a ′ D,3 touches the acoustic lifetime limit, we are still able to observe a recurrent interaction.This link (a D,1 → a ′ D,3 ) vanishes clearly as soon as we separate the pulse pairs by the acoustic lifetime (see from Figure 4 D).In this case, we can only observe nearest neighbour interactions a D,i → a ′ D,i+1 .OREO achieves in our HNLF a maximum dynamic range of ≈ 20 %.A numerical study of the HNLF behavior is depicted in Figure 7 spectively.
The set of equations ( 3) and ( 4) use the one dimensional electric field E, the refractive index n, the speed of light c 0 , the polarization P , the acoustic damping term Γ ′ and the acoustic velocity v ac .The polarization P and the driving term ⃗ f of the acoustic field can connected to electrostiction, giving equations ( 5) and ( 6), respectively.
With the density of the waveguide b 0 and the electrostrictive constant γ e .
Applying a plain wave ansatz for both the optical and acoustic wave, gives a set of nonlinear partial differential equations.Those cannot be solved analytically without applying several approximations.According to reference 3 , one get a set of linear coupled partial differential equations by using the rotating frame and slowly-varying wave approximation, yielding the known coupled mode equations of the SBS process, which include two optical equations for the data wave a D and the control wave a C and one equation for the acoustic field b: With the effective refractive index of the waveguide n eff , the speed of light c 0 , the elecostrictive constant γ e , the density of the waveguide b 0 , the acoustic group velocity in the waveguide v ac ,  ∆ω ̸ = 0.The coupled mode equations can also be derived from the Hamiltonian given in equation ( 1) of the main text by applying the Heisenberg equation 8 .
Next, we numerically study the dynamics of the coupled mode equations (CME) (7) with the technique presented in Reference 9 .This method uses the characteristics of the data and control wave n eff c 0 ± z = 0 and solve the CME along the c 0 n eff t ± z-direction via an implicit Runge-Kutta scheme.As this approach only tackles the optical part of the CME, we also employ the Euler method 10 to solve the acoustic part of the CME.Therefore, we drop the space-derivative of the acoustic mode, assuming a stationary acoustic field.In order to run the simulation we feed the parameters shown in Table S1 into the numerical framework.
Figure 7 depicts the simulation results of OREO in a highly nonlinear fiber.Although, we observe an overall agreement with the measurements results of the HNLF (see Fig. 4), the simulation show a linear behavior of the acoustic link for 2.5 ns (compare Fig. 4 B and Fig. 7 A).our experiment by decreasing σ exp , e.g., doubling the precision corresponds to σ exp /2.In total, we generate a dataset of size 27000, which we feed into a RANDOMFORST (RFC) in the same way as for the experiment, which then returns a predictive accuracy.
Figure 7 shows the results of the numerical study for different pulse widths, deadtimes, and experimental precision at each of two acoustic lifetimes.The results suggest that the experimental precision has the highest impact on the RFC's accuracy.A five time increase in precision compares to the measurement results of the ab-measurement, which we discussed in the main text.In the most optimized case, the RFC achieves an accuracy of 92 % for the pulse width and deadtime used in the experiment.In addition, in the case of the standard acoustic lifetime of 8.8 ns as measured in experiment (Fig. 7 A), we observe that a shorter deadtime dt is beneficial for the performance of OREO.However, the influence of the investigated deadtime span decreases for the case of increased acoustic lifetime due to the slower acoustic decay.This explains why the different markers are less distributed in Figure 7 B. In addition, it suggests that a high acoustic lifetime could allow OREO to capture even longer pattern, i.e., more complex context carried by the optical domain.In this case, the precision of OREO can be pushed to 97 %

Figure 3 :
Figure3: Applying OREO as an acceptor to predict patterns of optical pulses.A -Schematic illustration of how the acoustic link can be used by an optical evaluation pulse (Eval) to predict a pattern from optical pulses, which have been launched into the optical fiber before.The example shows a ab-pattern.B and C -Confusion matrix of a RANDOM FOREST (RFC) which is used to classify the data set with and without SBS.The RFC achieves an almost perfect classification rate as soon as OREO provides the recurrent feedback.D -Schematic illustration of the three pulse pattern recognition task.This case shows the cab-pattern.E shows the confusion matrix of the RFC, using 30 % of the data set for training.The RFC achieves an accuracy of 45 % and outperforms a simple guess by 11-times.The accuracy of the RFC is mostly limited by experimental precision.F -RFC confusion matrix using simulated data to study OREO's performance with experimental optimization.In this case, we can achieve an accuracy of 92 %.The simulation are based on a frequency matched SBS process and is described in more detail in the supplementary material.

Figure 1 B
and C show the results, where a C,2 and a C,3 only are active, respectively.The colored curve in both Figures shows the behavior of the corresponding data pulse a D,2 and a D,3 .A green square in each plot marks the region, where of τ = 250 ps, yielding C SF = 1.33 GROPS s −1 .The frequency selectiveness of Brillouin scattering allows OREO to process M frequencies simultaneously, extending the performance of OREO further to C MF = M • C SF .The number of frequencies that can be utilized by the scheme depends on the spectral width of the pulses.For example, a channel separation of 3.6 GHz and a photodetector with a bandwidth of 100 GHz boost the performance of the discussed example by a factor of 28 to C MF ≈ 28 • C SF = 36.4GROPS s −1 .
Γ b , the optical wave vectors k c,d , the acoustic wavevector q and the frequency relation between the fields ω d = ω c + Ω + ∆ω.Note that this set of equations capture also the acoustic phase introduced by an optical detuning from the Brillouin frequency: 1,2, Dirk Englund 3 , and Birgit Stiller 1,2, * 1 Max-Planck-Institute for the Science of Light, Staudtstr.2, 91058 Erlangen, Germany 2 Department of Physics, Friedrich-Alexander-Universität Erlangen-Nürnberg Staudtstr.7, 91058 Erlangen, Germany 3 Research Laboratory of Electronics, Massachusetts Institute of Technology, Cambridge, Massachusetts 02139, USA * Corresponding author.Email: birgit.stiller@mpl.mpg.de

Table S1 :
Simulation parameters for the spectral-analysis of SBS.