Introduction

In the contemporary big data era, there is an evident and necessary escalation in global data traffic demand. This is primarily due to applications like high-resolution live streaming, virtual meetings with multiple participants, immersive gaming, and real-time analytics, all of which require immediate access to large data volumes. Consequently, there is a marked increase in network bandwidth requirements. However, existing network technologies struggle to meet these expanding capacity needs, with traditional wireless systems facing limitations like bandwidth constraints, high latency, susceptibility to interference, and elevated data traffic. In this context, free-space optical (FSO) communication emerges as a promising candidate for wireless data transmission. FSO communication uses optical wavelength, from ultraviolet to infrared, achieving data transmission through the atmosphere without requiring a physical communication link. By leveraging the advantage of a broader spectrum range in optical bands and compact spatial confinement from a laser beam, FSO communication has already provided the ability to cater to high-capacity optical data transmission demand.

In recent years, optical beams carrying orbital angular momentum (OAM), first introduced by Allen et al. in 19921, have attracted attention in the research field of FSO communication. With the advantage of its inherent orthogonality, which leads to the ability to co-propagate with no interference and ideally unlimited topological charge states, the orthonormal basis becomes theoretically unbounded. This property makes OAM-carrying beams significantly improve the data transmission rate, giving rise to potential applications in optical manipulation, imaging, and FSO communication. However, the orthogonality of the OAM beam is no longer preserved in the presence of atmospheric turbulence2. The helical wavefront of the OAM beams would be drastically distorted while propagating through the turbulent medium since beams broaden and wander randomly due to the effect of inhomogeneities of the refractive index, resulting in unrecoverable power loss and inter-channel crosstalk of the FSO communication system.

Machine learning plays a crucial role in optics, photonics, and computer science. In OAM FSO communication, Timothy Doster and Abbie T. Watnik introduced detecting the active OAM modes in a transmission link using a convolutional neural network (CNN)3. Wenjie Xiong et al. investigated the use of a cylindrical lens for CNN-based OAM modes recognition to demodulate OAM shift-keying (OAM-SK) signals, achieving 99.53% mode detection accuracy4. However, one needs to mention that in CNNs, such as AlexNet, VGGNet, and GoogleNet, convolutional layers consume more than 85% of the whole runtime5, training on large data sets with more convolutional layers leads to much more time-consuming computation with high latency and power consumption.

To improve the computational efficiency with simultaneous low-energy cost, FSO computing is considered a competitive candidate. An optical matrix multiplication based on a 4F optical system was introduced in 1993 to demonstrate matrix inner-product multiplication by placing a computer-generated holographic mask at the co-focal plane between two lenses6. After two decades of development, kernels generated by state-of-the-art digital micromirror devices (DMDs) outperform the older hologram mask to simulate convolution computation, making optical convolutional acceleration promising.

In this work, we experimentally generate the multiplexed OAM-coded signal and introduce the free-space propagation under simulated atmospheric turbulence conditions. We complete the Fourier optics convolution right after the generated turbulence-affected multiplexed OAM beams via a DMD 4F-based system and the construction of a hybrid optical-electronic neural network (OECNN) system. We test our neural network by demultiplexing the 4-bit OAM-coded signals. Results show the proposed OECNN performs a demultiplexing accuracy of 72.84% under strong turbulence conditions with 3.2 times training time faster than designed all electrical convolutional neural network (ECNN).

Results and discussion

Orbital angular momentum

The OAM-carrying beams with helical wavefront, being potentially employed in various applications, which are generally called vortex beams, could be best described in terms of Laguerre-Gaussian (LG) beams which have an azimuthal phase term of exp(ilθ), representing an on-axis phase singularities of order l, where θ is the azimuthal angle and l is known as topological charge of the beam, ideally taking infinite value, which determines the number of times the phase should change by 2π on one single rotation around the beam axis7. So far, various state-of-the-art methods for the generation of LG beams have been successfully developed, such as using spiral phase plates8, diffractive phase holograms9, cylindrical lens pairs10, metamaterials11, q-plates12and more. In this study, we use the spatial light modulator (SLM) with the phase-modulated fork grating pattern to convert the input Gaussian-like beam to the OAM beam. For the detailed generation procedure see section Supplementary Note 1. The example of experimentally generated LG modes see Supplementary Fig. 1. We experimentally generate and use single-ringed LG modes (\(L{G}_{p}^{l}\)) where p = 0 (\(L{G}_{0}^{l}\)) to achieve the simplest and clearest optical ring intensity formation.

When LG beams with the same waist position and parameters are coaxially superimposed, they interfere and produce a vortex structure, resulting in complex patterns of bright and dark regions. Here, we demonstrate the superposition of two co-propagating LG beams with topological charge l1 and l2, where the formation of complex amplitude at z = 0 can be written as:

$$L{G}_{0}^{l}=L{G}_{0}^{l1}+L{G}_{0}^{l2}.$$
(1)

By multiplexing \(L{G}_{0}^{l1}\) and \(L{G}_{0}^{l2}\), they interfere at ∣l2 − l1∣ azimuthal positions, resulting in a transverse intensity profile of ∣l2 − l1∣ petal-like patterns. An example of the experimental generated transverse intensity profile of two multiplexed LG beams is shown in Fig. 1. For all three cases, the circular symmetric intensity profile comprises fourteen (∣l2 − l1∣) bright petals, where Fig. 1b (\(L{G}_{0}^{-7}\) + \(L{G}_{0}^{7}\)) maintains no peripheral vortices, Fig. 1a (\(L{G}_{0}^{-8}\) + \(L{G}_{0}^{6}\)) and Fig. 1c (\(L{G}_{0}^{-6}\) + \(L{G}_{0}^{8}\)) show rotated intensity (counterclockwise and clockwise, respectively) due to the effect of Gouy phase difference between two LG beams with different absolute value of topological charge (∣l2 − l1∣)13. Theoretically, the complex field distribution of the multiplexed LG beams with unlimited integer topological charge numbers n at z = 0 can be written as:

$$L{G}_{0}^{l}=L{G}_{0}^{l1}+L{G}_{0}^{l2}+L{G}_{0}^{l3}+...+L{G}_{0}^{ln},$$
(2)

the petal-like patterns of multiplexed LG beams with n superimposed numbers are related to the corresponding Gouy phase differences among selected OAM modes, generating unique intensity profiles.

Fig. 1: Experimentally generated intensity profile of the multiplexed orbital angular momentum (OAM) beams.
figure 1

a–c are multiplexed OAM beams with superimposed topological charges l1 = −8 and l2 = 6, l1 = −7 and l2 = 7, and l1 = −6 and l2 = 8, respectively. A grayscale color bar and a scale bar of length 1 mm are drawn.

Full size image

Atmospheric turbulence

In FSO communication, one critical challenge that needs to be addressed is atmospheric turbulence. The refractive index of the atmosphere undergoes stochastically variations spatially and temporally due to the temperature inhomogeneities and atmospheric pressure, which cause the wavefront to be randomly distorted as it propagates where manifested as intensity fluctuation, beam wandering, and scattering. In the OAM-carrying FSO communication link, atmospheric turbulence-induced aberrated helical phase front results in the channel crosstalk and information blending among multiplexed OAM modes. This would significantly decrease the transmission efficiency, impairing the stability and reliability of FSO communication systems14. The most widely accepted theory of atmospheric turbulent flow was first put forward by Kolmogorov in 194115. Since then, the theory of turbulent fluctuation could be directly related to refractive-index variation by introducing the Kolmogorov turbulence model16,17. Hitherto, many other practical models that improve the agreement between theory and experimental measurement are proposed and used, such as Tatarskii18, von Kármán19, modified von Kármán20 and Hill model21. In this study, we use the modified von Kármán atmospheric turbulence model as turbulence simulation; for the detailed introduction of the turbulence model see section Supplementary Note 2; The example of a computer-generated random atmospheric turbulence phase screen using the modified von Kármán turbulence model see Supplementary Fig. 2. The atmospheric turbulence is simulated by uploading the generated phase-modulated turbulence screens onto the SLM, affecting the incident multiplexed OAM beams.

Fourier optics convolution

In recent years, deep learning has advanced significantly with the utilization of CNNs, which are pivotal in image-related tasks such as object detection and classification. However, traditional electronic-based computing systems for CNN inference require numerous computationally intensive convolution operations, often demanding high-performance hardware and generating substantial thermal loads. The 4F optics system sidesteps these limitations by completing Fourier optics convolution passively in the frequency domain. The digital micromirror device (DMD) serves as an optical gate, modulating the incoming light based on the applied convolutional kernels. The output sensor, typically a camera or a photodiode array, captures the convolved images and saves them as data to be further processed electronically; for the detailed introduction see section Supplementary Note 3. The 4F system’s optical nature drastically reduces computing latency and power consumption. It stands as an innovative, high-speed alternative for performing all electronic CNN inference and offers new possibilities in integrated photonics computing.

Neural network system performance

We first generate OAM-coded beams with two multiplexed topological charge numbers and collect the dataset under four different atmospheric turbulence scenarios with original, weak (D/r0 = 0.15), medium (D/r0 = 0.3), and strong (D/r0 = 1.5), respectively. We choose three indistinguishable cases where the topological charge numbers are consecutive integers (l1 = 6, l2 = − 8; l1 = 7, l2 = − 7, and l1 = 8, l2 = − 6) and two distinguishable cases where the topological charge numbers have higher difference value (l1 = 9, l2 = − 8, and l1 = 10, l2 = − 10).

For each case, 1000 images are collected under each turbulence level. 20,000 images are generated after SLM1. After being convolved with 16 kernels, 320,000 images are captured by the camera and collected as the dataset. On the subsequent electronic layers side, the dataset is packed in groups of 16. We label these five OAM-coded beams as five classes. For each class, we randomly shuffled and split the collected 1000 images with the training set of 800 and the validation set of 200 to guarantee that both sets are completely independent. We compare the classification accuracy between the OECNN and ECNN. Since both neural networks are trained on the dataset with unique intensity patterns, we define the demultiplexing accuracy as the classification accuracy of the neural network.

We run both neural networks over 50 epochs five times, and the averaged results are shown in Fig. 2. The demultiplexing accuracy of the ECNN is 94.58%, 89.34%, 86.84%, and 84.32% for four different levels of atmospheric turbulence (Fig. 2a), with a full training time of 604.75s. The OECNN shows the demultiplexing accuracy of 87.96%, 83.39%, 81.87%, and 74.23% under the preset turbulence level order (Fig. 2b), with a full training time of 454.7s. The training time of OECNN is 1.3 times faster than ECNN. Then, we generate the 3-bit OAM-coded signal (topological charge l = − 7, l = − 5, and l = 6 are applied) with 8 classes in full to train both neural networks. The ECNN (Fig. 2c) shows the demultiplexing accuracy of 96.38%, 94.61%, 91.88%, 89.84%, with a full training time of 1004.9s. The OECNN (Fig. 2d) shows the demultiplexing accuracy of 89.52%, 85.87%, 84.07%, 76.13%, with a full training time of 504.8s. The training time of OECNN is 2 times faster than ECNN. At last, we generate and train the 4-bit OAM-coded signal (topological charges l = − 7, l = − 5, l = − 2, and l = 6 are applied) with 16 classes in total on both neural networks. The ECNN (Fig. 2e) shows the demultiplexing accuracy of 94.53%, 92.34%, 90.13%, 82.61%, with a full training time of 1954.8s. The OECNN (Fig. 2f) shows the demultiplexing accuracy of 86.74%, 82.87%, 81.61%, 72.84%, with a full training time of 604.85s, where OECNN shows 3.2 times training time faster than ECNN. As the number of classes exerts minimal impact on the calculation of total floating point operations (FLOPs), the total number of FLOPs of ECNN is averagely calculated as 26,843,340 FLOPs. The total number of FLOPs of OECNN is 9,298,451 FLOPs; for complete calculation and comparisons see Supplementary Table 1 in section Supplementary Note 4.

Fig. 2: Orbital angular momentum (OAM)-coded signal demultiplexing accuracy.
figure 2

a-b, c-d, and e-f are the demultiplexing accuracy results of the electronic convolutional neural network (ECNN) and the optical-electronic convolutional neural network (OECNN) with the 5-class, 8-class, and 16-class OAM-coded signals, respectively. Red, purple, green, and blue curves represent the turbulence-free, weak, medium, and strong atmospheric turbulence conditions, respectively. The shaded area represents the range of min-max results from 5 different runs.

Full size image

An apparent divergence could be observed in the comparative analysis of the demultiplexing accuracy between ECNN and OECNN. The demultiplexing accuracy of OECNN manifests a decrement of around 5% to 8% relative to its ECNN counterpart, even up to 13% under strong atmospheric turbulence levels. A closer inspection of the validation accuracy curve for the OECNN revealed a continued increase, even as the designated 50 epochs concluded, insinuating an ongoing learning process. In contrast, the validation curve associated with the ECNN stabilized, indicating cessation in its learning after 50 epochs. This discrepancy can be attributed to the inherent characteristics of the datasets. The OECNN-based datasets possess higher levels of distortion and noise due to the intrinsic systematic error of the experimental setup, especially gained from the beam aberration and distortion caused by the undesired tilted angle of the SLM and DMD. In addition, certain pixel losses during the experimental pixel-wise multiplication process are disregarded due to the physical pixel limitation of the SLM and DMD. For each class, a dataset comprising only 1000 images is employed. The limited data volume is considered to potentially curtail the achievable demultiplexing accuracy, especially when encountering circumstances where multiple applied topological charge numbers are consecutive, rendering the intensity pattern of OAM modes more analogous to one another, resulting in increased homogeneity and reducing their discernibility.

Besides the comparative analysis on demultiplexing accuracy, the comparison of the training time between the ECNN and the OECNN also yields noteworthy results. The training efficiency gain is particularly pronounced as the dataset’s class number increases. This trend illustrates the OECNN’s enhanced capability in managing extensive, large-scale datasets, a critical factor in practical neural network-based OAM communication scenarios. Furthermore, in evaluating our OECNN, we observed that the convolution layer accounts for 65% of the total FLOPs. This result indicates that while convolution remains a major component of the CNN’s computational process, its proportion is moderately lower compared to the mainstream neural networks with more complex structures, such as AlexNet and VGGNet, which are characterized by multiple convolution layers with a larger number of kernels and sizes, allocate a higher percentage of their computational budget to convolution operations5. This variance is primarily due to our OECNN being designed with one single convolution layer to realize the complete Fourier optics convolution operation integrating with the current optical setup. Despite this, our findings affirm the convolution layers are the main consumers of computational resources during inference tasks. Furthermore, our results underscore the potential application of optical-electronic/all-optical acceleration in more complex, convolution-intensive neural network architectures, demonstrating significant prospects for future enhancements in processing efficiency.

Conclusions

In conclusion, we have demonstrated a free-space multiplexed OAM-coded signal demultiplexing scheme utilizing intensity-based image recognition via a hybrid OECNN system. We show the validity of generating multiplexed OAM-coded beams and phase-modulated atmospheric turbulence simulation. We realize the Fourier optics convolution operation by utilizing DMD in the Fourier plane. The designed ECNN and OECNN are trained and compared. The proposed OECNN performs the demultiplexing accuracy of 86.74% of the 4-bit OAM-coded signal dataset under turbulence-free conditions and 72.84% under the strong atmospheric turbulence scenario, where the lower demultiplexing accuracy compared with the turbulence-free situation corroborates the vulnerability of the phase for the OAM-carrying beam and the states the challenge for OAM FSO long-distance communication. OECNN performs 3.2 times faster training time than ECNN under 16-class OAM-coded signal demultiplexing tasks, highlighting its potential for superior performance in comprehensive input dataset training tasks, marking a significant advancement over ECNN. Optimization of the neural network structure will be conducted. Considering the limitations that our OECNN currently uses a single Fourier optics convolution layer integrated with one input, meanwhile being subjected to the inherent physical limitation of SLM and DMD, the utilization of metasurfaces emerges as a viable approach for constructing an advanced high-throughput OAM-based integrated optical neural network acceleration system22,23. The detailed discussion of the system performance analysis and the future direction scheme is shown in Supplementary Figs. 3 and 4 in section Supplementary Note 4. By employing metasurfaces composed of subwavelength antenna arrays to simultaneously generate OAM-coded signals array and complete massively parallel Fourier optics convolution, the integration of metasurface holds promising for a groundbreaking enhancement in optical neural network processing speed and data throughput, thereby significantly advancing the capabilities of the OAM-based optical neural network communication system.

Methods

Our lab experiment equipment setup and scheme are shown in Fig. 3a, b. A fiber-coupled laser with 1550 nm wavelength is driven with 100 mW output power. After being coupled and collimated into the free-space by a fiber collimator, the incident Gaussian-like beam is expanded by the beam expander with a 15 mm beam size. A beam splitter is utilized to vertically reflect the expanded beam to distribute it onto the entire active region of the reflective phase-only SLM. The first SLM is programmed to load computer-generated combined fork phase-grating patterns with the desired superimposed topological charge numbers. Python code for generating fork phase grating pattern is provided in Supplementary Software 1. Tip/tilt-mounted SLM is fine-tuned to reflect the generated OAM beam in the original direction to maintain the alignment of the propagation path. A pinhole is employed after the first SLM to filter out the background noise and sort the phase-modulated first-order diffracted OAM-coded beam from the zeroth-order non-phase-modulated beam. The sample of the 4-bit OAM-coded signals under turbulence-free conditions is shown in Fig. 4. After propagating 0.3 m, the OAM-coded beam is projected onto the second SLM, which is loaded with the computer-generated turbulence phase screens based on the modified von Kármán atmospheric model. Atmospheric turbulence phase screens with three different turbulence levels D/r0 = 0.15, D/r0 = 0.3, and D/r0 = 1.5 are applied, representing weak, medium, and strong turbulence levels, respectively, where D represents the physical dimension of the SLM, and r0 is the Fried parameter. In this study, given the fixed physical dimension of the SLM, we define the weak turbulence condition (D/r0 = 0.15) as corresponding to a propagation distance of 1 km under clear sky conditions, where atmospheric turbulence is minimal. The medium turbulence condition (D/r0 = 0.3) corresponds to a propagation distance of 1 km under windy conditions and lower altitudes, where the interaction of faster, erratic air movement with obstacles like buildings and terrain creates increased turbulence. This effect is further amplified by thermal gradients caused by the sun’s heating of the Earth’s surface. Python code for generating atmospheric turbulence phase mask is provided in Supplementary Software 2. The strong turbulence condition (D/r0 = 1.5) corresponds to a propagation distance of 2 km near ground level, where intense thermal activity leads to significant turbulence. In this scenario, the larger temperature differences between the warmer ground and cooler air above cause stronger and more unstable air currents, resulting in much higher turbulence. Upon each turbulence level, thousands of stochastically pre-generated atmospheric turbulence phase screens are sequentially loaded onto the second SLM with 60 Hz to increase the variety of turbulence scenarios. Several examples of various levels of turbulence-affected multiplexed OAM beams are shown in Fig. 5. After being reflected by the second SLM, the turbulence-affected OAM-coded beams are Fourier transformed as passing through the first Fourier lens at the focal plane where the DMD is placed. In the digital Fourier transform implementations, the pixel of the Fourier spatial kernel is considered to be the same as the Fourier transformed image to realize the pixel-wise multiplication. However, in our Fourier optics system, the Fourier spatial kernel pixel is constrained by the resolution of SLM:

$$F{T}_{{{{{{{{\rm{kernel}}}}}}}}{{{{{{{\rm{pixel}}}}}}}}}=\frac{\lambda f}{4{\Delta }^{2}},$$
(3)

where λ is the wavelength of the light, f is the focal length of the lens, and Δ is the pixel pitch of the SLM with the super-pixel of 4. In our experiment, we use a 1550 nm laser source, two lenses with 150 mm focal length (f = 150 mm), and the pixel pitch of the SLM is 8 μm. Therefore, we have the optics spatial frequency filter with a 908 × 908 pixel size to match the digital to the optical Fourier optics convolution. The pre-trained spatial kernels, loaded onto the DMD with 320 Hz, serve as a spatial filter set in the frequency domain, filtering the spatial frequency components of the input OAM-coded beams. Kernels are integrated sequentially with the spatial frequency components of the incoming Fourier-transformed beams, executing pixel-wise multiplications. After passing through the second Fourier lens, the spatially-frequency-filtered beams are inverse Fourier transformed into the real-space and subsequently captured by the camera with 48 frames per second, realizing the transition from the optical to electronic domain. The images captured by the camera are collected as electronic output to a max-pooling layer and two fully connected layers. These layers comprise a hidden layer consisting of 256 neurons equipped with a ReLU activation function and a classification layer with the corresponding number of neurons, finalizing the construction of the hybrid optical-electronic neural network. Python codes for generating designed ECNN and OECNN are provided in Supplementary Software 3, and Supplementary Software 4, respectively.

Fig. 3: Experimental implementation of the orbital angular momentum (OAM)-coded signal generation and Fourier optics convolution.
figure 3

a Laboratory experiment setup. A laser source with a 1500 nm wavelength is coupled, expanded, and projected onto the first spatial light modulator (SLM) loaded with computer-generated combined fork phase grating patterns, generating OAM-coded beams with desired topological charges. After being reflected by the second SLM, which is loaded with the atmospheric turbulence phase masks, the turbulence-affected OAM-coded beams complete the Fourier optics convolution via pixel-wise multiplications with pre-trained kernels loaded on the digital micromirror device (DMD), which is placed at the focal plane. The spatially-frequency-filtered components are inverse Fourier transformed after passing through the second lens and captured by the camera as electronic output to the electronic max-pooling layer and two fully connected layers. FL Fourier lens. A scale bar of length 25.4 mm is drawn. b Experimental setup scheme. FC fiber coupler, BE beam expander, BS beam splitter, P pinhole, L lens.

Full size image
Fig. 4: Experimentally generated intensity profile of 4-bit orbital angular momentum (OAM)-coded signal under turbulence-free condition.
figure 4

OAM modes with topological charge l = −7, −5, 2, and 6 are selected. Each panel (a–p) represents the coded 4-bit string from (0000) to (1111) where the selective OAM modes are active. A grayscale color bar and a scale bar of length 1 mm are drawn.

Full size image
Fig. 5: Experimentally generated intensity profile of the multiplexed orbital angular momentum (OAM) beams passing through various simulated atmospheric turbulence conditions.
figure 5

Multiplexed OAM beams with two different topological charges (l1 = −8, l2 = 6), (l1 = −7, l2 = 7), (l1 = −6, l2 = 8), (l1 = −8, l2 = 8), and (l1 = −10, l2 = 10) are shown, respectively. a–e, f–j, k–o, p–t are turbulence-affected multiplexed OAM beams under turbulence-free, weak (D/r0 = 0.15), medium (D/r0 = 0.3), and strong (D/r0 = 1.5) turbulence level, respectively. A grayscale color bar and a scale bar of length 1 mm are drawn.

Full size image