Turbulence-resilient pilot-assisted self-coherent free-space optical communications using automatic optoelectronic mixing of many modes

In free-space optical communications that use both amplitude and phase data modulation (for example, in quadrature amplitude modulation (QAM)), the data are typically recovered by mixing a Gaussian local oscillator with a received Gaussian data beam. However, atmospheric turbulence can induce power coupling from the transmitted Gaussian mode to higher-order modes, resulting in a significantly degraded mixing efficiency and system performance. Here, we use a pilot-assisted self-coherent detection approach to overcome this problem. Specifically, we transmit both a Gaussian data beam and a frequency-offset Gaussian pilot tone beam such that both beams experience similar turbulence and modal coupling. Subsequently, a photodetector mixes all corresponding pairs of the beams’ modes. During mixing, a conjugate of the turbulence-induced modal coupling is generated and compensates the modal coupling experienced by the data, and thus the corresponding modes of the pilot and data mix efficiently. We demonstrate a 12 Gbit s−1 16-QAM polarization-multiplexed free-space optical link that is resistant to turbulence. A transmission scheme for free-space optical communications is shown to be highly robust against turbulence.

C ompared with radio, free-space optical (FSO) communications have gained substantial interest due to their higher data capacity and lower probability of interception [1][2][3] . Often, an amplitude-only-modulated Gaussian data beam (for example, as in pulse-amplitude modulation (PAM)) is transmitted and recovered 2 ; since data are encoded as distinct amplitude levels, the data constellation points of PAM lie on a one-dimensional line in the two-dimensional in-phase (I) and quadrature (Q) constellation 4 . Alternatively, FSO systems can benefit from simultaneously recovering the data beam's amplitude and phase to enable complex modulation formats 5,6 such as quadrature amplitude modulation (QAM) 7 . Since data are encoded as distinct vectors, QAM I/Q constellation points can be arranged in a two-dimensional array 4 . In comparison with PAM of the same number of constellation points (that is, modulation order) and average power per bit, QAM is generally less demanding in terms of the optical signal-to-noise ratio (OSNR) of the transmitted data due to its larger Euclidean distance in the two-dimensional I/Q constellation 4 . This advantage tends to be more pronounced as the modulation order increases 4 . In addition, phase recovery can enable various digital signal processing (DSP) functions 8 that might benefit future FSO systems 6,9 (for example, compensation for hybrid fibre/FSO systems 6 and adaptive probabilistic shaped modulations 9 ).
Intensity modulation/direct detection (IM/DD) FSO links typically receive amplitude-encoded data by directly detecting the beam's intensity levels, yet phase information is not readily recovered 2,5,10 . Alternatively, FSO systems can recover both amplitude and phase by using coherent detection, which mixes the data beam with a receiver Gaussian local oscillator (LO) beam 5,9,11 . However, atmospheric turbulence generally limits coherent detection because it induces power coupling of the data beam from the Gaussian mode to other Laguerre-Gaussian (LG) spatial modes [12][13][14] . Such turbulence-induced modal coupling can significantly degrade the data-LO mixing efficiency due to 'mode mismatch' between the LO and data beams 12,13,15,16 . Without turbulence, the photodetector (PD) efficiently mixes the data and the LO since they typically occupy the same single-Gaussian mode 17 , and hence are 'mode matched' in their spatial distributions 18,19 . With turbulence, however, significant power of the data beam can be coupled into higher-order LG modes and degrade the mixing efficiency by >20 dB (refs. 12,13,15 ) since data power coupled to orthogonal higher-order modes does not efficiently mix with the Gaussian LO 15,20 .
To enable amplitude and phase recovery in turbulent links, various modal-coupling mitigation approaches have been demonstrated [21][22][23][24][25] . One technique uses adaptive optics to couple the data power back into the Gaussian mode by measuring the distortion using a wavefront sensor and applying a DSP-calculated conjugate phase to the beam by a wavefront corrector 21 . Another technique uses multi-mode digital coherent combining [22][23][24][25] , wherein much of the data power in higher-order modes is captured by either a multi-mode fibre 22,23,25 or an array of single-mode fibre (SMF) apertures 24 . Subsequently, the power from each of the multiple modes is recovered by a separate coherent detector and combined using DSP [22][23][24][25] . The performance depends on the number of recovered Turbulence-resilient pilot-assisted self-coherent free-space optical communications using automatic optoelectronic mixing of many modes modes, and the complexity of the detection system tends to increase with the number of detected modes 22,23,25 . Since turbulence may induce coupling to a large number of modes, a laudable goal towards achieving simultaneous amplitude and phase recovery would be to automatically compensate for such power coupling without additional data processing and do so in a single element that efficiently scales to recover all captured modes.
In this article, we experimentally demonstrate the near-error-free transmission of a 12 Gbit s −1 16-QAM polarization-multiplexed (PolM) FSO link that is resilient to turbulence-induced LG modal power coupling for 200 random turbulence realizations. The amplitude and phase of the transmitted QAM data are retrieved using a pilot-assisted self-coherent detector. We transmit a Gaussian pilot beam with a frequency offset from the Gaussian data beam such that both beams experience similar turbulence-induced LG modal coupling. Subsequently, a single free-space-coupled PD mixes the received multi-mode data beam with the multi-mode pilot beam in 'self-coherent' detection 26 . During mixing, a conjugate of the turbulence-induced modal coupling of the pilot beam is automatically generated and used to compensate for the modal coupling in the data beam. Specifically, each data-pilot LG modal pair efficiently mixes and contributes to the intermediate frequency (IF) signal. Since the data and pilot experience similar modal coupling, our approach can simultaneously mix and recover nearly all of the captured data modes using a single PD. Experimental results for the turbulence strength (that is, ratio of the beam size to the Fried parameter) 2w 0 /r 0 ≈ 5.5 show an average mixing loss of ~3.3 dB.

Results
Concept of pilot-assisted self-coherent detection using optoelectronic mixing. In an FSO link, a fundamental Gaussian beam (that is, LG 0,0 (x,y)) carrying a data channel (denoted as S(t,f) with the carrier frequency f) is transmitted through a turbulent atmosphere. Owing to a random spatial and temporal refractive index distribution, the turbulence effects can induce a transverse, spatially dependent wavefront distortion to the Gaussian beam 27 . Moreover, since such distortion induces modal power coupling, the electrical field of the data beam (E data ) at the receiver aperture can be expressed as a superposition of LG modes 12,28 : where LG l,p (x,y) represents the electrical field of the LG mode 17 with an azimuthal index l and a radial index p; LG * l,p (x, y) dxdy is the complex coefficient of the corresponding LG l,p component in the wavefront, * denotes the conjugate of the modal electrical field, and the portion of the optical power coupled to the LG l,p mode is |a l,p | 2 ; and U(x, y) = ∑ l ∑ p a l,p LG l,p (x, y) represents the turbulence-induced LG modal coupling. Ideally, the complex weights a l,p for all modal components tend to satisfy l p a l,p 2 ∼ = 1 if the receiver aperture can collect almost the entire beam 28 . A turbulent IM/DD FSO link (that is, S(t,f) is amplitude-only encoded) may suffer from turbulence-induced modal-coupling loss if an SMF-coupled PD is used because higher-order modes are not efficiently captured by the SMF 13 . For a free-space-coupled PD, however, an IM/DD FSO link may not be significantly affected by modal coupling if the receiver aperture can collect most of the distorted beam 29 . This free-space-coupled PD can utilize the detected optical intensity (that is, |S(t,f)| 2 ) to recover the amplitude-encoded data, but the beam's phase information is not readily recoverable.
As shown in Fig. 1a, coherent-detection FSO links can recover both the amplitude and phase of the data although they suffer from performance degradation caused by turbulence-induced modal coupling. Here, the transmitted data S(t,f) contain both amplitudeand phase-encoded data (for example, 16-QAM data). By way of a simple illustrative example, the continuous-wave LO at the receiver in a single-PD heterodyne coherent detector has an optical frequency offset Δf from the data carrier (denoted as C(f − Δf)) and is a Gaussian beam (that is, C(f − Δf)·LG 0,0 (x,y)). The square-law mixing in the PD of the coherent receiver results in a photocurrent 26,30 where Re[·] is the real part of a complex element; I is the generated photocurrent; |C(f − Δf)| 2 and |S(t,f)| 2 are the direct current (d.c.) and the signal-signal beating interference (SSBI) photocurrent, respectively; and 2Re[S(t,f)C*(f − Δf)] generates the desired signal-LO beating (SLB) photocurrent. However, the Gaussian-mode LO does not mix efficiently with the multiple-LG-mode data beam due to the mode mismatch between their LG spectra, which is expressed as 15 where orthogonality amongst the LG modes ensures that ∫ ∫ LG 0,0 (x, y) LG * 0,0 (x, y) dxdy = 1 and ∫ ∫ LG l,p (x, y) LG * 0,0 (x, y) dxdy = 0, given that l ≠ 0 or p ≠ 0. Equation (3) shows that only the portion of the transmitted power that remains LG 0,0 after turbulence can be efficiently mixed with the LO and utilized for recovering the QAM data. Such modal-coupling loss can result in severe degradation of the mixing IF power and thus the recovered data quality 20 . We note that this mixing-efficiency degradation in coherent detection can occur for a PD that is: (1) free-space-coupled due to orthogonality between the higher-order modes and the Gaussian LO 15,20 and (2) SMF-coupled due to power in the higher-order modes not being efficiently coupled into the fibre 13 . Figure 1b illustrates the simultaneous recovery of the amplitude and phase of QAM data by utilizing pilot-assisted self-coherent detection, which automatically compensates for the turbulence-induced modal coupling. In addition to the Gaussian data beam, we transmit a co-axial Gaussian beam carrying a continuous-wave pilot tone with a frequency offset Δf, producing a frequency gap between the pilot and data beams of roughly the channel bandwidth (B) to avoid SSBI. The electrical fields of the data and pilot beams are likely to experience similar turbulence-induced distortion and modal coupling due to their frequency difference being orders of magnitude smaller than their carrier frequencies 27 . This similar distortion produces automatic 'mode matching' between the beams, such that the electric field of the pilot tone is 31 : Importantly, a turbulence-induced LG-coupling conjugate U* is automatically generated from the pilot to compensate for the modal coupling experienced by the distorted data beam, and the total generated photocurrent is: where S(t,f)C*(f − Δf) generates the desired signal-pilot beating (SPB) photocurrent at an IF of Δf. The modal coupling is (ideally) corrected in an automatic fashion and the mixing efficiency is: Mixing efficiency ∝ U (x, y) U * (x, y) dxdy = l p a l,p LG l,p (x, y) l′ p′ a * l′,p′ LG * l′,p′ (x, y) dxdy = l p l′ p′ a l,p LG l,p (x, y) a * l′,p′ LG * l′,p′ (x, y) dxdy = l p a l,p 2 ∼ = 1, where each LG l,p component of the data beam is efficiently mixed with the corresponding LG l,p component of the pilot beam. Consequently, almost all the captured optical power carried by higher-order LG spatial modes can contribute to the IF signal and can be automatically recovered using a single square-law free-space PD. The recovered QAM data can thus exhibit resilience against modal-coupling loss due to the efficient mixing between the data and pilot beams.
We note that the pilot-assisted self-coherent approach shares some similarities with both IM/DD and coherent detection: (1) similar to IM/DD, our approach does not use a receiver-based LO; and (2) similar to coherent detection, our approach recovers the amplitude and phase by mixing an 'LO-like' transmitter-generated pilot with the data beam and is often called 'self-coherent detection' 32,33 . Notably, the pilot in our self-coherent system would experience similar FSO channel loss as the data beam, which may be noteworthy in longer-distance FSO links, whereas the LO in coherent detection would not 6 .
Generally, the OSNR needed to achieve a desired bit error rate (BER) depends on both the modulation formats and the detection approaches 4,32,34 . When comparing our self-coherent detection with heterodyne coherent detection for amplitude-and phase-encoded data, the transmitted power of self-coherent detection is shared between the pilot and data beams, resulting in self-coherent detection being more OSNR-demanding compared with coherent detection (without turbulence effects) 32 . For example, to achieve a given BER for the same QAM order, our self-coherent approach is likely to require an OSNR of around 3 dB higher when the carrier (that is, LG 0,0 LG l,p modal coupling LG 0,0 LG l,p modal coupling LG 0,0 LG l,p modal coupling

Fig. 1 | Concept of simultaneous amplitude and phase recovery of QAM data in turbulent FSo links. a,
The performance of coherent detection can be significantly degraded by turbulence-induced LG modal-coupling effects. A fundamental Gaussian beam (that is, LG 0,0 mode) carrying 16-QAM data is transmitted through a turbulent atmosphere. Owing to the turbulence-induced LG modal power coupling, the received data beam will contain many LG modes. In an LO-based heterodyne coherent detector, only the LG 0,0 mode can be efficiently mixed with the LO and recovered, resulting in degradation of the recovered data quality. This is true for both free-space-coupled and SMF-coupled PDs since the LO is typically single-Gaussian mode. Rx, receiver. b, Pilot-assisted self-coherent detection can automatically compensate for the turbulence-induced LG modal-coupling effects. In the pilot-assisted self-coherent detector, we transmit an additional continuous-wave pilot, which experiences similar turbulence-induced LG coupling as the data beam. During optoelectronic mixing of the pilot and data beams in a square-law free-space detector, a conjugate of the turbulence experienced by the pilot is automatically generated and compensates the turbulence experienced by the data beam. Therefore, almost all the data LG modes can be efficiently mixed with the pilot to enable simultaneous amplitude and phase recovery of QAM data. In a and b, the frequency offset Δf is greater than the data bandwidth B to avoid the SSbI. SSbI, signal-signal beating interference; SLb, signal-LO beating; SPb: signal-pilot beating.
pilot)-to-signal power ratio (CSPR) is ~1 compared with heterodyne coherent detection 32,33 . When comparing our amplitude-and-phase self-coherent approach with amplitude-only IM/DD, the OSNR advantage of self-coherent QAM over IM/DD PAM (with the same modulation order) becomes more pronounced as the modulation order increases (for example, conventionally regarded to be many decibels for ≥16-QAM) 4,32-34 .
In longer-distance FSO links, the required optical power per bit for a desired BER can be a limiting factor 10,16 . Since the transmitted power is shared between the pilot and data beams, self-coherent detection will probably have a lower signal-to-noise ratio (SNR) compared with free-space-coupled IM/DD with the same received optical power and receiver thermal noise. Moreover, the SNR advantage of QAM over PAM diminishes as the modulation order decreases 4 . Consequently, IM/DD may have a better BER performance than pilot-assisted self-coherent detection for low modulation orders, such as 2-PAM 16,34 . We also note that IM/DD may have a better performance than self-coherent detection under lower SNR conditions even at higher modulation formats 4,32,34 .
Moreover, since atmospheric turbulence tends not to induce significant depolarization effects 35 , our pilot-assisted system should be compatible with PolM techniques by transmitting pilot-data pairs on each orthogonal polarization. Experimental results for a PolM system are shown later in Figs. 4 and 5.
Our approach transmits a pilot along with the data, and the pilot serves to help probe the turbulence and create a conjugate of the distortion from modal coupling. In optical communications, we note that pilot-assisted techniques have been demonstrated to probe a channel's signature and apply a conjugate of that signature to help mitigate various channel impairments, including cross-phase modulation 36 and laser phase noise 37 . More specifically, it has been shown via simulation that turbulence-induced modal crosstalk can be reduced by mixing a pilot beam and data-carrying LG beams in a mode-division-multiplexed FSO link 38 . In that approach, the pilot acquires the turbulence signature, is split into multiple copies at the receiver, and generates a conjugate of the turbulence for each of the LG data beams in separate PDs.
Experimental setup of free-space optical communications with emulated turbulence. We experimentally demonstrate pilot-assisted self-coherent detection in a 12 Gbit s −1 PolM 16-QAM 1-m-long FSO link with emulated turbulence. Figure 2 shows the experimental setup (see the Methods section for more details). The strengths (that is, the ratio of the beam size 2w 0 to the Fried parameter r 0 ) of the weaker and stronger turbulence effects are 2w 0 /r 0 ≈ 2.2 and 5.5, respectively.
We emulate atmospheric turbulence effects using a single rotatable phase plate. Generally, turbulence effects can be more accurately emulated using multiple phase plates 27 . To address our emulation accuracy, we simulate the optical and electrical mixing power loss using single and multiple random phase screen (RPS) models; the simulation results show similar loss distributions and trends for both 1-RPS and 5-RPS models (see Supplementary Figs. 1 and 2 for more details).

Characterization of optical and electrical mixing power loss.
We measure the turbulence-induced optical power loss and electrical mixing power loss of the pilot-assisted self-coherent detector for each polarization at 1,000 random realizations of the emulated turbulence. For both X and Y polarizations, Fig. 3a shows LG l,p spectrum measurement (signal on λ 1 turned off) that stronger turbulence induces <2 dB of optical power loss for self-coherent detection since the free-space-coupled PD can capture most of the power; we note that free-space-coupled IM/DD systems are likely to have similar captured power loss. As shown in Fig. 3b, the self-coherent detector has an electrical mixing power loss of <3 dB and <6 dB for 99% weaker and 90% stronger turbulence realizations among 1,000 random turbulence realizations, respectively. The relatively low mixing power loss for self-coherent detection is due to efficient mixing of the pilot and data beams, which is likely to recover almost all the data power from the captured modes. As discussed, turbulence-induced modal coupling can result in significant power loss for 'mode-selective' SMF-coupled IM/DD or coherent detectors. Figure 3a shows that the optical power loss for SMF-coupled systems ranges from ~2 to ~22 dB and from ~7 to ~30 dB under ~2.2 and ~5.5 turbulence strengths, respectively. Among the 1,000 emulated turbulence realizations, Fig. 3b shows that the coherent detector can suffer from a mixing power loss of ~28 dB for 99% and 90% of weaker and stronger turbulence, respectively. This mixing loss is due to the SMF-coupled detector not efficiently capturing the power coupled to higher-order modes 13 .
To help further validate our experimental results, we simulate the self-coherent system using 1-RPS (see Supplementary Equations (1)-(6) for simulation details). As shown in Fig. 3c, the simulation results indicate that self-coherent detection suffers <4 dB of average optical and electrical mixing power loss as the turbulence strength 2w 0 /r 0 is increased from ~1 to ~7. Moreover, the plotted experimental results are generally in agreement with the simulation.
Turbulence-resilient 12 Gbit s −1 16-QAM PolM free-space optical transmission. We demonstrate 12 Gbit s −1 PolM FSO transmission under emulated turbulence effects, with each polarization carrying 1.5 Gbaud 16-QAM data. The transmitted total optical power per polarization (including pilot and data beams) is ~7 dBm. The transmitted CSPR values are ~1.1 and ~1 for X and Y polarizations, respectively. Figure 4 shows the recovered 16-QAM constellations using the self-coherent detector under example realizations of the weaker and stronger turbulence. We measure the turbulence-induced LG spectra for l and p indices of −5 to +5 and 0 to 10, respectively. The complex wavefront is measured using off-axis holography (see Methods) 39 . With no turbulence effects, Fig. 4a shows that the pilot-assisted self-coherent detector can achieve a near-error-free performance and recover an error vector magnitude (EVM) of ~8% for the 16-QAM data. Under one random realization of weaker turbulence, the measured LG spectrum of Fig. 4b shows that the data power is mainly coupled to the neighbouring LG modes. Under two different random realizations of stronger turbulence, Fig. 4c,d show that turbulence effects can induce a power loss of >25 dB and that power can be coupled to a large number of LG modes. The performance of the self-coherent detector is not severely affected by these turbulence effects and the 16-QAM data can be recovered with EVM values from ~8% to ~10% for both realizations. This turbulence resiliency is due to the automatic modal-coupling compensation by the pilot-data mixing, enabling almost all captured LG modes to be efficiently recovered.
To elucidate the effects of turbulence-induced modal coupling on coherent detection, we also show the recovered 16-QAM data for an SMF-coupled heterodyne coherent detector in Fig. 4; the recovered data quality degrades for both polarizations, from EVM values of ~7.5% without turbulence (Fig. 4a) to >16% for stronger turbulence (Fig. 4c,d). This degradation is due to data power coupled to higher-order modes that is not efficiently captured by the SMF 13 .
We also measure the electrical spectra for the self-coherent and coherent detectors under these example turbulence realizations. Compared with the case of no turbulence, there is a ~3 dB and ~18 dB SNR degradation of the IF signal measured for the self-coherent and coherent detectors, respectively, under the turbulence realizations of Fig. 4 (see Supplementary Fig. 3 for more details). Figure 5 shows measured BER values for the pilot-assisted self-coherent detector under 200 random realizations of weaker and stronger turbulence. Results show that the self-coherent detector can achieve BER values below the 7% forward error correction limit for all realizations. Since turbulence can cause strong modal-coupling-induced power loss, the performance of the coherent detector can degrade and does not achieve the 7% forward error correction limit for some realizations.
We further characterize the performance of the self-coherent detector by measuring the BER as a function of the transmitted power. We find power penalties of ~3 dB for both polarizations under one realization of the stronger turbulence (see Supplementary Fig. 4).

Enhancing spectral efficiency using Kramers-Kronig detection.
In our self-coherent approach, a frequency gap between the pilot and data beams is needed to avoid SSBI. This gap is roughly equal to the data bandwidth, such that our spectrum is around 2× the data bandwidth. However, this frequency gap can be reduced to increase the spectral efficiency using SSBI mitigation techniques 40,41 such as Kramers-Kronig (KK) detection 6,41 . Therefore, we demonstrate a reduction of the data-pilot gap to ~0.1 GHz (IF ≈ 0.9 GHz) using KK detection (see Supplementary Fig. 5  LG spectrum (2w 0 /r 0 ≈ 2.2) LG spectrum (2w 0 /r 0 ≈ 5.5) LG spectrum (2w 0 /r 0 ≈ 5.5)

Fig. 4 | Experimental results of turbulence-induced
LG modal power coupling and recovered 16-QAM data qualities using the pilot-assisted self-coherent detector. a, No turbulence distortion. b, One example realization (R1) of the weaker turbulence distortion (2w 0 /r 0 ≈ 2.2). c,d, Two different example realizations (R1 (c) and R2 (d)) of the stronger turbulence distortion (2w 0 /r 0 ≈ 5.5). For each of the four realizations, we measure the LG modal power spectrum (two indices −5 ≤ l ≤ +5 and 0 ≤ p ≤ 10) and recover the 16-QAM data constellations. In this demonstration of PolM FSO data transmission, each polarization carries a 6 Gbit s −1 16-QAM signal. To indicate the effects of turbulence-induced modal coupling on a coherent-detection FSO system with the single-Gaussian-mode LO, the performance using an LO-based heterodyne coherent detector is also shown. pol., polarization; EVM, error vector magnitude.
under example realizations of weaker and stronger turbulence. Using KK detection, the spectral efficiency of the pilot-assisted approach could be increased by roughly 2×. Importantly, the KK scheme typically utilizes a stronger pilot than the non-KK approach. Hence, it is typically less power efficient than the non-KK pilot-assisted approach 41 , resulting in a trade-off between power efficiency and spectral efficiency.

Discussion
The following issues are interesting to consider: (i) Our 1.5 GHz baud rate is limited by the ~3.5 GHz bandwidth of the PD. However, free-space-coupled PDs with a bandwidth of ~49 GHz have been reported 42 , making >100 Gbit s −1 possible. (ii) We use LG modes to analyse modal coupling. However, we could utilize other bases (for example, Hermite-Gaussian 22 ). Importantly, we do not need to specify a priori the basis used because our approach is 'automatic' and the pilot and data can be described in different bases. (iii) We note that differential-phase-shift-keyed (DPSK) systems are also referred to as 'self-coherent' 43,44 . In DPSK systems: (1) data are typically encoded in the optical phase difference between neighbouring symbols; (2) the received data beam is split into two copies of which one is delayed; (3) these copies are coherently combined using a Mach-Zehnder interferometer; and (4) both Mach-Zehnder interferometer output branches are detected by two PDs simultaneously to recover the differential-encoded data 43 . Different from our pilot-assisted approach, almost all the captured optical power in DPSK systems contains data 44 . However, to recover the amplitude and phase of QAM data, differential systems typically utilize a more complex receiver than that of the pilot-assisted approach 43,45 .
Interestingly, it might be possible to use multi-mode mixing as described in this paper to achieve automatic turbulence resiliency in a differential, high-order QAM system. (iv) A beam diverges with the link distance. Consequently, both the data and pilot beams can suffer from truncation by a limited-size receiver aperture causing power loss for longer-distance links 46 . Moreover, truncation can cause power coupling to higher-order modes 46 . These higher-order modes tend to be automatically mixed by the pilot-assisted self-coherent detection since the pilot and data beams experience similar truncation effects. (v) We use a free-space-coupled PD. Can our approach use fibre-coupled PDs? One possibility might be to use a multi-mode fibre-coupled PD 10 such that many modes are captured and then impinge on the PD. (vi) Although FSO propagation is dependent on a beam's carrier frequency, it is likely that beam divergence and turbulence-induced spatial distortions are similar for the pilot and data beams. This is because their typical frequency difference (<1 nm) is substantially smaller than their carrier frequencies (~1.55 μm) 27,47 .
This paper has described the concept and experimental/simulation results of pilot-assisted self-coherent links to automatically mitigate modal coupling for recovering the amplitude and phase of data. However, there are important questions for further study as to limits and dependencies of our approach, including: (1) the frequency dependence of spatial distortions and (2) its effectiveness as a function of distance, divergence, turbulence strength, signal bandwidth and signal-carrier frequency separation.

online content
Any methods, additional references, Nature Research reporting summaries, source data, extended data, supplementary infor- To indicate the effects of turbulence-induced modal coupling on a coherent-detection FSO system with the single-Gaussian-mode LO, the performance using an SMF-coupled LO-based heterodyne coherent detector is also shown. In the PolM FSO data transmission, each polarization carries a 6 Gbit s −1 16-QAM data signal. Note that we measure the bER performance for one polarization at a time due to limitations of our measurement setup. Therefore, the bER values for X and Y polarizations with the same realization label may correspond to different turbulence realizations and are difficult to be compared directly. mation, acknowledgements, peer review information; details of author contributions and competing interests; and statements of data and code availability are available at https://doi.org/10.1038/ s41566-021-00877-w.

Methods
Experimental details of free-space optical communications in emulated turbulence. As shown in Fig. 2, we transmit a pair of data-carrying and pilot Gaussian beams on both X and Y polarizations. A 6 Gbit s −1 16-QAM data channel at a wavelength of λ 1 ≈ 1.55 μm is generated, amplified using an erbium-doped fibre amplifier (EDFA) and equally split into two copies. One copy is delayed using a >15 m SMF to decorrelate the data channels and two independent data channels are individually combined with another pilot tone at a wavelength of λ 2 (with a frequency offset of ~2.6 GHz from λ 1 , Δλ ≈ 0.02 nm). The polarizations of the signals and pilots are adjusted and subsequently combined using a polarization beam combiner to transmit PolM 16-QAM signals. The total optical power including the pilot and data beams is ~7 dBm for each of the polarizations. The optical signal is coupled to free space using an optical collimator (Gaussian beam size of diameter 2w 0 ≈ 2.2 mm), is distorted using a rotatable turbulence emulator (see the section 'Experimental emulation of atmospheric turbulence effects') and then propagates in free space for ~1 m. In this demonstration, we emulate different strengths of atmospheric turbulence using two separate turbulence emulators with different Fried parameters r 0 of 1.0 mm and 0.4 mm. The emulated turbulence distortion for the transmitted Gaussian beam is characterized by the ratio of the beam size to the Fried parameter 27 , and these are 2w 0 /r 0 ≈ 2.2 and 5.5 for the two emulators.
At the receiver, we demultiplex one polarization at a time using a half-wave plate cascaded with a polarizer. The receiver has an aperture diameter of ~10 mm. We measure the spatial amplitude and phase profiles of the turbulence-distorted beam and calculate its LG decomposition using off-axis holography 39 (see the section 'Off-axis holography for complex wavefront measurement'). After polarization demultiplexing, the distorted beam is equally split into two copies that are sent to the pilot-assisted self-coherent detector and a single-PD LO-based heterodyne coherent detector.
In the pilot-assisted self-coherent detector, the entire spatial profiles of the distorted data and pilot beams are focused into a free-space-coupled InGaAs PD (3 dB bandwidth <3.5 GHz) using an aspheric lens with a focal length of 16 mm and a numerical aperture of ~0.79. The coupling efficiency of the received Gaussian beam, defined as the ratio of the optical power detected by the PD over the total received optical power by the receiver aperture (without turbulence effects), is measured to be >92%. The generated photocurrent is recorded using a real-time digital oscilloscope and the I-Q information of the data channel is subsequently retrieved using off-line DSP algorithms (see the section 'Digital signal processing for retrieving the I-Q information at the receiver'). The Nyquist-shaped 16-QAM data channel has a symbol rate of 1.5 GHz with a roll-off factor of 0.1, expanding the data's spectrum to ~1.7 GHz. To avoid SSBI effects, we set the IF (that is, the difference between the pilot and data beams' carrier frequencies) at Δf ≈ 2.6 GHz, which includes a frequency gap of ~1.8 GHz between the pilot and data beams. Thus, the total transmitted pilot-assisted signal spectrum is ~3.5 GHz, which is roughly twice that of the data spectrum (see Supplementary Fig. 3 for more details).
At the single-PD LO-based heterodyne coherent detector (the pilot λ 2 is turned off), we set the same IF value as the pilot-assisted self-coherent receiver. The distorted Gaussian beam is coupled into an SMF via a collimator (aperture diameter ≈ 3.5 mm), amplified using an EDFA, and mixed with an LO (at the same wavelength λ 2 as the pilot) at the SMF-coupled PD. The received optical signal is amplified by the EDFA to meet the power sensitivity requirement of the SMF-coupled PD. The electrical signal is subsequently recorded using a real-time digital oscilloscope and processed to retrieve the data channel's I-Q information using the same off-line DSP algorithms as the pilot-assisted self-coherent detector. Note that we measure the optical power loss and electrical mixing power loss of this detector (shown in Fig. 3) without using the EDFA inside this receiver. The mixing power loss is measured at the IF of ~2.6 GHz in the electrical domain.
To evaluate the effectiveness of the pilot-assisted self-coherent detector for various turbulence scenarios, we measure the BER values of the 16-QAM data channels carried by both polarizations over 200 random turbulence realizations. To measure turbulence-induced modal-power-coupling effects on an SMF-coupled coherent detector, we also measure the BER performance for the LO-based heterodyne coherent detector over 200 random turbulence realizations. Note that we measure the BER performance for one polarization at a time due to the limitations of our measurement setup. Therefore, the BER values for X and Y polarizations with the same realization label may correspond to different turbulence realizations and are difficult to be compared directly.
Experimental emulation of atmospheric turbulence effects. We experimentally emulate the turbulence-induced distortion by utilizing glass plates (Lexitek), the refractive index distributions of which are fabricated to emulate Kolmogorov power spectrum statistics 20,27 . Two rotatable glass plates are used separately in the experiment with different Fried parameters (r 0 ) of 1.0 mm (weaker turbulence effects) and 0.4 mm (stronger turbulence effects). Different 'random' turbulence realizations are implemented by rotating the single glass plate to different orientations. The diameter of the transmitted Gaussian beam is 2w 0 ≈ 2.2 mm. The data-carrying Gaussian beams are distorted by the glass plate and then propagate in free space for a distance of ~1 m before reaching the receiver. The strength of the turbulence distortion is given by the ratio of the beam diameter to the Fried parameter 27 , that is, 2w 0 /r 0 . For a proof-of-concept demonstration, we investigate the performance of the pilot-assisted self-coherent detector at two different turbulence strengths (2w 0 /r 0 ≈ 2.2 and 5.5). Under even stronger turbulence effects, the self-coherent FSO systems may suffer from strong beam-wandering effects and subsequent optical power loss 48 . A beam pointing and tracking system can be used to compensate for these beam-wandering effects 49 .
In this demonstration, we use a single phase plate to emulate the turbulence distortions for this ~1 m FSO link. However, a multiple-phase-plate emulation can generally provide a higher accuracy for emulating the volume atmospheric turbulence effects 27 . To illustrate the validity of our emulation method, we simulate 1-RPS and 5-RPS turbulence effects; similar trends for turbulence-induced system degradations were found (see Supplementary Figs. 1 and 2 for more details). We note that our turbulence emulation provides an approximation of the Gaussian beam's propagation in a turbulent medium and may not fully reflect the effects of real atmospheric turbulence. To further enhance the accuracy of turbulence emulation, some advanced modelling or emulation methods could potentially be applied 27,50 .
Off-axis holography for complex wavefront measurement. We use off-axis holography to measure the complex wavefront (that is, the amplitude and phase) of the distorted Gaussian beam and its corresponding LG spectrum. An off-axis reference Gaussian beam (beam diameter ~7 mm) on the same wavelength as the distorted pilot Gaussian beam is incident on the infrared camera with a tilted angle. We record the off-axis interferogram and apply digital image processing to extract the complex wavefront (see Supplementary Fig. 6 for more details). The data-carrying beam is turned off when we measure the complex wavefront of the turbulence-distorted pilot beam.
After the complex wavefront of the distorted Gaussian beam is obtained, we decompose it into a two-dimensional LG modal spectrum in which the two indices l and p range from −5 to +5 and from 0 to 10, respectively, as expressed in equation (7) 28 : Erec (x, y) LG * l,p (x, y) dxdy, where E rec (x,y) and LG l,p (x,y) are the measured complex field of the distorted Gaussian beam and the theoretical complex field of an LG l,p mode, respectively. The ratio of optical power coupling to the LG l,p mode is given by |a l,p | 2 .
Digital signal processing for retrieving the I-Q information at the receiver.
The detected electrical signal is sampled using a real-time oscilloscope (20 GHz bandwidth and 50 gigasamples per second sampling rate) and recorded for off-line DSP. The recorded signals from the pilot-assisted self-coherent detector and the single-PD LO heterodyne coherent detector are processed using the same DSP procedures. Each signal is filtered using a root-raised-cosine finite impulse response filter with a roll-off factor of 0.1, and the filtered signal is subsequently equalized using a constant modulus algorithm. After equalization with the constant modulus algorithm, carrier frequency offset estimation and carrier phase recovery are sequentially performed to reduce the frequency and phase difference between the signal and the LO (or pilot). Finally, the EVM and BER values of the demodulated signal are calculated to evaluate the quality of the data transmission. The EVM of the detected signal is calculated using equation (8) as follows 7 : where the x i and xi represent the transmitted and recovered data symbols, respectively, and N is the total number of detected symbols. In this demonstration, ~180,000 symbols are collected to calculate the EVM and BER values of the 16-QAM data signals.

Data availability
All data, theory detail, simulation detail that support the findings of this study are available from the corresponding authors upon reasonable request.