Main

Compared with radio, free-space optical (FSO) communications have gained substantial interest due to their higher data capacity and lower probability of interception1,2,3. Often, an amplitude-only-modulated Gaussian data beam (for example, as in pulse-amplitude modulation (PAM)) is transmitted and recovered2; since data are encoded as distinct amplitude levels, the data constellation points of PAM lie on a one-dimensional line in the two-dimensional in-phase (I) and quadrature (Q) constellation4. Alternatively, FSO systems can benefit from simultaneously recovering the data beam’s amplitude and phase to enable complex modulation formats5,6 such as quadrature amplitude modulation (QAM)7. Since data are encoded as distinct vectors, QAM I/Q constellation points can be arranged in a two-dimensional array4. In comparison with PAM of the same number of constellation points (that is, modulation order) and average power per bit, QAM is generally less demanding in terms of the optical signal-to-noise ratio (OSNR) of the transmitted data due to its larger Euclidean distance in the two-dimensional I/Q constellation4. This advantage tends to be more pronounced as the modulation order increases4. In addition, phase recovery can enable various digital signal processing (DSP) functions8 that might benefit future FSO systems6,9 (for example, compensation for hybrid fibre/FSO systems6 and adaptive probabilistic shaped modulations9).

Intensity modulation/direct detection (IM/DD) FSO links typically receive amplitude-encoded data by directly detecting the beam’s intensity levels, yet phase information is not readily recovered2,5,10. Alternatively, FSO systems can recover both amplitude and phase by using coherent detection, which mixes the data beam with a receiver Gaussian local oscillator (LO) beam5,9,11. However, atmospheric turbulence generally limits coherent detection because it induces power coupling of the data beam from the Gaussian mode to other Laguerre–Gaussian (LG) spatial modes12,13,14. Such turbulence-induced modal coupling can significantly degrade the data–LO mixing efficiency due to ‘mode mismatch’ between the LO and data beams12,13,15,16. Without turbulence, the photodetector (PD) efficiently mixes the data and the LO since they typically occupy the same single-Gaussian mode17, and hence are ‘mode matched’ in their spatial distributions18,19. With turbulence, however, significant power of the data beam can be coupled into higher-order LG modes and degrade the mixing efficiency by >20 dB (refs. 12,13,15) since data power coupled to orthogonal higher-order modes does not efficiently mix with the Gaussian LO15,20.

To enable amplitude and phase recovery in turbulent links, various modal-coupling mitigation approaches have been demonstrated21,22,23,24,25. One technique uses adaptive optics to couple the data power back into the Gaussian mode by measuring the distortion using a wavefront sensor and applying a DSP-calculated conjugate phase to the beam by a wavefront corrector21. Another technique uses multi-mode digital coherent combining22,23,24,25, wherein much of the data power in higher-order modes is captured by either a multi-mode fibre22,23,25 or an array of single-mode fibre (SMF) apertures24. Subsequently, the power from each of the multiple modes is recovered by a separate coherent detector and combined using DSP22,23,24,25. The performance depends on the number of recovered modes, and the complexity of the detection system tends to increase with the number of detected modes22,23,25. Since turbulence may induce coupling to a large number of modes, a laudable goal towards achieving simultaneous amplitude and phase recovery would be to automatically compensate for such power coupling without additional data processing and do so in a single element that efficiently scales to recover all captured modes.

In this article, we experimentally demonstrate the near-error-free transmission of a 12 Gbit s−1 16-QAM polarization-multiplexed (PolM) FSO link that is resilient to turbulence-induced LG modal power coupling for 200 random turbulence realizations. The amplitude and phase of the transmitted QAM data are retrieved using a pilot-assisted self-coherent detector. We transmit a Gaussian pilot beam with a frequency offset from the Gaussian data beam such that both beams experience similar turbulence-induced LG modal coupling. Subsequently, a single free-space-coupled PD mixes the received multi-mode data beam with the multi-mode pilot beam in ‘self-coherent’ detection26. During mixing, a conjugate of the turbulence-induced modal coupling of the pilot beam is automatically generated and used to compensate for the modal coupling in the data beam. Specifically, each data–pilot LG modal pair efficiently mixes and contributes to the intermediate frequency (IF) signal. Since the data and pilot experience similar modal coupling, our approach can simultaneously mix and recover nearly all of the captured data modes using a single PD. Experimental results for the turbulence strength (that is, ratio of the beam size to the Fried parameter) 2w0/r0 ≈ 5.5 show an average mixing loss of ~3.3 dB.

Results

Concept of pilot-assisted self-coherent detection using optoelectronic mixing

In an FSO link, a fundamental Gaussian beam (that is, LG0,0(x,y)) carrying a data channel (denoted as S(t,f) with the carrier frequency f) is transmitted through a turbulent atmosphere. Owing to a random spatial and temporal refractive index distribution, the turbulence effects can induce a transverse, spatially dependent wavefront distortion to the Gaussian beam27. Moreover, since such distortion induces modal power coupling, the electrical field of the data beam (Edata) at the receiver aperture can be expressed as a superposition of LG modes12,28:

$${E}_{{\rm{data}}}\left(t,f,x,y\right)=S\left(t,f\right)U(x,y)=S\left(t,f\right)\sum _{l}\sum _{p}{a}_{l,p}{{\rm{LG}}}_{l,p}\left(x,y\right),$$
(1)

where LGl,p(x,y) represents the electrical field of the LG mode17 with an azimuthal index l and a radial index p; \({a}_{l,p}=\int\int U\left(x,y\right){{\rm{LG}}}_{l,p}^{* }\left(x,y\right){\rm{d}}x{\rm{d}}y\) is the complex coefficient of the corresponding LGl,p component in the wavefront, * denotes the conjugate of the modal electrical field, and the portion of the optical power coupled to the LGl,p mode is |al,p|2; and \(U(x,y)=\sum _{l}\sum _{p}{a}_{l,p}{{\rm{LG}}}_{l,p}\left(x,y\right)\) represents the turbulence-induced LG modal coupling. Ideally, the complex weights al,p for all modal components tend to satisfy \(\sum _{l}\sum _{p}{\left|{a}_{l,p}\right|}^{2}\cong 1\) if the receiver aperture can collect almost the entire beam28.

A turbulent IM/DD FSO link (that is, S(t,f) is amplitude-only encoded) may suffer from turbulence-induced modal-coupling loss if an SMF-coupled PD is used because higher-order modes are not efficiently captured by the SMF13. For a free-space-coupled PD, however, an IM/DD FSO link may not be significantly affected by modal coupling if the receiver aperture can collect most of the distorted beam29. This free-space-coupled PD can utilize the detected optical intensity (that is, |S(t,f)|2) to recover the amplitude-encoded data, but the beam’s phase information is not readily recoverable.

As shown in Fig. 1a, coherent-detection FSO links can recover both the amplitude and phase of the data although they suffer from performance degradation caused by turbulence-induced modal coupling. Here, the transmitted data S(t,f) contain both amplitude- and phase-encoded data (for example, 16-QAM data). By way of a simple illustrative example, the continuous-wave LO at the receiver in a single-PD heterodyne coherent detector has an optical frequency offset Δf from the data carrier (denoted as C(f − Δf)) and is a Gaussian beam (that is, C(f − Δf)·LG0,0(x,y)). The square-law mixing in the PD of the coherent receiver results in a photocurrent26,30

$$\begin{array}{l}I\propto \int\int {\left|C\left(f-\Delta f\right){{\rm{LG}}}_{0,0}\left(x,y\right)+S\left(t,f\right)U\left(x,y\right)\right|}^{2}{\rm{d}}x{\rm{d}}y\\ ={\left|C\left(f-\Delta f\right)\right|}^{2}+{\left|S\left(t,f\right)\right|}^{2}+2{\rm{Re}}\left[S\left(t,f\right){C}^{* }\left(f-\Delta f\right)\right]\\ \int\int U\left(x,y\right){{\rm{LG}}}_{0,0}^{* }\left(x,y\right){\rm{d}}x{\rm{d}}y,\end{array}$$
(2)

where Re[·] is the real part of a complex element; I is the generated photocurrent; |C(f − Δf)|2 and |S(t,f)|2 are the direct current (d.c.) and the signal–signal beating interference (SSBI) photocurrent, respectively; and 2Re[S(t,f)C*(f − Δf)] generates the desired signal–LO beating (SLB) photocurrent. However, the Gaussian-mode LO does not mix efficiently with the multiple-LG-mode data beam due to the mode mismatch between their LG spectra, which is expressed as15

$$\begin{array}{l}\int\int U\left(x,y\right){{\rm{LG}}}_{0,0}^{* }\left(x,y\right){\rm{d}}x{\rm{d}}y\\ =\int\int \sum _{l}\sum _{p}{a}_{l,p}{{\rm{LG}}}_{l,p}\left(x,y\right){{\rm{LG}}}_{0,0}^{* }\left(x,y\right){\rm{d}}x{\rm{d}}y={a}_{0,0},\end{array}$$
(3)

where orthogonality amongst the LG modes ensures that \(\int\int {{\rm{LG}}}_{0,0}\left(x,y\right){{\rm{LG}}}_{0,0}^{* }\left(x,y\right){\rm{d}}x{\rm{d}}y=1\) and \(\int\int {{\rm{LG}}}_{l,p}\left(x,y\right){{\rm{LG}}}_{0,0}^{* }\left(x,y\right){\rm{d}}x{\rm{d}}y=0\), given that l ≠ 0 or p ≠ 0. Equation (3) shows that only the portion of the transmitted power that remains LG0,0 after turbulence can be efficiently mixed with the LO and utilized for recovering the QAM data. Such modal-coupling loss can result in severe degradation of the mixing IF power and thus the recovered data quality20. We note that this mixing-efficiency degradation in coherent detection can occur for a PD that is: (1) free-space-coupled due to orthogonality between the higher-order modes and the Gaussian LO15,20 and (2) SMF-coupled due to power in the higher-order modes not being efficiently coupled into the fibre13.

Fig. 1: Concept of simultaneous amplitude and phase recovery of QAM data in turbulent FSO links.
figure 1

a, The performance of coherent detection can be significantly degraded by turbulence-induced LG modal-coupling effects. A fundamental Gaussian beam (that is, LG0,0 mode) carrying 16-QAM data is transmitted through a turbulent atmosphere. Owing to the turbulence-induced LG modal power coupling, the received data beam will contain many LG modes. In an LO-based heterodyne coherent detector, only the LG0,0 mode can be efficiently mixed with the LO and recovered, resulting in degradation of the recovered data quality. This is true for both free-space-coupled and SMF-coupled PDs since the LO is typically single-Gaussian mode. Rx, receiver. b, Pilot-assisted self-coherent detection can automatically compensate for the turbulence-induced LG modal-coupling effects. In the pilot-assisted self-coherent detector, we transmit an additional continuous-wave pilot, which experiences similar turbulence-induced LG coupling as the data beam. During optoelectronic mixing of the pilot and data beams in a square-law free-space detector, a conjugate of the turbulence experienced by the pilot is automatically generated and compensates the turbulence experienced by the data beam. Therefore, almost all the data LG modes can be efficiently mixed with the pilot to enable simultaneous amplitude and phase recovery of QAM data. In a and b, the frequency offset Δf is greater than the data bandwidth B to avoid the SSBI. SSBI, signal–signal beating interference; SLB, signal–LO beating; SPB: signal–pilot beating.

Figure 1b illustrates the simultaneous recovery of the amplitude and phase of QAM data by utilizing pilot-assisted self-coherent detection, which automatically compensates for the turbulence-induced modal coupling. In addition to the Gaussian data beam, we transmit a co-axial Gaussian beam carrying a continuous-wave pilot tone with a frequency offset Δf, producing a frequency gap between the pilot and data beams of roughly the channel bandwidth (B) to avoid SSBI. The electrical fields of the data and pilot beams are likely to experience similar turbulence-induced distortion and modal coupling due to their frequency difference being orders of magnitude smaller than their carrier frequencies27. This similar distortion produces automatic ‘mode matching’ between the beams, such that the electric field of the pilot tone is31:

$$\begin{array}{l}{E}_{{\rm{pilot}}}\left(f-\Delta f,x,y\right)=C\left(f-\Delta f\right)U\left(x,y\right)\\ =C\left(f-\Delta f\right)\sum _{l}\sum _{p}{a}_{l,p}{{\rm{LG}}}_{l,p}\left(x,y\right).\end{array}$$
(4)

Importantly, a turbulence-induced LG-coupling conjugate U* is automatically generated from the pilot to compensate for the modal coupling experienced by the distorted data beam, and the total generated photocurrent is:

$$\begin{array}{l}I\propto \int \int {\left|C\left(f-\Delta f\right)U\left(x,y\right)+S\left(t,f\right)U\left(x,y\right)\right|}^{2}{\rm{d}}x{\rm{d}}y\\ ={\left|C\left(f-\Delta f\right)\right|}^{2}+{\left|S\left(t,f\right)\right|}^{2}+2{\rm{Re}}\left[S(t,f){C}^{* }\left(f-\Delta f\right)\right]\\ \int \int U\left(x,y\right){U}^{* }\left(x,y\right){\rm{d}}x{\rm{d}}y,\end{array}$$
(5)

where S(t,f)C*(f − Δf) generates the desired signal–pilot beating (SPB) photocurrent at an IF of Δf. The modal coupling is (ideally) corrected in an automatic fashion and the mixing efficiency is:

$$\begin{array}{l}{\rm{Mixing}}\,{\rm{efficiency}}\propto \int\int U\left(x,y\right){U}^{* }\left(x,y\right){\rm{d}}x{\rm{d}}y\\ =\int\int {\sum }_{l}{\sum }_{p}{a}_{l,p}{{\rm{LG}}}_{l,p}\left(x,y\right){\sum }_{l{\prime} }{\sum }_{p{\prime} }{a}_{l{\prime} ,p{\prime} }^{* }{{\rm{LG}}}_{l{\prime} ,p{\prime} }^{* }\left(x,y\right){\rm{d}}x{\rm{d}}y\\ ={\sum }_{l}{\sum }_{p}{\sum }_{l{\prime} }{\sum }_{p{\prime} }\int\int {a}_{l,p}{{\rm{LG}}}_{l,p}\left(x,y\right){a}_{l{\prime} ,p{\prime} }^{* }{{\rm{LG}}}_{l{\prime} ,p{\prime} }^{* }\left(x,y\right){\rm{d}}x{\rm{d}}y\\ ={\sum }_{l}{\sum }_{p}{\left|{a}_{l,p}\right|}^{2}\cong 1,\end{array}$$
(6)

where each LGl,p component of the data beam is efficiently mixed with the corresponding LGl,p component of the pilot beam. Consequently, almost all the captured optical power carried by higher-order LG spatial modes can contribute to the IF signal and can be automatically recovered using a single square-law free-space PD. The recovered QAM data can thus exhibit resilience against modal-coupling loss due to the efficient mixing between the data and pilot beams.

We note that the pilot-assisted self-coherent approach shares some similarities with both IM/DD and coherent detection: (1) similar to IM/DD, our approach does not use a receiver-based LO; and (2) similar to coherent detection, our approach recovers the amplitude and phase by mixing an ‘LO-like’ transmitter-generated pilot with the data beam and is often called ‘self-coherent detection’32,33. Notably, the pilot in our self-coherent system would experience similar FSO channel loss as the data beam, which may be noteworthy in longer-distance FSO links, whereas the LO in coherent detection would not6.

Generally, the OSNR needed to achieve a desired bit error rate (BER) depends on both the modulation formats and the detection approaches4,32,34. When comparing our self-coherent detection with heterodyne coherent detection for amplitude- and phase-encoded data, the transmitted power of self-coherent detection is shared between the pilot and data beams, resulting in self-coherent detection being more OSNR-demanding compared with coherent detection (without turbulence effects)32. For example, to achieve a given BER for the same QAM order, our self-coherent approach is likely to require an OSNR of around 3 dB higher when the carrier (that is, pilot)-to-signal power ratio (CSPR) is ~1 compared with heterodyne coherent detection32,33. When comparing our amplitude-and-phase self-coherent approach with amplitude-only IM/DD, the OSNR advantage of self-coherent QAM over IM/DD PAM (with the same modulation order) becomes more pronounced as the modulation order increases (for example, conventionally regarded to be many decibels for ≥16-QAM)4,32,33,34.

In longer-distance FSO links, the required optical power per bit for a desired BER can be a limiting factor10,16. Since the transmitted power is shared between the pilot and data beams, self-coherent detection will probably have a lower signal-to-noise ratio (SNR) compared with free-space-coupled IM/DD with the same received optical power and receiver thermal noise. Moreover, the SNR advantage of QAM over PAM diminishes as the modulation order decreases4. Consequently, IM/DD may have a better BER performance than pilot-assisted self-coherent detection for low modulation orders, such as 2-PAM16,34. We also note that IM/DD may have a better performance than self-coherent detection under lower SNR conditions even at higher modulation formats4,32,34.

Moreover, since atmospheric turbulence tends not to induce significant depolarization effects35, our pilot-assisted system should be compatible with PolM techniques by transmitting pilot–data pairs on each orthogonal polarization. Experimental results for a PolM system are shown later in Figs. 4 and 5.

Our approach transmits a pilot along with the data, and the pilot serves to help probe the turbulence and create a conjugate of the distortion from modal coupling. In optical communications, we note that pilot-assisted techniques have been demonstrated to probe a channel’s signature and apply a conjugate of that signature to help mitigate various channel impairments, including cross-phase modulation36 and laser phase noise37. More specifically, it has been shown via simulation that turbulence-induced modal crosstalk can be reduced by mixing a pilot beam and data-carrying LG beams in a mode-division-multiplexed FSO link38. In that approach, the pilot acquires the turbulence signature, is split into multiple copies at the receiver, and generates a conjugate of the turbulence for each of the LG data beams in separate PDs.

Experimental setup of free-space optical communications with emulated turbulence

We experimentally demonstrate pilot-assisted self-coherent detection in a 12 Gbit s−1 PolM 16-QAM 1-m-long FSO link with emulated turbulence. Figure 2 shows the experimental setup (see the Methods section for more details). The strengths (that is, the ratio of the beam size 2w0 to the Fried parameter r0) of the weaker and stronger turbulence effects are 2w0/r0 ≈ 2.2 and 5.5, respectively.

Fig. 2: Experimental setup for 12 Gbit s−1 16-QAM PolM FSO link.
figure 2

At the transmitter, a pilot–data-channel pair is transmitted on each of the orthogonal polarizations. The PolM Gaussian beams then propagate through a rotatable turbulence emulator. At the receiver, an FM is used to send the distorted beams for LG spectrum measurement via off-axis holography. Equal copies of the received beams are detected by the pilot-assisted self-coherent detector and single-PD LO-based heterodyne coherent detector. During the detection of the heterodyne coherent receiver, the pilot is turned off. The same DSP algorithms are applied to both receivers to retrieve the 16-QAM data. AWG, arbitrary waveform generator; Mod., modulator; EDFA, erbium-doped fibre amplifier; PC, polarization controller; PBC, polarization beam combiner; pol., polarization; Col., collimator; MR, mirror; HWP, half-wave plate; FM, flip mirror; BS, beam splitter; FS PD, free-space-coupled photodetector; SMF, single-mode fibre.

We emulate atmospheric turbulence effects using a single rotatable phase plate. Generally, turbulence effects can be more accurately emulated using multiple phase plates27. To address our emulation accuracy, we simulate the optical and electrical mixing power loss using single and multiple random phase screen (RPS) models; the simulation results show similar loss distributions and trends for both 1-RPS and 5-RPS models (see Supplementary Figs. 1 and 2 for more details).

Characterization of optical and electrical mixing power loss

We measure the turbulence-induced optical power loss and electrical mixing power loss of the pilot-assisted self-coherent detector for each polarization at 1,000 random realizations of the emulated turbulence. For both X and Y polarizations, Fig. 3a shows that stronger turbulence induces <2 dB of optical power loss for self-coherent detection since the free-space-coupled PD can capture most of the power; we note that free-space-coupled IM/DD systems are likely to have similar captured power loss. As shown in Fig. 3b, the self-coherent detector has an electrical mixing power loss of <3 dB and <6 dB for 99% weaker and 90% stronger turbulence realizations among 1,000 random turbulence realizations, respectively. The relatively low mixing power loss for self-coherent detection is due to efficient mixing of the pilot and data beams, which is likely to recover almost all the data power from the captured modes.

Fig. 3: Characterization of optical and mixing power loss for the pilot-assisted self-coherent detector under different turbulence strengths.
figure 3

To indicate the effects of turbulence-induced modal coupling on a coherent-detection FSO system with the single-Gaussian-mode LO, we also show the optical and mixing power loss of an SMF-coupled LO-based heterodyne coherent detector. a, Experimentally measured histograms of optical power loss under two different turbulence distortions (2w0/r0 ≈ 2.2 and 5.5) for X (left) and Y (right) polarizations. Note that free-space-coupled IM/DD systems are likely to have similar captured power loss as the pilot-assisted self-coherent detector. b, Experimentally measured histograms of mixing power loss (in the electrical domain) under two different turbulence distortions (2w0/r0 ≈ 2.2 and 5.5) for X (left) and Y (right) polarizations. The mixing power loss is measured at the IF of ~2.6 GHz in the electrical domain. In a and b, 1,000 different turbulence realizations are measured for each polarization. c, Simulated average optical power loss (top) and average electrical mixing power loss (bottom) results for different turbulence strengths from 1 to 7. The average values of experimentally measured data points (including both X and Y polarizations) are also plotted.

As discussed, turbulence-induced modal coupling can result in significant power loss for ‘mode-selective’ SMF-coupled IM/DD or coherent detectors. Figure 3a shows that the optical power loss for SMF-coupled systems ranges from ~2 to ~22 dB and from ~7 to ~30 dB under ~2.2 and ~5.5 turbulence strengths, respectively. Among the 1,000 emulated turbulence realizations, Fig. 3b shows that the coherent detector can suffer from a mixing power loss of ~28 dB for 99% and 90% of weaker and stronger turbulence, respectively. This mixing loss is due to the SMF-coupled detector not efficiently capturing the power coupled to higher-order modes13.

To help further validate our experimental results, we simulate the self-coherent system using 1-RPS (see Supplementary Equations (1)–(6) for simulation details). As shown in Fig. 3c, the simulation results indicate that self-coherent detection suffers <4 dB of average optical and electrical mixing power loss as the turbulence strength 2w0/r0 is increased from ~1 to ~7. Moreover, the plotted experimental results are generally in agreement with the simulation.

Turbulence-resilient 12 Gbit s−1 16-QAM PolM free-space optical transmission

We demonstrate 12 Gbit s−1 PolM FSO transmission under emulated turbulence effects, with each polarization carrying 1.5 Gbaud 16-QAM data. The transmitted total optical power per polarization (including pilot and data beams) is ~7 dBm. The transmitted CSPR values are ~1.1 and ~1 for X and Y polarizations, respectively. Figure 4 shows the recovered 16-QAM constellations using the self-coherent detector under example realizations of the weaker and stronger turbulence. We measure the turbulence-induced LG spectra for l and p indices of −5 to +5 and 0 to 10, respectively. The complex wavefront is measured using off-axis holography (see Methods)39.

Fig. 4: Experimental results of turbulence-induced LG modal power coupling and recovered 16-QAM data qualities using the pilot-assisted self-coherent detector.
figure 4

a, No turbulence distortion. b, One example realization (R1) of the weaker turbulence distortion (2w0/r0 ≈ 2.2). c,d, Two different example realizations (R1 (c) and R2 (d)) of the stronger turbulence distortion (2w0/r0 ≈ 5.5). For each of the four realizations, we measure the LG modal power spectrum (two indices −5 ≤ l ≤ +5 and 0 ≤ p 10) and recover the 16-QAM data constellations. In this demonstration of PolM FSO data transmission, each polarization carries a 6 Gbit s−1 16-QAM signal. To indicate the effects of turbulence-induced modal coupling on a coherent-detection FSO system with the single-Gaussian-mode LO, the performance using an LO-based heterodyne coherent detector is also shown. pol., polarization; EVM, error vector magnitude.

With no turbulence effects, Fig. 4a shows that the pilot-assisted self-coherent detector can achieve a near-error-free performance and recover an error vector magnitude (EVM) of ~8% for the 16-QAM data. Under one random realization of weaker turbulence, the measured LG spectrum of Fig. 4b shows that the data power is mainly coupled to the neighbouring LG modes. Under two different random realizations of stronger turbulence, Fig. 4c,d show that turbulence effects can induce a power loss of >25 dB and that power can be coupled to a large number of LG modes. The performance of the self-coherent detector is not severely affected by these turbulence effects and the 16-QAM data can be recovered with EVM values from ~8% to ~10% for both realizations. This turbulence resiliency is due to the automatic modal-coupling compensation by the pilot–data mixing, enabling almost all captured LG modes to be efficiently recovered.

To elucidate the effects of turbulence-induced modal coupling on coherent detection, we also show the recovered 16-QAM data for an SMF-coupled heterodyne coherent detector in Fig. 4; the recovered data quality degrades for both polarizations, from EVM values of ~7.5% without turbulence (Fig. 4a) to >16% for stronger turbulence (Fig. 4c,d). This degradation is due to data power coupled to higher-order modes that is not efficiently captured by the SMF13.

We also measure the electrical spectra for the self-coherent and coherent detectors under these example turbulence realizations. Compared with the case of no turbulence, there is a ~3 dB and ~18 dB SNR degradation of the IF signal measured for the self-coherent and coherent detectors, respectively, under the turbulence realizations of Fig. 4 (see Supplementary Fig. 3 for more details).

Figure 5 shows measured BER values for the pilot-assisted self-coherent detector under 200 random realizations of weaker and stronger turbulence. Results show that the self-coherent detector can achieve BER values below the 7% forward error correction limit for all realizations. Since turbulence can cause strong modal-coupling-induced power loss, the performance of the coherent detector can degrade and does not achieve the 7% forward error correction limit for some realizations.

Fig. 5: Experimentally measured BER performance of the pilot-assisted self-coherent detector over 200 different emulated turbulence random realizations.
figure 5

a, Weaker turbulence effects, X polarization. b, Weaker turbulence effects, Y polarization. c, Stronger turbulence effects, X polarization. d, Stronger turbulence effects, Y polarization. To indicate the effects of turbulence-induced modal coupling on a coherent-detection FSO system with the single-Gaussian-mode LO, the performance using an SMF-coupled LO-based heterodyne coherent detector is also shown. In the PolM FSO data transmission, each polarization carries a 6 Gbit s−1 16-QAM data signal. Note that we measure the BER performance for one polarization at a time due to limitations of our measurement setup. Therefore, the BER values for X and Y polarizations with the same realization label may correspond to different turbulence realizations and are difficult to be compared directly.

We further characterize the performance of the self-coherent detector by measuring the BER as a function of the transmitted power. We find power penalties of ~3 dB for both polarizations under one realization of the stronger turbulence (see Supplementary Fig. 4).

Enhancing spectral efficiency using Kramers–Kronig detection

In our self-coherent approach, a frequency gap between the pilot and data beams is needed to avoid SSBI. This gap is roughly equal to the data bandwidth, such that our spectrum is around 2× the data bandwidth. However, this frequency gap can be reduced to increase the spectral efficiency using SSBI mitigation techniques40,41 such as Kramers–Kronig (KK) detection6,41. Therefore, we demonstrate a reduction of the data–pilot gap to ~0.1 GHz (IF ≈ 0.9 GHz) using KK detection (see Supplementary Fig. 5 for more details); the recovered 16-QAM data exhibit EVM values of <12% for both polarizations under example realizations of weaker and stronger turbulence. Using KK detection, the spectral efficiency of the pilot-assisted approach could be increased by roughly 2×. Importantly, the KK scheme typically utilizes a stronger pilot than the non-KK approach. Hence, it is typically less power efficient than the non-KK pilot-assisted approach41, resulting in a trade-off between power efficiency and spectral efficiency.

Discussion

The following issues are interesting to consider:

  1. (i)

    Our 1.5 GHz baud rate is limited by the ~3.5 GHz bandwidth of the PD. However, free-space-coupled PDs with a bandwidth of ~49 GHz have been reported42, making >100 Gbit s−1 possible.

  2. (ii)

    We use LG modes to analyse modal coupling. However, we could utilize other bases (for example, Hermite–Gaussian22). Importantly, we do not need to specify a priori the basis used because our approach is ‘automatic’ and the pilot and data can be described in different bases.

  3. (iii)

    We note that differential-phase-shift-keyed (DPSK) systems are also referred to as ‘self-coherent’43,44. In DPSK systems: (1) data are typically encoded in the optical phase difference between neighbouring symbols; (2) the received data beam is split into two copies of which one is delayed; (3) these copies are coherently combined using a Mach–Zehnder interferometer; and (4) both Mach–Zehnder interferometer output branches are detected by two PDs simultaneously to recover the differential-encoded data43. Different from our pilot-assisted approach, almost all the captured optical power in DPSK systems contains data44. However, to recover the amplitude and phase of QAM data, differential systems typically utilize a more complex receiver than that of the pilot-assisted approach43,45. Interestingly, it might be possible to use multi-mode mixing as described in this paper to achieve automatic turbulence resiliency in a differential, high-order QAM system.

  4. (iv)

    A beam diverges with the link distance. Consequently, both the data and pilot beams can suffer from truncation by a limited-size receiver aperture causing power loss for longer-distance links46. Moreover, truncation can cause power coupling to higher-order modes46. These higher-order modes tend to be automatically mixed by the pilot-assisted self-coherent detection since the pilot and data beams experience similar truncation effects.

  5. (v)

    We use a free-space-coupled PD. Can our approach use fibre-coupled PDs? One possibility might be to use a multi-mode fibre-coupled PD10 such that many modes are captured and then impinge on the PD.

  6. (vi)

    Although FSO propagation is dependent on a beam’s carrier frequency, it is likely that beam divergence and turbulence-induced spatial distortions are similar for the pilot and data beams. This is because their typical frequency difference (<1 nm) is substantially smaller than their carrier frequencies (~1.55 μm)27,47.

This paper has described the concept and experimental/simulation results of pilot-assisted self-coherent links to automatically mitigate modal coupling for recovering the amplitude and phase of data. However, there are important questions for further study as to limits and dependencies of our approach, including: (1) the frequency dependence of spatial distortions and (2) its effectiveness as a function of distance, divergence, turbulence strength, signal bandwidth and signal-carrier frequency separation.

Methods

Experimental details of free-space optical communications in emulated turbulence

As shown in Fig. 2, we transmit a pair of data-carrying and pilot Gaussian beams on both X and Y polarizations. A 6 Gbit s−1 16-QAM data channel at a wavelength of λ1 ≈ 1.55 μm is generated, amplified using an erbium-doped fibre amplifier (EDFA) and equally split into two copies. One copy is delayed using a >15 m SMF to decorrelate the data channels and two independent data channels are individually combined with another pilot tone at a wavelength of λ2 (with a frequency offset of ~2.6 GHz from λ1, Δλ ≈ 0.02 nm). The polarizations of the signals and pilots are adjusted and subsequently combined using a polarization beam combiner to transmit PolM 16-QAM signals. The total optical power including the pilot and data beams is ~7 dBm for each of the polarizations. The optical signal is coupled to free space using an optical collimator (Gaussian beam size of diameter 2w0 ≈ 2.2 mm), is distorted using a rotatable turbulence emulator (see the section ‘Experimental emulation of atmospheric turbulence effects’) and then propagates in free space for ~1 m. In this demonstration, we emulate different strengths of atmospheric turbulence using two separate turbulence emulators with different Fried parameters r0 of 1.0 mm and 0.4 mm. The emulated turbulence distortion for the transmitted Gaussian beam is characterized by the ratio of the beam size to the Fried parameter27, and these are 2w0/r0 ≈ 2.2 and 5.5 for the two emulators.

At the receiver, we demultiplex one polarization at a time using a half-wave plate cascaded with a polarizer. The receiver has an aperture diameter of ~10 mm. We measure the spatial amplitude and phase profiles of the turbulence-distorted beam and calculate its LG decomposition using off-axis holography39 (see the section ‘Off-axis holography for complex wavefront measurement’). After polarization demultiplexing, the distorted beam is equally split into two copies that are sent to the pilot-assisted self-coherent detector and a single-PD LO-based heterodyne coherent detector.

In the pilot-assisted self-coherent detector, the entire spatial profiles of the distorted data and pilot beams are focused into a free-space-coupled InGaAs PD (3 dB bandwidth <3.5 GHz) using an aspheric lens with a focal length of 16 mm and a numerical aperture of ~0.79. The coupling efficiency of the received Gaussian beam, defined as the ratio of the optical power detected by the PD over the total received optical power by the receiver aperture (without turbulence effects), is measured to be >92%. The generated photocurrent is recorded using a real-time digital oscilloscope and the I–Q information of the data channel is subsequently retrieved using off-line DSP algorithms (see the section ‘Digital signal processing for retrieving the I–Q information at the receiver’). The Nyquist-shaped 16-QAM data channel has a symbol rate of 1.5 GHz with a roll-off factor of 0.1, expanding the data’s spectrum to ~1.7 GHz. To avoid SSBI effects, we set the IF (that is, the difference between the pilot and data beams’ carrier frequencies) at Δf ≈ 2.6 GHz, which includes a frequency gap of ~1.8 GHz between the pilot and data beams. Thus, the total transmitted pilot-assisted signal spectrum is ~3.5 GHz, which is roughly twice that of the data spectrum (see Supplementary Fig. 3 for more details).

At the single-PD LO-based heterodyne coherent detector (the pilot λ2 is turned off), we set the same IF value as the pilot-assisted self-coherent receiver. The distorted Gaussian beam is coupled into an SMF via a collimator (aperture diameter ≈ 3.5 mm), amplified using an EDFA, and mixed with an LO (at the same wavelength λ2 as the pilot) at the SMF-coupled PD. The received optical signal is amplified by the EDFA to meet the power sensitivity requirement of the SMF-coupled PD. The electrical signal is subsequently recorded using a real-time digital oscilloscope and processed to retrieve the data channel’s I–Q information using the same off-line DSP algorithms as the pilot-assisted self-coherent detector. Note that we measure the optical power loss and electrical mixing power loss of this detector (shown in Fig. 3) without using the EDFA inside this receiver. The mixing power loss is measured at the IF of ~2.6 GHz in the electrical domain.

To evaluate the effectiveness of the pilot-assisted self-coherent detector for various turbulence scenarios, we measure the BER values of the 16-QAM data channels carried by both polarizations over 200 random turbulence realizations. To measure turbulence-induced modal-power-coupling effects on an SMF-coupled coherent detector, we also measure the BER performance for the LO-based heterodyne coherent detector over 200 random turbulence realizations. Note that we measure the BER performance for one polarization at a time due to the limitations of our measurement setup. Therefore, the BER values for X and Y polarizations with the same realization label may correspond to different turbulence realizations and are difficult to be compared directly.

Experimental emulation of atmospheric turbulence effects

We experimentally emulate the turbulence-induced distortion by utilizing glass plates (Lexitek), the refractive index distributions of which are fabricated to emulate Kolmogorov power spectrum statistics20,27. Two rotatable glass plates are used separately in the experiment with different Fried parameters (r0) of 1.0 mm (weaker turbulence effects) and 0.4 mm (stronger turbulence effects). Different ‘random’ turbulence realizations are implemented by rotating the single glass plate to different orientations. The diameter of the transmitted Gaussian beam is 2w0 ≈ 2.2 mm. The data-carrying Gaussian beams are distorted by the glass plate and then propagate in free space for a distance of ~1 m before reaching the receiver. The strength of the turbulence distortion is given by the ratio of the beam diameter to the Fried parameter27, that is, 2w0/r0. For a proof-of-concept demonstration, we investigate the performance of the pilot-assisted self-coherent detector at two different turbulence strengths (2w0/r0 ≈ 2.2 and 5.5). Under even stronger turbulence effects, the self-coherent FSO systems may suffer from strong beam-wandering effects and subsequent optical power loss48. A beam pointing and tracking system can be used to compensate for these beam-wandering effects49.

In this demonstration, we use a single phase plate to emulate the turbulence distortions for this ~1 m FSO link. However, a multiple-phase-plate emulation can generally provide a higher accuracy for emulating the volume atmospheric turbulence effects27. To illustrate the validity of our emulation method, we simulate 1-RPS and 5-RPS turbulence effects; similar trends for turbulence-induced system degradations were found (see Supplementary Figs. 1 and 2 for more details). We note that our turbulence emulation provides an approximation of the Gaussian beam’s propagation in a turbulent medium and may not fully reflect the effects of real atmospheric turbulence. To further enhance the accuracy of turbulence emulation, some advanced modelling or emulation methods could potentially be applied27,50.

Off-axis holography for complex wavefront measurement

We use off-axis holography to measure the complex wavefront (that is, the amplitude and phase) of the distorted Gaussian beam and its corresponding LG spectrum. An off-axis reference Gaussian beam (beam diameter ~7 mm) on the same wavelength as the distorted pilot Gaussian beam is incident on the infrared camera with a tilted angle. We record the off-axis interferogram and apply digital image processing to extract the complex wavefront (see Supplementary Fig. 6 for more details). The data-carrying beam is turned off when we measure the complex wavefront of the turbulence-distorted pilot beam.

After the complex wavefront of the distorted Gaussian beam is obtained, we decompose it into a two-dimensional LG modal spectrum in which the two indices l and p range from −5 to +5 and from 0 to 10, respectively, as expressed in equation (7)28:

$${a}_{l,p}=\int\int {E}_{{\rm{rec}}}\left(x,y\right){{\rm{LG}}}_{l,p}^{* }\left(x,y\right){\rm{d}}x{\rm{d}}y,$$
(7)

where Erec(x,y) and LGl,p(x,y) are the measured complex field of the distorted Gaussian beam and the theoretical complex field of an LGl,p mode, respectively. The ratio of optical power coupling to the LGl,p mode is given by |al,p|2.

Digital signal processing for retrieving the I–Q information at the receiver

The detected electrical signal is sampled using a real-time oscilloscope (20 GHz bandwidth and 50 gigasamples per second sampling rate) and recorded for off-line DSP. The recorded signals from the pilot-assisted self-coherent detector and the single-PD LO heterodyne coherent detector are processed using the same DSP procedures. Each signal is filtered using a root-raised-cosine finite impulse response filter with a roll-off factor of 0.1, and the filtered signal is subsequently equalized using a constant modulus algorithm. After equalization with the constant modulus algorithm, carrier frequency offset estimation and carrier phase recovery are sequentially performed to reduce the frequency and phase difference between the signal and the LO (or pilot). Finally, the EVM and BER values of the demodulated signal are calculated to evaluate the quality of the data transmission. The EVM of the detected signal is calculated using equation (8) as follows7:

$${\rm{EVM}}=\sqrt{\frac{1}{N{\max }_{i}{\left|\widehat{{x}_{i}}\right|}^{2}}{\sum }_{i=1}^{N}{\left|{x}_{i}-\widehat{{x}_{i}}\right|}^{2} }\times 100 \%,$$
(8)

where the xi and \(\widehat{{x}_{i}}\) represent the transmitted and recovered data symbols, respectively, and N is the total number of detected symbols. In this demonstration, ~180,000 symbols are collected to calculate the EVM and BER values of the 16-QAM data signals.