Introduction

Quantum key distribution (QKD)1,2,3 promises an information-theoretically secure symmetric key distribution for distant partners. The past three decades have witnessed rapid development of QKD technologies and the growth of QKD network deployment globally, which have been employed in various security applications4,5,6,7,8,9,10,11. Within the QKD family, continuous-variable (CV) QKD benefits from the use of off-the-shelf commercial telecom components12,13 and provides a cost-effective alternative in metropolitan networks. Twenty years since the pioneering GG02 protocol14 was proposed, the theory15,16,17,18,19,20 and experimentation21,22,23 of CVQKD have made remarkable progress.

Moreover, the tremendous breakthroughs of local oscillator (LO) schemes since 201524,25 have pushed CVQKD into a new stage, in which techniques from modern digital coherent communication have been brought in26,27,28,29. We call this stage digital CVQKD. Specifically, digital signal processing (DSP) significantly improves the signal-to-noise ratio (SNR) by compensating for channel drifting and device impairments, which greatly simplifies physical systems. This paves the way for an ultra-high secret key rate with tens of GHz of bandwidth in the CVQKD system. However, this phenomenon also complicates the security analysis.

Two barriers exist in the security analysis of a digital CVQKD system. One is the discrete modulation format, and the other is DSP. The former results from the destruction of estimating the covariance matrix directly from the measurement results and was recently solved with the semidefinite programming30,31,32 or other novel methods33,34. The other barrier is the difficulty of constructing an appropriate measurement operator to describe the output of DSP.

Specifically, a single-point quadrature measurement of each state in one ensemble is sufficient for reliable tomography of single-mode states. However, for the tomography of continuous-mode states, the extraction of quadrature information involves multi-point sampling and processing. Therefore, a time-domain description of system’s behavior should be introduced, which is beyond the traditional single-mode description.

Here, to narrow the gap between practical systems and theoretical models, we develop a generalized security proof framework for continuous-mode systems processed by linear DSP algorithms. The key step is the temporal-mode (TM) construction using DSP results, which is suitable for the analysis of high-speed, multi-point sampling systems. Specifically, in continuous-mode formalism, time-domain field operators can be introduced by Fourier transformation, based on which the generalized receiver can be well modeled. By properly calibrating the shot-noise unit (SNU), we model the linear processing of sampled data by recombination of time-domain field operators, which defines a specific TM field operator35,36,37. Consequently, the security of DSP is reduced to the security of a specific single TM measurement. Then, the rest of the analysis is compatible with traditional methods.

Moreover, the results show that the mismatch between the measured state’s TM and the receiver’s TM leads to inefficiency in detection. The mission of the DSP algorithm is to merge this mismatch, thus improving the detection efficiency, which coincides with improving SNR in its classical correspondence.

Our work provides a feasible way of analyzing the security and performance of a continuous-mode system processed by linear DSP algorithms, so it could provide important guidance for the DSP design of digital CVQKD. Linear DSP toolboxes are expected to be directly employed in CVQKD, reinforcing the importance of our work.

Results

Temporal modes of continuous-mode states

We start by introducing the basics of continuous-mode formalism of quantum optics and then describing the state preparation phase. Recall that in traditional CVQKD analysis, the coherent state is represented by the creation and annihilation operators in terms of a single-mode field, given by \({\hat{a}}_{i}^{{\dagger} },{\hat a_i}\). By contrast, in a practical system, high-speed modulation inevitably introduces a nonuniform temporal waveform, so continuous-mode formalism of field operators35,37,38 should be introduced, which is widely used in studying continuous-mode quantum optics. By transforming the annihilation and creation operators from their discrete-mode counterparts, the continuous-mode field operators are defined as \({\hat a_i}\to {\left({{\Delta }}\omega \right)}^{\frac{1}{2}}\hat{a}\left(\omega \right)\) and \(\hat a_i^\dagger \to {\left({{\Delta }}\omega \right)}^{\frac{1}{2}}{\hat{a}}^{{\dagger} }\left(\omega \right)\), where Δω denotes the mode spacing, which satisfies the commutation relation \([\hat{a}{(\omega ),{\hat{a}}^{{\dagger} }}({\omega }^{{\prime} })]=\delta (\omega -{\omega }^{{\prime} })\). In the time domain, it is useful to define the Fourier transforms of \(\hat{a}\left(\omega \right)\), namely \(\hat{a}\left(t\right)\), given by \(\hat{a}\left(t\right)=\frac{1}{\sqrt{2\pi }}\int\,d\omega \hat{a}\left(\omega \right)\exp \left(-i\omega t\right)\). The creation operator \({\hat{a}}^{{\dagger} }\left(t\right)\) follows a similar definition.

Based on this, the photon-wavepacket creation operator \({\hat{A}}_{{\xi }_{i}}^{{\dagger} }\)35,39 can be defined as

$${\hat{A}}_{{\xi }_{i}}^{{\dagger} }=\int\,dt{\xi }_{i}\left(t\right){\hat a^\dagger }\left( t \right),$$
(1)

in which the wavepacket \({\xi }_{i}\left(t\right)\) usually reads \({\xi }_{i}^{0}\left(t\right){e}^{-i\omega t}\), as an envelope \({\xi }_{i}^{0}\left(t\right)\) with a carrier eiωt. It is also known as the TM field operator36,37 if \({\xi }_{i}\left(t\right)\) meets the orthonormalization that \(\int\,dt{\xi }_{i}^{* }\left(t\right){\xi }_{j}\left(t\right)={\delta }_{ij}\), for different i, j. The TM operators also obey the commutation relation, which reads

$$\left[{\hat{A}}_{{\xi }_{i}},{\hat{A}}_{{\xi }_{j}}^{{\dagger} }\right]={\delta }_{ij}.$$
(2)

It is then important to define the photon-wavepacket coherent state \({\left\vert {\gamma }_{i}\right\rangle }_{{\xi }_{i}}\) on ξi-TM, as35

$${\left\vert {\gamma }_{i}\right\rangle }_{{\xi }_{i}}={\hat{D}}_{{\xi }_{i}}\left({\gamma }_{i}\right)\left\vert 0\right\rangle =\exp \left({\gamma }_{i}{\hat{A}}_{{\xi }_{i}}^{{\dagger} }-{\gamma }_{i}^{* }{\hat{A}}_{{\xi }_{i}}\right)\left\vert 0\right\rangle ,$$
(3)

where γi denotes the displacement parameter, and \({\left\vert {\gamma }_{i}\right\vert }^{2}\) represents the average photon number. The photon-wavepacket coherent state obeys the eigenvalue equation \({\hat{A}}_{{\xi }_{i}}{\left\vert {\gamma }_{i}\right\rangle }_{{\xi }_{i}}={\gamma }_{i}{\left\vert {\gamma }_{i}\right\rangle }_{{\xi }_{i}}\). Under this notation, the quadrature operator with the phase angle θ can be defined as

$${\hat{X}}_{{\xi }_{i}}^{\theta }={\hat{A}}_{{\xi }_{i}}^{{\dagger} }\exp \left(i\theta \right)+{\hat{A}}_{{\xi }_{i}}\exp \left(-i\theta \right).$$
(4)

In a digital CVQKD system, coherent states are generated by widely used in-phase/quadrature (I/Q) modulation. As shown in Fig. 1(a), assuming that \({\left\vert {\gamma }_{0}\right\rangle }_{{\xi }_{L}}\) is the photon-wavepacket coherent state fed to the I/Q modulator, \({\left\vert {\gamma }_{m}\right\rangle }_{{\xi }_{L}}\) are then the state of the I or Q arm after the balanced beam splitter (BS), where \({\gamma }_{m}={\gamma }_{0}{\mathrm{/}\sqrt{2}}\). Then, each arm performs the intensity modulation with a certain waveform, which is modeled by a time-dependent BS with transmitivity \(\eta \left(t\right)\) related to modulation40,41. Assuming that the data encoded on the I and Q components in the i-th period are \(\left\{{x}_{i}\right\}\) and \(\left\{{p}_{i}\right\}\) and that their normalized waveform envelopes are \({\xi }_{{{{\rm{I}}}}}\left(t\right)\) and \({\xi }_{{{{\rm{Q}}}}}\left(t\right)\), we can rewrite \(\sqrt{{\eta }_{{{{\rm{I}}}}}\left(t\right)}{\xi }_{L}^{0}\left(t\right)={x}_{i}{\xi }_{{{{\rm{I}}}}}\left(t\right)\) on the I path and \(\sqrt{{\eta }_{{{{\rm{Q}}}}}\left(t\right)}{\xi }_{L}^{0}\left(t\right)={p}_{i}{\xi }_{{{{\rm{Q}}}}}\left(t\right)\) on the Q path. Then, the I and Q path’s output states are transformed into \({\left\vert {x}_{i}{\gamma }_{m}\right\rangle }_{{\xi }_{{{{\rm{I}}}}}}\) and \({\left\vert {p}_{i}{\gamma }_{m}\right\rangle }_{{\xi }_{{{{\rm{Q}}}}}}\). Finally, after passing through another balanced BS and a proper attenuator, the output photon-wavepacket coherent state is \({\left\vert {x}_{i}+i{p}_{i}\right\rangle }_{{\xi }_{{{{\rm{out}}}}}}\) if \({\xi }_{{{{\rm{I}}}}}\left(t\right)={\xi }_{{{{\rm{Q}}}}}\left(t\right)={\xi }_{{{{\rm{out}}}}}\left(t\right)\) with properly calibrated modulation. This means that in the entanglement-based scheme, the output coherent states can be seen as two-mode squeezed states, with ξout-TM being measured on one mode with a heterodyne measurement. As for \({\xi }_{{{{\rm{I}}}}}\left(t\right)\ne {\xi }_{{{{\rm{Q}}}}}\left(t\right)\), the output state can be decomposed into two orthogonal TMs with different but correlated displacement parameters, which we leave for further investigations.

Fig. 1: Models of digital CVQKD.
figure 1

a The transmitter prepares a photon-wavepacket coherent state (PWCS) with an arbitrary form of envelope \({\xi }_{L}\left(t\right)\); via the I/Q modulation, the quadratures of the PWCS are modulated by Gaussian distributed random numbers. The carrier can be either a continuous wave or pulsed coherent state. b Measured states are fed to a practical receiver, interfering with the local oscillator (LO) at a balanced beam splitter (BS) and detected by a band-limited homodyne detector (HD) (modeled by an ideal HD and a filter), followed by the sampling and DSP devices. The mismatch between the measured state’s temporal mode (TM) and the receiver’s TM is equivalent to a BS with a transmitivity of η. c In the experiment, channel transmission causes a transformation of TM, which, if ignored, will cause a degradation in system performance. PM phase modulator.

Measurement, sampling, and data processing

On the receiver side, the input state is first measured by a practical homodyne detector with limited bandwidth and then sampled by an analog-to-digital converter (ADC). After this, the sampled data go through a series of DSP algorithms, and the final data output from DSP are assumed to represent the quadrature measurement result, which can be used to construct the covariance matrix and then calculate the secret key rate. We only consider linear DSP algorithms here because the transmitted quantum light is extremely weak, so no obvious optical nonlinear effects occur. Thus, linear compensation algorithms are sufficient to recover the signal. When we use the above continuous-mode formalism notation, mapping the outputs of linear DSP to quadrature measurements is surprisingly natural; the crucial step is normalization with properly calibrated SNU. To paint a clear picture of this, we first ignore the imperfections of the homodyne detector and finite-resolution issue of ADC, and we discuss the trusted detection model considering the detector’s efficiency and noise in the Methods section.

The receiver can be modeled as an ideal homodyne detector, followed by a filter, as shown in Fig. 1b. Assuming the filter has an impulse response function (IRF), namely, \(g\left(t\right)\), the photocurrent flux operator of a homodyne detector is given by35,39

$$\hat{f}\left(t\right)=\left[{\hat{a}}^{{\dagger} }\left(t\right){\hat{a}}_{{{{\rm{LO}}}}}\left(t\right)+{\hat{a}}_{{{{\rm{LO}}}}}^{{\dagger} }\left(t\right)\hat{a}\left(t\right)\right]* g\left(t\right),$$
(5)

where * denotes the convolution. The photon wavepacket of the local oscillator (LO) is given by \({\alpha }_{{{{\rm{LO}}}}}\left(t\right)={\mu }_{{{{\rm{LO}}}}}^{1{{\mathrm{/}}}2}{\xi }_{{{{\rm{LO}}}}}\left(t\right)\exp \left(-i{\omega }_{{{{\rm{LO}}}}}t+i\theta \right)\), where μLO denotes the average number of photons contained in an envelope \({\xi }_{{{{\rm{LO}}}}}\left(t\right)\) for a pulsed LO, or a time period as defined. Because LO is considered a classical field with enough photons, the fluctuation in the measurement output mainly comes from the signal quadrature measurement part. Therefore, the photocurrent flux after taking the average over LO is more useful, given by \({\hat{f}}_{{{{\rm{LO}}}}}\left(t\right)=\left\langle {\alpha }_{{{{\rm{LO}}}}}\left(t\right)\right\vert \hat{f}\left(t\right)\left\vert {\alpha }_{{{{\rm{LO}}}}}\left(t\right)\right\rangle\).

Considering ADC as the integral sampling process with integral time Δts, the sampled data at time t0 are

$${\hat{D}}_{{t}_{0}}=\frac{1}{{{\Delta }}{t}_{{{{\rm{s}}}}}}\int\nolimits_{{t}_{0}}^{{t}_{0}+{{\Delta }}{t}_{{{{\rm{s}}}}}}dt{\hat{f}}_{{{{\rm{LO}}}}}\left(t\right),$$
(6)

in which the electronics amplification is ignored.

Multiple sampling points may exist within one period Tr. Generally, types of linear data processing exist: (i) directly choose one sampled data of each period as final data of this period, for instance, the sampling point near the peak of the envelope of measured state; (ii) using the sampled data within one period to define the final data of this period, for instance, calculating the weighted averaging of all sampled data within the same period; and (iii) generally, a DSP algorithm may use the sampled data from multiple periods, for instance, the root raised cosine (RRC) filter42 introduces a convolution over multiple periods.

For a DSP algorithm involving N sampled data, the output data at the time corresponding to the tj sampling time could be

$${\hat{D}}_{{t}_{j}}^{N}={f}_{{{{\rm{dsp}}}}}\left({\hat{D}}_{{t}_{j-k+1}},...{\hat{D}}_{{t}_{j-k+N}}\right)=\mathop{\sum }\limits_{i=1}^{N}{f}_{{{{\rm{dsp}}}}}^{i}\,{\hat{D}}_{{t}_{j-k+i}}$$
(7)

with linear expansion, where \({f}_{{{{\rm{dsp}}}}}^{i}\) and k are real numbers determined by DSP algorithms. After simplification, Eq. (7) is given by

$${\hat{D}}_{{t}_{j}}^{N}\,=\,\frac{{\mu }_{{{{\rm{LO}}}}}^{1{{\mathrm{/}}}2}}{{{\Delta }}{t}_{{{{\rm{s}}}}}}\int\,{G}_{{{{\rm{dsp}}}}}^{{t}_{j}}\left(\tau \right){\hat{X}}^{{\alpha }_{{{{\rm{LO}}}}}}\left(\tau \right)d\tau ,$$
(8)

where

$${G}_{{{{\rm{dsp}}}}}^{{t}_{j}}\left(\tau \right)=\mathop{\sum }\limits_{i=1}^{N}{f}_{{{{\rm{dsp}}}}}^{i}\int\nolimits_{{t}_{j-k+i}}^{{t}_{j-k+i}+{{\Delta }}{t}_{{{{\rm{s}}}}}}g\left(t-\tau \right)dt$$
(9)

is related to the detector’s IRF, sampling points, and the DSP algorithm. In addition, \({X}^{{\alpha }_{{{{\rm{LO}}}}}}\left(\tau \right)\) is the intermediate quadrature operator related to the LO’s features, which is given by

$$\hat{{X}^{{\alpha }_{{{{\rm{LO}}}}}}}\left(\tau \right)={\xi }_{{{{\rm{LO}}}}}\left(\tau \right){e}^{-i\left({\omega }_{{{{\rm{LO}}}}}\tau -\theta \right)}{\hat{a}}^{{\dagger} }\left(\tau \right)+{{{\rm{h.c.}}}}$$
(10)

SNU calibration and normalization

To normalize the output data from DSP, one key step is to define and calibrate the SNU, which is the most distinguishable phase from classical optical communication. Considering \({\hat{D}}_{{t}_{j}}^{N}\) as the final data for the period in which tj lies, we can easily verify that for the vacuum input, the mean is \(\left\langle 0\right\vert {\hat{D}}_{{t}_{j}}^{N}\left\vert 0\right\rangle =0\), and the variance \({\sigma }_{{{{\rm{SNU}}}}}^{2}=\left\langle 0\right\vert {\hat{D}}_{{t}_{j}}^{N}{\hat{D}}_{{t}_{j}}^{N}\left\vert 0\right\rangle\) is

$${\sigma }_{{{{\rm{SNU}}}}}^{2}=\frac{{\mu }_{{{{\rm{LO}}}}}}{{{\Delta }}{{t}_{{{{\rm{s}}}}}}^{2}}\int{\left\vert {\xi }_{{{{\rm{LO}}}}}\left(\tau \right)\right\vert }^{2}{\left[{G}_{{{{\rm{dsp}}}}}^{{t}_{j}}\left(\tau \right)\right]}^{2}d\tau .$$
(11)

For normalization, the sampled data \({\hat{D}}_{{t}_{j}}^{{{{\rm{SNU}}}}}\) are divided by \({\sigma }_{{{{\rm{SNU}}}}}=\sqrt{{\sigma }_{{{{\rm{SNU}}}}}^{2}}\), which gives

$${\hat{D}}_{{t}_{j}}^{{{{\rm{SNU}}}}}\,=\,{e}^{i\theta }\int\,d\tau \frac{{G}_{{{{\rm{dsp}}}}}^{{t}_{j}}\left(\tau \right){\xi }_{{{{\rm{LO}}}}}\left(\tau \right){e}^{-i{\omega }_{{{{\rm{LO}}}}}\tau }}{{\sigma }_{{{{\rm{cal}}}}}}{\hat{a}}^{{\dagger} }\left(\tau \right)\,+\,{{{\rm{h.c.}}}},$$
(12)

where \({\sigma }_{{{{\rm{cal}}}}}=\sqrt{\int\,d\tau {\vert {\xi }_{{{{\rm{LO}}}}}(\tau )\vert }^{2}{[{G}_{{{{\rm{dsp}}}}}^{{t}_{j}}(\tau )]}^{2}}\) is the rescaled factor when calibrating output data by SNU. It can be verified that the coefficient function of \(\hat{a}\left(\tau \right)\) is a normalized photon-wavepacket function, which is

$${{{\Xi }}}_{{{{\rm{DSP}}}}}^{{t}_{j}}\left(\tau \right)=\frac{1}{{\sigma }_{{{{\rm{cal}}}}}}{\xi }_{{{{\rm{LO}}}}}\left(\tau \right){G}_{{{{\rm{dsp}}}}}^{{t}_{j}}\left(\tau \right)\exp \left(-i{\omega }_{{{{\rm{LO}}}}}\tau \right),$$
(13)

with the normalization condition \(\int\,d\tau {\vert {{{\Xi }}}_{{{{\rm{DSP}}}}}^{{t}_{j}}(\tau )\vert }^{2}=1\). This introduces \({{{\Xi }}}_{{{{\rm{DSP}}}}}^{{t}_{j}}\)-TM, which is jointly defined by the LO, filter, sampling, and DSP algorithms. Then, we can further define its creation operator as

$${\hat{A}}_{{{{\Xi }}}_{{{{\rm{DSP}}}}}^{{t}_{j}}}^{{\dagger} }=\int\,d\tau {{{\Xi }}}_{{{{\rm{DSP}}}}}^{{t}_{j}}\left(\tau \right){\hat{a}}^{{\dagger} }\left(\tau \right).$$
(14)

Consequently, a simplified form of Eq. (12) in terms of the \({{{\Xi }}}_{{{{\rm{DSP}}}}}^{{t}_{j}}\)-TM operators is rewritten as

$${\hat{D}}_{{t}_{j}}^{{{{\rm{SNU}}}}}\,=\,{\hat{A}}_{{{{\Xi }}}_{{{{\rm{DSP}}}}}^{{t}_{j}}}^{{\dagger} }\exp \left(i\theta \right)\,+\,{\hat{A}}_{{{{\Xi }}}_{{{{\rm{DSP}}}}}^{{t}_{j}}}\exp \left(-i\theta \right)\,=\,{\hat{X}}_{{{{\Xi }}}_{{{{\rm{DSP}}}}}^{{t}_{j}}}^{\theta },$$
(15)

which shares the same form as Eq. (4).

Therefore, the final data (output from DSP and being normalized) can be treated as a quadrature measurement of \({{{\Xi }}}_{{{{\rm{DSP}}}}}^{{t}_{j}}\)-TM. As long as the data represent a quadrature measurement result, they can be used to construct the covariance matrix and are thus compatible with traditional security analysis methods. The abovementioned analysis also highlights one key point of SNU calibration, which is that the sampled data of vacuum input should be processed by the same DSP as the usual signal input case before the data are used to calculate the variance of shot noise.

Another important issue is that for a DSP involving sampled data exceeding one period, the possible crosstalk should be avoided. In this case, the TMs related to different periods should be orthogonal, that is, \(\int\,{{{\Xi }}}_{{{{\rm{DSP}}}}}^{{t}_{i}}{{{\Xi }}}_{{{{\rm{DSP}}}}}^{{t}_{j}* }=0\), where ti, tj belong to two different periods. This also coincides with classical DSP’s purpose, as crosstalk lowers the SNR. For instance, the RRC pulse shaping and filtering methods are commonly used to improve spectrum efficiency. Moreover, the RRC filter is designed with no intersymbol interference for different optimal sampling points, which is associated with the TMs related to different optimal sampling points being orthogonal.

This completes our security analysis framework for linear DSP, which can be summarized in two key points. One is properly calibrating SNU, which naturally leads the final data to be a quadrature measurement result with respect to the TM defined jointly defined by LO, detector, sampling, and DSP. The second is to avoid the complex measurement model introduced by intersymbol crosstalk, where TMs corresponding to different periods should be orthogonal. If these two conditions apply, the final data can be directly used to construct the covariance matrix, and then we can calculate the secret key rate through the current security analysis method.

Below, two examples are given to analyze the performance after considering the continuous-mode scenario, including the mode-matching issue and transmission-dispersion issue.

Projection on the measurement temporal mode

Besides security, our analysis also provides insights into imperfect detection efficiency, related to the mismatch between the measured state’s TM and the receiver’s TM. The whole measurement process under the TM representation is equivalent to the projection of the measured state’s TM to the receiver’s TM, and vice versa.

Now, consider a case where the measured state is a coherent state on ξA-TM, which is different from the receiver’s \({{{\Xi }}}_{{{{\rm{DSP}}}}}^{{t}_{j}}\)-TM. Using the Gram–Schmidt orthogonalization, we can define a third TM from \({{{\Xi }}}_{{{{\rm{DSP}}}}}^{{t}_{j}}\)-TM and orthogonal to ξA-TM, denoted as Ψ-TM, which leads to the following decomposition of the creation operator:

$${\hat{A}}_{{{{\Xi }}}_{{{{\rm{DSP}}}}}^{{t}_{j}}}^{{\dagger} }=\sqrt{\eta }{\hat{A}}_{{\xi }_{{{{\rm{A}}}}}}^{{\dagger} }+\sqrt{1-\eta }{\hat{A}}_{{{{\Psi }}}_{\perp }}^{{\dagger} },$$
(16)

where \(\eta ={[\int{dt}{{{\Xi }}}_{{{{\rm{DSP}}}}}^{{t}_{j},* }\left(t\right){\xi }_{{{{\rm{A}}}}}\left(t\right)]}^{2}\le 1\) denotes the mode-matching coefficient. With further examination of the first-order and second-order moments, the abovementioned decomposition can be modeled by an extra BS at the receiver side, with transmitivity η, quantifying the matching degree between ξA-TM and \({{{\Xi }}}_{{{{\rm{DSP}}}}}^{{t}_{j}}\)-TM. A detailed derivation can be found in the Methods section. Here, η < 1 means an extra loss induced by the mode mismatch, which decreases the performance of the system. This degradation is rather covert, different from the physical components introduced by loss, that is, fiber coupling loss and the non-unit quantum efficiency of photodiodes. The closer η is to 1, the better the performance of a DSP algorithm.

Here, we take the weighted averaging scenario to further illustrate the mode-matching issue in the time domain. In this case, the DSP function is given by \({f}_{{{{\rm{dsp}}}}}^{i}={w}_{i}\), where wi is the weight of the i-th sampling point within one period. The measurement results can be described by Fig. 2. A sampled data point measures not only the signal at one time point but also convoluted nearby signals around the sampling point. Therefore, an intuitive understanding of mode matching is that the sum of all sampled data covers a certain signal area. In Fig. 2a, the bandwidth of the detector is large enough that the IRF is approximate to the δ-function, and then ultra-dense sampling is needed. By contrast, if the bandwidth gradually decreases, the IRF becomes wider, and fewer sampling points are required to achieve a similar mode-matching degree, as shown in Fig. 2b, c. If the detector’s IRF is similar to the measured state’s envelope, a single-sampling point is enough, as in Fig. 2d.

Fig. 2: Weighted averaging of sampled data.
figure 2

Assume the LO is a continuous-wave (CW) laser. a The bandwidth of the detector is large enough, and the impulse response function (IRF) is approximate to the δ-function. The final data are recovered by weighted averaging of all of the sampled data within one period, and the weight follows the shape of the measured state’s envelope. b, c The bandwidth of the detector is limited, resulting in a widened IRF, which convolutes the signal around each sampling point. d If the detector’s IRF is similar to the measured state’s envelope, single-point sampling can represent the final data.

We also note that, the discussion of system’s side information is an interesting research topic. We discuss it in two scenarios: (1) Alice performs a good calibration of the modulation variance of the transmitted signal; (2) Alice performs a poor calibration of the modulation variance, for example, neglecting optical modes that may exist on Alice’s side.

For the first scenario, all of the optical modes in the spectrum are taken into account, including the part outside the bandwidth of Bob’s detector. Because the information of the whole transmitter is included in the variance of Alice’s TM, any energy that Bob does not detect (either the energy beyond the detector’s bandwidth or the energy loss caused by the mode mismatch) contributes to the channel loss estimated from covariance matrix, which will be considered as caused by the eavesdropper. Therefore, this is not a side channel, but rather a performance degradation.

For the second scenario, Alice fails to perform a proper calibration, which will open a source-flaw-related side channel. For example, part of the signal’s energy of the sideband is not included when calibrating the modulation variance of Alice. This issue has been extensively studied in the single-mode case43,44,45,46,47. Note that this is not caused by the continuous-mode model proposed by our work, but by the improper calibration method of the modulation variance at the transmitter. To avoid this issue, three alternative methods are proposed: (1) Add a power meter at Alice’s side to calibrate Alice’s overall energy and determine the modulation variance of the whole TM; (2) avoid information leakage through single-sideband modulation43; (3) take the leakage information into account in the overall security analysis44,45,46,47.

Therefore, in the practical security analysis of a CVQKD system, especially when the homodyne detector is used to calibrate Alice’s modulation variance (such as using Bob’s device to perform back-to-back test for engineering convenience), we should deal with this phenomenon more carefully. We will investigate this issue in detail in the future.

Continuous-mode transmission

The measured state’s TM is determined by the modulation at the transmitter side, which is controlled, and also the channel transmission, which is not controlled. Therefore, the DSP needs to be designed or adjusted according to the channel condition. We take channel dispersion as an example to show how it influences the measurement, which is significant in a high-speed system. Considering a coherent state with a narrow Gaussian wavepacket \({\xi }_{{{{\rm{in}}}}}\left(t\right)\), after passing through the channel, its output envelope is transformed with a transfer function \(h\left(t,z\right)\), given by \({\xi }_{{{{\rm{out}}}}}\left(t\right)={\xi }_{{{{\rm{in}}}}}\left(t\right)* h\left(t,z\right)\), where \(h\left(t,z\right)\) is the channel transfer function in the time domain, and its Fourier transform is given by \({{{\mathscr{F}}}}\left[h\left(t,z\right)\right]=\exp \left(-{k}_{i}z+i{k}_{1}{{\Omega }}z+i\frac{{k}_{2}}{2}{{{\Omega }}}^{2}z\right)\), in which the second-order Taylor expansion of the real part of the wave vector is considered, z is the transmission distance, Ω is the Fourier frequency, and k1 and k2 denote the inverse group velocity and second-order dispersion coefficient, respectively. Only for the ultrashort period, the influence of third-order nonlinear dispersion needs to be considered. It can be seen that the output state’s TM varies with increasing distance, as in Fig. 3a. Therefore, if the DSP does not consider this TM varying, there will be an increasing extra loss as the transmission distance increases, which will decrease the maximal transmission distance, as simulated in Fig. 3b. This is where self-adaptive algorithms apply.

Fig. 3: Simulation results of the channel dispersion effect.
figure 3

a Evolution of the envelope of a Gaussian pulse with a full width at half-maximum (FWHM) of 20 ps in optical fibers, ignoring the fiber loss, where β2 = − 20.4 ps2/km is considered, which is a typical value of standard G.652 telecom fiber at 1550 nm. b Secret key rates for narrow-pulse (Gaussian pulse) propagation considering fiber-dispersion-induced TM mismatch, where β2 = 0 ps2/km and β2 = −20.4 ps2/km are considered, which are typical values of no dispersion channel and the standard G.652 telecom fiber at 1550 nm. In the simulation, we use the single-sampling-point scheme, and the sampling time accuracy is considered. The sampling time t = 0 ps is assumed as the peak of Gaussian pulse. It is shown that the increasing mode mismatch reduces the secret key rate while also relaxing the accuracy requirement of the sampling time.

Discussion

In this study, we have developed a generalized practical system model with continuous-mode formalism of quantum optics, based on which the IQ modulation at the transmitter side and band-limited homodyne detection with the sampling process at the receiver side can be well described. Then, with proper calibration of SNU, the output data of a linear DSP can be modeled by the quadrature measurement result with respect to a specific TM, jointly defined by the LO, filter, sampling, and DSP algorithms. This immediately results in good compatibility with traditional security analysis methods, which completes the security proof of linear DSP algorithms. Linear DSP toolboxes are expected to be directly employed in CVQKD, which highlights the importance of our work.

In addition to the security, our work also provides a method to analyze the performance of a DSP algorithm through a factor quantifying the matching degree between the measured state’s TM and the receiver’s TM. Moreover, interesting concepts like the DSP-induced fast fading-channel effect can be further analyzed to explore the practical limitations of a CVQKD system. By the guidance of our work, secure and better-performing DSP algorithms can be designed, which will exploit the significant potential of digital CVQKD to achieve ultra-high secret key-generation speed and cost-effective implementations.

Methods

Trusted detection model considering DSP

Our model mainly deals with the DSP part, which is cascaded after the trusted physical detection process. The input of the DSP modular is actually the output of the practical trusted detector, namely, \({\hat{X}}_{{{{\rm{out}}}}}\), as shown in Fig. 4 (a). From this point of view, our analysis is actually a complement to the existing trusted model after considering the time-related information and its processing.

Fig. 4: The proposed detection model considering non-ideal efficiency and electronic noise.
figure 4

a The PM version of our detection model. b The equivalent EB version of our detection model.

Since we restrict the discussed DSP algorithms to linear algorithms, the processing of the output signal is equivalent to the independent processing of the incident signal and the electronic noise. Then the output of DSP shares the same form as the output of the current trusted model, given by

$$\begin{array}{ll}&{\hat{X}}_{{{{\rm{DSP}}}}}={f}_{{{{\rm{dsp}}}}}\left({\hat{X}}_{{{{\rm{out}}}}}\right)\\ &=\sqrt{{\eta }_{{{{\rm{D}}}}}}{f}_{{{{\rm{dsp}}}}}\left({\hat{X}}_{{{{\rm{in}}}}}\right)+\sqrt{1-{\eta }_{{{{\rm{D}}}}}}{f}_{{{{\rm{dsp}}}}}\left({\hat{X}}_{v}\right)+{f}_{{{{\rm{dsp}}}}}\left({X}_{{{{\rm{el}}}}}\right)\\ &\mathop{=}\limits^{{{{\rm{PM}}}}}\sqrt{{\eta }_{{{{\rm{D}}}}}}{\hat{X}}_{{{{\Xi }}}_{{{{\rm{DSP}}}}}^{{t}_{j}}}^{\theta }+\sqrt{1-{\eta }_{{{{\rm{D}}}}}}{\hat{X}}_{{{{\Xi }}}_{{{{\rm{DSP}}}},v}^{{t}_{j}}}+{X}_{{{{\rm{el}}}}}^{{{{\rm{DSP}}}}}\end{array}$$
(17)
$$\mathop{=}\limits^{{{{\rm{EB}}}}}\sqrt{{\eta }_{{{{\rm{D}}}}}}{\hat{X}}_{{{{\Xi }}}_{{{{\rm{DSP}}}}}^{{t}_{j}}}^{\theta }+\sqrt{1-{\eta }_{{{{\rm{D}}}}}}{\hat{X}}_{{{{\rm{EPR}}}}}^{{{{\rm{DSP}}}}},$$
(18)

where \({f}_{{{{\rm{dsp}}}}}\left(\cdot \right)\) denotes the linear DSP function, and \({\hat{X}}_{{{{\Xi }}}_{{{{\rm{DSP}}}}}^{{t}_{j}}}^{\theta }\) is the incident TM defined by a specific DSP algorithm. The result of the prepare-and-measure (PM) scheme (Fig. 4a) is given in Eq. (17), where \({\hat{X}}_{{{{\Xi }}}_{{{{\rm{DSP}}}},v}^{{t}_{j}}}\) refers to the TM of a vacuum input, and the equivalent electronic noise \({X}_{{{{\rm{el}}}}}^{{{{\rm{DSP}}}}}\) is the broadband electronic noise filtered by the same DSP algorithm. The result of the equivalent entanglement-based (EB) scheme (Fig. 4b) is given in Eq. (18), where \({\hat{X}}_{{{{\rm{EPR}}}}}^{{{{\rm{DSP}}}}}\) is the equivalent trusted mode introduced by our model.

Therefore, the trusted model of a practical detector with DSP can be simplified as Fig. 4b, as long as we re-calibrate the variance of the equivalent electronic noise \({v}_{{{{\rm{el}}}}}^{{{{\rm{DSP}}}}}\).

In experiments, the practical calibrating steps are actually very similar to the current method, given by the following two steps:

  1. (1)

    Turn off the quantum input signal, turn off the LO, and directly sample the output of the detector, which corresponds to the measurement data of the electronic noise;

  2. (2)

    Process the collected data by the same DSP function as measured signal and then use the processing result to calculate the variance of the electronic noise.

After the calibrated electronic noise variance is obtained, the variance of the trusted EPR mode introduced in the EB scheme can be obtained, given by \({V}_{{{{\rm{EPR}}}}}^{{{{\rm{DSP}}}}}=1+{v}_{{{{\rm{el}}}}}^{{{{\rm{DSP}}}}}{{\mathrm{/}\left(1\; -{\eta }_{{{{\rm{D}}}}}\right)}}\), referred to as Bob’s input.

Derivation of the mode-matching coefficient

Assume that the measured state at Bob’s input is an unknown wavepacket coherent state \({\left\vert \gamma \right\rangle }_{{\xi }_{{{{\rm{A}}}}}}\) related to the wavepacket \({\xi }_{{{{\rm{A}}}}}\left(t\right)\). To obtain the equivalent performance of mode matching, we first decompose the receiver’s basis function \({{{\Xi }}}_{{{{\rm{DSP}}}}}^{{t}_{j}}\left(t\right)\) into the input basis \({\xi }_{{{{\rm{A}}}}}\left(t\right)\) and its orthogonal basis \({{{\Psi }}}_{\perp }\left(t\right)\). Then we show the mode-matching coefficient.

Using the Gram–Schmidt process, we can map the receiver’s operator from the basis \({{{\Xi }}}_{{{{\rm{DSP}}}}}^{{t}_{j}}\left(t\right)\) to the set of bases \(\left\{{\xi }_{{{{\rm{A}}}}}\left(t\right),{{{\Psi }}}_{\perp }\left(t\right)\right\}\). In this transformation, the basis \({{{\Xi }}}_{{{{\rm{DSP}}}}}^{{t}_{j}}\left(t\right)\) represents the measurement mode-matched basis function. The Gram–Schmidt process is given as follows:

  • Step 1: The overlapping of two bases (also called the mode-matching coefficient) is defined as

    $$\sqrt{\eta }=\int\,dt{{{\Xi }}}_{{{{\rm{DSP}}}}}^{{t}_{j},* }\left(t\right){\xi }_{{{{\rm{A}}}}}\left(t\right).$$
    (19)
  • Step 2: The mode-matched basis is written as \({\zeta }_{1}\left(t\right)={\xi }_{{{{\rm{A}}}}}\left(t\right)\) directly.

  • Step 3: The second orthonormal basis is calculated by

    $$\begin{array}{ll}{\zeta }_{2}\left(t\right)&={{{\Xi }}}_{{{{\rm{DSP}}}}}^{{t}_{j}}\left(t\right)-\frac{\int\,dt{{{\Xi }}}_{{{{\rm{DSP}}}}}^{{t}_{j},* }\left(t\right){\zeta }_{1}\left(t\right)}{\int\,dt{\zeta }_{1}^{* }\left(t\right){\zeta }_{1}\left(t\right)}{\zeta }_{1}\left(t\right)\\ &={{{\Xi }}}_{{{{\rm{DSP}}}}}^{{t}_{j}}\left(t\right)-\sqrt{\eta }{\xi }_{{{{\rm{A}}}}}\left(t\right).\end{array}$$
    (20)

It is easy to verify that

$$\begin{array}{ll}&\int\,dt{\zeta }_{2}^{* }\left(t\right){\zeta }_{2}\left(t\right)\\ &=\int\,dt\left[{{{\Xi }}}_{{{{\rm{DSP}}}}}^{{t}_{j},* }\left(t\right)-\sqrt{\eta }{\xi }_{{{{\rm{A}}}}}^{* }\left(t\right)\right]\left[{{{\Xi }}}_{{{{\rm{DSP}}}}}^{{t}_{j}}\left(t\right)-\sqrt{\eta }{\xi }_{{{{\rm{A}}}}}\left(t\right)\right]\\ &=1+\eta -2\sqrt{\eta }\int\,dt{\xi }_{{{{\rm{A}}}}}\left(t\right){{{\Xi }}}_{{{{\rm{DSP}}}}}^{{t}_{j}}\left(t\right)\\ &=1-\eta .\end{array}$$
(21)

After normalizing the basis function, we can obtain

$$\begin{array}{ll}{{{\Psi }}}_{\perp }\left(t\right)&=\frac{{\zeta }_{2}\left(t\right)}{\sqrt{\int\,dt{\zeta }_{2}^{* }\left(t\right){\zeta }_{2}\left(t\right)}}\\ &=\frac{1}{\sqrt{1-\eta }}\left({{{\Xi }}}_{{{{\rm{DSP}}}}}^{{t}_{j}}\left(t\right)-\sqrt{\eta }{\xi }_{{{{\rm{A}}}}}\left(t\right)\right).\end{array}$$
(22)

The receiver’s basis function is then decomposed as

$${{{\Xi }}}_{{{{\rm{DSP}}}}}^{{t}_{j}}\left(t\right)=\sqrt{\eta }{\xi }_{{{{\rm{A}}}}}\left(t\right)+\sqrt{1-\eta }{{{\Psi }}}_{\perp }\left(t\right),$$
(23)

and based on this, we can define two temporal modes (TMs) given by

$${\hat{A}}_{{\xi }_{{{{\rm{A}}}}}}^{{\dagger} }=\int\,dt{\xi }_{{{{\rm{A}}}}}\left(t\right){\hat{a}}^{{\dagger} }\left(t\right),$$
(24)
$${\hat{A}}_{{{{\Psi }}}_{\perp }}^{{\dagger} }=\int\,dt{{{\Psi }}}_{\perp }\left(t\right){\hat{a}}^{{\dagger} }\left(t\right).$$
(25)

Now, the measurement results can be rewritten as

$${\hat{D}}_{{t}_{j}}^{{{{\rm{SNU}}}}}={\hat{X}}_{{{{\Xi }}}_{{{{\rm{DSP}}}}}^{{t}_{j}}}^{\theta }=\sqrt{\eta }{\hat{X}}_{{\xi }_{{{{\rm{A}}}}}}+\sqrt{1-\eta }{\hat{X}}_{{{{\Psi }}}_{\perp }},$$
(26)

where \({\hat{X}}_{\xi }={\hat{A}}_{\xi }+{\hat{A}}_{\xi }^{{\dagger} }\) denotes the quadrature operator of ξ-TM.

Moments of measured data

Now, let us investigate the first-order and second-order moments of the final data. The mean value (first-order moment) is given by

$$\begin{array}{ll}{d}_{{{{\rm{out}}}}}&\,=\,\left\langle \gamma \right\vert {\hat{D}}_{{t}_{j}}^{{{{\rm{SNU}}}}}{\left\vert \gamma \right\rangle }_{{\xi }_{{{{\rm{A}}}}}}\,=\,\left\langle \gamma \right\vert \left(\sqrt{\eta }{\hat{X}}_{{\xi }_{{{{\rm{A}}}}}}+\sqrt{1-\eta }{\hat{X}}_{{{{\Psi }}}_{\perp }}\right){\left\vert \gamma \right\rangle }_{{\xi }_{{{{\rm{A}}}}}}\\ &=\sqrt{\eta }\left\langle \gamma \right\vert {\hat{X}}_{{\xi }_{{{{\rm{A}}}}}}{\left\vert \gamma \right\rangle }_{{\xi }_{{{{\rm{A}}}}}}+\sqrt{1-\eta }\left\langle \gamma \right\vert {\hat{X}}_{{{{\Psi }}}_{\perp }}{\left\vert \gamma \right\rangle }_{{\xi }_{{{{\rm{A}}}}}}\\ &=\sqrt{\eta }{d}_{{{{\rm{in}}}}},\end{array}$$
(27)

where \(\left\langle \varphi \right\vert \hat{A}\left\vert \varphi \right\rangle\) is the expectation value of \(\hat{A}\) in the state φ, and din denotes the mean value of the input mode.

The variance of the final data can be obtained and is

$$\begin{array}{ll}{\sigma }^{2}&=\left\langle \gamma \right\vert {\hat{D}}_{{t}_{j}}^{{{{\rm{SNU}}}}}{\hat{D}}_{{t}_{j}}^{{{{\rm{SNU}}}}}{\left\vert \gamma \right\rangle }_{{\xi }_{{{{\rm{A}}}}}}-\left\langle \gamma \right\vert {\hat{D}}_{{t}_{j}}^{{{{\rm{SNU}}}}}{\left\vert \gamma \right\rangle }_{{\xi }_{{{{\rm{A}}}}}}^{2}\\ &=\eta \left\langle \gamma \right\vert {\hat{X}}_{{\xi }_{{{{\rm{A}}}}}}{\hat{X}}_{{\xi }_{{{{\rm{A}}}}}}{\left\vert \gamma \right\rangle }_{{\xi }_{{{{\rm{A}}}}}}+2\sqrt{\eta \left(1-\eta \right)}\left\langle \gamma \right\vert {\hat{X}}_{{\xi }_{{{{\rm{A}}}}}}{\hat{X}}_{{{{\Psi }}}_{\perp }}{\left\vert \gamma \right\rangle }_{{\xi }_{{{{\rm{A}}}}}}\\ &\quad+\left(1-\eta \right)\left\langle \gamma \right\vert {\hat{X}}_{{{{\Psi }}}_{\perp }}{\hat{X}}_{{{{\Psi }}}_{\perp }}{\left\vert \gamma \right\rangle }_{{\xi }_{{{{\rm{A}}}}}}\\ &=\eta {V}_{{{{\rm{in}}}}}+\left(1-\eta \right)\cdot 1,\end{array}$$
(28)

where Vin is the variance of the input mode.

From Eq. (26), we can see that the measurement results are equivalent to a mode-matching loss added before the receiver side, which is modeled by a beam splitter (BS). After the first-order and second-order moments of measured data are given, it is more intuitive to see that the transmittance of equivalent BS is η. In the above discussion, we assume that \(\left\vert \gamma \right\rangle\) is a TM coherent state to simplify the calculation of Eqs. (27) and (28). While one can further exam that, for an arbitrary input state, Eqs. (27) and (28) still hold. The above derivations also hold considering heterodyne detection.