The rapid growth in the amount of data being transferred within data centres, combined with the slowdown in Moore’s Law, creates challenges for the future scalability of electronically switched data-centre networks. Optical switches could offer a future-proof alternative, and photonic integration platforms have been demonstrated with nanosecond-scale optical switching times. End-to-end switching time is, however, currently limited by the clock and data recovery time, which typically takes microseconds, removing the benefits of nanosecond optical switching. Here we show that a clock phase caching technique can provide clock and data recovery times of under 625 ps (16 symbols at 25.6 Gb s−1). Our approach uses the measurement and storage of clock phase values in a synchronized network to simplify clock and data recovery versus conventional asynchronous approaches. We demonstrate the capabilities of our technique using a real-time prototype with commercial transceivers and validate its resilience against temperature variation and clock jitter.
Subscribe to Journal
Get full journal access for 1 year
only $8.25 per issue
All prices are NET prices.
VAT will be added later in the checkout.
Rent or Buy article
Get time limited or full article access on ReadCube.
All prices are NET prices.
Source data are provided with this paper.
Singh, A. et al. Jupiter rising: a decade of Clos topologies and centralized control in Google’s datacenter network. ACM SIGCOMM Comput. Commun. Rev. 45, 183–197 (2015).
Markov, I. Limits on fundamental limits to computation. Nature 512, 147–154 (2014).
Dorren, H. et al. Challenges for optically enabled high-radix switches for data center networks. J. Lightwave Technol. 33, 1117–1125 (2015).
Ghiasi, A. Large data centers interconnect bottlenecks. Opt. Express 23, 2085–2090 (2015).
Krishnamoorthy, A. et al. From chip to cloud: optical interconnects in engineered systems. J. Lightwave Technol. 35, 3103–3115 (2017).
Testa, F. & Pavesi, L. Optical Switching in Next Generation Data Centers (Springer, 2017).
Ballani, H. et al. Bridging the last mile for optical switching in data centers. In 2018 Optical Fiber Communication Conference (OFC) W1C.3 (OSA, 2018).
Clark, K. et al. Sub-nanosecond clock and data recovery in an optically-switched data centre network. In 2018 European Conference on Optical Communication (ECOC) 1–3 (IEEE, 2018).
Chen, C. P. et al. Programmable dynamically-controlled silicon photonic switch fabric. J. Lightwave Technol. 34, 2952–2958 (2016).
Cheng, Q., Wonfor, A., Wei, J. L., Penty, R. V. & White, I. H. Low-energy, high-performance lossless 8 × 8 SOA switch. In Proc. 2015 Optical Fiber Communication Conference (OFC) Th4E.6 (OSA, 2015).
Shi, K. et al. System demonstration of nanosecond wavelength switching with burst-mode PAM4 transceiver. In Proc. 2019 European Conference on Optical Communication (ECOC) 1–3 (IEEE, 2019).
Xilinx. DS893 (v1.12) Virtex UltraScale Architecture Data Sheet: DC and AC Switching Characteristics (2019); https://www.xilinx.com/support/documentation/data_sheets/ds893-virtex-ultrascale-data-sheet.pdf
Banu, M. & Dunlop, A. E. Clock recovery circuits with instantaneous locking. Electron. Lett. 28, 2127–2130 (1992).
Terada, J. et al. A 10.3 Gb/s burst-mode CDR using a ΔΣ DAC. IEEE J. Solid-State Circuits 43, 2921–2928 (2008).
Rylyakov, A. et al. A 25 Gb/s burst-mode receiver for low latency photonic switch networks. IEEE J. Solid-State Circuits 50, 3120–3132 (2015).
Yang, Y., Wen, Y. J., Nirmalathas, A., Liu, H. F. & Novak, D. Optical clock recovery at line rates via injection locking of a long cavity Fabry–Pérot laser diode. IEEE Photon. Technol. Lett. 16, 1561–1563 (2004).
Ozkaya, I. et al. A 56 Gb/s burst-mode NRZ optical receiver with 6.8 ns power-on and CDR-lock time for adaptive optical links in 14 nm FinFET CMOS. In Proc. 2018 IEEE International Solid-State Circuits Conference (ISSCC) 266–268 (IEEE, 2018).
Bostica, B., Burzio, M., Gambini, P. & Zucchelli, L. in Photonic Networks (ed. Prati, G.) 362–376 (Springer, 1997).
Timing and Synchronization Aspects in Packet Networks G.8261 (ITU, 2008); https://www.itu.int/rec/T-REC-G.8261-201908-I/en
Lee, K. S., Wang, H., Shrivastav, V. & Weatherspoon, H. Globally synchronized time via datacenter networks. In Proc. 2016 ACM SIGCOMM Conference 454–467 (ACM, 2016).
Slavík, R. et al. Ultralow thermal sensitivity of phase and propagation delay in hollow core optical fibres. Sci. Rep. 5, 15447 (2015).
Data Center Networking Equipment—Issues and Best Practices TC9.9 (ASHRAE, 2016); https://tc0909.ashraetcs.org/documents/ASHRAE_TC0909_Power_White_Paper_22_June_2016_REVISED.pdf
Soleimani, S., Afzali-Kusha, A. & Forouzandeh, B. Temperature dependence of propagation delay characteristic in FinFET circuits. In Proc. Int. Conference on Microelectronics (ICM) 276–279 (IEEE, 2008).
Ma, L.-S., Jungner, P., Ye, J. & Hall, J. L. Delivering the same optical frequency at two places: accurate cancellation of phase noise introduced by an optical fiber or other time-varying path. Opt. Lett. 19, 1777–1779 (1994).
Lord, A., Blank, L. C., Boggis, J. M., Bryant, E. & Stallard, W. A. Theory of control mechanism for an optically time-division-multiplexed system. Electron. Lett. 24, 2011–2012 (1988).
Ellis, A. D., Widdowson, T., Phillips, I. D. & Pender, W. A. High speed OTDM networks employing electro-optic modulators. Trans. Inst. Electron. Inf. Commun. Eng. Sect. E E81-C, 1301–1308 (1998).
Zhang, Q., Liu, V., Zeng, H. & Krishnamurthy, A. High-resolution measurement of data center microbursts. In Proc. 2017 Internet Measurement Conference (IMC) 78–85 (ACM, 2017).
IEEE Standard for Ethernet—Amendment 3: Physical Layer Specifications and Management Parameters for 40 Gb/s and 100 Gb/s Operation over Fiber Optic Cables 802.3bm (IEEE, 2015); https://standards.ieee.org/standard/802_3-2018.html
Lipínski, M., Włostowski, T., Serrano, J. & Alvarez, P. White rabbit: a PTP application for robust sub-nanosecond synchronization. In Proc. 2011 IEEE International Symposium on Precision Clock Synchronization for Measurement, Control and Communication (IEEE, 2011); https://doi.org/10.1109/ISPCS.2011.6070148
Bousonville, M. et al. New phase stable optical fiber. In Proc. Beam Instrumentation Workshop 2012 (BIW2012) 101–103 (JACoW, 2012).
40-Gigabit-Capable Passive Optical Networks 2 (NG-PON2) G.989.2 (ITU, 2015); https://www.itu.int/rec/T-REC-G.989.2-201902-I/en
Aijaz, A., Dohler, M., Aghvami, A. H., Friderikos, V. & Frodigh, M. Realizing the tactile internet: haptic communications over next generation 5G cellular networks. IEEE Wirel. Commun. 24, 82–89 (2016).
Takesue, H. et al. Quantum key distribution over a 40-dB channel loss using superconducting single-photon detectors. Nat. Photon. 1, 343–348 (2007).
Torres-Company, V. & Weiner, A. M. Optical frequency comb technology for ultra-broadband radio-frequency photonics. Laser Photon. Rev. 8, 368–393 (2014).
Kuo, B. P.-P., Myslivets, E., Alic, N. & Radic, S. Wavelength multicasting via frequency comb generation in a bandwidth-enhanced fiber optical parametric mixer. J. Lightwave Technol. 29, 3515–3522 (2011).
Zhang, M. et al. Broadband electro-optic frequency comb generation in a lithium niobate microring resonator. Nature 568, 373–377 (2019).
We acknowledge financial support from Microsoft, Inphi Inc. and EPSRC grants EP/R041792/1 and EP/R035342/1 and Royal Society Paul Instrument Fund PIF/R1/180001. Eblana Photonics provided the lasers used in this work. We thank P. Watts for helpful discussion at the early stages of the work and E. Vonhof for assistance in the generation of the figures.
A patent application, entitled ‘Phase Caching for Fast Data Recovery’, has been filed by Microsoft Technology Licensing, LLC with the US Patent and Trademark Office on 27 October 2017, on the technology described in this Article. This patent is currently pending (patent no. US20190132112A1).
Publisher’s note Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
B, Data centre traffic pattern; C, Impact of traffic on throughput.
A, 48 hour stability; B, Tolerance to rapid temperature change.
A, Impact of rate-of-change of temperature on clock phase caching; B, Minimum required phase update rate as a function of rate-of-change of temperature.
A, Impact of sinusoidal jitter on clock phase caching; B, Impact of Gaussian jitter on clock phase caching.
Clock phase caching scalability.
About this article
Cite this article
Clark, K.A., Cletheroe, D., Gerard, T. et al. Synchronous subnanosecond clock and data recovery for optically switched data centres using clock phase caching. Nat Electron 3, 426–433 (2020). https://doi.org/10.1038/s41928-020-0423-y