SKA will produce the largest data flow ever seen in astronomy1. Just the first phase of SKA (SKA1) — accounting for only 10% of the full scale — will create data products with a growth rate of ~1 exabyte per year once operational, requiring a computing capacity of ~300 petaflops. The vast volume of the SKA data and diverse locations of science users require a coordinated network of geographically distributed SKA regional centres (SRCs). China, as one of the founding members of SKA, plans to host an SRC, offering a platform for multi-disciplinary science research, in-depth data processing, long-term storage and advanced technique development. Exaflop-scale computing and exabyte-level data storage pose unprecedented challenges to the data collection, transportation, analysis, curation and storage. Therefore, prototyping is essential before the full-scale SRC construction begins.

The China SKA regional centre prototype (CSRC-P, Fig. 1) deployment and integration tests were carried out at the Shanghai Astronomical Observatory in August 2019, rendering it the first completed SRC prototype in the world. CSRC-P contains a small-scale yet fully functional high-performance computing (HPC) system that includes compute nodes, a parallel file system, high-speed networks, user login and management systems. The design of the CSRC-P is science-oriented with consideration for the complex structure of the SKA data, the various requirements of computing and storage resources from different science projects and highly concurrent data flow. The overall system adopts the idea of application–system co-design: hardware and software components are selected to achieve the best performance with limited power consumption, based on the application characteristics, for example, computation, memory access and input/output (I/O)2.

Fig. 1: The China SKA regional centre prototype.
figure 1

It is the first completed SKA regional centre prototype in the world.

Full size image

One of the main roles of the CSRC-P is to understand the requirements of various scientific tasks involving differing computing architectures, in order to provide an assessment for a future large-scale expansion and costing budget. Both general-purpose — Intel x86 CPU — and heterogeneous — graphics processing unit (GPU) and Advanced RISC Machine (ARM) — computing architectures have been adopted in CSRC-P, including sixteen multi-core CPU nodes (Xeon 6140 processor), three CPU+GPU nodes (a total of sixteen NVIDIA Tesla V100 cards), and twelve ARM nodes (Huawei Kunpeng 920 processor). The total computing power is 268 teraflops. Heterogeneous computing architectures are gaining increasing attention in fields such as data-intensive scientific computing and artificial intelligence. Astronomy applications using GPU accelerators have also seen a remarkable recent growth. ARM processors are being used in SKA HPC for the first time and benchmark tests show that the overall performance of the ARM cluster is comparable with the Intel-processor cluster. In addition to having a higher number of processing cores, ARM processors offer many other significant advantages, including higher memory bandwidth and lower power consumption. The computing speed of the HPC is also affected by other components, for example, memory, system I/O, storage and inter-node network. For example, utilization of all-flash NVMe storage in CSRC-P results in an increase of 46% in memory bandwidth and 66% in I/O bandwidth compared with traditional clusters. In order to support the long-term persistent operation of SRCs, designing high-performance, low-power consumption (≥35 gigaflops per watt) computing architectures and optimizing astronomical algorithms will be the focus of future work.

The CSRC-P represents an integration of advanced computer hardware design and cutting-edge SKA algorithms and software, going beyond just a pure HPC system. With the deployment of the built-in software environments, CSRC-P has reduced some of the SKA precursor/pathfinder data. The Murchison Widefield Array GLEAM and the Australian SKA Pathfinder DINGO pipelines have been successfully installed on CSRC-P. Currently, CSRC-P has established an end-to-end connection with the Pawsey Supercomputing Centre in Perth, Western Australia with a maximum data transfer rate of 5 Gbps. Future cloud-based SRCs will not only satisfy the needs of big-data processing, but also flexibly adapt to the ever-increasing data flow and system upgrades. Software solutions based on ‘container’ technology (such as Docker) are being developed to facilitate new users.

The commissioning of CSRC-P is expected to take place in 2020. The pioneering work and practical operational experience of CSRC-P will be valuable for improving the design and future large-scale expansions of SKA regional centres. CSRC-P aims to provide scientists worldwide with necessary computing resources, high-quality data products and convenient technical support to conduct SKA early science and to understand the data challenges. As the prototyping work continues, the evolving and increasing requirements from scientific users will enable the CSRC-P to expand its resources and capabilities accordingly.

CSRC-P is supported by the China SKA science and engineering teams, Ministry of Science and Technology of China, SKA-China Office, SKA Organisation and Chinese Academy of Sciences.