Abstract
The clinical treatment planning of coronary heart disease requires hemodynamic parameters to provide proper guidance. Computational fluid dynamics (CFD) is gradually used in the simulation of cardiovascular hemodynamics. However, for the patientspecific model, the complex operation and high computational cost of CFD hinder its clinical application. To deal with these problems, we develop cardiovascular hemodynamic point datasets and a dual sampling channel deep learning network, which can analyze and reproduce the relationship between the cardiovascular geometry and internal hemodynamics. The statistical analysis shows that the hemodynamic prediction results of deep learning are in agreement with the conventional CFD method, but the calculation time is reduced 600fold. In terms of over 2 million nodes, prediction accuracy of around 90%, computational efficiency to predict cardiovascular hemodynamics within 1 second, and universality for evaluating complex arterial system, our deep learning method can meet the needs of most situations.
Introduction
Coronary heart disease (CHD) has become a leading cause of global mortality^{1,2}, with more than 50% of these cases being related to coronary stenosis^{3}. In order to achieve successful therapeutic effects, CHD clinical treatment plans require a variety of hemodynamic parameters to provide proper guidance. Currently, pressure fieldbased fractional flow reserve (FFR) is the gold standard for the clinical diagnosis of myocardial ischemia severity caused by stenosis^{4,5,6,7}. The treatment regimen guided by FFR has been proven safe and effective^{8,9}. For patients with severe myocardial ischemia, revascularization is typically required. Coronary artery bypass grafting (CABG) is the most commonly used revascularization procedure^{10}. Velocity fieldbased postoperative blood flow changes in the grafts and the distal end of the stenotic coronary artery are thought to be the most direct parameter for assessing the influence of CABG^{11,12}. However, the application rate of hemodynamic parameters in clinical practice is low, mainly due to its high measurement cost and potential risks during catheter insertion. Taking invasive FFR as an example, the price of the pressure guide wire required for measurement is relatively high. In addition, the use of vasodilator drugs such as adenosine may increase the time and cost of interventional surgery^{13}, and may also increase the surgical risk of patients with adenosine sensitivity or asthma^{14}. Therefore, obtaining hemodynamic parameters, including velocity and pressure inexpensively and noninvasively is crucial for the support of CABG and the treatment of CHD.
A large number of previous studies have used computational fluid dynamics (CFD) to obtain cardiovascular hemodynamics^{15,16,17,18,19}. Based on the patient’s cardiovascular geometry, provided by medical imaging data (e.g., MRI, CT, etc.) and given boundary conditions, CFD can inexpensively and noninvasively obtain solutions for velocity and pressure through the conservation of mass and momentum under isothermal and incompressible assumptions. However, the cost to model cardiovascular hemodynamics with available computational resources is very high^{20}. When subjects’ personalized CFD boundary conditions (e.g., the inlet is set to pulsatile flow, and the outlet pressure is an invasive measured value) are used to calculate the hemodynamics of complex cardiovascular models with small grafts and coronary branches, even highperformance computational clusters usually require several hours of iteration to ensure the accuracy of the model. Even with simplified boundary conditions (e.g., steady flow at the inlet and zero pressure at the outlet), CFD methods require a calculation time around 10 min. In addition, each patient’s cardiovascular geometry is unique. This means that the CFD procedure will need to be completed separately and repeatedly for each patient. The high computational cost of CFD hinders its clinical application to the treatment of CHD, such as the inability to provide surgical guidance. Therefore, it has become increasingly necessary to develop a cardiovascular hemodynamic calculation method that can reduce calculation costs while ensuring model accuracy.
The development of deep learning, one of many machine learning methods, provides a new way to solve the above problems. Deep learning detects distributed representation features of data by constructing neural networks and combining lowlevel features to form more abstract highlevel features or attribute features, thus completing the task of classification or regression^{21,22,23,24,25,26,27}. Advanced deep learning algorithms and highperformance GPU servers can greatly reduce computing times while ensuring high accuracy. Due to the development of deep learning techniques, some studies have introduced its application to predict 2D/3D flow fields from geometrical features. For example, Guo et al. put forward a calculation method of 2D flow around simple geometric models based on convolutional neural networks^{28}. And Liang et al. proposed a deep learning method to predict 3D simplified thoracic aortic hemodynamics^{29}. However, the research concerning predictions of hemodynamics via deep learning is still very limited^{30}. The main limitations of these studies are: (1) most studies focus on 2D flow fields, which have a limited scope of application^{31,32,33,34}, (2) the 3D flow field model only appears in ideal geometry, and sample resolution in the dataset is too low to represent complex flow field distributions and geometric structures^{28,29}. For CABG surgery, a cardiovascular model with small grafts and coronary branches has an intricate geometry and internal flow field distribution. Therefore, in this study, in order to accurately predict complex 3D cardiovascular hemodynamics with limited samples, new requirements to adapt to the flexibility and high resolution of the input geometry have been imposed on datasets and deep learning networks, which is also the main technical problem and contribution of this study. Concerning the dataset, each sample must have enough spatial resolution to resolve complex flow field and model geometry. Therefore, it is necessary to find a new, highresolution sample representation format. And concerning the network, it is necessary to propose a new network structure that can effectively handle the new sample format.
In this study, with the aim of predicting 3D hemodynamics in the real cardiovascular systems of patients with coronary stenosis (e.g., geometry containing aorta, coronary arteries, and bypass graft), we have proposed a new deep learning method that could predict the velocity field and pressure field based on the geometric features of the model. We collected cardiovascular data with small branches from computed tomography angiography (CTA) performed on 110 patients with CHD for model expansion and simulation of CABG surgery. Under certain boundary conditions, a CFD method was used to obtain the hemodynamic results (i.e., velocity and pressure field) of all models. Later, we converted the CFD results into highdensity 3D point clouds. The point cloud inherited the ability of CFD results to characterize the geometric structure and flow field distribution of the model, which could characterize the complex flow field distribution and geometry of real cardiovascular models with high resolution^{35,36}. On this basis, preoperative and postoperative cardiovascular hemodynamic point datasets were established, respectively. We also proposed a new deep learning network based on the PointNet structure^{37}, which could effectively resolve the disorder of point clouds and introduce spatial relationships. By extracting and integrating global and local features of the point cloud, the network could analyze and reproduce the relationship between vessel geometry in the point cloud datasets and the corresponding hemodynamics. The deep learning network only needs to be trained once. Next, when we input cardiovascular geometry information from a new patient, the corresponding 3D hemodynamic parameter prediction results could be obtained within 1 second. In order to verify the accuracy of our deep learning method, we define error functions (ERR), normalized mean absolute error (NMAE), and mean relative error (MRE) to evaluate the difference between the two methods. Based on the acquired hemodynamic results, we further calculated and compared the preoperative FFR and the postoperative blood flow of the graft and the distal stenotic coronary artery. Statistical analysis results showed that the predicted results of deep learning were in agreement with the traditional CFD method, but the calculation time was reduced 600fold. Our deep learning method aims to realize the prediction of velocity and pressure fields before and after CABG surgery instead of CFD. To the best of our knowledge, this study represents the first report describing deep learning techniques that can effectively and accurately predict 3D hemodynamics of complex cardiovascular system with small grafts and coronary branches with limited data.
Results
Prediction results of velocity field
When the preoperative and postoperative velocity field datasets were used as inputs to the proposed deep learning network, the loss function value versus epochs was made available (as seen in Supplementary Fig. 1). The loss function fully converged.
We compared the prediction results of deep learning with CFD. Figure 1 displays the streamline diagram of a 3D velocity field distribution and a crosssectional view of velocity magnitude contour in the same areas. It showed that the velocity fields obtained by the two methods had good reliability. Our deep learning method could predict the distribution of velocity fields in the entire cardiovascular model before and after CABG, which included not only general attributes such as laminar blood flow, but also the occurrence of complex vortex structures.
We calculated the mean value of the predicted velocity field ERR of the 100 models in the test sets, as shown in Table 1. The result showed that the prediction accuracy of the coronary arteries (NMAE < 3%, MRE < 5%) and grafts (NMAE < 1.5%, MRE < 2.5%) was higher than that of the aorta and superior aortic branch artery (Preoperative: NMAE = 6.02%, MRE = 9.77%; Postoperative: NMAE = 6.01%, MRE = 9.74%). This was mainly due to the larger magnitude of flow and the complicated flow field distribution by vortex and flow separation in both the aorta and superior aortic branch artery parts. We give detailed explanations and feasible improvements in the prediction error analysis section below.
Deep learning prediction results (shown in Fig. 1 and Table 1) could reflect the effect of CABG on the distribution of internal flow field in the cardiovascular system. It could accurately reproduce velocity fields in the small lesion coronary and reconstructed grafts, which meant that it could not only reflect the preoperative ischemic condition of LAD branches with different stenosis rates, but also signal the postoperative improvement of blood supply. In addition, it could be seen from the predicted results that CABG had a big impact on the flow field of the grafts and LAD with stenosis but had little influence on the flow field of other parts. The proposed network could effectively identify significant and nonsignificant disturbances of the graft on the flow field, which highlighted its high performance.
Prediction results of pressure field
This study aimed to develop a universal deep learning method. The same network structure could accomplish predicting hemodynamic parameters with different attributes, which could then be proven via the analysis results of the velocity field and pressure field.
Different from the velocity, the pressure in the flow field was scalar, that was, the pressure at a point had the same value in all directions. There were different vector components of velocity vector in X, Y, and Z directions. Therefore, the pressure datasets as the network input contained less information than the velocity field datasets, which was reflected in the convergence speed of the loss function value versus epochs (as seen in Supplementary Fig. 1). The loss function converged faster.
Figure 2 displays a 3D pressure distribution cloud map obtained from deep learning and CFD method, with a crosssectional view of the same part. The pressure fields obtained by the two methods were also in agreement. Our deep learning method could accurately replicate the pressure distribution of the entire cardiovascular model with small grafts and coronary branches.
We calculated the mean value of the predicted pressure field ERR of the 100 models in the test set, as shown in Table 2. The prediction accuracy of pressure fields of coronary artery (NMAE < 2.5%, MRE < 4%) and grafts (NMAE < 1.5%, MRE < 2%) was also higher than that of the aorta and superior aortic branch artery (Preoperative: NMAE = 4.30%, MRE = 7.61%; Postoperative: NMAE = 4.28%, MRE = 7.35%), as explained in the prediction error analysis section below.
Based on the velocity and pressure field, we calculated important clinical indicators related to CABG surgery: preoperative FFR of the lesion LAD and the postoperative blood flow of the graft and the distal stenotic coronary artery. The performance of our deep learning method could be further evaluated by comparing the indicators obtained by the two methods, which is described in detail below.
Deep learning improves computing efficiency
After the training was completed, and when the point coordinate space information of the cardiovascular model in the test set was input to the network, the hemodynamics of the query point could be obtained within 1 second using a NVidia GeForce GTX 1080 Ti GPU. For the CFD method, the calculation time of one model on an Intel Xeon Gold 6148 2.4 Ghz × 2 CPU server was about 10 min. Deep learning improved the computational efficiency of a single model 600fold. Although it took some time (about 40 h) to complete network training, this process only needs to be completed once to predict the hemodynamic of all models in the given test set. Compared to the traditional CFD method, where each model requires independent simulation calculations, the computational cost of deep learning is far less than CFD.
Together with the accuracy analysis of the results, the proposed deep learning network could efficiently and accurately predict 3D hemodynamics of complex cardiovascular system with small grafts and coronary branches. This also meant that deep learning has broad application prospects, such as the possibility of application in the early planning or even realtime support of CABG.
Calculate FFR and improved flow based on prediction results
In order to further confirm the accuracy and clinical utility of our deep learning method, we calculated the preoperative FFR value and postoperative blood flow value of the grafts and the distal end of the stenosis using the hemodynamic results acquired from CFD and deep learning, respectively. The FFR was defined as the ratio of the mean pressure at a crosssection 3 cm downstream of the stenosis to the mean pressure at the LAD coronary entry region^{38}. Improved blood flow was defined as graft inlet flow, which was calculated based on the diameter and velocity of the graft inlet section. Figure 3a, b were the scatter plots of FFR and improved flow on each model obtained from the two methods, which showed that the correlation between the FFR and improved flow attained from these methods was excellent (FFR: r = 0.9580, P < 0.001; Flow: r = 0.9734, P < 0.001). Also, the Bland–Altman analysis result is as shown in Fig. 3c, d: 97 sets of FFR data and 97 sets of improved flow data fall within the 95% confidence interval (FFR: −0.07780–0.09254; flow: −1.282–0.8568), which confirmed that the clinical indicators calculated by these methods were in agreement.
Our deep learning method reduced the computational time of hemodynamics to 1 s, and its output was a point cloud format which was easy to postprocess. On this basis, the calculation time of clinical indicators such as FFR and improvement of blood flow could also be reduced to a few seconds while ensuring high accuracy, which further confirmed the superiority of our deep learning method.
Prediction error analysis
We extracted regions with large prediction error function values (MRE > 10%) in the entire cardiovascular model. These regions were highly consistent with the vortex regions in the CFD calculation results, as shown in Fig. 4a, b. Vortexes were mainly distributed in the aorta and superior aortic branch artery rather than the coronary artery and graft. Points in the aorta and superior aortic branch artery region accounted for more than 99% of the query point cloud, and more than 30% of the points in the whole region were located in the vortex region, which was the main source of prediction errors for the cardiovascular model. We extracted the points only in the vortex region which was defined with Eigen Helicity method, level 0.005, actual value 44.89 s^{−1} for predicted results and calculated the error as shown in Fig. 4c. The points in the coronary artery and graft part only accounted for 1% of the query point cloud, and only about 10% of them were in the vortex region, which had little effect on the prediction errors. The vortex distribution also explained why the graft and coronary parts had higher prediction accuracy.
Compared to laminar flow, the vortex flow part was extremely complicated. Previous studies that used deep learning to predict complex vortexes required much more training data than ours, even in 2D space^{39,40,41}. Taking Lee’s research as an example^{39}, in a 2D plane with a size of 250 × 250 (grid cells), 500,000 vortex samples were needed to train the network. The number of samples was far more than that of this study. However, the complexity of the vortex (2D) was lower than that of this study (3D). Therefore, we hypothesized that the limited dataset of this study was not sufficient to fully characterize the characteristics of vortex, which could lead to a decrease in the accuracy of the prediction results. To verify this theory, we fixed the test set and increased the size of the training set from 10% to 100% and then calculate the MRE of the vortex region, as shown in Fig. 4c. Even at the maximum of the training set, the MRE of the vortex part still had a downward trend. With a larger dataset, accuracy would also improve. The collection of more cardiovascular models of patients with coronary artery stenosis to build a richer dataset is necessary as more data will lead to higher prediction accuracy and better model performance.
Discussion
In this study, we used deep learning to predict 3D hemodynamics of complex cardiovascular systems with small grafts and coronary branches before and after CABG surgery. Our results showed that calculation results of the deep learning and CFD methods were highly consistent, and the calculation efficiency was improved 600fold. This study proved that deep learning could achieve efficient and accurate predictions of 3D hemodynamics in complex models, which also means that it has great application value in scientific research and clinical fields.
The comparative analysis of our deep learning method against previous deep learning approaches is available in Supplementary Table 1. As mentioned above, there was no previous research that achieved 3D hemodynamic prediction of complex models such as thoracic aortic tree, including the small grafts and coronary branches. Our work made up for this gap, which was mainly due to the use of the point cloud to create datasets and propose a highperformance deep learning network. Previous studies typically required normalized flow field data to help deep learning networks obtain correspondence between model geometry and flow field distribution^{24,25}. In other words, regular spatial relationships (e.g., orthogonalization) are introduced into the flow field data by interpolation or other methods. Therefore, the spatial resolution of flow field data depends on space segmentation size during normalization. When there are big differences in the size of different parts of the model, such as the radius of the aorta which is much larger than that of the coronary artery and the graft in this study, it will be very difficult to choose the appropriate space segmentation size. Large segmentation sizes cannot characterize small parts of the model. Small segmentation size results in a large amount of data exceeded the upper limit of server processing. For example, when using Guo et al.’s normalization method to process the cardiovascular models in this study^{28}, we should place the model in a threedimensional space cuboid. The length, width, and height of the cuboid should be the maximum value in the corresponding direction of the model. For a cardiovascular model with a length of 80 mm, a width of 120 mm and a height of 95 mm, when using 0.1 mm as the segmentation size, one 3D cardiovascular model needs to have 800 × 1200 × 950 = 836,000,000 normalized points to contain the mean shape, which is unacceptable for network and GPU processors. Liang et al. proposed a normalization method for deep learning^{29}. However, it was only suitable for large ideal aorta, not for the whole complex cardiovascular system, especially for small coronary branches. In addition, Liang et al. normalized the thoracic aortic models of different patients to the same meshes with only 80,100 nodes. However, in this study, the mesh independence test (as seen in Supplementary Fig. 4) showed that for the thoracic aortic, when the number of meshes exceeded one million, the CFD simulation results can be considered to be stable. Therefore, the deep learning method proposed by Liang et al. still has great limitations in the application range and data resolution. Due to limitations of resolution and network performance, most of the previous research objects are simple idealized flow fields. Unlike previous approaches, we utilized highdensity 3D point clouds to build datasets. A point cloud is the connection point of CFD meshes and is generally called a node. It is directly output by the mesh result and does not require normalization or other processing. The point cloud can represent complex or small features of the model under the appropriate mesh setting^{42,43}. In this study, the mesh independence test results confirmed that the model contains about 2 million points that could represent the complex structure of the entire cardiovascular system, of which 0.4 million were surface model point clouds and 1.6 million were internal query point clouds. Each point has only spatial coordinates and hemodynamic information, which means there is no connection or interaction between them. Thus, it can store a lot of useful information with a small amount of data unlike a connected point set. In order to resolve the disorder of the point cloud and introduce the spatial relationship, we propose a dual sampling channel network structure based on PointNet. Since there is no connectivity information between the points, there is no specific input sequence for the points. That is to say, when N points are used as input to the network, due to the different input sequence, there may be N! input permutations, that is, the disorder of the point cloud. Symmetric function can ensure that the output is the same regardless of the order of input, to resolve point disorder^{37}. The dual sampling channel can extract the geometry of the model point cloud of the surface as the global geometry feature, and the internal query point cloud distribution as the local flow field feature. Global features convey the outer geometry information within the model, which can help the query point cloud to obtain its position inside the model. Under the uniform CFD boundary conditions, the position of the query point is corresponding to the flow field. The local flow field characteristics and the corresponding position information can be used as teacher signals to help the network learn the hemodynamic values of a specific position. In this way, the spatial relationship is effectively introduced to help the network attain correspondence between the model geometry and the flow field distribution.
Our deep learning method is highly universal, which is not limited to guiding the implementation of CABG and the treatment of CHD. It can analyze and reproduce the relationship between complex cardiovascular geometry and hemodynamics in a given dataset, which can be extended to the hemodynamic simulation of other organs and tissues, or even the flow field research under experimental conditions, such as replacing the steady flow 3D PIV experiment with sufficient data. From a technical perspective, our deep learning method is highly practical. For different properties of hemodynamic parameters, the prediction can still be completed without adjusting the network structure, which was difficult to achieve in the past^{28,29}. The analysis results of the velocity and pressure fields confirm that the same network structure can achieve high accuracy predictions for physical fields with and without spatial components. In addition, point cloud, as a conventional data format, can be accepted by most of analysis software (e.g., ANSYS and Python) which makes the processing of point cloud data relatively easy. In terms of resolution, universality, accuracy, and computational efficiency, our deep learning method can meet the needs of most situations. We also noticed that the four datasets (preoperative, postoperative, velocity, and pressure fields) need to be trained separately as inputs, which increased the computational cost and operational complexity of deep learning to a certain extent. Therefore, we will explore potential improvements due to similarities in features between the different fields and application scenarios in further work. For example, by merging four datasets (with different labels), all prediction results can be output in one training session.
The biggest limitation of this study is the lack of clinical data. In CFD simulation, there is no boundary condition information for patients. Currently, we adopted constant values on inlets and outlets, which have been widely used among a number of geometries^{44,45,46}. Therefore, the simulation results should include differences from real hemodynamics. In future approaches that include boundary conditions, another input channel will be required on the network. This input channel will take the patient’s personalized boundary conditions as the input, and together with the model point cloud as the teaching signal to control the training process. In the analysis of prediction accuracy, we only compared the prediction results of deep learning with CFD, but lack of comparison with clinically measured data of patients (such as invasive FFR). Itu et al.^{47} and Tesche et al.^{48} proved that under the premise of good consistency between the FFR calculated by deep learning and CFD, compared with the invasive FFR, the FFR values calculated by these three methods were also with good consistency, which we intend to address in the future. The data for this study comes from a project optimizing the treatment plan of coronary stenosis. Therefore, our datasets do not contain information on other cardiovascular diseases such as coronary aneurysms or aortic diseases. In addition, the point cloud data used in this study is extracted from the CFD meshing result. In the point cloud extraction process, we deleted the connection relationship between the grids. Although the point cloud can reproduce the CFD flow field prediction results, it also brings potential limitations, such as the loss of correlation information between different nodes in the original CFD results and the introduction of the disorder of point clouds. Based on the above limitations and prediction error analysis results, in future work, we need to increase the number and type of patients in datasets to include the characteristics of different cardiovascular diseases and improve the accuracy of predictions. Regarding the datasets with several types of disease, we also need to establish the quantitative methodology to evaluate the variety of geometry as a training data. Secondly, we need to collect physiological information of patients to build the datasets under personalized boundary conditions. Based on this study, we aim to build a network with multiple constraints, multiple channels of input, and multiple sampling layers in parallel. It can help us use deep learning to achieve the prediction of highdimensional flow field such as fluid–structure interaction (FSI), etc. Thirdly, the uncertainty of vessel wall identification should be noted as the common limitation in imagebased analysis including CFD. Present study exhibits the flow estimation on the point clouds which generated on segmented blood vessel. Then, the flow field strongly depends on the quality of vessel wall segmentation. Though CFD results from the same STL file can exhibit good consistency among different research groups^{49}, still segmentation process from DICOM images can lead to variability in geometry^{50}. To overcome this uncertainty of real geometry, the establishment of stable segmentation method or normalization of hemodynamic parameters will be required.
Methods
Ethics approvals
The experimental scheme and related details of this study were approved by the Institutional Ethics Committee of People’s Hospital (Beijing, China) and Tohoku University (Sendai, Miyagi, Japan). All experiments were carried out in accordance with relevant guidelines and regulations. We explained research content to the subjects in detail and obtained their written informed consent.
Data collection
The patient data used in this study was based on the project ‘Biomechanics study on quantitative relationships between coronary artery stenosis and myocardial ischemia^{51,52,53}, which focused on the diagnosis and optimization of coronary stenosis surgical procedures. The CTA data for 110 patients with LAD stenosis who had visited the People’s hospital since 2018 was collected and collated by professional clinicians with a 128slice CT scanner (Brilliance iCT, Philips Healthcare, The Netherlands). 3D model reconstruction was also performed by the clinician. We obtained 110 STL cardiovascular models as raw data.
Model geometric parameters modification
The deep learning dataset, which only contained 110 real cardiovascular models, had a very limited amount of information, which was far from enough to represent the relationship between the geometry of the model and the corresponding hemodynamics. Therefore, based on the statistical results of previous cardiovascular morphology studies^{47,48,54,55,56,57,58,59}, the geometric parameters of the 110 original cardiovascular models were adjusted to increase the number of models. For each parameter, we randomly selected one value within the given range as the modification basis of the original model, as shown in Table 3. Based on this method, we extended one original model into nine new models, which meant that the total number of models increased to 1100, as shown in Supplementary Fig. 2.
Simulated operation of CABG and CFD simulation
After model expansion, we performed the simulation implementation of the CABG operation and the CFD simulation.
As the most patients did not have undergone CABG surgery, the virtual bypass surgery was performed except for undergone CABG case (as seen in Supplementary Fig. 3). With the agreements of clinicians, the left internal mammary artery (LIMA) with diameter of 2 mm was deployed using modeling software Mimics (Materialize NV, BE).
According to the generation of geometry, tetrahedron numerical meshes with boundary layers were generated by ANSYSMeshing (ANSYS, Canonsburg, USA). Total mesh number was selected to have the number of nodes from 2.83 to 3.01 million based on meshindependence test.
Steady flow simulation was performed on ANSYSCFX (ANSYS, Canonsburg, USA). Blood flow was simplified to be an incompressible Newtonian fluid with 1050 kg/m^{3} density and 0.0035 Pa·s viscosity. Velocity boundary of 1.125 m/s was imposed on the inlet assuming the peak wave velocity of cardiac cycle^{60}. Outlet boundary was set as zero pressure condition. Noslip condition was assigned to all wall boundaries.
More detail is summarized in the Supplementary Method.
Creation of datasets and proposal of deep learning network
Using simulation software (e.g., ANSYS, OpenFOAM, etc.), the highdensity 3D point cloud form of the four groups—preoperative, postoperative, velocity, and pressure fields—of the CFD simulation results could be directly output (i.e., they could be represented as a set of points {P_{i}  i = 1, ..., N} in space). Each point Pi was a vector containing spatial coordinates (x, y, z) and hemodynamic parameters at that point. Pi was the connection point of CFD meshes (usually called node). CFD meshes generation was only related to the geometry of the model. Therefore, the distribution of points depended only on the geometry of the model. The position of points in the model was fixed, we could not change its spatial distribution. What we could do is to directly extract and analyze the points in a certain position through the simulation software.
We divided each group of point cloud data into two sets: a training set and a test set. The training set included simulation results of 1000 cardiovascular models based on the original 100 models. In order to ensure the generalization of the deep learning network, the test set consisted of the CFD results of 100 cardiovascular models that were expanded from the 10 original models which were different from the training set. Based on this, the four groups—preoperative, postoperative, velocity, and pressure fields—of hemodynamic datasets were established, respectively. These four datasets were used independently to train four separate networks. Hence, we obtain four optimal network configurations to further predict the corresponding hemodynamics.
In the case of certain boundary conditions, the values of flow velocity and pressure at each point were jointly determined by the overall shape of the model and its specific spatial coordinates. This was also the basic principle for CFD to resolve the simulation results via the NavierStokes and continuity equations. The segmentation network structure of PointNet^{37} could realize feature extraction and hemodynamic prediction of point clouds. This study inherited the concepts of global feature and local feature proposed by the original PointNet, and optimized the network structure. Since the original PointNet had only one single input channel, global features, and local features were extracted from the same and all input points, which could help the PointNet identify the relationship between these two features. However, it was inevitable that there would be duplication between the two features, and then some effective and specific information would be lost. In order to solve this problem, a network structure with double input and double sampling channels was proposed in this study. The structure and parameter setting were shown in Fig. 5. For each model in the dataset, we extracted two types of point clouds. One was the model point cloud, which only included spatial coordinates for the outermost points of the cardiovascular model. These points could represent the global features of the overall model geometry. The second was the query point cloud, which included the remaining points inside the cardiovascular model. These points contained local features, such as the spatial coordinates of each point and its corresponding hemodynamics. The 3D deep learning network had two independent input channels that corresponded to these two point clouds. Two feature extraction parts were directly connected to their respective input and sampling channels. This effectively enhanced the extraction of effective and specific information from these two features, and improved the prediction accuracy. For the same point cloud data, the comparison of prediction MRE from the original single channel PointNet and our dual channel network was shown in Table 4. The prediction ERR of our dual channel network was lower than that of the original PointNet.
Global features were the global geometric information of the model. Local features referred to the location of each point and the corresponding flow field distribution inside the model. These two features both contained the geometric features of the same cardiovascular model (commonality). The two features also had different effective information (difference). The network needed to extract commonality and difference and learned the correlation between them to further realize the flow field prediction.
Based on the above principle, the network construction scheme was as follows:
To enhance the commonality and correlation, the first two feedforward fullyconnected layers (FC1 and FC2) of the two feature extraction sections shared weights, which meant they shared the same underlying feature extraction methods. In order to evaluate the necessity of sharing weights, we compared the network without shared weights with the results of this study, as shown in Table 5. The results showed that the shared weight could effectively reduce the prediction error. The two feature extraction sections also had independent feedforward fullyconnected layers (FC3 and FC4), which further enhanced the ability of the network to recognize the effective and specific information (difference) of global features and local features. After FC3 and FC4, the global and local features contained in the two point clouds were represented as a N * 512 and N * 128dimensional vectors, respectively. We concatenated the two vectors to form an N * 640dimensional vector. This vector contained both the global features of the model point cloud and the local features of the query point cloud, which helped the network further integrate the correlation between the two features. The last part of the network was feedforward fullyconnected layers (FC5 and FC6), which were used to yield hemodynamic results.
For other details of the network, we added a Maxpooling layer as a symmetric function in the feature extraction part of the model point cloud, which could aid in resolving the disorder of the input point cloud^{37}. We used the mean absolute error as the regression loss function^{24,61}. We used the Adam optimizer with specific parameters: learning rate = 0.001, ϵ = 0.001, ρ1 = 0.9, ρ2 = 0.999, and δ = 1E−8^{62}. The 3D deep learning network was trained by TensorFlow (v2.0.0rc, Python 3.6 on a Nvidia GeForce GTX 1080 Ti GPU). The preoperative and postoperative datasets needed to be separately trained as inputs for the network. During the training process, we saved the optimal network parameter configurations for both training sets. After that, while only inputting the spatial coordinates in the test set, the network could recognize the hemodynamic prediction output of query point cloud.
Definition of error functions
Referring to the evaluation criteria of previous studies, NMAE^{29} and MRE^{28} were defined as error functions to evaluate the accuracy of deep learning network predictions based on the CFD results, as shown in equation (1) and equation (2):
N represented the number of selected query points. i was the spatial sequence of the 3D point cloud. P_{i} and \(\hat P_i\) represented the flow velocity or pressure value at a certain point calculated by CFD and deep learning, respectively. MaxP and MinP represented the maximum and minimum magnitude of the corresponding hemodynamic parameters among all points in the selected area, respectively. NMAE can characterize the error of the deep learning prediction result relative to the true value of the overall flow field (CFD result). MRE can characterize the error of the deep learning prediction value relative to the true value at all query points of the model. The definition of the error function draws on previous studies. The comparative analysis results are shown in Supplementary Table 1. In this study, ERR is designed to evaluate the velocity or pressure fields represented by point clouds. For other parameters (such as FFR calculated by pressure field, etc.), new ERR should be defined according to the specific situation. In these definitions, each of the points of different cardiovascular parts can affect ERR with the same weight. However, the number of points and the magnitude of velocity and pressure must have a great difference among aorta, coronary arteries, and bypass graft. In order to avoid the impact of this variation on the evaluation results, local ERR (The model was divided into several parts, and the ERR value of a certain part, such as the left anterior descending branch, was called local ERR.) values were obtained to assess the prediction accuracy on smalltolarge parts. We calculated the ERR values of the proximal and distal end of left anterior descending artery (LAD), graft, right coronary artery (RA), the left circumflex branch (LCX), the aorta and superior aortic branch artery, respectively. Regarding the LAD, the proximal and distal ends were divided by stenosis. When there were multiple stenosis in the LAD branch, the stenosis with highest degree was selected.
Statistics and reproducibility
All ERR calculations were based on the velocity or pressure results of 100 models in the test set. The definition of query point cloud number (i) was defined in equation (1) and equation (2). This study took the average value of ERR of 100 models. The standard deviation was used to calculate the error bars.
Reporting summary
Further information on research design is available in the Nature Research Reporting Summary linked to this article.
Data availability
Data analyzed during the current study are available from the corresponding author upon reasonable request. Restrictions apply to the sharing of patient data that supports the findings of this study. With the approval of the Institutional Ethics Committee of People’s Hospital, the patient’s data can be authorized for use by qualified researchers. The Source data underlying the graphs and charts presented in the main figures (from Fig. 1 to Fig. 4) can be accessed at: https://doi.org/10.6084/m9.figshare.13295915.v1^{63}.
Code availability
All source code described in this project can be accessed at: https://doi.org/10.5281/zenodo.4287103^{64}.
References
Wallace, D. & Wallace, R. In RighttoWork Laws and the Crumbling of American Public Health 61–69 (Springer, Cham, 2018).
Tuppo, E. et al. Changes in coronary heart disease incidence and mortality in New Jersey 2000–2017. https://doi.org/10.7282/t3fzmb8d61 (2019).
Maddox, T. M. et al. Nonobstructive coronary artery disease and risk of myocardial infarction. JAMA 312, 1754–1763 (2014).
Saka, Y. et al. Clinical comparison study between a newly developed opticalbased fractional flow reserve device and the conventional fractional flow reserve device. Coron. Artery Dis. 31, 342–347 (2020).
Wong, C. et al. Validation of a novel ‘Wireless’ fractional flow reserve measurement during coronary angiography. Hear. Lung Circ. 27, S487 (2018).
Wang, W. et al. A fastfractional flow reserve simulation method in a patient with coronary stenosis based on resistance boundary conditions. Comput. Model. Eng. Sci. 116, 163–173 (2018).
Yue Feng, Y. L. Study on the Influence of right atrial pressure on the numerical calculation of fractional flow reserve. Mol. Cell. Biomech. 16, 31–32 (2019).
Zhang, Z., Li, K. & Tian, J. Efficacy and safety outcomes of fractional flow reserve in guiding clinical therapy of nonSTsegment elevation myocardial infarction compared with angiography alone in elderly Chinese patients. Clin. Interv. Aging 11, 1751–1754 (2016).
Shin, J. et al. Fractional flow reserve guided coronary revascularization in lung transplant recipients. J. Hear. Lung Transpl. 38, S323 (2019).
Squiers, J. J. & Mack, M. J. Coronary artery bypass graftingfifty years of quality initiatives since Favaloro. Ann. Cardiothorac. Surg. 7, 516–520 (2018).
Amin, S., Werner, R. S., Madsen, P. L., Krasopoulos, G. & Taggart, D. P. Intraoperative bypass graft flow measurement with transit time flowmetry: a clinical assessment. Ann. Thorac. Surg. 106, 532–538 (2018).
Handa, T., Orihashi, K., Nishimori, H. & Yamamoto, M. Maximal blood flow acceleration analysis in the early diastolic phase for aortocoronary artery bypass grafts: a new transittime flow measurement predictor of graft failure following coronary artery bypass grafting. Surg. Today 46, 1325–1333 (2016).
Zhuang, B., Wang, S., Zhao, S. & Lu, M. Computed tomography angiographyderived fractional flow reserve (CTFFR) for the detection of myocardial ischemia with invasive fractional flow reserve as reference: systematic review and metaanalysis. Eur. Radiol. 30, 712–725 (2020).
Westra, J. et al. Diagnostic performance of in‐procedure angiography‐derived quantitative flow reserve compared to pressure‐derived fractional flow reserve: the FAVOR II Europe‐Japan study. J. Am. Heart Assoc. 7, e009603 (2018).
Zhang, M. et al. Haemodynamic effects of stent diameter and compaction ratio on flowdiversion treatment of intracranial aneurysms: a numerical study of a successful and an unsuccessful case. J. Biomech. 58, 179–186 (2017).
Zhang, M., Anzai, H., Chopard, B. & Ohta, M. Towards the patientspecific design of flow diverters made from helixlike wires: an optimization study. Biomed. Eng. Online 15, 159 (2016).
Hoi, Y. et al. Effects of arterial geometry on aneurysm growth: threedimensional computational fluid dynamics study. J. Neurosurg. 101, 676–681 (2004).
Qiao, A., Liu, Y., Li, S. & Zhao, H. Numerical simulation of physiological blood flow in 2way coronary artery bypass grafts. J. Biol. Phys. 31, 161–182 (2005).
Fu, Y., Qiao, A., Yang, Y. & Fan, X. Numerical simulation of the effect of pulmonary vascular resistance on the hemodynamics of reoperation after failure of one and a half ventricle repair. Front. Physiol. 11, 207 (2020).
Taylor, C. A., Fonte, T. A. & Min, J. K. Computational fluid dynamics applied to cardiac computed tomography for noninvasive quantification of fractional flow reserve: scientific basis. J. Am. Coll. Cardiol. 61, 2233–2241 (2013).
Li, G. et al. Pulsewavepattern classification with a convolutional neural network. Sci. Rep. 9, 1–11 (2019).
Huang, G. B., Lee, H. & LearnedMiller, E. Learning hierarchical representations for face verification with convolutional deep belief networks. 2012 IEEE Conference on Computer Vision and Pattern Recognition. 2518–2525 (2012).
Simonyan, K. & Zisserman, A. Very deep convolutional networks for largescale image recognition. arXiv:1409.1556v6. Preprint at https://arxiv.org/abs/1409.1556v6 (2015).
Jha, D. et al. Enhancing materials property prediction by leveraging computational and experimental data using deep transfer learning. Nat. Commun. 10, 1–12 (2019).
Varadarajan, A. V. et al. Predicting optical coherence tomographyderived diabetic macular edema grades from fundus photographs using deep learning. Nat. Commun. 11, 1–8 (2020).
Coenen, A. et al. Diagnostic accuracy of a machinelearning approach to coronary computed tomographic angiography–based fractional flow reserve: result from the MACHINE consortium. Circ. Cardiovasc. Imaging 11, e007217 (2018).
Wang, Z.Q. et al. Diagnostic accuracy of a deep learning approach to calculate FFR from coronary CT angiography. J. Geriatr. Cardiol. JGC 16, 42 (2019).
Guo, X., Li, W. & Iorio, F. Convolutional neural networks for steady flow approximation. In Proceedings of the 22nd ACM SIGKDD international conference on knowledge discovery and data mining, 481–490 (2016).
Liang, L., Mao, W. & Sun, W. A feasibility study of deep learning for predicting hemodynamics of human thoracic aorta. J. Biomech. 99, 109544 (2020).
Kutz, J. N. Deep learning in fluid dynamics. J. Fluid Mech. 814, 1–4 (2017).
Miyanawala, T. P. & Jaiman, R. K. An efficient deep learning technique for the NavierStokes equations: Application to unsteady wake flow dynamics. arXiv:1710.09099. Preprint at https://arxiv.org/abs/1710.09099 (2017).
Wang, Z. et al. Model identification of reduced order fluid dynamics systems using deep learning. Int. J. Numer. Methods Fluids 86, 255–268 (2018).
Lye, K. O., Mishra, S. & Ray, D. Deep learning observables in computational fluid dynamics. J. Comput. Phys. 401, 109339 (2020).
Lee, S. & You, D. Prediction of laminar vortex shedding over a cylinder using deep learning. arXiv:1712.07854v1. Preprint at https://arxiv.org/abs/1712.07854v1 (2017).
Graham, L. N., Ellison, K., Herman, B. K. & Riddell, C. S. Visualization and storage algorithms associated with processing point cloud data. U.S. Patent 7, 804498 (2010).
You, H., Feng, Y., Ji, R. & Gao, Y. Pvnet: A joint convolutional network of point cloud and multiview for 3d shape recognition. In Proceedings of the 26th ACM international conference on Multimedia 1310–1318 (2018).
Qi, C. R., Su, H., Mo, K. & Guibas, L. J. Pointnet: Deep learning on point sets for 3d classification and segmentation. In Proceedings of the IEEE conference on computer vision and pattern recognition 652–660 (2017).
Pijls, N. H. et al. Measurement of fractional flow reserve to assess the functional severity of coronaryartery stenoses. N. Engl. J. Med. 334, 1703 (1996).
Lee, S. & You, D. Datadriven prediction of unsteady flow over a circular cylinder using deep learning. J. Fluid Mech. 879, 217–254 (2019).
Duriez, T., Brunton, S. L. & Noack, B. R. Machine Learning Controltaming Nonlinear Dynamics and Turbulence Ch. 6 (Springer, Cham. Press, Switzerland, 2017).
Ling, J., Kurzawski, A. & Templeton, J. Reynolds averaged turbulence modelling using deep neural networks with embedded invariance. J. Fluid Mech. 807, 155–166 (2016).
Layton, W. & Lenferink, H. W. J. A multilevel mesh independence principle for the Navier–Stokes equations. SIAM J. Numer. Anal. 33, 17–30 (1996).
Frey, P. J. & Alauzet, F. Anisotropic mesh adaptation for CFD computations. Comput. Methods Appl. Mech. Eng. 194, 5068–5082 (2005).
Boutsianis, E. et al. CFD and PTV steady flow investigation in an anatomically accurate abdominal aortic aneurysm. J. Biomech. Eng. 131, 011008–011023 (2008).
Vinoth, R. et al. Steady and Transient Flow CFD Simulations in an Aorta Model of Normal and Aortic Aneurysm Subjects. In The Proceedings of the International Conference on Sensing and Imaging. 29–43 (2019).
Martin, J. D. CFD Analysis comparing steady flow and pulsatile flow through the aorta and its main branches. ASME 2016 International Mechanical Engineering Congress and Exposition. 3, IMECE201667155, V003T04A064 (2016).
Itu, L. et al. A machinelearning approach for computation of fractional flow reserve from coronary computed tomography. J. Appl. Physiol. 121, 42–52 (2016).
Tesche, C. et al. Coronary CT angiography–derived fractional flow reserve: machine learning algorithm versus computational fluid dynamics modeling. Radiology 288, 64–72 (2018).
Radaelli, A. G. et al. Reproducibility of haemodynamical simulations in a subjectspecific stented aneurysm model—a report on the Virtual Intracranial Stenting Challenge 2007. J. Biomech. 41, 2069–2081 (2008).
Berg, P. et al. Multiple aneurysms anatomy challenge 2018 (MATCH): phase I: segmentation. Cardiovasc. Eng. Technol. 9, 565–581 (2018).
Ge, X. et al. Modelbased analysis of the sensitivities and diagnostic implications of FFR and CFR under various pathological conditions. Int. J. Numer. Method. Biomed. Eng. https://doi.org/10.1002/cnm.3257 (2019).
Li, B., Wang, W., Mao, B. & Liu, Y. A method to personalize the lumped parameter model of coronary artery. Int. J. Comput. Methods 16, 1842004 (2019).
Wang, W. et al. Numerical simulation of instantaneous wavefree ratio of stenosed coronary artery. Int. J. Comput. Methods 16, 1842009 (2019).
Roy, S., Gupta, A., Nanrah, B. K., Verma, M. & Saha, R. Morphometric study of left coronary artery trunk in adult human cadavers: a study on the eastern region population. J. Clin. diagnostic Res. JCDR 8, 7 (2014).
Kassab, G. S. & Fung, Y.C. B. The pattern of coronary arteriolar bifurcations and the uniform shear hypothesis. Ann. Biomed. Eng. 23, 13–20 (1995).
Malkawi, A. H. et al. Morphology of aortic arch pathology: implications for endovascular repair. J. Endovasc. Ther. 17, 474–479 (2010).
Canaud, L. et al. Proximal fixation of thoracic stentgrafts as a function of oversizing and increasing aortic arch angulation in human cadaveric aortas. J. Endovasc. Ther. 15, 326–334 (2008).
Campens, L. et al. Reference values for echocardiographic assessment of the diameter of the aortic root and ascending aorta spanning all age categories. Am. J. Cardiol. 114, 914–920 (2014).
Hager, A. et al. Diameters of the thoracic aorta throughout life as measured with helical computed tomography. J. Thorac. Cardiovasc. Surg. 123, 1060–1066 (2002).
Febina, J., Sikkandar, M. Y. & Sudharsan, N. M. Wall Shear Stress Estimation of Thoracic Aortic Aneurysm Using Computational Fluid Dynamics. Comput. Math. Methods Med. https://doi.org/10.1155/2018/7126532 (2018).
Chai, T. & Draxler, R. R. Root mean square error (RMSE) or mean absolute error (MAE)?–Arguments against avoiding RMSE in the literature. Geosci. Model Dev. 7, 1247–1250 (2014).
Kingma, D. P. & Ba, J. Adam: A Method for Stochastic Optimization. Comput. Sci. https://doi.org/10.1063/1.4902458 (2014).
Li, G. Figures’ source data.xlsx. figshare. Dataset. https://doi.org/10.6084/m9.figshare.13295915.v1 (2020).
Li, G. 3D Cardiovascular systerm flow prediction. Zenodo. https://doi.org/10.5281/zenodo.4287103 (2020).
Acknowledgements
This research is partially supported by the Creation of a development platform for implantable/wearable medical devices by a novel physiological data integration system of the Program on Open Innovation Platform with Enterprises, Research Institute and Academia (OPERA) from the Japan Science and Technology Agency (JST). This work is also supported by the JSPS KAKENHI with the Grant Number JP18K18355, the GrantinAid [A] (No16H01805), the GrantinAid [C] (17K01444), the GrantinAid [C] (19K04163), the National Natural Science Foundation of China (11772015), and the National Natural Science Foundation of China (11832003, 11772016).
Author information
Authors and Affiliations
Contributions
G.L., H.W., Y.L., and A.Q. acquired the data. G.L., H.W., M.Z., S.T., A.Q., M.O., Y.L., and H.A. created and designed this study; G.L., H.W., M.Z., S.T., and H.A. performed the experiments and analyzed the data. All of the authors discussed and coauthored the manuscript. The contributions of G.L. and H.W. were equal.
Corresponding author
Ethics declarations
Competing interests
The authors declare no competing interests.
Additional information
Publisher’s note Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
Supplementary information
Rights and permissions
Open Access This article is licensed under a Creative Commons Attribution 4.0 International License, which permits use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons license, and indicate if changes were made. The images or other third party material in this article are included in the article’s Creative Commons license, unless indicated otherwise in a credit line to the material. If material is not included in the article’s Creative Commons license and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder. To view a copy of this license, visit http://creativecommons.org/licenses/by/4.0/.
About this article
Cite this article
Li, G., Wang, H., Zhang, M. et al. Prediction of 3D Cardiovascular hemodynamics before and after coronary artery bypass surgery via deep learning. Commun Biol 4, 99 (2021). https://doi.org/10.1038/s42003020016381
Received:
Accepted:
Published:
DOI: https://doi.org/10.1038/s42003020016381
This article is cited by

Recent developments in modeling, imaging, and monitoring of cardiovascular diseases using machine learning
Biophysical Reviews (2023)

Angular difference in human coronary artery governs endothelial cell structure and function
Communications Biology (2022)
Comments
By submitting a comment you agree to abide by our Terms and Community Guidelines. If you find something abusive or that does not comply with our terms or guidelines please flag it as inappropriate.