Graph Laplacian-based spectral multi-fidelity modeling

Low-fidelity data is typically inexpensive to generate but inaccurate, whereas high-fidelity data is accurate but expensive. To address this, multi-fidelity methods use a small set of high-fidelity data to enhance the accuracy of a large set of low-fidelity data. In the approach described in this paper, this is accomplished by constructing a graph Laplacian from the low-fidelity data and computing its low-lying spectrum. This is used to cluster the data and identify points closest to the cluster centroids, where high-fidelity data is acquired. Thereafter, a transformation that maps every low-fidelity data point to a multi-fidelity counterpart is determined by minimizing the discrepancy between the multi- and high-fidelity data while preserving the underlying structure of the low-fidelity data distribution. The method is tested with problems in solid and fluid mechanics. By utilizing only a small fraction of high-fidelity data, the accuracy of a large set of low-fidelity data is significantly improved.


A Gradient of the objective function
In this section, we evaluate an explicit expression for the gradient of the loss function J = J data + ωJ reg in Eq. ( 17) with respect to the parameters α α α.
The gradient of the data misfit term is given by In deriving this expression we have made use of Here δ i j is the Kronecker delta.The gradient of the regularization term is given by (A.4) Combining (A.1) and (A.3) we have the expression for the total gradient, (A.5)

B Positive-definiteness of the Hessian of the objective function
The Hessian can be obtained by differentiating the gradient in (A.1) and (A.3) with respect to the optimization parameters.This yields, where z i ≡ w w w i −u u u i ||w w w i −u u u i || • w w w i .In the equation above, the first two terms emerge from the data-misfit term, while the third term is the contribution of the regularization term.The inner product of the Hessian with an arbitrary realization of parameters denoted by α writes In the equation above, the term ∂ 2 z i ∂ α pq ∂ α uv is a symmetric fourth order tensor and therefore has real eigenvalues.We let . ., N be the smallest eigenvalue.Therefore, we have

C Solution for a canonical problem
We make the following assumptions on the data: 1.The low-fidelity data ū u u i are distributed among M distinct clusters.
2. C(i) is the cluster number for the i−th data point.Without loss of generality, we may assume that points are numbered such that C(i) = i for i = 1, . . ., M.

3.
For any point i, the high-fidelity data is related to the low-fidelity data via the transformation where δ δ δ j , j = 1, . . ., M, are the rigid translations for each cluster.
4. For i = 1, . . ., M, the high-fidelity data u u u i is available and is used to construct the data matching term.Using assumptions 2 and 3, we note that these points, Under the assumptions above, the proposed method admits a multi-fidelity solution whose error vanishes as the regularization parameter approaches zero.
Proof.In the limit of infinite data, the graph Laplacian of the low-fidelity graph yields M eigenfunctions with null eigenvalues 1 such that, That is, the j-th eigenfunction attains a value of +1 for all points that belong to the j-th cluster and a value of −1 for all other points.
For the proposed method we consider the choice K = N = M. Further, we consider the special case where the unknown parameters are given by, α jm = αδ jm .With this choice, from Eq. (C.3) and ( 16) we conclude that From Eq. ( 15) this yields, For i = 1, . . ., M, the difference between the multi-fidelity expression ( 14) and the high-fidelity data (C.2) is given by Where γ(α) ≡ exp(−2α) 1+(M−1) exp(−2α) .Using this expression in the definition of the data matching term (18), we arrive at, Using (C.4) in Eq. ( 19) we conclude that the regularization term is given by, where we have used the fact the eigenvalues for the eigenfunctions considered in this expansion are zero.Therefore the total objective function is equal to, Further, it is easily verified that Setting dJ dα = 0 to find the stationary point of the objective function, we arrive at, In the limit ω → 0, to leading order this equation yields the solution Using this solution in the expression of the data matching term we note that J data (α) = O(ω), which tends to zero as ω → 0. Also J reg (α) = O(ω 2 ln 2 ω) which also tends to zero as ω → 0. Thus both the data matching and regularization terms tend to zero as the regularization parameter tends to zero.
We now show that with choice of α, for any point indexed by i, the coordinates predicted by the multi-fidelity approximation tend to the high-fidelity coordinates as the regularization is reduced.To accomplish this we compute Therefore, Clearly this difference tends to zero as ω → 0.

D L-curve criterion for determining the regularization parameter
The L-curve is a plot of the data misfit loss J data (α α α * (ω); ω) versus the regularization loss J reg (α α α * (ω); ω) obtained after solving the minimization problem (17) for different values of regularization parameter ω.It can be used to visualize the balance between the two terms and provides a way to tune the regularization parameter.The optimal value ω * is indeed chosen to be the one corresponding to the elbow of the L-curve, i.e. the point of maximum curvature.In fact, the elbow of the curve separates the regions where the final solution is either dominated by a large data misfit error or by a large regularization loss.Therefore, any perturbation of the parameter ω * would lead to a significant increase in one loss term or the other.Because the minimization problem (17) is inexpensive to solve, we can use the L-curve criterion to determine the optimal value of ω.In

E Effect of the selection strategy of high-fidelity data
In Step 2 of the method, we propose a strategy to select the parameters values at which to acquire high-fidelity data.The strategy consists in determining the low-fidelity data points that are the closest to the centroids of the clusters arising within the low-fidelity data, and then employ the high-fidelity model to compute their counterpart.We want to compare the performance of this strategy against a random selection of the low-fidelity points for which we acquire a high-fidelity version.To do that, we consider the problem of predicting the traction on a soft body and the results obtained in Section 2.1.Then, we solve the same problem with a random selection of the parameters at which to compute high-fidelity data points.The number of high-fidelity data points used is fixed, N = 30, and the problem is solved 200 times with different random selections.Then, we compute the mean and standard deviation of the error for each component as defined in (4), and display them in Table E. 1.We notice that the errors committed with a random selection is larger than the errors attained with the proposed selection approach, but is still considerably smaller than the low-fidelity errors.This suggests that the proposed selection strategy ought to be preferred to a random choice when possible, but that the multi-fidelity method can be successfully applied even when the set of high-fidelity data is predetermined.

F Traction on a soft material with a stiff inclusion
To generate the multi-fidelity data set, we selected a value of τ = 3 • 10 −10 , which corresponds to the smallest non-zero eigenvalue, and a spectrum cutoff of K = 90.To determine the value of the regularization parameter in Eq. ( 17) we make use of the L-curve method to study the data misfit loss versus the regularization loss for different values of ω ∈ [10 −9 , 10 −6 ] (see Figure D.1).The optimal value corresponding to the elbow of the curve is found to be ω * = 8.53 • 10 −9 .The low-and high-fidelity models used to solve the elasticity problem described in Section 2.1 are finite element solvers that differ by their mesh density, i.e. the number of elements used to discretize the computational domain.In

G Aerodynamic coefficients for a family of NACA airfoils
In Section 2.2 we consider the problem of predicting the aerodynamic coefficients of 4-digit NACA airfoils for different Reynolds number and angles of attack.The multi-fidelity transformation is found with a spectrum cutoff of K = 210, a value of τ = 9.07 • 10 −4 , and a regularization parameter ω * = 8.87 • 10 −7 .The resulting multi-fidelity data points are shown in Figure 5a as a point cloud in the 3-dimensional data space of the lift, drag and moment coefficients (C L , C D and C M , respectively).
To better visualize the data at the three levels of fidelity, we select the case of the NACA 0012 airfoil profile at Reynolds Re = 6 • 10 6 .In Figure G.1 we plot the graphs of the aerodynamic coefficients versus angle of attack for the three models, including experimental data available for lift and drag coefficients 2 .We observe that the high-fidelity CFD simulations match the experimental results within 10% error for lift and 15% error for drag.In Figure G.2 we show the computational mesh used to perform these simulations.
By analyzing the lift and drag coefficients curves, we note that the multi-fidelity model could learn and retain the low-fidelity trend, and adjust the magnitudes in light of the sparse high-fidelity data.For the moment coefficient, the low-fidelity data shows a significant disagreement with the high-fidelity results.Nonetheless, the predictions of the multi-fidelity model more closely match the high-fidelity data in magnitude.The prediction of the trend is also correct for angles of attack smaller than 8 degrees.It is verified that none of the high-fidelity data points related to the NACA 0012 airfoil were used in constructing the multi-fidelity model.We conclude that the multi-fidelity model could correct the data structures in the 3-dimensional data space using the few select high-fidelity data points available, and has increased the accuracy for all low-fidelity points.

H Definition of the data space
As discussed in Section 4.2, we consider problems where the solution for a given instance of input parameters µ µ µ is a set of quantities of interest q q q(µ µ µ).
The data space u u u where the low-fidelity graph is constructed is defined as u u u(µ µ µ) = R R R(µ µ µ, q q q(µ µ µ)), with R R R being a restriction operator that extracts the appropriate components of µ µ µ and q q q.The choice of R R R is problem dependent.It should include all the variables directly related to the predictions one wants to make, i.e. the subset of the relevant components of q q q, and the parameters that would help identifying clusters and structures in the data.
For example, including bifurcation parameters in the data space is important, as small changes in their value can cause significant changes in the topology or qualitative nature of the physical solution.Similarly, parameters that identify different and distinct regimes of the solution, e.g. the Reynolds number in fluid dynamics, provide valuable information about different data points, and can help separating the point cloud.This is especially important when the performance of the low-fidelity model strongly depend upon certain parameters.If the error of the low-fidelity data significantly differs based on a parameter, adding it to the data space will make sure that the transformation (14) will act accordingly.
In Figure H.1 we show a schematic of a simple case where adding the parameter µ 1 to the data space (q 1 , q 2 ) leads to a clear separation of the data points.In this illustrative example, including the parameter in the data space allows for a better selection of the high-fidelity data to acquire and a more appropriate treatment of the two clusters.
It is important to notice that the transformation (14) does not lead to any displacements along the parameters directions, as the high-fidelity data points will have by construction the same input parameters.Hence, the multi-fidelity transformation will act only on the space of the quantities of interests, as it should.

H.1 Numerical results with augmented data space
In this Section we analyze the results of the numerical problems proposed in Section 2, when the input parameters are included in data space.
For the problem of traction on soft body (Section 2.1), with input parameters µ µ µ = [x c , y c , θ , a, b] and quantities of interest q q q = [ f 1 , f 2 , f 3 , f 4 , σ max yy ], we consider the case where the data space is formed by concatenating all inputs and all outputs, i.e. u u u = [µ µ µ, q q q].
On the other hand, for the problem of the aerodynamic coefficients of NACA airfoils (Section 2.2), the input parameters and the set of quantities of interest are µ µ µ = [η, x η , t, α, Re] and q q q = [C L , C D , C M ], respectively.In this case, we include only the Reynolds number in the data space, i.e. u u u = [Re,C L , C D , C M ].
For both problems, we apply once again the proposed multi-fidelity method considering the graph constructed in the 7/8 augmented data space.The numerical results are reported in Table H.1.We note that in these particular numerical experiments, using only the quantities of interest in the definition of the data space (that is u u u = q q q) leads to better results in almost all cases.SpecMF (u u u = q q q) 0.52

Figure C. 1 .
Figure C.1.Schematic of a canonical problem with the data set partitioned into M = 3 clusters.

Figure D. 1
we show the L-curve used to determine the regularization parameter for the traction on soft material problem in Section 2.1.

Figure D. 1 .
Figure D.1.L-curve for the traction problem in Section 2.1.The optimal value for the regularization parameter is the one corresponding to the elbow of the curve, marked with a triangle in the graph.
Figure F.1 we show a comparison between the two meshes.The low-and high-fidelity solutions of the problem show similar trends, but the low-fidelity solutions tend to underestimate the magnitude of traction field.In Figure F.2 we show a sample of low-and high-fidelity solutions for different instances of input parameters.

Figure F. 1 .Figure F. 2 .
Figure F.1.Comparison between the two meshes of the low-and high-fidelity finite element models used to solve the traction on soft material problem.

Figure G. 1 .
Figure G.1.Comparison of the curves of the lift, drag and moment coefficients versus angle of attack for the airfoil NACA 0012 at Re = 6 • 10 6 .The low-fidelity data are indicated with dashed orange line, the multi-fidelity data with a dash-dot blue line, the high-fidelity data with a solid red line, and the experimental data are marked with a green upside-down triangle.

Figure G. 2 .
Figure G.2. Example of a mesh used for the high-fidelity CFD simulations.

Figure H. 1 .
Figure H.1.Schematic of a case where adding the parameter µ 1 to the data space (q 1 , q 2 ) separates the data set into two distinct clusters, C 1 and C 2 .
without any restriction on the data misfit, we have