## Abstract

This paper presents a novel physics-driven spatiotemporal regularization (STRE) method for high-dimensional predictive modeling in complex healthcare systems. This model not only captures the physics-based interrelationship between time-varying explanatory and response variables that are distributed in the space, but also addresses the spatial and temporal regularizations to improve the prediction performance. The STRE model is implemented to predict the time-varying distribution of electric potentials on the heart surface based on the electrocardiogram (ECG) data from the distributed sensor network placed on the body surface. The model performance is evaluated and validated in both a simulated two-sphere geometry and a realistic torso-heart geometry. Experimental results show that the STRE model significantly outperforms other regularization models that are widely used in current practice such as Tikhonov zero-order, Tikhonov first-order and L1 first-order regularization methods.

## Introduction

Linear regression is a widely used approach for modeling the relationship between explanatory variables * x*’s and response variable

*by the linear function,*

**y***=*

**y***+*

**Rx***, in which*

**ε***is a parameter matrix characterizing the model details. Linear regression has widespread applications in various fields such as engineering, healthcare, economics and social science, for predictive modeling, experimental design, or system optimization. Regression parameters are often estimated based on the static data set of explanatory and response variables. However, rapid advancement of distributed sensing and imaging technology brings the proliferation of high-dimensional spatiotemporal data, i.e.,*

**R***=*

**y***) and*

**y**(s, t*=*

**x***) in healthcare systems. Traditional regression is not generally applicable for predictive modeling in these complex structured systems.*

**x**(s, tFor example, Fig. 1 shows the distribution of electric potentials * y(s, t*) acquired by the ECG sensor network placed on the body surface, also named body surface potential mapping (BSPM)

^{1,2}. Medical scientists call for the estimation of electric potentials

*) on the heart surface from BSPM*

**x**(s, t*) so as to investigate cardiac pathological activities (e.g., tissue damages in the heart)*

**y**(s, t^{3,4,5,6}. However, spatiotemporally varying data and complex torso-heart geometries defy traditional regression modeling and regularization methods.

In general, high-dimensional predictive modeling (i.e., * y(s, t*) =

*) +*

**Rx**(s, t*) poses several challenges including*

**ε**(1) **Physics-based derivation of parameter matrix** * R*: Traditional regression modeling estimates parameter matrix

*based on the readily available data set of [*

**R***]. However, distributed sensing or imaging of spatiotemporal systems provides only the surface profiles*

**x**,**y***) such as BSPMs. It is often difficult to directly measure heart-surface potential mappings*

**y**(s, t*). As such, inferring*

**x**(s, t*) needs a better knowledge of parameter matrix*

**x**(s, t*. Fortunately, physical laws define the mechanisms of electrical propagation from the heart to the body surface. This, in turn, enables the derivation of parameter matrix*

**R***using physics-based principles (i.e., divergence theorem, Green’s theorem).*

**R**(2) **Ill-conditioned system**: Linear systems involving high-dimensional data * y(s, t*) and

*) are commonly ill-conditioned. This is partly caused by unobserved*

**x**(s, t*), and partly due to the fact that parameter matrix*

**x**(s, t*is rank deficient (i.e.,*

**R***rank(*) <

**R***min*{

*dim(*),

**x***dim(*)}). The condition number of

**y***(i.e.,*

**R***cond(*) = ||

**R***||||*

**R**

**R**^{−1}||) is also shown to be large in high-dimensional predictive modeling (e.g., inverse ECG problems

^{7,8}). Moreover, the derivation of

*depends, to a great extent, on deterministic physics-based principles and the numerical analysis of complex geometries but does not account for real-world uncertainties. Such uncertainties may be introduced by simplified physical assumptions, geometric variations, measurement noises and other extraneous factors. As a result, high-dimensional prediction models cannot always match satisfactorily with data from real-world experiments.*

**R**(3) **Spatiotemporal regularization**: Ill-conditioned systems make the prediction more sensitive to noise factors (e.g., * ε*) and approximation errors in parameter matrix

*. For example, measurement noises can potentially cause a small change Δ*

**R***in the observed data*

**y***). Considering the estimation of*

**y**(s, t*changes to*

**x***+ Δ*

**x***, we will have the changes in the solution expressed as . Because of the large condition number*

**x***cond(*), the pseudo-inverse solution of may be completely different. As such, there is an urgent need to develop new statistical approaches that leverage physics-based principles and observed data to account for uncertainties and tackle the ill-conditioned problems. Although

**R***) and*

**x**(s, t*) are spatially distributed and dynamically evolving over time, they have spatial and temporal correlations. Very little has been done to develop new spatial regularization methods that handle approximation errors through spatial correlations of dynamic profiles on the complex geometry (e.g., the heart surface), as well as new temporal regularization methods to increase model robustness to measurement noises and other uncertainty factors.*

**y**(s, tThis paper presents a new spatiotemporal regularization model to tackle these research challenges and address ill-condtioned problems in high-dimensional predictive modeling. Our contributions in the present investigation are as follows:

(1) High-dimensional systems involve complex geometries, which challenge the derivation of parameter matrix * R*. We developed realistic models of torso-heart geometries, numerically discretized them with the boundary element method, and then utilized physical laws (i.e., divergence theorem and Green’s theorem) to derive the parameter matrix.

(2) As physics-based models are deterministic and do not account for real-world uncertainties, we developed a physical-statistical approach that integrates physics-derived parameter matrix * R* with a spatiotemporal regularization (STRE) method to build the high-dimensional prediction model. This approach leverages data from actual experiments to improve spatial and temporal regularity of the solutions, thereby making the final prediction closer to reality.

(3) The proposed STRE model involves quadratic programming and high-dimensional data, which cannot be solved analytically. Iterative algorithms are commonly used such as the multiplicative update method which, however, requires the nonnegative constraint of * x(s, t*). As such, they are not generally applicable because the electric field involves both positive and negative potentials. We developed a new method of dipole multiplicative update, which is inspired by the dipole assumption in electrodynamic physics. This new idea overcomes the drawbacks of existing multiplicative update methods, and provides a generalized approach to solve spatiotemporal regularization problems.

(4) Few, if any, previous works focused on both spatial and temporal regularizations in inverse and forward ECG problems. We evaluated and validated the proposed STRE model in simulation as well as a real-world case study to map electric potentials from the body to the heart surface. Experimental results show that our method not only effectively tackles the ill-conditioned problems in high-dimensional predictive modeling, but also outperforms those regularization models widely used in current practice (i.e., Tikhonov zero-order, Tikhonov first-order and L1 first-order regularization methods). This research work provides a new and effective approach to investigate disease-altered electric potentials from the body to the heart surface.

The remainder of this paper is organized as follows: Section II introduces the research background. Section III presents our research methodology. Section IV describes the experimental design. Experimental results are shown in section V. Section VI concludes this paper.

## Research Background

## Ill-conditioned systems

The high-dimensional predictive model, * y(s, t*) =

*) +*

**Rx**(s, t*, where*

**ε***) and*

**x**(s, t*) are spatiotemporal data, is generally ill-conditioned. For example, the inverse ECG problem in healthcare (i.e., mapping the potential distribution on the heart surface from the body surface)*

**y**(s, t^{7,8}is ill-conditioned. The condition number of the parameter matrix

*(i.e.,*

**R***cond(*) = ||

**R***||||*

**R**

**R**^{−1}||) is a measure of relative sensitivity of the solution

*) to the observed data*

**x**(s, t*) (i.e., ), which is shown to be large in prediction models that involve high-dimensional data and complex structured systems. The large value of*

**y**(s, t*cond(*) indicates that the prediction model is highly sensitive to changes in

**R***). The pseudo-inverse solution of in traditional regression methods (i.e., ) is unreliable and sensitive to uncertainty factors. Therefore, additional physical or statistical constraints are required to guarantee the norm of the solution to be regular and increase the reliability of the high-dimensional prediction model.*

**y**(s, t## Regularization Methods

Statistical regularization models such as Tikhonov and L1 regularization methods^{7,8,9,10} were proposed to address the ill-conditioned parameter matrix * R*, increase the model reliability and improve the prediction accuracy.

The objective function of Tikhonov regularization is formulated as

while the L1 regularization is formulated as

where ||·||_{2} and ||·||_{1} denote the L2- and L1-norm, respectively, *λ* is the regularization parameter, and Γ represents the mathematical operator constraining * x(s, t*). Note that Γ is the identity matrix in zero-order Tikhonov and L1 regularization methods (also known as ridge regression

^{11}and LASSO

^{12}in statistics), which directly penalize the magnitude of the estimator.

Zero-order regularization is effective to shrink unreliable components of the estimator and achieve sparse solutions for high-dimensional predictive modeling. However, they are limited in the ability to handle measurement noises or approximation errors in ill-conditioned systems. Therefore, first-order regularization methods were proposed to address such limitations by constraining the gradient of the solution * x(s, t*). Note that Γ is a discretized gradient operator in the first-order regularization methods. One of the most commonly used gradient operators is a bidiagonal matrix

^{9,10}expressed as

which is a central-difference approximation for the first-order derivative. However, this approximation does not account for the complex geometries of space-time dynamic systems, and is only effective for one-dimensional data. Most of previous works aligned * x(s, t*) in one column as {

**x**(s_{1}|

*t*),

**x**(s_{2}|

*t*), …,

**x**(s_{N}|

*t*)}

^{T}, and then applied the bidiagonal gradient matrix. Note that the alignment of spatiotemporal data in one column is not an effective way (maybe even incorrect) to compute the spatial gradients. As such, regularization results are not as satisfactory as expected.

In the inverse ECG problem, another commonly used gradient operator is the normal derivative-operator of the potential distribution on the heart surface, Γ* x(s, t*) = ∂

*)/∂*

**x**(s, t*, where*

**n***) denotes the dynamic potential distribution on the heart surface and*

**x**(s, t*denotes the surface-normal vector*

**n**^{7,8}. However, this operator only includes the normal derivative of

*), but ignores the gradient component on the the heart surface (i.e., ∂*

**x**(s, t*)/∂*

**x**(s, t*, where*

**τ***denotes the surface-tangent vector) and does not take into account the spatial correlations between adjacent regions. It is worth mentioning that spatiotemporal data from distributed sensing and imaging are generally spatially distributed and have spatial correlations*

**τ**^{13,14}. In the existing first-order regularization methods, the gradient operator Γ does not account for the spatial correlations or complex geometries of space-time systems. Thus, it is imperative to develop new regularization models to handle the approximation errors and improve the spatial regularity of the solution in high-dimensional predictive modeling.

In addition, space-time systems are dynamically varying over time and have temporal correlations. For example, the human heart is a typical spatiotemporal system with cardiac electrical activities dynamically varying in both space and time^{15,16}. Messnarz *et al*.^{17} proposed a spatiotemporal approach to reconstruct cardiac electric potentials. Spatial correlation is addressed by a surface gradient of the solution that is approximated using a symmetric matrix. The temporal constraint is formulated on the assumption that electric potentials on the heart surface are monotonically nondecreasing during the depolarization phase. However, the geometry of heart surface is highly complex, and thus a symmetric matrix tends to be limited in the ability to approximate the surface gradient. Moreover, the nondecreasing assumption in the temporal constraint may not be generally applicable to high-dimensional predictive modeling. Thus, there is an urgent need to design a novel spatiotemporal regularization method with the ability to effectively improve the spatial and temporal regularities in space-time systems.

## Research Methodology

As shown in Fig. 2, modern industries are increasingly investing in distributed sensing and imaging technology to cope with complexity in space-time dynamic systems. This brings large amount of spatiotemporal data (e.g., potential mappings in cardiology). This section presents a new physics-driven spatiotemporal regularization (STRE) approach for high-dimensional predictive modeling. First, we derive the parameter matrix * R* by integrating the boundary element method with divergence theorem and Green’s theorem. Second, we investigate the spatial regularization that handles approximation errors through spatial correlations of dynamic profiles on the complex geometry (i.e., heart surface), as well as the temporal regularization to increase model robustness to measurement noises. Finally, we develop a new generalized method of dipole multiplicative update to solve the objective function of the proposed STRE model.

## Physics-based Derivation of Parameter Matrix *R*

The observed data * y(s, t*) are generally obtained from the surface of a complex structured system such as BSPMs. Inferring the internal dynamic variable

*) (e.g., electric potential distributions on the heart surface) of these systems depends on the high-dimensional predictive modeling*

**x**(s, twhere * R* is the parameter matrix characterizing the interrelationship between

*) and*

**x**(s, t*).*

**y**(s, tIn the human body system, the heart represents the bioelectric source, and the torso is modeled as a homogeneous and isotropic volume conductor whose boundary consists of body surface *S*_{B} and heart surface *S*_{H}^{18,19}. Electric potentials * x(s, t*) on the heart surface and

*) on the body surface are related by the Laplace’s equations derived from physics-based principles (i.e., divergence theorem and Green’s theorem). Solving for the parameter matrix*

**y**(s, t*involves tackling this Laplace’s equation and calculating complex surface integrations, which are difficult to solve analytically in realistic torso-heart geometry. Thus, boundary element method (BEM)*

**R**^{20,21}is implemented to discretize

*S*

_{B}and

*S*

_{H}into triangle meshes, and divide the surface integrals into a series of numerical integrations over the triangle elements. Thus, the parameter matrix

*is expressed as*

**R**^{18,19}

where the coefficient matrices, * A*'s and

*'s depend entirely on the torso-heart geometry. The rows of*

**M**

**A**_{BB},

**A**_{BH}and

**M**_{BH}correspond to the locations of different nodes on the body triangle-mesh

*S*

_{B}. Similarly, the rows of

**A**_{HH},

**A**_{HB}and

**M**_{HH}represent the locations of different nodes on the heart triangle-mesh

*S*

_{H}. The different columns of all the matrices correspond to locations of triangle elements on the surface of integration.

However, inferring * x(s, t*) in complex structured systems is an ill-conditioned problem, because the parameter matrix

*is often with a large condition-number*

**R**^{7,8}. Moreover, several assumptions have been made when deriving matrix

*. For examples, the human body is modeled as a homogeneous volume conductor, and geometrical variations over time are assumed to be negligible. These assumptions may not hold true in real-world situations and will introduce uncertainties when predicting*

**R***)*

**x**(s, t^{22,23}. Thus, obtaining a numerically robust solution of high-dimensional predictive modeling calls for the integration of physics-based principles with new statistical regularization methods.

## Spatial and Temporal Regularization

The spatiotemporal data acquired by distributed sensing and imaging systems are generally distributed in the space and have spatial correlations. In existing regularization methods, the constraint operator Γ or the penalty term does not account for the spatial correlations or the geometries of complex systems, but rather align the mesh nodes in one column or take the normal derivative operator. As such, they are limited in the ability to improve the spatial regularity. In this investigation, we propose to define the constraint operator Γ to be a spatial Laplacian operator Δ_{s} to overcome the drawbacks in existing methods.

The matrix Δ_{s} is computed by determining the Laplacian at each mesh node. In a two-dimensional square lattice with a lattice constant *d* as shown in Fig. 3(a), *x*_{i} denotes the value of dynamic variable * x(s, t*) at node

*p*

_{i}= (

*u*

_{i},

*v*

_{i}), where (

*u*

_{i},

*v*

_{i}) are location coordinates. According to Taylor’s theory,

*x*

_{i}is approximated as the sum of

*x*

_{0}and its derivatives at node

*p*

_{0}= (

*u*

_{0},

*v*

_{0}):

Adding the above four equations yields

Thus, the Laplacian of *x*_{0} at node *p*_{0} is expressed as

where . Finally, the surface Laplacian of this square lattice is

However, real-world geometries are complex and are generally discretized into irregularly triangulated meshes using the boundary element method^{20,21} as shown in Fig. 3(b). Unlike the 2D square lattice, the Euclidean distance between different pairs of nodes is not a constant on the 3D triangle mesh. Thus, we estimate the Laplacian at each mesh node by linear interpolation. In this 3D triangle mesh, *x*_{t}(*i*) denotes the value of dynamic variable * x(s, t*) at node

*p*

_{i}at time

*t*, and

*d*

_{ij}is the distance between

*p*

_{i}and

*p*

_{j}. Using linear interpolation, the value at the location which is along the edge of

*p*

_{i}and

*p*

_{j}, and away from

*p*

_{i}, as shown in Fig. 3(b), is expressed as

where is the average of *d*_{ij}’s over the neighbor nodes *p*_{j}’s of *p*_{i}, and these neighbors *p*_{j}’s are the vertices of the triangles that include *p*_{i} as one of the vertices. Thus, the Laplacian of *x*_{t}(*i*) at *p*_{i} in a 3D triangle mesh is defined as

where *n*_{i} is the number of neighbor nodes *p*_{j}’s of *p*_{i}; , denotes the average of over these *p*_{j}’s. According to Eq. (8), we define the elements of the Laplacian matrix Δ_{s} as

Therefore, the spatial regularity of three-dimensional triangle mesh at node *p*_{i} is defined as

where *N* is the total number of mesh nodes.

In addition, spatiotemporal data * x(s, t*) and

*) are dynamically evolving over time and have temporal correlations. However, few, if any, previous works have effectively dealt with the temporal regularization for high-dimensional predictive modeling in space-time systems (i.e.,*

**y**(s, t*) =*

**y**(s, t*) +*

**Rx**(s, t*, the two-body dynamic prediction problem). Therefore, we propose to define the temporal regularity as*

**ε**where *T* denotes the length of the overall time span of the spatiotemporal data, and *w* is a time window. Temporal correlation is stronger when two time points are close to teach other, and electric potentials at two time points that are far away from each other tend to have bigger differences. Therefore, the time window *w* is often chosen to be a small number. Adding the temporal constraints in Eq. (11) to our regularization model is conducive to increase the model robustness to measurement noises in the time domain.

## Spatiotemporal Regularization (STRE) Model

Combining the parameter matrix, spatial and temporal regularization as described in previous subsections, we formulate our STRE model by the following objective function

where *λ*_{s} and *λ*_{t} are the spatial and temporal regularization parameters, which can be chosen by the L-curve method^{24} or cross validation. By adding both the spatial and temporal regularization into the objective function, the proposed model will not only handle the approximation errors in * R*, but also increase the model robustness to measurement noises in the time domain. Therefore, it is expected that the proposed STRE method will greatly improve the performance of high-dimensional predictive modeling in space-time systems.

This objective function involves both spatial and temporal correlations, and is difficult to be solved analytically. Iterative algorithms are commonly used such as the multiplicative update method which, however, requires nonnegative constraint of * x(s, t*)

^{25,26}. As such, they are not generally applicable because both negative and positive electric potentials exist on the heart or body surface. Here, we develop a dipole multiplicative update method to solve the proposed STRE model, inspired by the dipole assumption in electrodynamic physics. In this method,

**x**_{t}is split into its positive part and negative part , which are defined as and . Thus,

**x**_{t}can be denoted as . To simplify notation, we use

**y**_{t}and

**x**_{t}to denote

*) and*

**y**(s, t*) here and later on. Then the term that only depends on*

**x**(s, t*vectx*

_{t}in the objective function becomes

where * I* is an identity matrix whose dimension is the same as the Laplacian matrix Δ

_{s}. We substitute into Eq. (13) and define

where matrix **A**^{+} and **A**^{−} are the positive and negative parts of matrix * A*, whose definition is similar to that of or . We then obtain the update rules shown in Table 1. See the detailed proof in Appendix B.

## Experimental Design

In the present investigation, the proposed STRE model is implemented to predict the time-varying distribution of electric potentials on the heart surface from real-world sensor data of electric potentials on the body surface. The model performance is evaluated and validated in both a simulated two-sphere geometry and a realistic torso-heart geometry.

## Simulation Studies in a Two-sphere Geometry

Figure 4 shows the simulated two-sphere geometry that is formed by two concentric spheres. Each sphere is triangulated with 364 triangles and 184 nodes, which generates a 184 × 184 parameter matrix * R*. A time-varying three-dimensional current dipole

**p**(

**t**) = (

*p*

_{x}(

*t*),

*p*

_{y}(

*t*),

*p*

_{z}(

*t*)) is placed at the center of the two-sphere geometry, which is defined as

where time *t* ranges from 0 ms to 300 ms. Thus, the dynamic distributions of electric potentials on the inner surface * x(s, t*) and outer surface

*) are calculated analytically by the equations*

**y**(s, t^{27}:

where *σ* = 1 is the electric conductivity inside the outer sphere, **r**_{H}(*s*) and **r**_{B}(*s*) denote the location vectors from the center to the inner and outer spheres, respectively, and *r*_{H} = 1.0 and *r*_{B} = 1.5 are the radii of the two spheres.

The proposed STRE model is implemented to predict the electric potentials on the inner sphere based on electric potentials * y(s, t*) on the outer sphere calculated by Eq. (17). Regularization parameters

*λ*

_{s}= 0.015 and

*λ*

_{t}= 0.5 are chosen by the L-curve method

^{24}, and time window

*w*is specified to be 2. In our simulation studies, Gaussian noises with mean zero and variance (i.e., ) are added to

*). Five different noise levels (i.e., 10%, 20%, 30%, 40%, 50%) are added at each time, which correspond to noises with standard deviations*

**y**(s, t*σ*

_{ε}= 0.1, 0.2, 0.3, 0.4, 0.5, respectively. At each noise level, the predicted potentials on the inner sphere will be compared with the true data (i.e., reference potentials) calculated by Eq. (16).

## Real-world Case Studies in a Realistic Torso-heart Geometry

Furthermore, we conduct experiments in the realistic torso-heart geometry, as shown in Fig. 5. The data of electric potentials (whose recording length is a complete cycle of heartbeat and *t* ranges from 0ms to 1000 ms) on the heart and body surfaces, and the torso-heart geometry are obtained from the Center for Integrative Biomedical Computing (CIBC) at the University of Utah^{28}. In this torso-heart geometry, the heart surface consists of 257 nodes and 510 triangles, while the torso surface is formed by 771 nodes and 1538 triangles. The BSPM * y(s, t*) are acquired from 367 sensors, which are located at 367 nodes on the body surface. Thus, a 367 × 257 parameter matrix

*is generated. The STRE model is implemented to predict the potential distribution on the heart surface from the BSPM*

**R***). Regularization parameters*

**y**(s, t*λ*

_{s}= 2.0 and

*λ*

_{t}= 0.005 are chosen by the L-curve method

^{24}, and the time window

*w*is specified to be 2.

Similarly, five different noise levels (i.e., 0.6%, 1.3%, 6.3%, 12.6%, 25.3%) are added to the electric potentials on the body surface * y(s, t*) to simulate the real-world uncertainties in this torso-heart geometry. The five noise levels are with standard deviations

*σ*

_{ε}= 0.005, 0.01, 0.05, 0.1, 0.2, respectively. The estimated electric potentials on the heart surface from high-dimensional predictive modeling will be benchmarked with real-world sensor data of reference potentials.

## Performance Evaluation

The performance metric, relative error (RE), is used to evaluate the model performance, i.e.,

where and * x(s, t*) denote the estimator and reference results, respectively. The performance of our STRE model is benchmarked with Tikhonov zero-order (Tikh_0th), Tikhonov first-order (Tikh_1st) and L1 first-order (L1_1st) regularization methods. In these first-order regularization methods, the matrix Γ is defined as the normal derivative operator of the electric potentials on the inner surface

^{7,8}. The methods to solve Tikhonov and L1 regularizations are described in Appendix A.

## Results and Discussions

## Experimental Results in the Two-sphere Geometry

Figure 6(a) shows the comparisons of relative error (RE) between the proposed STRE model and other regularization methods (i.e., Tikhonov zero-order, Tikhonov first-order and L1 first-order methods) in the two-sphere geometry, when there is no noise on the potential map * y(s, t*) of the outer sphere. Note that the proposed STRE model yields the RE of 0.006, which is significantly smaller than that obtained from Tikh_0th, Tikh_1st and L1_1st, which are 0.1475, 0.1026, and 0.1025, respectively.

Figure 6(b) shows the variations of RE for different regularization methods with respect to the noise level added to the potential map * y(s, t*) of the outer sphere. In the present investigation, we replicated the experiment 20 times for each noise level, and thus the resulted RE is shown with a corresponding error bar (i.e., the standard deviation of RE). When the noise level increases from

*σ*

_{ε}= 0.1 to

*σ*

_{ε}= 0.5, the RE monotonically increases for all the methods. Specifically, the RE increases from (0.0670 ± 0.00057) to (0.0769 ± 0.0034) for the proposed STRE model, from (0.1557 ± 0.00058) to (0.2080 ± 0.005) for Tikh_0th, from (0.1037 ± 0.00031) to (0.1538 ± 0.0031) for Tikh_1st, and from (0.1046 ± 0.0004) to (0.1569 ± 0.0041) for L1_1st. Notably, the STRE model yields the smallest RE for all noise levels, and achieves the slowest increase of RE with respect to the noise level among various regularization methods.

Furthermore, Fig. 7(a) shows the reference mapping of the true potential distribution on the inner sphere calculated by Eq. (16), whose value ranges from −2.5 *mV* to 2.5 *mV*. Note that the potential distribution on the inner sphere is dynamically varying over time, and Fig. (7) illustrates the mapping at *t* = 150 *ms*. Figure 7(b) shows the predicted potential mappings on the inner sphere by different methods when there is no noise on the potential map * y(s, t*) of the outer sphere. Note that the predicted potential mapping by the STRE yields a smaller RE of 0.006 compared to that of Tikh_0th (i.e., 0.1475), Tikh_1st (i.e., 0.1026) and L1_1st (i.e., 0.1025), which achieves the best performance to predict the reference potential mapping shown in Fig. 7(a). Figure 7(c) shows the predicted potential mappings on the inner sphere by different methods with noise level

*σ*

_{ε}= 0.5 in

*) of the outer sphere. Notably, the predicted potential mappings by Tikh_0th, Tikh_1st and L1_1st under this noise level show different color patterns from the results under the condition of no noise, and their RE’s are 0.208, 0.1528 and 0.1569, respectively. However, the predicted mapping by the proposed STRE model closely preserves the color patterns of the results with no noise, and yields the smallest RE of 0.0769.*

**y**(s, tAs shown in Figs 6 and 7, the proposed STRE model achieves the best performance among these regularization methods when predicting the dynamic potential distribution on the inner sphere in this two-sphere geometry. The model performance of Tikh_0th is the worst among all the methods, which is due to the fact that zero-order regularization method does not account for the spatial or temporal correlations in the data, but rather penalizes the magnitude of the estimator to achieve sparse solutions. The RE’s of Tikh_1st and L1_1st are around the same level, which is because the gradient operators of these two regularization methods are the same (i.e., the normal derivative operator). In the regular spherical geometry, the normal derivative operator does account for the spatial correlations to some extent in this simulation study, and thus these two first-order methods perform better than Tikh_0th. However, the temporal correlations are not well considered in Tikh_1st or L1_1st, and their RE’s are higher compared to that of the proposed STRE model. Experimental results show that the proposed STRE model achieves the smallest RE and increases the model robustness to measurement noises by improving both the spatial and temporal regularities of the solution.

## Experimental Results in the Realistic Torso-heart Geometry

Figure 8(a) shows the comparisons of relative error (RE) between the proposed STRE model and other regularization methods (i.e., Tikhonov zero-order, Tikhonov first-order and L1 first-order methods) in the realistic torso-heart geometry, when there is no additional noise on the potential map * y(s, t*) of the body surface. In the present investigation, our STRE model yields a much smaller RE of 0.0997 compared to that of Tikh_0th (i.e., 0.2488), Tikh_1st (i.e., 0.2839) and L1_1st (i.e., 0.2735). Note that the RE’s of all the methods in this realistic torso-heart geometry are relatively bigger compared to the results in the simulated two-sphere geometry when no extra noise is added to

*). This is mainly due to the fact that*

**y**(s, t*) are real-world BSPM data with measurement noises and other uncertainty factors in the inverse ECG problem, while that in the simulated two-sphere geometry are clean data calculated analytically by Eq. (17).*

**y**(s, tFigure 8(b) shows the variations of RE with respect to the noise level for different regularization methods. Although there are already measurement noises in the sensor data of potential map * y(s, t*) on the body surface, we added different levels of noises to increase the real-world uncertainties on

*). In the present investigation, we also replicated the experiment 20 times for each noise level, and thus each resulted RE is shown with a corresponding error bar (i.e., standard deviation of RE). When the noise level increases from*

**y**(s, t*σ*

_{ε}= 0.005 to

*σ*

_{ε}= 0.2, the RE monotonically increases for all the methods. Specifically, the RE increases from (0.2386 ± 0.0105) to (0.4933 ± 0.0175) for the proposed STRE model, from (0.5570 ± 0.0025) to (0.8521 ± 0.0086) for Tikh_0th, from (0.9720 ± 0.0115) to (2.8261 ± 0.1835) for Tikh_1st, and from (1.2481 ± 0.0082) to (2.8994 ± 0.1849) for L1_1st, respectively. It is worth mentioning that the RE’s increase dramatically when adding noises to

*) on the body surface, compared to the results in the simulated two-sphere geometry. This is mainly due to the fact that the realistic torso-heart geometry is much more complex and irregular. As such, the resulted high-dimensional prediction model tends to be more sensitive to noises. Nevertheless, our STRE model yields the smallest RE for all noise levels, and achieves the slowest increase of RE with respect to the noise level among various regularization methods in this realistic torso-heart geometry.*

**y**(s, tFurthermore, Fig. 9(a) shows the reference mappings of measured potential distribution on the heart surface, whose value ranges from −15 *mV* to 15 *mV*. Note that the potential distribution on the heart surface is dynamically varying over time, and Fig. (9) illustrates the heart-surface potential mapping when *t* = 50 *ms*. Figure 9(b) shows the predicted potential mappings on the heart surface by different methods, when there is no additional noise on the potential map * y(s, t*) of the body surface. Note that the proposed STRE yields the RE of 0.997, which is significantly smaller than that of Tikh_0th (i.e., 0.2488), Tikh_1st (i.e., 0.2839) and L1_1st (i.e., 0.2735), and yields the best performance to predict the reference potential mapping shown in Fig. 9(a). Figure 9(c) shows the predicted potential mappings by different methods with the noise level

*σ*

_{ε}= 0.005 in

*) on the body surface. It is worth mentioning that the predicted potential mappings by Tikh_0th, Tikh_1st and L1_1st under this noise level show significantly different color patterns from Fig. 9(a) and (b). Their RE’s are 0.557, 0.927 and 1.248, respectively. However, the STRE model yields the smallest RE of 0.2386 and approximately preserves the color patterns in real-world data of potential mapping on the heart surface.*

**y**(s, tAs shown in Figs 8 and 9, the proposed STRE model achieves the best performance among various regularization methods when predicting the dynamic potential distribution on the heart surface in this realistic torso-heart geometry. The inferior performance of Tikh_0th, Tikh_1st and L1_1st is due to the fact that they neither effectively address the spatial regularity in the inverse ECG problem nor take into account the temporal correlations of the space-time systems. It may be noted that the RE’s of Tikh_1st and L1_1st are higher than that of Tikh_0th, which is not the case in the simulated two-sphere geometry. This is because the realistic torso-heart geometry is more complex and irregular than the simulated two-sphere geometry. The normal derivative operator in Tikh_1st and L1_1st can address the spatial correlations to some extent in the regular two-sphere geometry, but will lead to incorrect approximations in the complex heart geometry. As such, this causes additional errors to the solution in the prediction model. The proposed STRE model effectively addresses both spatial and temporal regularities of the solution, thereby yielding the smallest RE and increasing the model robustness to measurement noises or real-world uncertainties.

## Conclusions

Advanced sensing and imaging technology lead to the proliferation of spatiotemporal data * x(s, t*) and

*). This poses significant challenges for high-dimensional predictive modeling (i.e.,*

**y**(s, t*) =*

**y**(s, t*) +*

**Rx**(s, t*) in complex systems (e.g., solving the inverse ECG problem). First, inferring*

**ε***) needs a better knowledge of parameter matrix*

**x**(s, t*that characterizes the physics-based interrelationship between*

**R***) and*

**x**(s, t*). Second, ill-conditioned systems make the predictions more sensitive to measurement noises and approximation errors in*

**y**(s, t*. Third, very little has been done to develop new spatial regularization methods that handle approximation errors, as well as new temporal regularization methods to increase model robustness to measurement noises. Thus, there is an urgent need to tackle these research challenges and address ill-conditioned problems in high-dimensional predictive modeling.*

**R**In this paper, we developed a physics-driven spatiotemporal regularization (STRE) model for predicting dynamic behaviors in space-time systems. First, we developed realistic models of torso-heart geometry, and utilized the boundary element method and physics-based principles (i.e., divergence theorem, Green’s theorem) to derive the parameter matrix * R*. Second, we developed a physical-statistical approach that integrates physics-derived parameter matrix

*with a spatiotemporal regularization method to build the high-dimensional predictive model. Third, we designed a new method of dipole multiplicative update, inspired by the dipole assumption in electrodynamic physics, to solve the generalized spatiotemporal regularization problems.*

**R**The proposed STRE model is implemented to predict potential distribution on the heart surface using BSPM data. The model performance is evaluated and validated in both a simulated two-sphere geometry and a realistic torso-heart geometry. Experimental results show that our method not only effectively tackles the ill-conditioned problems in high-dimensional predictive modeling, but also outperforms those regularization models widely used in current practice (i.e., Tikhonov zero-order, Tikhonov first-order and L1 first-order regularization methods). The present research work provides a new and effective approach to investigate disease-altered electric potentials on the heart surface in healthcare systems.

## Additional Information

**How to cite this article**: Yao, B. and Yang, H. Physics-driven Spatiotemporal Regularization for High-dimensional Predictive Modeling: A Novel Approach to Solve the Inverse ECG Problem. *Sci. Rep.* **6**, 39012; doi: 10.1038/srep39012 (2016).

**Publisher's note:** Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

## References

- 1.
Rudy, Y. & Burnes, J. E. Noninvasive electrocardiographic imaging.

*Annals of Noninvasive Electrocardiology***4**, 340–359 (1999). - 2.
Rudy, Y. Electrocardiographic imaging: a noninvasive imaging modality for characterization of intramural myocardial activation.

*Journal of Electrocardiology***32**, 1–6 (1999). - 3.
Yang, H., Kan, C., Liu, G. & Chen, Y. Spatiotemporal differentiation of myocardial infarctions.

*IEEE Transactions on Automation Science and Engineering***10**, 938–947 (2013). - 4.
Yang, H., Bukkapatnam, S. T., Le, T. & Komanduri, R. Identification of myocardial infarction (mi) using spatio-temporal heart dynamics.

*Medical Engineering & Physics***34**, 485–497 (2012). - 5.
Rudy, Y. Cardiac repolarization: insights from mathematical modeling and electrocardiographic imaging (ecgi).

*Heart Rhythm***6**, S49–S55 (2009). - 6.
Oster, H. S., Taccardi, B., Lux, R. L., Ershler, P. R. & Rudy, Y. Noninvasive electrocardiographic imaging reconstruction of epicardial potentials, electrograms, and isochrones and localization of single and multiple electrocardiac events.

*Circulation***96**, 1012–1024 (1997). - 7.
Ghosh, S. & Rudy, Y. Application of l1-norm regularization to epicardial potential solution of the inverse electrocardiography problem.

*Annals of Biomedical Engineering***37**, 902–912 (2009). - 8.
Shou, G., Xia, L., Liu, F., Jiang, M. & Crozier, S. On epicardial potential reconstruction using regularization schemes with the l1-norm data term.

*Physics in Medicine and Biology***56**, 57 (2010). - 9.
Jerosch-Herold, M., Swingen, C. & Seethamraju, R. T. Myocardial blood flow quantification with mri by model-independent deconvolution.

*Medical Physics***29**, 886–897 (2002). - 10.
Calamante, F., Gadian, D. G. & Connelly, A. Quantification of bolus-tracking mri: Improved characterization of the tissue residue function using tikhonov regularization.

*Magnetic Resonance in Medicine***50**, 1237–1247 (2003). - 11.
Hoerl, A. E. & Kennard, R. W. Ridge regression: Biased estimation for nonorthogonal problems.

*Technometrics***12**, 55–67 (1970). - 12.
Tibshirani, R. Regression shrinkage and selection via the lasso.

*Journal of the Royal Statistical Society. Series B (Methodological)*267–288 (1996). - 13.
Chen, Y. & Yang, H. Sparse modeling and recursive prediction of space-time dynamics in stochastic sensor networks.

*IEEE Transactions on Automation Science and Engineering***13**, 215–226 (2016). - 14.
Kan, C. & Yang, H. Network models for monitoring high-dimensional image profiles. In

*Proceedings of 2015 IEEE International Conference on Automation Science and Engineering (CASE)*, 1078–1083 (Gothenburg, Sweden, 2015). - 15.
Yang, H., Bukkapatnam, S. T. & Komanduri, R. Spatiotemporal representation of cardiac vectorcardiogram (vcg) signals.

*Biomedical Engineering Online***11**, 16–30 (2012). - 16.
Yang, H. Multiscale recurrence quantification analysis of spatial cardiac vectorcardiogram signals.

*IEEE Transactions on Biomedical Engineering***58**, 339–347 (2011). - 17.
Messnarz, B., Tilg, B., Modre, R., Fischer, G. & Hanser, F. A new spatiotemporal regularization approach for reconstruction of cardiac transmembrane potential patterns.

*IEEE Transactions on Biomedical Engineering***51**, 273–281 (2004). - 18.
Barr, R. C., Ramsey, M. & Spach, M. S. Relating epicardial to body surface potential distributions by means of transfer coefficients based on geometry measurements.

*IEEE Transactions on Biomedical Engineering*1–11 (1977). - 19.
Horáček, B. M. & Clements, J. C. The inverse problem of electrocardiography: A solution in terms of single-and double-layer sources on the epicardial surface.

*Mathematical Biosciences***144**, 119–154 (1997). - 20.
Yao, B., Pei, S. & Yang, H. Mesh resolution impacts the accuracy of inverse and forward ecg problems. In

*Proceedings of 2016 IEEE Engineering in Medicine and Biology Society (EMBC)*, 1–4 (Orlando, FL, 2016). - 21.
Chen, Y. & Yang, H. Numerical simulation and pattern characterization of spatiotemporal dynamics on fractal surfaces for the whole-heart modeling applications.

*European Physical Journal B (Complex Systems)***89**, 1–16 (2016). - 22.
Joseph, V. R. & Yan, H. Engineering-driven statistical adjustment and calibration.

*Technometrics***57**, 257–267 (2015). - 23.
Chang, C.-J. & Joseph, V. R. Model calibration through minimal adjustments.

*Technometrics***56**, 474–482 (2014). - 24.
Hansen, P. C. & O’Leary, D. P. The use of the l-curve in the regularization of discrete ill-posed problems.

*SIAM Journal on Scientific Computing***14**, 1487–1503 (1993). - 25.
Sha, F., Lin, Y., Saul, L. K. & Lee, D. D. Multiplicative updates for nonnegative quadratic programming.

*Neural Computation***19**, 2004–2031 (2007). - 26.
Lee, D. D. & Seung, H. S. Learning the parts of objects by non-negative matrix factorization.

*Nature***401**, 788–791 (1999). - 27.
Peters, M. & Wieringa, H. The influence of the volume conductor on electric source estimation.

*Brain Topography***5**, 337–345 (1993). - 28.
Burton, B. M.

*et al.*A toolkit for forward/inverse problems in electrocardiography within the scirun problem solving environment. In*Proceedings of 2011 IEEE Engineering in Medicine and Biology Society (EMBC)*, 267–270 (Boston, MA, 2011).

## Acknowledgements

This work is supported in part by the National Science Foundation (CMMI-1646660, CMMI-1617148, CMMI-1619648, and IOS-1146882). The authors also thank Harold and Inge Marcus Career Professorship (HY) for additional financial support.

## Author information

## Affiliations

### Complex Systems Monitoring, Modeling and Control Laboratory, The Pennsylvania State University, University Park, 16802, USA

- Bing Yao
- & Hui Yang

## Authors

### Search for Bing Yao in:

### Search for Hui Yang in:

## Contributions

H.Y. conceived the study and contributed to the design of the study, data modeling, data interpretation, and revised the manuscript. Y.B. contributed to the development of algorithms, evaluated the data, performed the data analysis, and drafted the manuscript. All authors read and approved the final manuscript.

## Competing interests

The authors declare no competing financial interests.

## Corresponding author

Correspondence to Hui Yang.

## Supplementary information

## PDF files

## Comments

By submitting a comment you agree to abide by our Terms and Community Guidelines. If you find something abusive or that does not comply with our terms or guidelines please flag it as inappropriate.

This work is licensed under a Creative Commons Attribution 4.0 International License. The images or other third party material in this article are included in the article’s Creative Commons license, unless indicated otherwise in the credit line; if the material is not included under the Creative Commons license, users will need to obtain permission from the license holder to reproduce the material. To view a copy of this license, visit http://creativecommons.org/licenses/by/4.0/