Introduction

Currently, trajectory prediction methods have made significant progress in the fields of computer vision and machine learning1. These methods include techniques based on physical models2,3,4,5, statistical models, machine learning6 and deep learning7. Physical and statistical models can utilize the physical properties of the target and historical trajectory data to make predictions, while machine learning and deep learning methods make predictions by learning the modal and temporal dependencies of large amounts of trajectory data. In addition, some studies have fused multimodal information into trajectory prediction to improve accuracy8,9,10,11,12. Although some results have been achieved, they still face challenges such as complex scenarios, multi-target interactions, and uncertainty, which require further research and improvement.

Trajectory prediction research in China started earlier, and the traditional methods mainly focus on the trajectory prediction methods based on techniques such as rules, statistics and machine learning. For example, trajectory prediction methods based on mathematical models such as Kalman filter and particle filter are widely used in the fields of target tracking and traffic management13,14,15. In recent years, domestic scholars began to adopt deep learning methods for trajectory prediction research. By using techniques such as convolutional neural network (CNN), recurrent neural network (RNN) and attention mechanism, the accuracy and robustness of trajectory prediction have been improved. Meanwhile, some trajectory prediction models for specific scenarios have also achieved good results, such as traffic flow prediction and pedestrian behavior prediction16,17,18. Foreign trajectory prediction research emphasizes the mining and analysis of large-scale real data. By collecting and utilizing a large amount of data such as mobile devices and traffic monitoring systems, researchers apply machine learning and deep learning methods for trajectory prediction. In addition, fusing other data sources such as geographic information and social networks has become a trend. They build relevant models based on factors such as historical behavioral patterns, environmental characteristics and social interactions of the target individuals and use them to predict future movement trajectories. These include modeling methods based on Markov Decision Process (MDP) and Markov Random Field (MRF), etc.

Kalman filtering algorithm is a classical method for trajectory prediction. Kalman filtering algorithm is characterized by recursive computation, which can efficiently handle real-time data streams and is suitable for application scenarios that require real-time updating of state estimates.It can effectively suppress the influence of measurement noise and improve the accuracy and precision of landslide monitoring. In addition, Kalman filter can also detect the possible occurrence of landslide in advance by predicting the future state change trend19,20,21. Lu put forward application of Kalman filter model in the landslide deformation forecast22. Li put forward research on prediction of metro surface deformation based on ensemble Kalman filter23. However, it suffers from deficiencies such as fixed filtering parameters, so it is proposed to optimize Kalman filtering with an improved Gray Wolf algorithm.

Improving the Gray Wolf optimization algorithm

Grey Wolf Optimization (GWO) is an optimization algorithm inspired by the social behavior of grey wolves. The algorithm has some significant advantages and some potential disadvantages. First, one of the advantages of Grey Wolf Optimization algorithm is its simplicity and ease of implementation. The principles of the algorithm are relatively simple and easy to understand and implement. Second, the Gray Wolf Optimization algorithm has a strong global search capability. Inspired by the prey-seeking behavior of grey wolf packs, the algorithm is able to effectively explore potential optimal solutions in the entire search space. This makes it perform well in unconstrained continuous optimization problems and is able to find globally optimal solutions. In addition, the Gray Wolf Optimization algorithm typically has a fast convergence rate. By simulating the collaborative and competitive behavior of wolves to update the solution vector, the algorithm is able to quickly converge to a better solution, thus speeding up the optimization process. Finally, the Gray Wolf Optimization algorithm has some parameter adaptivity. It can automatically adjust the parameter values in the search process to improve the robustness and performance of the algorithm. This adaptability can adapt to the characteristics of different problems and reduce the difficulty of parameter adjustment. However, the Gray Wolf Optimization algorithm also has some potential drawbacks. First, it is more sensitive to the constraints of the problem. When dealing with constrained optimization problems, additional processing is required to ensure the feasibility of the results. Second, although the Gray Wolf Optimization algorithm has good global search capability, it may still fall into local optimal solutions in some complex problems. Finally, parameter tuning in the algorithm is relatively difficult; different problems may require different parameter settings, and experimentation and debugging are needed to obtain better performance.

Taking \(\alpha\) as the optimal solution (individual's fitness is optimal), the second best solution \(\beta\), the best solution \(\delta\), and the remaining candidate solution named \(\omega\). The hunting process is guided by \(\alpha \beta \delta\) and \(\omega\) follows these three wolves. That is, we always go to the three best solutions and then search around the region with the aim of finding better solutions and then updating \(\alpha \beta \delta\).

During the hunting process, the behavior of the gray wolves rounding up their prey is defined as follows:

Formula for the distance between an individual and a hunt:

$$D = \left| {C \cdot X_{P} (t) - X(t)} \right|$$
(1)

Gray Wolf location update formula:

$$X\left( {t + 1} \right) = X_{P} \left( t \right) - A \cdot D$$
(2)

coefficient vector:

$$A = 2a \cdot r_{1} - a$$
(3)
$$C = 2 \cdot r_{2}$$
(4)

where t is the number of iterations, D is a vector of distances between individuals and hunts, '\(\cdot\)' is not a dot product, it is a multiplication, \(X_{P}\) is a vector of hunts' locations, X is a vector of Gray Wolf locations, \(a\) is a control parameter (decreasing linearly from 2 to 0 with the number of iterations), and r1 and r2 are random vectors, modulo a random number between (0 and 1).

From the formula, it can be seen that after moving the Gray Wolf pack to \(\alpha\), the direction of movement is determined by its own position and the random vector C, and the movement step length is determined by the isolation distance from the Gray Wolf distance and the coefficient vector A, i.e., \(\alpha\) linear decrease implies the randomness and the magnitude of the movement step length, and the step length decreases with the number of iterations getting closer and closer to the optimal solution.

The mathematical model of individual Gray Wolf tracking prey location is described as follows:

$$\left\{ {\begin{array}{*{20}l} {D_{\alpha } = {\text{|C}}_{{1}} \cdot {\text{X}}_{\alpha } - {\text{X|}}} \hfill \\ {D_{\beta } = {\text{|C}}_{{2}} \cdot {\text{X}}_{\beta } - {\text{X|}}} \hfill \\ {D_{\delta } = {\text{|C}}_{{3}} \cdot {\text{X}}_{\delta } - {\text{X|}}} \hfill \\ \end{array} } \right.$$
(5)

where \(D_{\alpha }\), \(D_{\beta }\) and \(D_{\delta }\) denote the distances between \(\alpha\) and \(\beta\), \(\delta\) and other individuals, respectively; \(X_{\alpha } X_{\beta } X_{\delta }\) represents the current positions of \(\alpha\) and \(\beta\),\(\delta\), respectively; \(C_{1} C_{2} C_{3}\) is a random vector, and X is the current position of the Gray Wolf.

$$\left\{ {\begin{array}{*{20}l} {X_{1} = X_{\alpha } - A_{1} \cdot \left( {D_{\alpha } } \right)} \hfill \\ {X_{2} = X_{\beta } - A_{2} \cdot \left( {D_{\beta } } \right)} \hfill \\ {X_{3} = X_{\delta } - A_{3} \cdot \left( {D_{\delta } } \right)} \hfill \\ \end{array} } \right.$$
(6)
$$X_{t + 1} = \frac{{X_{1} + X_{2} + X_{3} }}{3}$$
(7)

Equation (6) defines the step length and direction of individual \(\omega\) in the wolf pack toward \(\alpha\) and \(\beta\), \(\delta\), respectively, and Eq. (7) defines the final position of \(\omega\).dierji.

The improved Gray Wolf algorithm is formulated as follows:

$$a_{1} \left( t \right) = a_{first} - \left( {a_{first} - a_{final} } \right) \cdot \sin \left( {\frac{1}{\mu }\left( {\frac{t}{{T_{\max } }}} \right)^{\lambda } \cdot \pi } \right)$$
(8)

where \(a_{first}\) and \(a_{final}\) are the initial and final values of the control parameters, respectively; \(\mu\) and \(\lambda\) are the regulation parameters, and \(T_{\max }\) is the maximum number of iterations. By adjusting the regulation parameters, the convergence speed of the local and global search of the Gray Wolf algorithm is thereby improved.

Gray Wolf optimization algorithm for improved Kalman filtering

Kalman filtering has a wide range of applications in state estimation and optimization problems, but it also has some shortcomings. Meanwhile, combining the Gray Wolf algorithm with Kalman filtering can make up for these shortcomings and bring the following benefits: firstly, Kalman filtering performs poorly in dealing with nonlinear and non-Gaussian problems, and it is easy to fall into local optimal solutions. And the Gray Wolf algorithm, as a global optimization algorithm, can provide stronger global search capability. By combining the two, the global search ability of the Gray Wolf algorithm can be used to solve the limitations of Kalman filtering in nonlinear problems, making the optimization process more comprehensive and accurate. Second, Kalman filtering is highly sensitive to measurement noise and system modeling errors. These errors may cause Kalman filtering to fail to accurately estimate the system state, thus affecting the accuracy of the optimization results. The Gray Wolf algorithm, on the other hand, can reduce the dependence on a single objective through the diversity search strategy, thus improving the robustness and stability of the algorithm. Therefore, introducing the Gray Wolf algorithm into Kalman filtering can enhance the tolerance of the algorithm to measurement noise and model error, and improve the accuracy and reliability of the optimization results.

In summary, combining the Gray Wolf algorithm with Kalman filtering can give full play to the advantages of both and make up for their respective shortcomings. Through the global search capability and robustness of the Gray Wolf algorithm, combined with the state estimation and correction process of Kalman filtering, more accurate, stable and globally optimized results can be achieved, which is suitable for state estimation and optimization tasks in complex problems and real-time systems.

In the measurement update proposed to be solved iteratively thus optimizing the calculation of the filter gain, the efficiency of the filter estimation is affected due to the presence of ambient noise effects. Therefore, an adaptive forgetting factor is introduced to correct the one-step prediction covariance matrix in real time in order to correct the filter gain matrix and improve the algorithm's target tracking efficiency. Calculate the corrected one-step prediction covariance matrix:

$$P^{\prime}_{k|k - 1} = \lambda_{k} P_{k|k - 1} = S_{k|k - 1} S_{k|k - 1}^{T}$$
(9)

where \(\lambda_{k}\) is the forgetting factor introduced at moment \(k\). The measurement covariance matrix is:

$$\Lambda_{k} = P_{Y,k|k - 1} = \frac{1}{m}\sum\limits_{i = 1}^{2n} {\left( {Y_{k|k - 1}^{i} - \mathop {\hat{Y}_{k|k - 1} }\limits^{{}} } \right)\left( {Y_{k|k - 1}^{i} - \mathop {\hat{Y}_{k|k - 1} }\limits^{{}} } \right)}^{T} + R_{k - 1}$$
(10)

The measurement covariance matrix will increase when there is uncertainty in the tracking target.

$$\Lambda_{k + 1}{\prime} = \left\{ {\begin{array}{*{20}l} {\eta_{1} \eta_{1}^{T} ,} \hfill & {k = 0} \hfill \\ {\frac{{\rho *\Lambda_{k}{\prime} + \eta_{k + 1} \eta_{k + 1}^{T} }}{1 + \rho },} \hfill & {k \ge 1} \hfill \\ \end{array} } \right.$$
(11)

where \(\eta_{k} = Y_{k} - \hat{Y}_{k|k - 1}\) is the measurement residual, \(0 \le \rho \le 1\) is the weight coefficient determined as a result of the system data, and the relationship between \(\Lambda_{k}{\prime}\) and \(\Lambda_{k}\) is:

$$\Lambda_{k}{\prime} = \tau_{k} \Lambda_{k}$$
(12)

where \(\Lambda_{k}{\prime} \tau_{k} = \max \{ 1,\frac{1}{m}tr(\Lambda_{k}{\prime} \Lambda_{k}^{ - 1} )\}\) is a scalar.

$$\lambda_{k} = \max \left\{ {1,\frac{{tr\left( {\Lambda_{k}{\prime} - R_{k} } \right)}}{{tr\left( {P_{Y,k|k - 1} - R_{k} } \right)}}} \right\}$$
(13)

The update step of the algorithm improved by the introduction of the adaptive forgetting factor is:

$$P_{k|k - 1} = S_{k|k - 1} S_{k|k - 1}^{T}$$
(14)

Covariance matrix after introduction of forgetting factor:

$$P_{k|k - 1}{\prime} = \lambda_{k} P_{k|k - 1} = S_{k|k - 1} S_{k|k - 1}^{T}$$
(15)
$$\chi_{k|k - 1}^{i} = \hat{X}_{k|k - 1} + S_{k|k - 1} \zeta_{i}$$
(16)

Covariance matrix:

$$P_{Y,k|k - 1} = \frac{1}{m}\sum\limits_{i = 1}^{2n} {Y_{k|k - 1}^{i} \left( {Y_{k|k - 1}^{i} } \right)^{T} } - \hat{Y}_{k|k - 1} \left( {\hat{Y}_{k|k - 1} } \right)^{T} + R_{k - 1}$$
(17)
$$P_{XY,k|k - 1} = \frac{1}{m}\sum\limits_{i = 1}^{2n} {\left( {\chi_{k|k - 1}^{i} - \hat{X}_{k|k - 1} } \right)\left( {Y_{k|k - 1}^{i} - \hat{Y}_{k|k - 1} } \right)}^{T}$$
(18)

Calculate the forgetting factor:

$$\Lambda_{k}{\prime} = \lambda_{k} P_{Y,k|k - 1}$$
(19)

Kalman gain:

$$K_{k} = P_{XY,k|k - 1} \left( {P_{Y,k|k - 1} } \right)^{ - 1}$$
(20)

Target status update:

$$\hat{X}_{k} = \hat{X}_{k|k - 1} + K_{k} \left( {Y_{k} - \hat{Y}_{k|k - 1} } \right)$$
(21)

State covariance matrix:

$$P_{k} = P_{k|k - 1} - K_{k} P_{Y,k|k - 1} K_{k}^{T}$$
(22)

The process noise covariance matrix Q and the measurement noise covariance matrix R in Kalman filtering are optimized by adjusting the parameters, which in turn update the time update and the measurement update of Kalman filtering.

Case study

The algorithm is applied to an open pit iron mine for monitoring. Two monitoring points are selected in the mine, and optimized Kalman filter approach based on improved Gray Wolf algorithm (IGWO-OKF) and variance compensation adaptive Kalman filter are used to process the monitoring data of open pit deformation.

Optimizing the Gray Wolf algorithm

The machine used in this design contains Intel Core i7 processor and the GPU is RTX 3060. in terms of software, matlab is used for simulation.

From Eq. (3), it can be known that the control parameter a is an important influence on the search parameter A, which directly affects the algorithm's local and global search ability. The four curves in Fig. 1 correspond to different parameter control strategies, which have different degrees of influence on the convergence speed of the algorithm and the solution results. From the analysis, the optimal regulation parameter, i.e., \(\mu = 2,\lambda = 5\), is selected.

Figure 1
figure 1

Variation curve of dynamic factors with different weights as a function of the number of iterations.

Full size image

The optimal value of the objective function found by PGWO is 13.1649, and the optimal value of the objective function found by IGWO is 7.5488e-11. Therefore, the performance and effect of IWGO are better than that of PGWO. It can be seen from Fig. 2 that the number of iterations in the parameter space, IGWO and PWGO will gradually converge to a better solution with the increase of iterations, and IGWO will use fewer iterations, with faster convergence speed and better effect.

Figure 2
figure 2

Graph of Gray Wolf iteration curves.

Full size image

IGWO optimized Kalman filtering

Variance Compensation Adaptive Kalman Filtering (VCAKF) is an improved Kalman filtering algorithm for nonlinear systems or systems with uncertainties. It improves the performance of the filter by adaptively estimating and compensating for the variance of measurement and process noise. In conventional Kalman filtering, the variance of the measurement and process noise is usually fixed, but in practical applications these may change over time or in the environment. Variance-compensated adaptive Kalman filtering better adapts to system dynamic changes and uncertainties by estimating and updating the noise variance in real time. Therefore, in this paper, the proposed improved Gray Wolf algorithm optimizes Kalman filtering with variance-compensated adaptive Kalman filtering to process the deformation monitoring data of an open pit mine, respectively.

According to the deformation data obtained by automatic deformation monitoring, the variance compensation adaptive Kalman filtering is applied to two deformation monitoring data on the hillside of an open-pit mine, and the filtered values are obtained. The data of the previous 9 periods are original data, and the data of the 10th period is predicted. Each period lasts 10 days. The results are compared with the actual observed values. The observation point is shown in Fig. 3.

Figure 3
figure 3

Layout of monitoring points.

Full size image

By comparing the actual observed values of deformation monitoring point 1 with the filtered values in Table 1, it can be seen that the maximum residual error of the variance-compensated adaptive Kalman filter in the X-direction coordinate is −0.46 mm, and the minimum value of the X-direction coordinate residual error is 0.05 mm; in the Y-direction coordinate residual error has a maximum value of 0.49 mm, and its minimum value of the coordinate residual error is 0.07 mm.

Table 1 Raw data and variance-compensated adaptive Kalman data for monitoring site 1.
Full size table

As can be seen in Table 2, comparing the filtered values of the variance-compensated adaptive Kalman filter with the actual observed values, the maximum residuals of the coordinates in the X-direction of point 2 are found to be 0.49 mm; the maximum residuals of the coordinates in the Y-direction are found to be 0.46 mm; the maximum residuals of the filtered values are found to be no more than 0.5 mm in the variance-compensated adaptive Kalman filtering of point 2, and it can be seen that The stability of the variance compensated adaptive Kalman filter is relatively strong, and the residual values are relatively stable.

Table 2 Raw data and variance-compensated adaptive Kalman data for monitoring site 2.
Full size table

Table 3, shows that comparing the actual observed value and the filtered value of the deformation monitoring point 1, it can be seen that the maximum residual value of the IGWO optimized Kalman filter in the X direction is 0.2 mm; the maximum residual value in the Y direction is 0.24 mm, and the difference of the coordinates can be seen that the IGWO optimized Kalman filter is closer to the real measurement value, and the range of error is obviously smaller, and the stability is enhanced.

Table 3 Monitoring site 1 raw data and IGWO optimized post-Kalman data.
Full size table

According to Table 4, comparing the filtered values of IGWO optimized Kalman filter with the actual observed values, it is found that the maximum residual value of the coordinates in the X direction of the No. 2 point is 0.25 mm; the maximum residual value of the coordinates in the Y direction is 0.22 mm, thus it can be seen that the stability of the IGWO optimized Kalman filter is relatively strong, and the residual value is relatively stable and has a good convergence.

Table 4 Monitoring site 2 raw data and IGWO optimized post-Kalman data.
Full size table

The following figure shows the measured and estimated data of the slope displacement trajectory. The predicted motion trajectory after simulation is shown in Fig. 4.

Figure 4
figure 4

Displacement prediction map.

Full size image

As shown in the figure the variance compensated adaptive Kalman filter starts to diverge in the fifth period, the proposed IGWO optimized Kalman filter has less filter fluctuation, more stable filter values and higher robustness than the variance compensated adaptive Kalman filter.

Conclusion

In order to improve the accuracy of landslide monitoring, a IGWO-optimized Kalman filter is proposed to optimize the covariance matrix of process noise and measurement noise in the Kalman filter using the improved Gray Wolf algorithm. The experimental results demonstrate that the use of the IGWO-optimized Kalman filter model achieves faster convergence to the actual observations and less volatility than the traditional variance-compensated adaptive Kalman filter. This indicates that the model has stronger convergence ability, better stability, and better filtering results and deformation prediction accuracy. In contrast, the variance-compensated adaptive Kalman filtering algorithm is inferior to the IGWO-optimized Kalman filtering model in terms of filtering effect and prediction, and its accuracy, consistency, and robustness are all lacking. The proposed filtering model can be applied in landslide monitoring, and in the future, it is planned to enhance the improvement on the measurement update and further optimize the Kalman filter.