Introduction

Due to the risks associated with the presence of operating errors and dynamic changes in the environment, the development of fully automated systems for robotic construction machines or construction robots is limited, although automatic control can help achieve a high precision and efficiency for many kinds of machines1,2. To promote the operation of robotic construction machines, it is necessary to enhance the environmental perception and implement intelligent assistance with automatic control for novice operators3,4,5. By optimizing the proportional integral derivative (PID) controller, Feng et al.6 proposed an improved ant colony optimization algorithm (IACO) to increase the tracking accuracy of hydraulic systems. Moreover7, a robust controller was designed using the µ-synthesis method to ensure the stability and performance of a hydraulic excavator.

Zhao et al. developed a construction robot system with physical human–robot interaction (PHRI), which could perform tasks in earthquake disaster sites8,9. In such systems, as the end effector operation may be dangerous to the operator, a novel force strategy associated with a master–slave control schematic is usually adopted.

Most engineering robot systems are actuated through hydraulic servo systems that suffer from disturbances and uncertainties. To attain high-performance trajectory tracking control in uncertain nonlinear systems, the robust adaptive control method10 has been widely used because of its flexibility and robustness. For example11, a fuzzy logic controller was designed to track the trajectory of an industrial robot with 2 degrees of freedom. Three particle swarm optimization algorithms with different cost functions were used to optimize the controller parameters. The fuzzy logic in the algorithm decreased the complexity of mathematical modelling, which is commonly a complex and time-consuming process. However, owing to the lack of comprehensive analyses, the algorithm lacked generality in multiple situations, even though satisfactory results were obtained in the specific experiments. Among such strategies, adaptive sliding mode control is a valuable control method12,13,14.

Adaptive sliding mode control is a new control strategy for nonlinear systems. This method combines the advantages of adaptive control and sliding mode control by introducing adaptive estimation into the sliding mode controller. According to the information uncertainty determined using the adaptive controller, the sliding mode controller can be adjusted to decrease the system uncertainty and conservativeness of the sliding mode control. In this manner, the system can maintain the robustness of sliding mode control to external disturbances and unmodelled dynamics, and the adaptive control strategy can help overcome the limitations of sliding mode control15,16,17,18. To suppress the motion disturbance of the actuator, a nonlinear robust dual-loop control scheme was proposed19. In addition to considering the motion disturbance of the actuator, the nonlinear characteristics and friction problems of the EHLS were considered. Furthermore, a continuous control set model predictive speed control (CCS-MPSC) based on the fast terminal cost index (FTCI) was proposed20 to enhance the system tracking performance.

Although sliding mode control is highly robust, it is based on the precise mathematical model of the object21,22,23,24. The “rough” mathematical model of the system must be predicted before the system sliding mode surface parameters and controller parameters can be determined. For nonlinear systems, the dynamics of the nonlinear function of the system must be identified before designing the sliding mode controller. The method of robot parameter identification to obtain accurate model parameters is of significance in the design of robot controllers including sliding mode controllers. To address the uncertain load disturbances of a hydraulic Stewart manipulator, sliding mode control based on the discontinuous projection adaptive law was developed to enhance the tracking performance25,26,27. Cao et al. used neural networks to adjust the switching gain online and realize system identification and parameter prediction. Moreover, the authors designed a neural network sliding mode controller for hybrid electric vehicles28. Using the Szász–Mirakyan operator as the basic function and by adjusting the polynomial coefficients through the adaptive law determined through stability analyses, a robust adaptive controller was designed to enhance the impedance control of a robot manipulator29. Other common methods to enhance the estimation performance include the use of neuro-fuzzy model auxiliary filters and time-varying parameters30,31,32,33,34.

The aim of this article is about a novel method to identificate the moment of inertia for a robot with stereo vision and robot dynamic and its application for the adaptive sliding mode control strategy of a construction robot. A satisfactory trajectory tracking controller must ensure that a robot can perform precise trajectory tracking. To this end, the dynamic characteristics of the robot must be considered35,36,37,38. To design the controller for a swing when a robot carries an object and increase the trajectory tracking accuracy, it is necessary to design a robust adaptive controller to address the uncertainty of the parameters. For a hydraulic cylinder system, the load equivalent mass is the most important element in controller design. However, for a swing, because the load mass is large and changes in real time according to the working conditions, the equivalent mass cannot be accurately obtained. Nevertheless, it can be assumed that the load does not change during the carrying process. Based on the inverse dynamic analysis of the robot arm and boom cylinders, the initial value of the equivalent mass can be obtained through system identification39. Subsequently, relatively accurate initial values of the controller parameters can be obtained, and the convergence and trajectory tracking accuracy of the controller can be enhanced.

According to the former issues, the following work was complemented: A novel nonlinear hydraulic dynamics model is established for the double actuated hydraulic cylinder system of the swing in a hydraulic robot and a sliding mode control strategy with parameter adaptive estimation is designed to improve the trajectory tracking performance. A method for estimating the initial value of the moment of inertia based on stereo vision and robot dynamics is proposed in this manuscript. The estimated initial value of the moment of inertia is used in the robust adaptive control strategy to improve the trajectory tracking performance which contributes a novel strategy SMI (sliding mode control with initial value). The main contribution of the paper includes: A method obtaining the inertial moment parameters of the robot by using stereo vision and robot dynamics is proposed. Through the discussion of the initial value of the nonlinear adaptive algorithm of swing of a new type of engineering robot, the effect of this method in improving the trajectory tracking accuracy during the robot grasping and releasing process is proved. Some specific problems in the use of this method are discussed, such as the effect of the combination of the moment of inertia parameter acquisition method and PD controller, the parameter convergence characteristics of the parameter adaptive method, and the advantages of this method compared with the results obtained by the robot dynamics method only and stereo vision only.

The paper is organized as follows: The background and related research are described in "Introduction" section. "System and Problem Description" section presents the system overview of the teleoperation system and problem description. "Sliding adaptive robust controller for the swing" section describes the sliding mode adaptive robust controller for the swing. The method to identify the system initial value is described in "Identification of the moment of inertia" section. The simulation experiment is described in "Simulation experiment" section. "Experiment" section describes the experiment performed to compare the performance of three controllers carrying objects between two points.

System and problem description

System description

The proposed teleoperation system consists of two parts: a pair of hydraulic PHRIs as the master and an engineering robot reconstructed from a hydraulic excavator as the slave, as shown in Fig. 1. Moreover, the system includes a control PC for each part.

Figure 1
figure 1

Teleoperation system.

Full size image

The four asymmetric hydraulic cylinders are driven by servo valves. The working pressure can be measured using pressure sensors installed in the rod and rod-less cavities of the hydraulic cylinder. The displacement of the working device can be determined using a displacement sensor installed outside the hydraulic cylinder.

The system uses cameras for monitoring. However, considering the limitations of the camera viewing angle and resolution, a 3D camera is used to construct a virtual scene and the work object. The general shape of the object can be determined through the 3D camera, although the volume and shape cannot be accurately identified due to the low camera resolution.

Figure 2 shows the actuation system of the swing. The double-cylinder actuated swing occupies a small space and does not need to drive the heavy chassis. The production cost of the structure increases because of the use of the double cylinders. However, because multi-point transmission and effective force transmission can be realized by the hydraulic system, this design exhibits a low energy consumption and compact structure.

Figure 2
figure 2

Swing DOF in the construction robot.

Full size image

Problem description

The moment of inertia parameter of the swing varies considerably when the work object is carried and unloaded. However, the parameter has a constant value during one procedure. This article designs a sliding adaptive robust trajectory tracking controller for the swing of a hydraulic construction robot. However, the strategy suffers from a low convergence speed when the moment of inertia parameter varies. Therefore, a moment of inertia identification algorithm is used.

The main problem can be divided into the following subproblems.

  1. 1.

    According to the dynamic characteristics of the robot, the gravity compensation algorithm is used to determine the weight of the object according to the information provided by the arm and boom force sensors. The shape and size information of the working object is obtained through computer vision. These data points are combined with the weight information to determine the inertia of the object relative to the swing.

  2. 2.

    To effectively suppress the uncertainties and disturbances, a sliding mode adaptive controller is designed for the hydraulic servo system of the swing. By combining the control strategy and initial value obtained by determining the moment of inertia, the parameter convergence and trajectory tracking performance can be enhanced.

Sliding adaptive robust controller for the swing

Mathematical model of the swing

The dynamic model for the hydraulic system shown in Fig. 2 is

$$F_{1} l_{1} \sin \theta_{1} + F_{2} l_{2} \sin \theta_{2} = m{\ddot{l}}_{1} + I{\ddot{\theta }} + m{\ddot{l}}_{2} + C{\dot{\theta }}$$
(1)

where m is the mass of the cylinder rod, I is the equivalent mass of the robotic arm and the cylinders, F1 and F2 denote the load forces of the cylinders, and C is the damping coefficient for swing.

In this case,

$$\begin{aligned} & \left( {P_{1} A_{1} - P_{2} A_{2} } \right)l_{1} \sin \theta_{1} + \left( {P_{1} A_{2} - P_{2} A_{1} } \right)l_{2} \sin \theta_{2} \\ & \quad = m\ddot{l}_{1} + I\ddot{\theta } + m\ddot{l}_{2} + {\text{C}}\dot{\theta } \\ \end{aligned}$$
(2)

where A1 and A2 are the areas of the two sides of the piston, and P1 and P2 denote the pressures in the two chambers of the cylinder.

As is shown in Fig. 3, ignoring manufacturing tolerances, when \(\theta { = 0}\), \(\angle {\text{A}}_{{1}} {\text{OB}}_{{1}} { = }\angle {\text{A}}_{2} {\text{OB}}_{2} { = }\theta_{0}\). Suppose that \(\angle {\text{A}}_{{1}} {\text{OB}}_{{1}} { = }\theta_{1}\), \(\angle {\text{A}}_{2} {\text{OB}}_{2} { = }\theta_{2}\) ,during the rotating process, the length of the hydraulic cylinder can be obtained by the law of cosines

$$l_{1} = \sqrt {l_{a1}^{2} + l_{a0}^{2} - 2l_{a1} l_{a0} \cos \theta_{1} }$$
(3)
$$l_{2} = \sqrt {l_{a1}^{2} + l_{a0}^{2} - 2l_{a1} l_{a0} \cos \theta_{2} }$$
(4)

where \(\theta_{1} = \theta_{0} - \theta\), \(\theta_{2} = \theta_{0} { + }\theta\).

Figure 3
figure 3

The swing schematic.

Full size image

The result calculated using the MATLAB program is shown in Fig. 4. The lengths l1 and l2 are small and exhibit similar amplitudes when the swing DOF varies from the centre position θ = 0°to the θ = 60° position.

$${\text{A}}_{1} l_{1} \sin \theta_{1} {\text{ + A}}_{2} l_{2} \sin \theta_{2} \approx {\text{A}}_{2} l_{1} \sin \theta_{1} + {\text{A}}_{1} l_{2} \sin \theta_{2} \approx {\text{A}}_{{\text{s}}}$$
(5)

and

$$\Delta l_{1} \approx \Delta l_{2} \approx y,$$
(6)
Figure 4
figure 4

l1 and l2 when the swing rotates. (Kinematics model of the double actuated swing).

Full size image

Subsequently, (2) can be written as

$$(P_{1} - P_{2} )A_{s} = 2m\ddot{y} + I\ddot{\theta } + {\text{C}}\dot{\theta }$$
(7)

Considering leakage and compressibility, the dynamics of the cylinder oil flow can be expressed as17

$$\begin{gathered} \frac{{V_{1} \dot{P}_{1} }}{{\beta_{e} }} = - A_{s} \dot{x}_{l} + Q_{1} - C_{t} (P_{1} - P_{2} ) \hfill \\ \frac{{V_{2} \dot{P}_{2} }}{{\beta_{e} }} = A_{s} \dot{x}_{l} - Q_{2} + C_{t} (P_{1} - P_{2} ) \hfill \\ \end{gathered}$$
(8)

where Ct is the internal leakage coefficient, V1 and V2 denote the total fluid volumes of the hydraulic cylinders, βe is the bulk modulus of the fluid, and Q1 and Q2 are the fluid flow rates of the cylinders. The displacement of the spool valve xv is

$$\begin{aligned} & Q_{1} { = }k_{q} x_{v} \sqrt {\Delta P_{1} } \\ & Q_{2} { = }k_{q} x_{v} \sqrt {\Delta P_{2} } \\ \end{aligned}$$
(9)

where kq is the flow gain coefficient of the servo valve.

The dynamics of servo valve have been incorporated in the controller design. Here, the valve dynamics are neglected and the servo valve opening xv is proportional to the control input, since a high-response servo valve is used.

$$x_{v} = k_{a} u$$
(10)

where ka is the servo amplifier gain and u is the servo valve control input signal.

The state variables are defined as

$${\mathbf{x}} = \left[ {\begin{array}{*{20}c} {x_{1} } & {x_{2} } & {x_{3} } & {x_{4} } \\ \end{array} } \right] = \left[ {\begin{array}{*{20}c} y & {\dot{y}} & {P_{1} } & {P_{2} } \\ \end{array} } \right]$$
(11)

According to (7–10), the dynamics are expressed in a state-space form as

$$\begin{aligned} & \dot{x}_{1} = x_{2} \\ & \dot{x}_{2} = \frac{1}{M}\left[ {(x_{3} - x_{4} )A_{s} - Bx_{2} - d} \right] \\ & \dot{x}_{3} = \frac{{\beta_{e} }}{{V_{0} }}\left( { - A_{s} x_{2} + k_{q} k_{a} \sqrt {\Delta P_{1} } u - C_{t} (x_{3} - x_{4} )} \right) \\ & \dot{x}_{4} = \frac{{\beta_{e} }}{{V_{0} }}\left( {A_{s} x_{2} { - }k_{q} k_{a} \sqrt {\Delta P_{2} } u{ + }C_{t} (x_{3} - x_{4} )} \right) \\ \end{aligned}$$
(12)

where B is the damping coefficient for cylinder, V0 is the total fluid volume of the two cylinders V0 = V1 = V2, and As is the total area of the cylinders.

M is the equivalent load mass obtained from the three terms at the right side of Eq. (7). M can be acquired by the \(M\ddot{l}_{1} = I\ddot{\theta }\) and the second order differential of (3) or (4). As M does not vary greatly when \(\left| \theta \right| < 60^{ \circ }\) , for simplify reason, M is set as a constant in the simulation experiment.

The time derivative of the second expression in (12) is

$$\dddot y = \frac{1}{M}\left[ {(\dot{x}_{3} - \dot{x}_{4} )A_{s} - B\dot{x}_{2} - \dot{d}} \right]$$
(13)

In (8), the difference in the third and fourth terms is

$$\begin{aligned} \dot{x}_{3} - \dot{x}_{4} = & \frac{{\beta_{e} }}{{V_{0} }}[( - 2A_{s} x_{2} + k_{q} k_{a} (\sqrt {\Delta P_{1} } + \sqrt {\Delta P_{2} } )u \\ & - 2C_{t} (x_{3} - x_{4} )] \\ \end{aligned}$$
(14)

Thus,

$$\begin{aligned} & \dddot y = - \frac{{2\beta_{e} A_{s}^{2} x_{2} }}{{MV_{0} }} - \frac{{2\beta_{e} A_{s} }}{{MV_{0} }}C_{t} (x_{3} - x_{4} ) - \frac{B}{M}\dot{x}_{2} - \frac{{\dot{d}}}{M} + \\ & \frac{{\beta_{e} }}{{MV_{0} }}k_{q} k_{a} (\sqrt {\Delta P_{1} } + \sqrt {\Delta P_{2} } )u \\ \end{aligned}$$
(15)

Controller design

The following unknown parameter set is defined:

$$\sigma_{1} { = } - \frac{{2\beta_{e} A_{s}^{2} }}{{MV_{0} }},\sigma_{2} { = } - \frac{{2\beta_{e} A_{s} C_{t} }}{{MV_{0} }},\sigma_{3} = - \frac{B}{M},\sigma_{4} { = } - \frac{1}{M},\sigma_{5} = \frac{{\beta_{e} }}{{MV_{0} }}k_{q} k_{a}$$

The state-space Eq. (15) is linearly parameterized as40

$$\begin{aligned} & f = \sigma_{1} x_{2} + \sigma_{2} \left( {x_{3} - x_{4} } \right){ + }\sigma_{3} \dot{x}_{2} + \sigma_{4} \dot{d} \\ & g = \sigma_{5} (\sqrt {\Delta P_{1} } + \sqrt {\Delta P_{2} } ) \\ \end{aligned}$$
(16)

Function f can be expressed as \(f = \hat{f} + \Delta f\), where \(\hat{f}\) is the nominal part, and Δf is the uncertain part, bounded as

$$\left| {\Delta f({\mathbf{x}},t)} \right| \le F({\mathbf{x}},t)$$
(17)
$$\hat{f} = \hat{\sigma }_{1} x_{2} + \hat{\sigma }_{2} \left( {x_{3} - x_{4} } \right){ + }\hat{\sigma }_{3} \dot{x}_{2} + \hat{\sigma }_{4} \dot{d}$$
(18)

where F is a known function.

The control gain g is confined to a certain constant range.

$$0 < \beta_{\min } < g < \beta_{\max }$$
(19)

The estimated value of g is

$$\hat{g} = \hat{\sigma }_{5} (\sqrt {\Delta P_{1} } + \sqrt {\Delta P_{2} } )$$
(20)

The control objective is to ensure that y asymptotically tracks the desired trajectory yd.

The tracking error is defined as

$$\tilde{y} = y - y_{d}$$
(21)

The sliding mode surface is defined as

$$s = \left( {\frac{d}{dt} + \lambda } \right)^{2} \tilde{y} = \ddot{\tilde{y}} + 2\lambda \dot{\tilde{y}} + \lambda^{2} \tilde{y}$$
(22)

Differentiating s with respect to t yields

$$\begin{aligned} \dot{s} = & \dddot y - \dddot y_{d} + 2\lambda \ddot{\tilde{y}} + \lambda^{2} \dot{\tilde{y}} \\ = & f + gu - \dddot y_{d} + 2\lambda \ddot{\tilde{y}} + \lambda^{2} \dot{\tilde{y}} \\ \end{aligned}$$
(23)

The Lyapunov function candidate is defined as

$$V(s) = \frac{1}{2}s^{2}$$
(24)

Thus,

$$\begin{aligned} \dot{V} = & s\dot{s} = s\{ \sigma_{1} x_{2} + \sigma_{2} (x_{3} - x_{4} ) + \sigma_{3} \dot{x}_{2} + \sigma_{4} \dot{d} \\ & + \left[ {\sigma_{5} (\sqrt {\Delta P_{1} } + \sqrt {\Delta P_{2} } )u - \dddot y_{d} + 2\lambda \ddot{\tilde{y}} + \lambda^{2} \dot{\tilde{y}}} \right]\} \\ \end{aligned}$$
(25)

The control input u is

$$\begin{aligned} & u = \frac{1}{{\hat{g}}}(\hat{u} - v) \\ & \hat{u} = - \hat{f} + \dddot y_{d} - 2\lambda \ddot{\tilde{y}} - \lambda^{2} \dot{\tilde{y}} \\ & v = ksat\left( {\frac{s}{\phi }} \right) \\ \end{aligned}$$
(26)

where f is the boundary layer, k is the control gain, and sat(Δ) is the saturation function, defined as

$${\text{sat}}(\Delta ) = \left\{ {\begin{array}{*{20}c} {1,{\kern 1pt} \quad \Delta > 1,} \\ {\Delta ,\quad \left| \Delta \right| < 1,} \\ { - 1,\quad {\kern 1pt} \Delta < - 1,} \\ \end{array} } \right.$$
(27)

To obtain the adaptation parameters, the following Lyapunov function is defined:

$$V_{1} (s) = \frac{1}{2}s^{2} + \frac{1}{2}\sum\limits_{i = 1}^{5} {\gamma_{i} \hat{\sigma }_{i}^{2} }$$
(28)

where \(\hat{\sigma }_{i}\) is the estimated value of \(\sigma_{i}\).

The estimation error is

$$\tilde{\sigma }_{i} { = }\sigma_{i} - \hat{\sigma }_{i}$$
(29)

The time derivative of V1 is

$$\begin{aligned} \dot{V}_{1} = & s\dot{s}{ + }\sum\limits_{i = 1}^{5} {\gamma_{i} \hat{\sigma }_{i} \dot{\hat{\sigma }}_{i} } = - \hat{\sigma }_{1} \left[ {s\sigma_{1} x_{2} { - }\gamma_{1} \dot{\hat{\sigma }}_{1} } \right] \\ & + \hat{\sigma }_{2} \left[ {s\left( {x_{3} - x_{4} } \right) - \gamma_{2} \dot{\hat{\sigma }}_{2} } \right] + \hat{\sigma }_{3} (\dot{x}_{2} - \gamma_{3} \dot{\hat{\sigma }}_{3} ) \\ & { + }\hat{\sigma }_{4} ({\text{s}}\dot{\hat{d}} + \gamma_{4} \dot{\hat{\sigma }}_{4} )\left[ {\sigma_{5} (\sqrt {\Delta P_{1} } + \sqrt {\Delta P_{2} } )u - \gamma_{5} \dot{\hat{\sigma }}_{5} } \right] \\ & - ksat\left( {\frac{s}{\phi }} \right)s + \sigma_{4} \dot{\hat{d}}s \\ \end{aligned}$$
(30)

To ensure that \(\dot{V}_{1} < 0\) , the adaptation laws are set as

$$\begin{aligned} & \dot{\hat{\sigma }}_{1} = \Pr oj_{{\hat{\sigma }_{1} }} \left( { - \frac{{s\sigma_{1} x_{2} }}{{\gamma_{1} }}} \right) \\ & \dot{\hat{\sigma }}_{2} = \Pr oj_{{\hat{\sigma }_{2} }} \left( { - \frac{{s\left( {x_{3} - x_{4} } \right)}}{{\gamma_{2} }}} \right) \\ & \dot{\hat{\sigma }}_{3} = \Pr oj_{{\hat{\sigma }_{3} }} \left( {\frac{{\hat{\sigma }_{3} \dot{x}_{2} }}{{\gamma_{3} }}} \right) \\ & \dot{\hat{\sigma }}_{4} { = }\Pr oj_{{\hat{\sigma }_{4} }} \left( { - \frac{{{\text{s}}\dot{\hat{d}}}}{{\gamma_{4} }}} \right) \\ & \dot{\hat{\sigma }}_{5} = \Pr oj_{{\hat{\sigma }_{5} }} \left( {\frac{{\sigma_{5} (\sqrt {\Delta P_{1} } + \sqrt {\Delta P_{2} } )us}}{{\gamma_{5} }}} \right) \\ \end{aligned}$$
(31)

Function Proj() is a discontinuous projection, defined as

$$\Pr oj_{{\hat{\sigma }_{{\text{i}}} }} ( \bullet ) = \left\{ {\begin{array}{*{20}l} {0{\kern 1pt} } \hfill & {{\kern 1pt} if\quad \hat{\sigma }_{{\text{i}}} = \sigma_{{{\text{imax}}}} \& \bullet > 0,} \hfill \\ 0 \hfill & {if\quad {\kern 1pt} \hat{\sigma }_{{\text{i}}} = \sigma_{{{\text{imin}}}} \& \bullet < 0,} \hfill \\ \bullet \hfill & {otherwise} \hfill \\ \end{array} } \right.$$
(32)

For any adaptation function, the projection mapping ensures that

$$\hat{\sigma }_{{\text{i}}} \in \left\{ {\hat{\sigma }_{{\text{i}}} :\sigma_{{{\text{imin}}}} \le \hat{\sigma }_{{\text{i}}} \le \sigma_{{{\text{imax}}}} } \right\}$$
(33)
$$\hat{\sigma }_{{\text{i}}} \left[ {\gamma_{{\text{i}}} proj_{{\hat{\sigma }_{{\text{i}}} }} (\gamma_{{\text{i}}}^{ - 1} \Delta_{i} ) - \Delta_{i} } \right] \le 0$$
(34)

Controller (26) and adaptation laws (31) ensure that the tracking errors asymptotically converge to zero, i.e., \(\tilde{x}(t) \to 0\) as time \(t \to \infty\).

Identification of the moment of inertia

Volume and shape identification by stereo vision

To get the moment of inertia of the object, the volume and shape is needed. For the experiment in the paper, a simple method by corner and depth for a rectangle stone object is used.

Figure 5 shows an image from a scenario that the stereo camera is placed at the top of the object, a stone covered by write papers as the work object is placed at the floor with black background.

Figure 5
figure 5

The corner extracting.

Full size image

Erode algorithm is applied to to remove the interference of small targets other than the targets. For example, small spots in the red circle of the left image may bring disturbance in the following process. Binarize for the eroded image is used for the 2-D grayscale image by thresholding, thus ROI (range of interest) is obtained.

Harris algorithm is applied to the corner extracting process. The result is shown as the red points in the right image. According to the four points in the green circles, the length, width and position of the object are obtained.

Zhang’s calibration algorithm is used for Stereo calibration. After rectifying the pair of stereo images from the two cameras, BlockMatching disparity algorithm is used the get the depth of ROI41. Then the height of the object is obtained. Then the volume and shape of the object can be obtained.

Figure 6 shows the procedure of the image process. Similar methods can be used for cylindrical objects to obtain the diameter and length of the object through the corners.

Figure 6
figure 6

Image processing in computer vision.

Full size image

The moment of inertia to swing

The swing DOF when an object is being carried pertains to a robot arm system with constant parameters that cannot be identified offline. As described in "Sliding adaptive robust controller for the swing" section, the main parameters in (10) are dependent on the equivalent mass M. The equivalent mass M is composed of the mass of the swing hydraulic cylinder push rod and moment of inertia I. Therefore, I is

$$I = I_{a} + I_{b} + I_{o}$$
(35)

where Ia is the moment of inertia of the arm and folk glove, and Ib is the moment of inertia of the linkage system with the swing and boom. Ia and Ib can be accurately obtained through offline identification. Io is the moment of inertia of the object being carried, which varies in a large range and cannot be accurately calculated offline. A new method is proposed that combines the stereo vision system and hydraulic cylinder information of the boom and arm to obtain Io.

The density ρ and approximate volume V of the working object can be derived as

$$I_{o} { = }\int {\rho {\text{V}}}$$
(36)

However, the shape and volume obtained by the vision system are considerably different from those of the work object, and thus, the direct use of these parameters may lead to a low estimation accuracy. The force obtained by the force sensor is highly accurate. If the mass of the work object can be obtained considering the force of the arm and boom, the moment of inertia can be corrected to obtain more accurate moment of inertia parameters.

The dynamic model for a system composed of booms, arms and folkgloves is29

$$\begin{aligned} \left[ {\begin{array}{*{20}l} {\tau_{b} } \hfill \\ {\tau_{a} } \hfill \\ \end{array} } \right] = & \left[ {\begin{array}{*{20}l} {m_{b} l_{gb} C_{b} + m_{a} l_{b} } \hfill & {m_{a} l_{ga} C_{a} } \hfill \\ 0 \hfill & {m_{a} l_{ga} C_{a} } \hfill \\ \end{array} } \right]\left[ {\begin{array}{*{20}l} {C\theta_{b} } \hfill \\ {C\theta_{ba} } \hfill \\ \end{array} } \right]{\mathbf{g}} \\ & -\left[ {\begin{array}{*{20}l} {m_{b} l_{gb} S_{b} } \hfill & {m_{a} l_{ga} S_{a} } \hfill \\ 0 \hfill & {m_{a} l_{ga} S_{a} } \hfill \\ \end{array} } \right]\left[ {\begin{array}{*{20}l} {S\theta_{b} } \hfill \\ {S\theta_{ba} } \hfill \\ \end{array} } \right]{\mathbf{g}} \\ \end{aligned}$$
(37)

where τa and τb denote the torques of the arm and boom, respectively, as shown in Fig. 7.

Figure 7
figure 7

Boom and arm parameter.

Full size image

\(C_{b} = \cos \varphi_{b}\), \(C_{{\text{a}}} = \cos \varphi_{a}\), \(S_{b} = \sin \varphi_{b}\), \(S_{a} = \sin \varphi_{a}\), \(C\theta_{b} = \cos \theta_{b}\), \(C\theta_{ba} = \cos (\theta_{b} - \theta_{a} )\), \(S\theta_{b} = \sin \theta_{b}\),\(S\theta_{ba} = \sin (\theta_{b} - \theta_{a} )\).

The boom angle θb is

$$\theta_{b} = x_{b} - \alpha_{b1} - \alpha_{b2}$$
(38)

The arm angle θa is

$$\theta_{a} = x_{a} - \pi + \alpha_{a1} + \alpha_{a2}$$
(39)

The boom push force is

$$f_{b} = \frac{{\tau_{b} }}{{c_{b} \sin x^{\prime}_{b} }}$$
(40)

The arm push force is

$$f_{a} = \frac{{\tau_{a} }}{{c_{a} \sin \left( {x_{a} + x^{\prime}_{a} } \right)}}$$
(41)

When the robot carries an object, the torques can be formulated as,

$$\begin{aligned} \left[ {\begin{array}{*{20}c} {\tau_{b} } \\ {\tau_{a} } \\ \end{array} } \right] = & \left[ {\begin{array}{*{20}c} {\begin{array}{*{20}c} {m_{b} l_{gb} C_{b} + m_{a} l_{b} { + }m_{o} l_{o} } \\ 0 \\ \end{array} } & {\begin{array}{*{20}c} {(m_{a} l_{ga} + m_{o} l_{o} )C_{a} } \\ {(m_{a} l_{ga} + m_{o} l_{o} )C_{a} } \\ \end{array} } \\ \end{array} } \right]\left[ {\begin{array}{*{20}c} {C\theta_{b} } \\ {C\theta_{ba} } \\ \end{array} } \right]{\mathbf{g}} \\ & - \left[ {\begin{array}{*{20}c} {\begin{array}{*{20}c} {m_{b} l_{gb} S_{b} } \\ 0 \\ \end{array} } & {\begin{array}{*{20}c} {(m_{a} l_{ga} + m_{o} l_{o} )S_{a} } \\ {(m_{a} l_{ga} + m_{o} l_{o} )S_{a} } \\ \end{array} } \\ \end{array} } \right]\left[ {\begin{array}{*{20}c} {S\theta_{b} } \\ {S\theta_{ba} } \\ \end{array} } \right]{\mathbf{g }} \\ \end{aligned}$$
(42)

Thus, the following equations can be used to obtain mo

$$\begin{aligned} \tau_{a} { = } & (m_{a} l_{ga} + m_{o} l_{{{\text{g}}o}} )C_{a} C\theta_{ba} {\text{g}} \\ & - (m_{a} l_{ga} + m_{o} l_{go} )S_{a} S\theta_{ba} {\text{g}} \\ \end{aligned}$$
(43)
$$\begin{aligned} & \tau_{b} = \left( {m_{b} l_{gb} C_{b} + m_{a} l_{b} + m_{o} l_{o} } \right)C\theta_{b} {\mathbf{g}} \\ & \quad - m_{b} l_{gb} S_{b} S\theta_{b} {\mathbf{g}} - \left( {m_{a} l_{ga} + m_{o} l_{o} } \right)S_{ab} S\theta_{ba} {\mathbf{g}} \\ \end{aligned}$$
(44)

where Lgo is the distance from the centre of the object to the swing, determined using the Digiclops system.

$$m_{o} = \frac{{\tau_{a} - m_{a} l_{ga} {\text{g}}\left( {C_{a} C\theta_{ba} - S_{a} S\theta_{ba} } \right)}}{{l_{go} {\text{g(}}C_{a} C\theta_{ba} - S_{a} S\theta_{ba} )}}{ = }m_{o1}$$
(45)
$$\begin{aligned} & m_{o} = \frac{{\tau_{b} - \left( {m_{b} l_{gb} C_{b} + m_{a} l_{b} } \right)C\theta_{b} {\mathbf{g}} + }}{{l_{o} {\mathbf{g}}\left( {C\theta_{b} - S_{ab} S\theta_{ba} } \right)}} \\ & \frac{{m_{b} l_{gb} S_{b} S\theta_{b} {\mathbf{g}} + m_{a} l_{ga} S_{ab} S\theta_{ba} {\mathbf{g}}}}{{}} = m_{o2} \\ \end{aligned}$$
(46)

Although mo can be obtained using Eqs. (45) and (46), the two equations are slightly different. Equation (45) is relatively simple and must be used when possible. However, when the arm is vertical to the ground, singularity appears, and τa becomes extremely small. In this case, it is difficult to obtain the force. Therefore, the following formula is used to define mo

$$m_{o} = \left\{ {\begin{array}{*{20}l} {m_{o1} } \hfill & {\left| {\theta_{ab} } \right| < 20^{ \circ } } \hfill \\ {m_{o2} } \hfill & {else} \hfill \\ \end{array} } \right.$$
(47)

Using mo, a rough initial value of the moment of in can be obtained through the following equation:

$$I_{o1} = \rho \int_{{l_{o\min } }}^{{l_{o\max } }} {V_{1} dl}$$
(48)

where lomin and lomax denote the minimum and maximum distances of the object from the swing, respectively. The mass of the object is assumed to be evenly distributed, and volume V1 is determined through the vision information. Io1 can be used as the initial value of the parameter identification.

Notice The most remarkable feature of this paper is that the combination of gravity estimation algorithm and stereo vision algorithm can provide more accurate estimation of moment of inertia than the method based on robot dynamics alone.

Discussion about the identification method combining with PD controller

The current robot joint control mostly adopts PD control because of its simple structure and excellent dynamic performance. Because of its phase advance ability, the dynamic response performance exceeds that of nonlinear control including sliding mode control in case of large deviation. But it also has many weaknesses, such as the large initial input, being sensitive to the friction force and to parameter uncertainty. PD nonlinear control algorithm which combines PD controller and nonlinear controller was proposed in many literatures27,40.

It is very easy to prove the effect of this method on PD controller. Figure 8 shows the closed loop step response of a common open-loop plant with transfer function

$${\text{G}} = \frac{1}{{0.5s^{2} + s}}$$
(49)
Figure 8
figure 8

Effect of mass parameter on PD controller.

Full size image

A PD controller is designed with the design goal of phase margin less than 15%. PD5 represents the closed-loop step response of a PD controller with the transfer function shown in (49).The close loop of the PD controller has a closed loop response of 55° phase margin and an overshoot of 13.6%.

Under the same target, with the accuracy of the mass parameter as 80% and 60% respectively, two PD controllers are designed with the following transfer functions.

$${\text{G}} = \frac{1}{{0.4s^{2} + s}}$$
(50)

and

$${\text{G}} = \frac{1}{{0.3s^{2} + s}}$$
(51)

These PD controllers are denoted PD4 and PD3, respectively.The phase margin of PD4 is 50.1°and the overshoot is 17.9%. The phase margin of PD3 is 44.5°and the overshoot is 23.8%. This is because the actual mass parameters are larger than those designed by the controller, so the overshoot and phase angle margin are both unfavourable.

During the process of grasping and placing, the change of its moment of inertia is often more than twice that of the robot arm. It is difficult to set the moment of inertia of the identification close to the true value. At this time, the identification algorithm combining vision and gravity has more important significance.

Simulation experiment

Sine trajectory tracking simulation

To verify the effect of the proposed scheme, the following simulation experiments are performed over the MATLAB Simulink platform. Three methods compound PID control (CPID), common sliding mode control (SM) and the proposed strategy, namely, sliding mode control with initial value (SMI), as shown in Figs. 9, 10 and 11, are used in the comparison experiments.

Figure 9
figure 9

Compound PID controller (CPID).

Full size image
Figure 10
figure 10

Sliding mode controller (SM).

Full size image
Figure 11
figure 11

Sliding mode controller with initial value identification (SMI).

Full size image

The CPID controller represents a practical controller in a linear system (Fig. 9). The zero and pole points are optimized through the feedforward and feedback PID controller, and the PID parameters can be adjusted in conjunction with the Routh stability criterion.

The sliding mode controller is shown in Fig. 10. The sliding mode controller with initial value identification is shown in Fig. 11.

The input modules on the desired trajectory output, such as the position, velocity and acceleration. The trajectory is y = sin(πt).

The plant built by SimMechanics in Fig. 12 is applied in the simulation program. The load features can be adjusted by setting the mass and gravity centre in the arm of the plant. Considering the work scenario, the load parameter is a constant value after it is determined. The commonly used friction force based on the LuGre model is introduced.

$$D = \upsilon_{0} z + \upsilon_{1} \frac{dz}{{dt}} + \upsilon_{2} \dot{\theta }$$
(52)

where z is the average deformation of the bristles, υ0 is the bristle stiffness coefficient, υ1 is the micro damping coefficient, and υ2 is the viscous coefficient. Random force disturbance ranges from ± 25N is added in the plant.

Figure 12
figure 12

SimMechanics plant.

Full size image

In order to obtain the guidelines for experimentation, the realistic model of the hydraulic parallel robot manipulator has been used. The system parameters are selected based upon their actual values and are given in Table 1.

Table 1 Parameters in the simulation program.
Full size table

In the simulation program, the real value of mo is 2681 kg m2, and the calculated values of the parameters are.

$$\sigma = \left[ { - 4547, - 0.0113,0.2984,3.73 \times 10^{ - 4} ,14.48} \right]$$
(53)

The initial parameter value is input to the adaptive parameter identification module by setting the parameters in the S function. To validate the performance of the initial value identification, the initial values of M and σ are set. In SM, the initial value of mo is set as the unloading moment of inertia, 723 kg m2; thus, the initial values of σ are

$$\sigma { = [} - {16861,} - {0}{\text{.0417,1}}{.1065,}0.0014{,53}{\text{.68]}}$$
(54)

In SMI, the initial value of mo is set as 2469 kg m2; thus, the initial values of σ are

$$\sigma { = [} - {4937}{\text{.4,}} - {0}{\text{.0122,0}}{.3240,4}{\text{.05}} \times {10}^{{ - {4}}} {,15}{\text{.718]}}$$
(55)

The reference trajectory is θd = 60 × sinπt (°).

According to the simulation experiment results shown in Fig. 13, the CPID method performs better at the beginning. But as the CPID does not have the parameter estimation function, the initial and final tracking errors are similar. Many shortcomings of PD controllers have been discussed in "Discussion about the identification method combining with PD controller" section, similar situation will happen to PID or CPID in this paper.

Figure 13
figure 13

Trajectory tracking result.

Full size image

The trajectory tracking error of the SM controller is considerably smaller than the steady-state error of the CPID and near to that of SMI. The controller can be considered to be very effective only in terms of trajectory tracking function. However, due to parameter coupling14, the parameters of the SM method cannot approach the target parameters, and thus, the performance of the controller is limited especially the initial performance. As is shown in Fig. 14.

Figure 14
figure 14

Parameter identification result.

Full size image

The SMI trajectory tracking results in the later stage are the best among the three methods because of the superior parameter estimation, which was achieved with the proposed initial value estimation method. However, the parameter identification results are slightly different from the true values because of the accuracy limitations of the estimation algorithm.

Carrying experiment simulation

The effect of the new parameter identification on PD controller has been described in "Discussion about the identification method combining with PD controller" section. Since PD controller and nonlinear controller can be designed and independently and the coupling of control can be realized by decoupling, to discuss the influence of initial value of SMI is enough. The step position response is used to simulate the object carrying process.

To validate the effect of initial moment of inertia on the proposed trajectory tracking control scheme during an object carrying task, a simulation in which a square wave represents the desired trajectory is performed.

Considering that the accuracy of the visual identification algorithm is affected by many factors, such as the grab position, the orientation of the target object, the uncertainty of the object's density and even the degree of illumination, to cover all the different situation is as impossible. This experiment only discussed the effect of identification accuracy, the following controller with different parameters are discussed.

  1. 1.

    SMI with the calculate value is described in "Moment of inertia identification experiment" section which represents the real value of the inertia parameter as SMI10.

  2. 2.

    A value of which the parameter is well identified as 0.8 times the mass, representing a smaller inertia parameter as SMI8.

  3. 3.

    For the identification result under bad conditions, the moment of inertia is 0.6 times of the calculated value as SMI6.

With the upper controllers, the carrying process with square wave as desired trajectory are simulated. the friction force and disturbance are the same as "Sine Trajectory tracking simulation" section.

From the result of Figs. 15 and 16, we can conclude that: The trajectory tracking performances of each controller are different at the first ten second. The accuracy mainly depends on the distance between the initial value and the true value. But after a certain period of time, each controller performs at almost the same accuracy. The influence of initial value is very small, especially after the step process. However, there is an error in the desired value due to imperfections in the identification algorithm that cannot be completely eliminated. The error is small, approximately 1% of the desired trajectory. Considering that the nonlinear algorithm mainly aims at the steady state situation, parameter changes and uncertainties, the effectiveness of the identification method in a sliding mode algorithm is proved.

Figure 15
figure 15

Position tracking result.

Full size image
Figure 16
figure 16

Tracking error.

Full size image

During the processes of picking up and unloading objects of the carrying task, the mass parameters often change dramatically. At least two processes, carry and unload, are included and repeated. The initial value identification method can effectively improve the accuracy of the initial stage. The practical significance of the initial value identification method in this paper is proved. SMI is a practical method for the situation where the object changes frequently, but it has little impact on the situation that one constant load mass repeats many times.

The experimental results demonstrate the importance of the initial value identification method for all control strategies parameter identification. In addition to trajectory tracking, this method can be used in other fields, such as energy-saving control.

Experiment

This section describes the moment of inertia identification experiment. The system setup and implementation issues are in "Experiment setup and implementation issues" section. The process and result of the identification experiment are in "Moment of inertia identification experiment" section.

Experiment setup and implementation issues

The experimental setup consists of the following hardware components.

  1. 1.

    Construction robot reconstruction for the 10 T LiuGong excavator.

  2. 2.

    Pointgrey Research Colour Digiclops, with three Sony 1/3" progressive scan CCD cameras.

  3. 3.

    Rexroth Huade 4 WRE16-10 servo valve.

  4. 4.

    Druck PTX1400 pressure transmitters with an operating pressure range of 25 MPa.

  5. 5.

    Miran LVDT20 rod-type displacement sensor.

  6. 6.

    ADVANTECH PCI-1710UL Data acquisition card (DAQ) with 12-bit data acquisition and 100 kHz sampling rate.

The hardware of the control system consists of two PC-compatible computers that communicate through a local area network. All analogue measurement signals (cylinder position, chamber pressure, supply pressure and load force) are fed back to the slave PC through four plug-in DAQ cards.

Moment of inertia identification experiment

To test the effect of the vision information combined with the gravity recognition algorithm of moment of inertia, as described in "Controller design" section, a three-dimensional vision and gravity compensation method was applied to grasp stone.

In the experiments with a 1.0 m long stone object, the object was grabbed in the middle position (0.5 m, denoted M) or edge position (0.9 m, denoted E), and the results are compared with the torque obtained using the robot dynamics method only and the vision method only.

The moment of inertia result is Isum. According to Table 2. The identification experiment results are shown in Table 2.

Table 2 Results of identification experiments.
Full size table

In Table 2, the calculated value column shows the manually measured results, which has the highest accuracy. The identified value column shows the result with the method in this paper combined with stereo vision in "The moment of inertia to swing" section and robot dynamics in "The moment of inertia to swing" section. The dynamics only column shows the results calculated with only the robot dynamics method, and the vision only column shows the results calculated with only the stereo vision information. The volume is calculated according to the parameters from computer vision.

From Table 2 and Fig. 17, we can conclude that:

Figure 17
figure 17

Results of identification experiments.

Full size image

In this experiment, the moment of inertia of the load is three times that of the mechanical arm. The significance of identification algorithm of moment of inertia is validated.

Identified value with the method combined with stereo vision robot dynamic performs best of the three algorithms. This shows the superiority of the identification method of moment of inertia combined with vision that proposed in this paper.

Regarding the target object as a mass point, a moment of inertia can also be identified with the robot dynamic algorithm as Dynamic only. Dynamic only performs better than Vision only because the accuracy of force sensor is better than vision. But in the experiment, when the folkglove grasps the middle position and edge position of the stone, the results are the same. The robot dynamics algorithm cannot show the change of the grasp position.

Although the precision is low, Vision only is a very effective method. Considering the high price and complex structure of the force sensor and vision will be widely used in intelligent control, the method is still an effective method in the future.

Due to the influence of many factors on the visual algorithm, the relationship between its accuracy and the object needs to be further studied.

Conclusion

This study focuses on a novel moment of inertia identification algorithm combined with stereo vision and robot dynamic and the adaptive robust sliding mode control scheme with the identified initial value for trajectory tracking of swing in a construction robot.

A novel control scheme that obtains the initial value of the moment of inertia of swing through the robot gravity identification algorithm and stereo vision information is proposed. The scheme can be used to overcome the limitations associated with the low convergence speed of the parameter identification algorithm and large change in the inertia caused by the change in the work object.

Simulation and online experiments are performed to validate the effect of the novel scheme. The sine trajectory tracking simulation experiment demonstrates the superiority of the SM algorithm over the CPID algorithm and necessity of obtaining the initial value for the sliding mode algorithm in SMI. Carrying simulation experiments involving the different accuracy validate the effectiveness of the identification method in a sliding mode algorithm. Moment of inertia identification experiment validated the feasibility and necessity of the new identification method. Many issues about the application of the identification method are discussed.

The method of identifying the moment of inertia based on the combination of stereo vision and robot dynamics proposed in this paper is applicable to all control strategies including mass parameter identification.