Upper limb modeling and motion extraction based on multi-space-fusion

Modeling and motion extraction of human upper limbs are essential for interpreting the natural behavior of upper limb. Owing to the high degrees of freedom (DOF) and highly dynamic nature, existing upper limb modeling methods have limited applications. This study proposes a generic modeling and motion extraction method, named Primitive-Based triangular body segment method (P-BTBS), which follows the physiology of upper limbs, allows high accuracy of motion angles, and describes upper-limb motions with high accuracy. For utilizing the upper-limb modular motion model, the motion angles and bones can be selected as per the research topics (The generic nature of the study targets). Additionally, P-BTBS is suitable in most scenarios for estimating spatial coordinates (The generic nature of equipment and technology). Experiments in continuous motions with seven DOFs and upper-limb motion description validated the excellent performance and robustness of P-BTBS in extracting motion information and describing upper-limb motions, respectively. P-BTBS provides a new perspective and mathematical tool for human understanding and exploration of upper-limb motions, which theoretically supports upper-limb research.

In the past decade, the field of medical robotics has gained momentum and technology has matured 1 .However, more than one billion people worldwide are disabled, and the number is growing rapidly 2 .Among them, more than 80% of people disabled due to stroke suffer from upper-limb disability, and only 10% can regain partial mobility of the upper limb after treatment 3 .Limb motion is highly correlated with neuronal activity (neural plasticity) 4,5 , which is a potential mechanism for upper-limb motion recovery 6,7 .Based on this, upper-limb rehabilitation robotics has flourished, providing strong support for upper-limb motion recovery 8 .Upper-limb rehabilitation robots have the advantages and potential of compensating for motor deficits, motion enhancement, restoration of upper-limb functions, and rehabilitation task performance [9][10][11][12] .
Despite this, there is still conflict between patients with upper limb disabilities and those undergoing medical rehabilitation.The upper-limb rehabilitation equipment (exoskeleton 6,12,13 and end-effector 12,14 ) widely accepted mode of rehabilitation, drives or compensates for the motion of the affected limb, and provides intense rehabilitative training.However, the global outbreak of COVID-19 has severely hampered the rehabilitation of disabled people 2 and has greatly reduced the efficiency of rehabilitation.Furthermore, moderate-to-severe patients cannot independently move the affected limb during the pre-rehabilitation period; they can only receive passive rehabilitation training.This model fails to play the key role of the healthy limb and is inefficient in rehabilitation.
The rehabilitation efficacy can be improved by organically integrating the upper-limb rehabilitation robot with the patient's sense of autonomy.Research on human 7,[15][16][17] and marmoset 4 brains has shown that self-directed actions can repair damaged neural circuits in the brain to some extent.Furthermore, observational learning leads to the activation of the mirror neural system in the observer's brain 18 (observed with fMRI 19 ) and had been shown to improve upper-limb motor function 18,20 .Mirroring motions 21 and observational learning are both initiated by the patient's brain; therefore, the rehabilitation effect can be reversed to act directly on the brain, thus increasing the rehabilitation efficacy.Therefore, extracting motion information from the upper limb and implementing it in the rehabilitation process can lead to optimal upper-limb rehabilitation effects.
Information regarding the motion of the human upper limb consists mainly of motion angles and holistic motion.For upper-limb rehabilitation motion, the holistic motion of the upper limb is of significant concern.The method of analyzing human upper-limb motion information can advance the research process of upper-limb motion recovery.The scope of the analysis should include the major human upper-limb motor bones: clavicle, humerus, forearm, and palm.
Numerous scholars have analyzed and explored upper-limb motion from different perspectives.Motion capture systems (e.g.VICON), which have become more popular in recent years and can provide accurate information on upper-limb motions, are generally utilized as experimental tools, and are not widely promoted because of their high cost and low interference resistance performance 22,23 .The Kinect depth camera 24 acquired 3D spatial points from different parts of the upper limb and processed the data that can be utilized to build a remote system for assessing upper-limb motility 25 , determining upper-limb kinematic parameters (joint range of motion, displacement in the local coordinate system, joint smoothness, upper-limb length, etc.) 26,27 , solving upper-limb joint motion angles [28][29][30] , combining sliding mode control algorithms for human-machine interaction 31 , applying Kalman filtering techniques to fuse multiple Kinect data and track human motion 32 , etc.While Kinect's human skeleton recognition technology offers the advantages of high flexibility, low cost, and non-invasiveness 33 , the number of human joint spatial points identified and extracted from the upper limb is very limited.Innovative approaches (multi-camera data fusion 32 and inertial sensor compensation 34,35 ) and wearable devices [36][37][38] have been developed to improve the accuracy of Kinect depth camera data and measure upper-limb motion angles.However, the simplification of the upper limb to six degrees of freedom (DOFs) and below is inconsistent with the physiological properties of the upper limb.Meanwhile, a sizable research has been conducted for modeling upperlimb motion, including dissecting shoulder motion during the design of the upper-limb exoskeleton (ARMin series [39][40][41] , HARMONY 42 , CLEVERarm 43 , WINDER 44 , ChARMin 45 , etc.) and developing Denavit-Hartenberg based models of upper limb forward and reverse kinematics and dynamics 46,47 , hybrid twist-based model of shoulder kinematics 48 , rigid body model describing the kinematics of the scapula relative to the sternum 49 , and a musculoskeletal model of the upper limb [50][51][52] , etc.The established kinematic models describe localized motions of the upper limb and thus cannot completely describe the overall upper-limb motions.
In this study, we propose a general modelling and kinematic angle-solving method for simplified models of upper limbs with 1-8 DOFs (rotational) and named Primitive-Based triangular body segment method (P-BTBS).P-BTBS defines a triangular primitive space (TPS), maps the problem in Euclidean space to TPS, is utilized for inverse solving of 1-8 DOFs of the upper limb, and provides a mathematical description of upper-limb motions.The results largely satisfied realistic motion of the upper limbs.The coordinate information of a maximum of six spatial points and only one coordinate system transformation is required to analyze and extract the upperlimb motion information.In engineering applications, the Kinect's human skeleton recognition technique is the optimal method for extracting 3D coordinates, and the six spatial points are largely consistent with those Figure 1.Overview of P-BTBS.The study targets of P-BTBS are the human upper limb (especially the abnormal upper limb, which is a global problem).The two dark blue triangles indicate the Euclidean space and Triangular primitive space (TPS), respectively.P-BTBS is aimed at dissecting upper-limb motions from the complex composition of the brain, nerves, muscles, and bones.The P-BTBS is suitable for various equipment and technologies (e.g.depth cameras and motion capture systems), and can be applied to a wide range of research areas (especially in the field of the medical robot).
identified by this technique.Therefore, the P-BTBS (Fig. 1) has generic nature of the equipment, technology, and study targets.The ingenious and convenient modelling and calculation method is in line with the physiological characteristics of the upper limb, providing new ideas for upper-limb motion research and theoretical support for upper-limb mirroring rehabilitation 21,53,54 , motor function assessment 42,55,56 , upper-limb motion recognition 36,57 , etc.In conclusion, this study achieved ingenious and convenient kinematic analysis and description that better matched the physiological characteristics of the human upper limb by utilizing multi-spatial fusion, and provided theoretical support to advance the engineering application of upper limb modelling and motion extraction in various research fields.

Sources of inspiration.
Inspired by the fact that, defect-free Sierpiński triangles can be self-assembled on a silver surface in the microscopic world 58 , this led us to realize that matter moves with the property of triangles.Then we started to think about whether this was a base motion, which was the origin of the TPS.

Simplified model of the upper limb.
As shown in Fig. 2a, the clavicle, scapula, humerus, forearm, and hand were the main components of the upper limb.The hand is composed of many bones including five metacarpals.The joint houses the main components of the upper-limb motion: the sternoclavicular joint (SC), elbow joint (EL), acromioclavicular joint (AC), glenohumeral joint (GH), scapulothoracic joint (ST), wrist joint (WR), and other joints in the palm.Both the EL and WR are composite joints.
To simulate the skeletal motions of the upper limb, the discontinuous line segments are divided, each of which is called the body segment (BS).In this article, the red, green, golden, and dark blue BSs indicate the clavicle, humerus, forearm, and metacarpal, respectively.The red, green, and golden BSs are equal to the length of the clavicle, humerus, and forearm, respectively, and the dark blue BS is equal to the length of the third metacarpal  1).
(longest metacarpal).The coordinate system was established using the International Society of Biomechanics (ISB) recommended coordinate system 59 .In numerous studies, scholars have mechanically simplified the joints of the upper limbs.The following simplifications were made in this study: 1. SC is a rotary joint 47,60 with two DOFs rotating around the X-and Y-axes (the basal joint of the entire upper limb and serves as a mechanical support).2. GH is a ball joint 47,61,62 with three DOFs rotating around the X-, Y-, and Z-axes (the largest range of motion joint in the upper limb and greatly extending upper limb motion).3. EL is a rotary joint 47,62 with one DOF rotating around the Z-axes (the composite joint structure contributes significantly to the stability of forearm motion).4. WR is a rotary joint 47,63 with two DOFs rotating around the X-and Z-axes (the composite joint structure contributes significantly to the sophisticated motions and stability of the palm).5.The clavicle is horizontal (parallel to the X-Z plane) when the arm is in its natural downward state.
The motions of the scapula are special among the limb bones because the motion of the joints connected to the scapula is somewhat complex (sliding and rotation of the SC; translation and rotation of the AC) 64 and therefore cannot be conventionally simplified.The scapula has many muscles attached to it, which are used to maintain the stability of the shoulder joint and even the head and neck position and to provide strength for the motion of the upper limbs.Therefore, the most prominent role of the scapula should be reflected in biomechanics 49 , without denying that the scapula increases the flexibility of the upper limb in terms of kinematics.In addition, the scapula is located under the skin, which makes observing its motion in a non-invasive way difficult 65 , therefore the visual motion of the scapula was not considered in this study, but the upper-limb motion was observed directly by P-BTBS.

Upper-limb modular motion model and coordinate system transformation.
The locations of all actual spatial points and bones are presented in Table 1.The length and spatial geometry relationships are shown in Fig. 3, and the angle between the clavicle and frontal plane was 20 degree 66 .The geometric features of the virtual spatial points can be obtained according to the definition of triangular spatial points, and the coordinates of the virtual spatial points are calculated using the coordinates of the actual spatial points.The spatial coordinates of P 1-2 and P 5-7 can be easily obtained as shown in Fig. 3.The spatial coordinates of P 11 and P 12 can be estimated using Eq.(1), which is obtained from the vertical and length relationships: where i = 11 or 12, l 10,13 denotes the length of vector − → L 10,13 .The local coordinate system was established using ISB's suggested method, with the origin located at point 3.As shown in Fig. 2a,c rotation w L R and translation P LORG transformation relationship exists between local and world coordinate systems.Rotational transformations follow the form of Z-Y-X Euler angles.The coordinates of spatial point 3 in the world coordinate system are (x 3 , y 3 , z 3 ) as a result of Eq. ( 2).
The rotation matrix of Euler angles as shown in Eq. ( 3), where cx = cos(x) and sx = sin(x); α, β, and γ denote the rotation angles around the Z, Y, and X axes, respectively.
In popular spatial position acquisition techniques (e.g.motion capture systems and depth cameras etc.) the posture of the world coordinate system can be adjusted.So, the singularity problem of Euler angles can therefore be mitigated by adjusting the position of the world coordinate system to avoid a 90-degree rotation of the intermediate axes.
(1)  3a and b, the coordinates of spatial point 4 are known in the local coordinate system, as shown in Eq. ( 4), where l 3,4 denotes the length of the vector − → L 3,4 .The transformation relationship for coordinate point 4 is given by Eq. ( 5), where W P 4 are the coordinates of point 4 in the world coordinate system and acquired by motion capture system.Substituting Eqs. ( 2), ( 3) and (4) into Eq.( 5), and a function of α, β, and γ can be determined.As shown in Fig. 6c and d, for the known world coordinate system β = 0°and γ = 90°.The value of α = 88.82° was determined using the Trust-Region-Dogleg algorithm.
Given the values of α, β, and γ, the local coordinates of all spatial points can be evaluated using Eq. ( 6).
The Euclidean space.The triangular transformation relationship of the simplified upper limb model is presented in Table 2.The triangles can be selected according to the DOFs required to estimate the motion angles and describe the motion of the upper limb, which reflects the generality of the model.If the required DOFs are equal to i, there exist C i 8 choices.The motion angles of the upper limb were estimated using Eqs.( 7), ( 8), ( 9), (10) and (11).The normal vectors of planes ∆ 5,8,9 , ∆ 6,8,9 , and ∆ 7,8,9 can be expressed using Eq. ( 12).
Substituting the normal vectors A, B , and C into Eq.( 13) to estimate θ 3-5 : The calculation of Eqs. ( 7) to (13) involves only trigonometric functions, which improve the speed of computer response and increase the robustness of P-BTBS.
The upper limb was divided into four segments: clavicle, humerus, forearm, and metacarpal, the lengths and numbers of which are denoted as L 1 , L 2 , L 3 , and L 4 , respectively.One body segment BS-h (h = 1, 2, 3, 4) is described by k spatial triangles, and the spatial triangle number k ranges from 1 to 3. One spatial triangle, ST (h,k) , can be characterized using three spatial points (i.e. points A, B, and C).These three points were selected in a clockwise order.As shown in Fig. 2b, the area of ST (h,k) is denoted as S (h,k), and the projected areas in the X-Y, X-Z, and Y-Z planes are S m , S n , and S u , respectively, and can be calculated using Eq. ( 14), where det(ST x ) denotes the determinant of matrix ST x .
Area S (h,k) can be estimated using the projected areas in Eq. ( 14), as shown in Eq. (15), where m , n , and u are the unit normal vectors of the X-Y-Z coordinate system.
The The area of the h-th BS can be defined as S BS-h , as shown in Eq. ( 16).
The area of each BS can be estimated using the matrix sum of squares, according to Eq. ( 16), and matrix E h was utilized to characterize the BS, as shown in Eq. ( 17), where P x = x x + y x + z x .P i and P j are the actual spatial points of BS-h, P d , …, P f are virtual spatial points of BS-h, and f i j are functions of x, y, and z.
Vector E P h denotes the third column of matrix E h , as shown in Eq. ( 18).
The body segment matrix E O in Euclidean space can be determined using Eq. ( 19), where L h denotes the length of the h-th BS.

Triangular primitive space (TPS).
As shown in Fig. 2b, the basic physical quantities in the coordinate system M-N-U in the TPS are the projected areas of the spatial triangle in the X-Y, X-Z, and Y-Z planes, respectively, as shown in Eq. (20).Therefore, a point P TPS in the TPS is equivalent to a spatial triangle in Euclidean space.
The metric distance between point P TPS (M i , N i , U i ) and origin point O M-N-U can be estimated using Eqs.( 14) and ( 20), as shown in Eq. ( 21).Thus, according to Eqs. ( 20) and ( 21) and the physical significance of the point in the TPS, there is no singularity issue.
In TPS, a BS is expressed as three points, which has four cases, as shown in Fig. 4.Each case has three points, (M i , N i , U i ), (M j , N j , U j ), and (M k , N k , U k ), where case 4 can be considered as three coincident points located at the origin of the coordinates.
A plane S TPS in TPS can be represented as a matrix S h TPS using three points, as shown in Eq. ( 22), and det(S h TPS ) = 0 is the equation for this plane.
( The body segment matrix E T in the TPS can be determined using the planes of the four body segments, as shown in Eq. ( 23).

Action representation and inverse solving.
The matrices representing the holistic motion of the upper limb, named the upper-limb body segment matrices, are obtained from the Euclidean space and TPS, respectively, as shown in Eqs. ( 19) and ( 23), respectively.
In scientific research and engineering applications, attention has been paid to investigating the upper-limb motion of different bones with diverse rotational DOFs according to various research scenarios and engineering needs.P-BTBS was used to build spatial triangles to estimate the rotation DOFs of bones of interest.A spatial triangle represents the spatial state of the corresponding bone, as shown in Fig. 5.
The coordinates of the points in Fig. 4 were first determined by solving matrices E T and S h TPS , then substituted into Eq.( 21) to estimate the area, |S (h,k) |, of the corresponding spatial triangle.Obtaining L h using the matrix E O is straightforward.24) to estimate the motion angle of the BS-1-Scapula, BS-3-Forearm, and BS-4-Metacarpal.θ 3-5 for BS-2-Humerus can be estimated by solving Eq. ( 25), which was optimized using Eq. ( 13), where The coordinates of BS-2-Humerus in the TPS can be calculated using E T with h = 2, i = 3, j = 4, and k = 5 by solving Eq. ( 22).Substituting the coordinates of BS-2-Humerus and L 2 into Eq.( 26) to estimate ψ 1-3 .

Simplified model of humeral motion.
Humeral motions can be divided into two categories when the initial posture of the arms is in a natural downward state.One category is the combined motion of the X and Y axes, where the X-Y plane is parallel to the sagittal plane, as shown in Fig. 6h.When the initial motion of the humerus tends to move away from the sagittal plane, humerus posture can be decomposed using rotational DOFs around the X and Y axes.This decomposition results from the fact that two DOFs in the X and Y axes can determine any position of the humeral end in front of the sagittal plane.
The other is the combined motion of the Y and Z axes, and the Y-Z plane is parallel to the frontal plane, as shown in Fig. 6i.When the initial motion of the humerus moves away from the frontal plane, humerus posture can be decomposed using rotational DOFs around the Y and Z axes.This decomposition results from the fact that two DOFs in the Y and Z axes can determine any position of the humeral end in front of the coronal plane.
Determining the motion tendency of the humerus during the initial stages is important.Moving from the initial state, the motion category can be determined by comparing the magnitudes of θ 4 and θ 5 .For θ 4 < θ 5 , the motion belongs to the first category.Otherwise, motion belongs to the second category.
Joint angle estimation during continuous motion.For healthy individuals, continuous motion is one of the most important motions in daily life.Continuous motion is a crucial rehabilitation aim for people undergoing upper-limb motion recovery.
To verify the performance of P-BTBS in solving joint angles during continuous upper-limb motion, motion capture experiments for seven DOFs upper limbs were performed.The experimental setup is illustrated in Fig. 6a and b, in which eight motion capture cameras were evenly distributed around the circumference.The circumference, height, and frame rate were approximately 3.8 m, 2 m, and 60 Hz, respectively.The participants stood at the center of the circumference.And the experimental procedure is shown in Supplementary Video 1.
The participants were in a good physical condition, without upper-limb injuries suffered within a month, and without strenuous exercise performed within a week.The skin near the marker point was first washed with medicinal alcohol before the experiment.The black bandages were then wrapped around key positions to minimize the effect of skin motion, as shown in Fig. 6e-g.Moreover, the black bandage was wrapped around the abdominal area to reduce the influence of the vertical motion of abdominal fat.The inclination angle was approximately 30°.The motion of a rigid body can be captured using a motion capture system with the highest accuracy.To obtain the exact values of the joint angles, several triangular rules were pasted onto the body as rigid bodies.
The P-BTBS is a general method for eight DOFs.To demonstrate the generality and advantage of handling highly redundant upper-limb motions, seven DOFs of the upper limb were chosen as the experimental target.Six upper-limb spatial points (M3, M4, M8, M9, M10, and M13) were selected, as shown in Fig. 6e-g.Seven spatial triangles were built, as shown in Fig. 5.The experimental motions are shown in Fig. 7a, where the initial state was not in naturally downward.The initial state was set up to avoid obscuring the side-marker points of the abdomen.The simplified model of humeral motion evidently shows that the experimental motions belong to the second category.( 24) .
Vol:.(1234567890)The experimental motions first started with the initial state and were lifted forward on the humerus to perform motion 1.The elbow joint was bent to perform motion 2, the humerus was rotated around the Y-axis to perform motion 3, and the wrist joint was inwardly bent to perform motion 4. Finally, experimental motion returned to its initial state and repeated the next set of cycles.Two sets of identical experimental motions were used.

Scientific
The coordinates of the six marker points were obtained from the motion capture system, and the raw data were smoothed using the sliding average filtering algorithm.The calculated values were estimated using P-BTBS.The exact values were acquired from the motion capture system and smoothed using a sliding average filtering algorithm.The motion capture system and P-BTBS differently defined rotation and the values could be opposite; therefore, the exact values were taken as the opposite.In accordance with the simplified model of humeral motion, the θ 5 of humerus was not significant in the second category, which does not help determine humeral posture.The angular variation of the remaining six DOFs is shown in Fig. 7b-g.
Root Mean Square Error (RMSE) was used as the error evaluation method in this study, which is widely used in engineering measurements.In practical applications, RMSE is sensitive to the data and measures the deviation between the measured value and the true value, thus reflecting the precision of the measurement method.
The experimental motions were executed twice, and all data had two cycles.The shoulder complex was moved passively throughout the experiment; therefore, the two rotation angles of the clavicle changed minimally.The motion angle θ 1 of the clavicle ranged from 0.8733° to 8.9923° with a RMSE of 1.3877°.A forward arm extension was included in the experimental motions, and the θ 2 of clavicle varied more than θ 1 .The motion angle θ 2 of the clavicle ranged from − 12.5627° to 1.4329° with an RMSE of 1.0119°.The motion angle θ 3 of the humerus ranged from 1.9542° to 38.7424° with an RMSE of 2.0933°.In its initial state, the humerus was lifted forward; therefore, the motion angle θ 4 of the humerus ranged from 36.2078° to 85.1273° with an RMSE of 2.2824°.The motion angle θ 6 of the forearm ranged from 2.5508° to 89.7897° with an RMSE of 0.9814°.The motion angle θ 7 of the hand ranged from − 51.3814° to 7.6649° with an RMSE of 1.8875°.
The exact values obtained from the motion capture system were the gold standard compared with the P-BTBS results and the mean value of RMSE was 1.6074°.Moreover, compared with the traditional vector method, P-BTBS estimated three to eight motion angles with one coordinate transformation, which can reduce the errors caused by multiple transformations.
More importantly, a maximum of six spatial points was required to estimate the eight motion angles of the upper limb.The locations of the six spatial points coincided with those of Kinect's human skeleton recognition technology 33 , which encourages the application of P-BTBS to most scenarios for estimating the spatial coordinates.
Description of upper-limb motions.Upper-limb motions are complex, unordered, and subconscious.
Explaining the subconscious actuation of brain such as picking up objects from a table is challenging.P-BTBS implements the function of describing upper-limb motions through mathematical methods, which provides a way to observe upper-limb motions.P-BTBS constructs a mathematical framework for multi-space fusion and derives models (Eqs.( 19) and ( 23)) for describing upper limb motions in Euclidean space (E O ) and TPS (E T ), respectively.This mathematical framework not only enriches the motion information mathematically, but more importantly reflects the real upper limb motions, which makes it possible to accurately describe these complex motions of the upper limb.
In the motion description experiments, the participants and experimental field layout were the same as those in the joint angle-estimation experiments.Four motions with the same wiggling amplitude were performed, as shown in Fig. 8a and b.The experimental procedure is shown in Supplementary Video 2. Fixed motions were not chosen because the high contingencies were not sufficient to demonstrate the ability of the P-BTBS to describe upper limb motions.Dynamic wiggling motions are more in line with real-world upper limb motions because of the dynamic nature of the motions in the real world.The four sets of experimental motions were designed because they encompassed the range of upper limb motions (front and side of the body) in everyday life.(d,e).The two angles of rotational motion of the humerus.As shown in the simplified model of humeral motion, the angle θ 5 has no contribution to the action before the frontal plane, so is not calculated and measured.(f,g).The angle of rotation motion of the forearm and palm.
The E O and E T of each frame were calculated in Euclidean space and TPS, respectively, whereas the features were analyzed only in Euclidean space.The independent variables in E T (i.e.M, N, and U) have physical significance only in TPS.Thus, E O and E T are numerical matrices of dimensions 4 × 4 and 6 × 6 in Euclidean space, respectively.
When the one-dimensional feature of P-BTBS was utilized to describe the motions, the 2-Norm of E T was calculated.The variation with 2-Norm characteristics of the four motions is shown in Fig. 8c.The 2-Norm of E T for each motion has a cyclical variation around its mean value.
When the two-dimensional feature of P-BTBS was utilized to describe the motions, the 2-Norm and eigenvalue mean of E O values were calculated.Each E O has four eigenvalues, which can be complex numbers and exist in pairs.Thus, the mean value of the four eigenvalues remains a real number.The mean values of the eigenvalues of E O and 2-Norm were utilized as the X and Y axes, respectively, as shown in Fig. 8d.A wiggling motion can be clustered in a region.
To verify the accuracy of E O and E T in describing upper-limb motions, a three-dimensional (3D) dimension reduction analysis was performed.The 2-Norm of E O , mean of eigenvalues of E O , and 2-Norm of E T construct a 3D feature for each frame.Principal components (PC) 1 and PC 2 are shown in the X and Y axes of the principal component analysis (PCA) diagram, respectively, with PC1 at 93% and PC2 at 5.3%, as shown in Fig. 8e.
One point in the PCA diagram corresponds to the motion frame.The four sets of wiggling motions corresponded to the four 95% confidence ellipses.The confidence ellipses of motions 3 and 4 were close but did not intersect.This is because motions 3 and 4 are similar, but fundamentally different (fixed relative posture of the humerus and forearm, only humerus rotates around the Y-axis), reflecting both the differences and similarities in upper-limb motions.

Figure 2 .
Figure 2. Overview of P-BTBS application object and method.(a) Four groups of bones and five joints (the scapulothoracic joint was not considered) are involved in this study.Each group of bones is given a distinctive color, and the body segments are represented by the corresponding color for the different bones.(b) A spatial triangle in Euclidean space consists of three spatial points with an area of S, the projected areas in the X-Y, X-Z, and Y-Z planes are S m , S n, and S u , respectively.A spatial triangle in the triangular primitive space (TPS) is represented by a point whose coordinates are (S m , S n , S u ).(c) The coordinates P W in the world coordinate system can be expressed as P L in the local coordinate system utilizing a coordinate system transformation (rotation w L R and translation P LORG ).Red points indicate actual spatial points on the joints (The locations are shown in Table1).

Figure 3 .
Figure 3.The definition of upper-limb modular motion model and the relationships between geometric features.Black points indicate virtual spatial points (calculated from actual spatial points), points 1, 2, and 3 are fixed points, and the remaining points are all moving points.Spatial point 3 is the origin of the local coordinate system.(a,b) Figures define the two spatial isosceles triangles of the clavicle and show the initial state of the natural dropping of the arms.(c,d) Figures define the three spatial isosceles triangles of the humerus, l 5,8 ⊥l 6,8 ⊥l 7,8 .(e,f) Figures define the spatial triangles of the forearm and two spatial isosceles triangles of the metacarpal, l 9,10 ⊥l 10,11 ⊥l 10,12 .The motion information represented by the spatial triangles is shown in Table2. https://doi.org/10.1038/s41598-023-36767-0www.nature.com/scientificreports/

Figure 4 .
Figure 4. Four upper-limb modeling cases in TPS.The coordinates of the three points in case 4 are (0, 0, 0).The plane of case 3 consists of a point, a vertical point on the M-axis, and origin.The plane of case 2 consists of two points and the origin.The plane of case 1 consists of three points.

Figure 5 .
Figure 5. Diagram of various motions of upper limb and the corresponding spatial triangles.Arbitrary selection of spatial triangles according to the required DOFs.(a) Attention to just one DOF of motion of the hand.(b) Attention to one DOF of clavicle and humerus each.(c) Attention to one DOF of the clavicle and two DOFs of the humerus.(d) Focus on two DOFs of the clavicle, one DOF of the forearm, and one DOF of the palm.

Figure 6 .
Figure 6.Experimental overview and humerus-motion simplified model.Motion capture lens (a) height and (b) layout.(c) Position and (d) posture of the world coordinate system.The red, green, and blue axes are the X, Y, and Z axes, respectively.The X-Y plane is parallel to the ground, and the local coordinate system is rotated 0° and 90° around the Y and X axes of the world coordinate system, respectively.(e-g) The six points labelled in the diagram are required for the P-BTBS, the remaining points are necessary for the motion capture system to calculate the exact values.The red triangle symbols show the placement of the triangular rulers.(h) and (i) The sagittal plane is parallel to the X-Y plane.The frontal plane is parallel to the Y-Z plane.The origin of the coordinate system is located at spatial point 3.

Figure 7 .
Figure 7.The experiment of upper-limb motion information extraction during continuous motion.The calculated and exact values are the results of the P-BTBS and motion capture system, respectively.The positive and negative results of subtracting the calculated value from the exact value are filled in the graph in light blue and light orange, respectively.The total error for each group of results is expressed as Root Mean Squared Error (RMSE).(a) Five consecutive upper-limb motions, repeated twice.The initial state is not in the natural downward state to avoid the upper limb obscuring the marker point during the initial.Action 1: a forward lift of the humerus.Action 2: a bend of the elbow.Action 3: a turn of the humerus around the Y axis.Action 4: an internal wrist snap, finishing with a return to the initial position and proceeding to the next motion.(b,c) Two angles of rotational motion of the clavicle.(d,e).The two angles of rotational motion of the humerus.As shown in the simplified model of humeral motion, the angle θ 5 has no contribution to the action before the frontal plane, so is not calculated and measured.(f,g).The angle of rotation motion of the forearm and palm.

Figure 8 .
Figure 8.The experiment of upper-limb motions description.(a,b) The four groups of motions are described.Each set of motions is performed with the same waggle amplitude.(c) The 2-Norm of E T matrices obtained in TPS is calculated for each frame, with each group of values having a cyclical variation around its mean value, and having a visible difference in the mean values.Just one point per ten data points is displayed, the rest are skipped.(d) The 2-Norm and eigenvalues' mean value of E O matrix in Euclidean space are calculated for each frame to form a 2D planar map.The points representing the different motions appear in different regions of the 2D plane.Just one point per twenty data points is displayed, the rest are skipped.(e) Principal component analysis (PCA) and data downscaling are performed on the three-dimensional features of each frame, with the three-dimensional features being the 2-Norm of E O , the eigenvalues' mean value of E O and the 2-Norm of E T , respectively.Just one point per twenty data points is displayed, the rest are skipped.

Table 1 .
Definition of actual spatial points.Note that point 8 defines the motion of the humerus.