Multi-scopic neuro-cognitive adaptation for legged locomotion robots

Dynamic locomotion is realized through a simultaneous integration of adaptability and optimality. This article proposes a neuro-cognitive model for a multi-legged locomotion robot that can seamlessly integrate multi-modal sensing, ecological perception, and cognition through the coordination of interoceptive and exteroceptive sensory information. Importantly, cognitive models can be discussed as micro-, meso-, and macro-scopic; these concepts correspond to sensing, perception, and cognition; and short-, medium-, and long-term adaptation (in terms of ecological psychology). The proposed neuro-cognitive model integrates these intelligent functions from a multi-scopic point of view. Macroscopic-level presents an attention mechanism with short-term adaptive locomotion control conducted by a lower-level sensorimotor coordination-based model. Macrosopic-level serves environmental cognitive map featuring higher-level behavior planning. Mesoscopic level shows integration between the microscopic and macroscopic approaches, enabling the model to reconstruct a map and conduct localization using bottom-up facial environmental information and top-down map information, generating intention towards the ultimate goal at the macroscopic level. The experiments demonstrated that adaptability and optimality of multi-legged locomotion could be achieved using the proposed multi-scale neuro-cognitive model, from short to long-term adaptation, with efficient computational usage. Future research directions can be implemented not only in robotics contexts but also in the context of interdisciplinary studies incorporating cognitive science and ecological psychology.

. Illustration of random selection strategy by considering the suspected nodes.

Learning Epoch
In the conventional model of topological learning model, using competitive learning 1 , some of them using fuzzy C-means which has local property with minimum convergence 2, 3 . However using the conventional competitive learning method for dynamic granularity of node with controlled random selection make the generated topological structure is not stable. Therefore, we use two-layer learning process which composed as fine-tuning and coarse tuning. The fine-coarse tuned learning strategy is also used for improving the effectiveness of multilabel classification framework 4 . This strategy also has been use for classify meaningful point cloud data 5 . In our proposed method uses two-step competitive learning method. Fine tuning is used to process random selection of 3D point cloud data associated with suspected node explained in Section Note S1-1. Coarse tuning is used to process the overall random selection of point cloud. After processing random selection. we update the winner node and its connected node in every selected random. The mathematical model can be seen in the following equations.
Where, s 1 (n) is the winner node in nth selected node calculated in Equation (10).

Strength's Node Calculation
To find the important area required to be increase the granurality of the node, we analyze the strength of the node by calculating the direction of normal vector and the magnitude of the normal vector as feature extraction. Some researcher use the eigen vector of 3D covariance matrix from assined point and its neighborhood to describe 3D local properties. The value of curvature is indicated by the minimum eigen value in eigen vector of covariance matrix 6,7 . The change of curvature also can be calculated by generated eigen value l 3 /(l 1 + l 2 + l 3 ) 8 . This method is efficient if the facet or triangulation information undefined.
Then, it composes less information which only composed as geometrical characteristic. In our proposed model, the facet or Figure 2. Illustration of strength node calculation by vector projection. A) Gray color represents unused information and blue color represent considered information related with ith node. B) Calculating the number of facet and vector unit between ith node and its neighbors. C) representing the normal vector unit in each facet. D) calculating the average vector value triangulation of the topological structure is defined. We calculate the properties based on vector projection. Therefore, normal vector of facet and strength of node can be acquired. The illustration of strength's node calaculation can be seen in SM Figure 2.
The strength value of ith node is calculated from these following equations Where w i is the normal vector of the ith node. It is calculated based on vector average of facet normal vector associated to ith node (v i, j ). N f i is the number of facet associated with ith node. Normal vector of jth facet in ith node calculated in this following equation H i is the facet information associated with ith node composed as the facet id and connected nodes.û i, j is the vector unit between jth and ith node.

Node Addition
The number of node's addition process is defined before hand. The node addition (rnode) is inserted in the center edge between a node (q) with highest maximum accumulated error and a node ( f ), with highest error among neighbor's nodes of q node, calculated as follow The accumulated error of q and f node is decrease 50%. Accumulated error value of r node is set as the error value of q node.

Node deletion
The node will be deleted if there is no connection with other nodes.

Flow of process
In step 0, the model perform the initialization process. We generates two nodes with random position in selected n dimension spaces (h 1 , h 2 , in R n ). The edge value is set zero (c i, j = 0, 8 i 8 j ). We set a value of set of node with 2 initial nodes (A = 1, 2). The number of node is set by 2 (r = 2).
In step 1, It generates random sampling (v) of the input data from sensors, with the number of sampling is set as l .
The value of t is set to 0.
In step 2,the learning process will be performed with coarse tuning. We process tth generated sample data Then we find two nearest node id (s 1 , s 2 ) from selected tth sampling data (v t ).
If the edge value between s 1 th node and s 2 th node is not connected, then the value set to 1, c s 1 ,s 2 = 1; c s 2 ,s 1 = 1;. Then the age of edge between s 1 th node and s 2 th node is set to 0 (g s 1 ,s 2 = 0). In step 3, we update the accumulated error of node s 1 .
In step 4, learning update is processed explained in Section Learning Epoch .
In step 5, we calculate the edge of the nodes. g i,s 1 = 1, g s 1 ,i = 1, if c i,s 1 = 1 and i 2 A. If the value ofg i,s 1 > g MAX , then the value of edge between ith node and s 1 th node will be disconnected (c i s 1 ). If some node there is no connection with other nodes, then the node will be deleted.
In step 6, we perform node addition including node deletion.

Note S2 -Affordance Detection Module
If there is a nonhomogeneous normal vector for any area, the AD module ask DA module to increase area's node density by sending the strength of node d in Eq. (14). For movement related commanding in microscopic level, AD module will send the object affordance information composed as centroid posision (C a ) calculated as , and object boundary size calculated as R a = max(h i C a ). Furthermore, the DA module also provide topological structure A with vector of curvature in each node w and strength of node d to LER module in mesoscopic level.

Representing Cortex and SC Integration
The accuracy of the affordance depends on the degree of awareness and attention. Depending on the accuracy of the perception, affordances can be wrongly perceived 9 . Awareness and attention are certainly closely related but perhaps not inseparable. When I are walking, our attention may not be focused on the path, however, the attention typically shifts toward the path when danger is noticed 10 . It seems that some unconscious knowledge tells the conscious self to "look out". In fact, the perceptual-cognitive system is continuously processing, though our awareness of it is minimal. This system is extremely efficient, shifting attention automatically when needed. In biological process point of view, it shows the integration of visual motor cortex and superior colliculus level. In our approach, the degree of attention is generated automatically by Dynamic Attention with Density Representation (see Section ). Outside the focus of attention, the accuracy of perceiving the affordance is lower, owing to the low information density. When a risky obstacle is detected, the attention focuses onto it, and perceptual accuracy increases because the information density increases (SM Figure 3). The affordance-perception process receives external sensory information from the attention process in the form of a topological structure.
When the density of node (a) in the non-homogeneous vector (N (non-H) ) calculated in Eq. (??) is lower than the threshold calculated in Eq. (16), the affordance model prompts the SC to increase the node density in a certain area explained in Section Note S1. The model sends the center of attention (C a ), the radius size (R a ) of non-homogeneous vectors calculated as follows:

Representing Affordance Detection and Affordance Effectivity Fit Integration
The pathway process from cortex to movement-related motor cortex shows the how the affordances are represented to the motor cortex. The visual cortex model sends information about the detected object or obstacle (size and position), where its size and position are represented by parameter C a , R a in Section Note S2-2. In our model, the non-homogeneous vector in the attention area will be detected as the object or obstacle during movement. After that, the affordance perception will serve the possible action and safe area by detecting normal vector to the vertical direction in the attention area.

Note S3 -Central Pattern Generator
The CPG model dynamically generates locomotion patterns. In cat studies, two-layered CPG has been observed, incorporating rhythm generators (RG) and pattern formation (PF) 11 . I have designed a single-model CPG where one RG represents one leg's movement pattern, and one PF neuron represents the activation of one muscle. Since each leg uses four muscles (flexor and extensor muscles of the hip and knee joints), there is one RG neuron and four PF neurons for each CPG structure. Our model uses two CPG pairs; every pair represents the forelimb and hindlimb for another pair. SM Figure 4 shows the complete design of the proposed CPG model. Rhythm-generator neurons generate oscillation signals responsible for controlling the patterns of a certain leg. All of the RG are interconnected and control the timing of the leg's swing action. I used the Matsuoka neural-oscillator model to generate a dynamic signal. The inner state of the RG neuron is as follows: The received-inhibition effect of its self-adaptation (v i ) and the signal from other RG neurons (y j ) influenced by synaptic weight (w i j ). y i is calculated as y i = max(x i , 0) and v i is calculated as follows: The inner-state and self-adaptation effects are respectively influenced by time constants t and T . The RG neuron is also influenced by sensory feedback (a i ) from ground-force afferent, swing-related afference, and nociceptors.
where, a i,0 is the basic stimulation of the ith neuron, w FR,i j and w SR,i j are the synaptic weights of the force afferent (F i ) and the swing-phase afferent (S i ) of the ith leg to the jth RG neuron. Swing-afferent feedback is influenced by the nociceptor (N i ); its weight is represented by w NS . The nociceptor (N i ) is a pain receptor that detects the condition of leg damage and sends damage stimuli to RG neurons, affecting sensory stimulation from the ground-force afferent to the RG neurons and from the 8/20 swing-related afferent to the RG neurons. Parameter G ST IM is the gain parameter controlling the relationship between speed stimulation S ST IM and the sensory network. The gain parameter is assumed to differently affect the forelimbs and hindlimbs. Based on preliminary tests, the G ST IM for hindlimbs is calculated as G ST IM = µ HL /(1 + exp ( l HL S ST IM + h HL )) and the G ST IM for hindlimbs is calculated as G ST IM = exp (log(µ HL )(l FL S ST IM h FL ) 2 ) Given the RG neurons control the movement-pattern phase for each leg, they transmit the firing signal (p i ) to all PF neurons in a certain leg to activate swing behavior. This is calculated as follows: where Q is the firing threshold. The value of h re f i is calculated as follows: When the RG signal is fired, R is subtracted from the h re f i (t) value of neuron i. R > 0 , g re f is a discount rate of h re f i , and 0  g re f  1. The firing value of the ith RG neuron p i (t) is generated for all PF neurons in the ith leg.
Upon receiving the firing signal from RG neurons (p i ), PF neurons perform the swing behavior by activating muscle stimulation. The PF neurons control the timing of each muscle contraction. The activation signal generated by PF neurons is calculated as follows: (To control the speed, speed stimulation is adjusted from parameter S ST IM , which influence Eqs. (17) and (19)) where, PF i neurons are calculated as PF i (t) = PF i (t 1) + p i , d Opt represents the starting control, calculated by µ = (30 f

Note S4 -Affordance Effectivity Fit (AEF)
The AEF is represented by artificial neural network is depicted in SM Figure 5. There are input parameters from the output of the AD module (C a and R a ) and internal sensory information, which are four 3D vectors (v l x , v l y , v l z ) representing the vectors of motion interrupt of the four legs; twelve parameters represent the joint angles (q 1 , q 2 , q 3 , ·· ·, q 12 ), and four parameters represent the touch sensor signals from the four feet (T 1 , T 2 , T 3 , T 4 ).
The output layer comprises two groups, activated alternately. The first group contains twelve parameters representing all of the joints' angular accelerations (Q 1 ,Q 2 ,Q 3 , ·· ·,Q 12 ). The output is generated when an interrupt command transfers to the CPG module in short-term adaptation. The output layer's second group is conveys walking provision information (s SAG , s COR , s DIR ), generated when there are behavior interruption transferred to the BC module in medium-term adaptation. Confidence node selection The selection of the node with stable normal vectors is calculated by the dynamic attention model (detailed in Section Note S1), with the selected node initially added directly to the map's structure. This node selection step is as follows: Surface matching Following localization and selection of appropriate nodes, the module matches the position and normal vector of nodes between the GNG and the map constructed. There are three steps: 1) Find the nearest map node to teach the GNG node, 2) 6D vector matching, and 3) Add new map nodes.
1. The module finds the map node nearest to teach the GNG node. It finds the map node with the position and normal vector closest to each GNG node. If the module cannot find it within the defined range, the node will not be processed for the 11/20 next step. Only nodes that have been assigned to the map are selected to be processed in the next step. Node assignment is represented in SM Figure 7b. For each selected GNG node, the difference in position and surface vector with the map node is calculated using Eqs. (23) and (24), where, l d i , j and v d i , j are the distance and degree of vector similarity between d i th GNG node and jth map's node. The module chooses the nearest node in a limited area that features a high similarity value for the surface vector.
2. Next, the assigned nodes are fitted to the map node. This involves calculating the different 3-D positions using Eq. 25, whereṡ(t) is the difference in position.
3. Next, the associated GNG node and map nodes are merged, enabling comparison between the two. Any GNG node that does not pair with a map node is added as a new map node.

Note S6 -Cognitive Map Building
At this level, a cognitive map is built using the topological structure-based map reconstruction generated at the microscopic level. The proposed cognitive map comprises segments of a location; for example, intersections or T-junctions. The topological map reconstruction generated using the localization and mapping module at the mesoscopic level is processed using multi-layer GNG in conjunction with the area of interest. Meanwhile, the cognitive map processes the topological map comprising the 3-D location node, the normal 3-D node vector, and the connection between them. This means the cognitive map is assigned a certain location but with more diverse environmental conditions. This preliminary model uses GNG to generate topological input, which requires map reconstruction from lower-level control. This process is represented in SM Figure 8. The robot's position and topological map information are obtained using localization and lower-level map modeling; then, the model analyzes the robot travel success rate from one node to another node on the topological map. Following this step, the topological cognitive map is generated.

Implementation in simulation
For the computer simulation, I used Open Dynamics Engines (ODE) to create an artificial room, corridors featuring certain obstacles, and a buggy-type robot with an artificial depth camera. The results for the proposed module in the simulated corridor and underground pipe are presented in SM Figures 9a and 9b, with the cognitive map represented by the white topological structure.

Real-world implementation
For real implementation, we created an artificial terrain, as shown in SM Figure 10, and then used a Kinect camera to reconstruct the map. Then, the map reconstruction data became the input for the cognitive map module, with the results of the cognitive map presented in SM Figure 11.  As shown in the Fig. 12, the robot can avoid the obstacle by changing the leg swing pattern and/or changing the footing position. Based on the information given by affordance detection model, when the interrupt is activated as depicted in Fig. 13, then the Affordance effectivity fit sends the interrupt signal at the joint angle level. As shown in Fig. 14, the step height changes due to the influence of the motion interrupt generator. The sample of pattern signals from CPG and joint angle signals is shown in Fig. 15.
The signal pattern is not influenced. Only the signal from the motor neuron pools is influenced. This mechanism can realize good performance as well. However, further discussion and development are required. When we disabled our improved process, the robot got stuck in the obstacle. Snapshots of this failure are shown in the Fig. 16.

Note S9 -Quantitative Experiments
We conducted some quantitative experiments to show the integration of multi-module: 1) avoiding a sudden obstacle, and 2) avoiding a static obstacle. We evaluate the response of obstacle detection and the accuracy of avoiding. Evaluating the accuracy of avoiding, we classify the outcome into three categories: obstacle totally avoided, obstacle avoided but touched, and avoidance failed. A link to associated video can be seen at https://www.dropbox.com/s/l6s9kw8afdamtrq/Quantitative_experiments.mp4?dl=0.

Robot Performance Avoiding Sudden Obstacle
We run the quadruped robot in a trotting gait at normal speed. During the swinging phase (front leg lifted), some pieces of wood are moved and placed, by hand, immediately in front of the robot. We repeated this activity 50 times with random positions and arrangements of the obstacle. The affordance detection process successfully clarified all of the obstacles presented this way. The performance results can be seen in Table. 1. In 28 of the trials, the robot totally avoided the obstacle, in 13 trials the robot avoided but touched the obstacle, and 9 times the robot failed to avoid it. The success rate for avoiding the obstacle thus reached 82 percent.

Robot Performance Avoiding Static Obstacle
We set the quadruped robot moving with a normal-speed walking gait. We put some obstacle across the robot's way, fifty times with random obstacle arrangements. In this experiment, the affordance detector also succeeded in clarifying the obstacle. The performance results can be seen in Table. 1. The robot successfully avoided the obstacle 23 times, avoided but touched the obstacle 21 times, and failed 6 times fails. Overall, the robot's success rate was 88 percent.