## Abstract

The fascinating collective behaviors of biological systems have inspired extensive studies on shape assembly of robot swarms. Here, we propose a strategy for shape assembly of robot swarms based on the idea of mean-shift exploration: when a robot is surrounded by neighboring robots and unoccupied locations, it would actively give up its current location by exploring the highest density of nearby unoccupied locations in the desired shape. This idea is realized by adapting the mean-shift algorithm, which is an optimization technique widely used in machine learning for locating the maxima of a density function. The proposed strategy empowers robot swarms to assemble highly complex shapes with strong adaptability, as verified by experiments with swarms of 50 ground robots. The comparison between the proposed strategy and the state-of-the-art demonstrates its high efficiency especially for large-scale swarms. The proposed strategy can also be adapted to generate interesting behaviors including shape regeneration, cooperative cargo transportation, and complex environment exploration.

## Introduction

In nature, groups of insects and animals can self-assemble various spatial shapes that are functional for the groups adapting to the environments^{1,2,3}. As a remarkable example, army ants can assemble shapes to transport food cooperatively or construct bridges using their bodies to overcome spatial gaps^{4,5}. These shape assembly behaviors are generated spontaneously by local interactions among the individuals. They exhibit strong adaptability to individual faults and can be easily scaled up to groups of thousands or millions of individuals^{1,3}.

The fascinating collective behaviors of biological systems have inspired extensive studies on shape assembly of robot swarms^{6,7,8,9}. One class of strategies widely studied in the literature are based on goal assignment in either centralized or distributed ways^{10,11,12}. Once a swarm of robots are assigned unique goal locations in a desired shape, the consequent task is simply to plan collision-free trajectories for the robots to reach their goal locations^{10} or conduct distributed formation control based on locally sensed information^{6,13,14}. It is notable that centralized goal assignment is inefficient to support large-scale swarms since the computational complexity increases rapidly as the number of robots increases^{15,16}. Moreover, when robots fail to function normally, additional algorithms for fault-tolerant detection and goal re-assignment are required to handle such situations^{17}. As a comparison, distributed goal assignment can support large-scale swarms by decomposing the centralized assignment into multiple local ones^{11,12}. It also exhibits better robustness to robot faults. However, since distributed goal assignments are based on locally sensed information, conflicts among local assignments are inevitable and must be resolved by sophisticated algorithms such as local task swapping^{11,12}.

Another class of strategies for shape assembly that have also attracted extensive research attention are free of goal assignment^{18,19,20,21}. For instance, the method proposed in ref. ^{18} can assemble complex shapes using thousands of homogeneous robots. An interesting feature of this method is that it does not rely on external global positioning systems. Instead, it establishes a local positioning system based on a small number of pre-localized seed robots. As a consequence of the local positioning system, the proposed edge-following control method requires that only the robots on the edge of a swarm can move while those inside must stay stationary. The method in ref. ^{19} can generate swarm shapes spontaneously from a reaction-diffusion network similar to embryogenesis in nature. However, this method is not able to generate user-specified shapes precisely. The method in ref. ^{21} can aggregate robots on the frontier of shapes based on saliency detection. The user-defined shape is specified by a digital light projector. An interesting feature of this method is that it does not require centralized edge detectors. Instead, edge detection is realized in a distributed manner by fusing the beliefs of a robot with its neighbors. However, since the robots cannot self-localize themselves relative to the desired shape, they make use of random walks to search for the edges, which would lead to random trajectories. Another class of methods that do not require goal assignment is based on artificial potential fields^{22,23,24,25}. One limitation of this class of methods is that robots may easily get trapped in local minima, making it difficult to assemble nonconvex complex shapes.

Here, we propose a strategy for shape assembly of robot swarms based on the idea of mean-shift exploration: when a robot is surrounded by neighboring robots and unoccupied locations, it would actively give up its current location by exploring the highest density of nearby unoccupied locations in the desired shape. This idea does not rely on goal assignment. It is realized by adapting the mean-shift algorithm^{26,27,28}, which is an optimization technique widely used in machine learning for locating the maxima of a density function. Moreover, a distributed negotiation mechanism is designed to allow robots to negotiate the final desired shape with their neighbors in a distributed manner. This negotiation mechanism enables the swarm to maneuver while maintaining a desired shape based on a small number of informed robots. The proposed strategy empowers robot swarms to assemble nonconvex complex shapes with strong adaptability and high efficiency, as verified by numerical simulation results and real-world experiments with swarms of 50 ground robots. The strategy can be adapted to generate interesting behaviors including shape regeneration, cooperative cargo transportation, and complex environment exploration.

## Results

The proposed shape assembly strategy consists of three components. The first is a human-swarm graphical interface that can specify user-defined shapes. The second is a distributed negotiation process that can autonomously reach an agreement among the robots on the final location and orientation of the swarm shape. The third, which is the core of the proposed strategy, is a distributed control algorithm. The three components are detailed as follows.

### A human-swarm graphical interface

The first step of shape assembly is to specify a desired geometric shape. We designed a human-swarm interface that allows a human operator to specify the desired shape by drawing or loading a binary graphical image (Fig. 1a). The graphical image is converted to a black-white binary grid, where the black cells in the grid correspond to the locations that should be occupied by the robots. The binary grid is then converted to a grayscale grid by the distance transformation algorithm^{29} so that the influence of the desired shape is gradually expanded and the robots can move into the shape more smoothly (Fig. 1a). With this human-swarm interface, the operator can specify the desired shape in a graphical way without specifying the physical parameters of the grid. The parameters such as the physical size of each cell can be autonomously generated. Moreover, the number of cells in the grid does not have to be the same as the number of robots because the proposed method can handle mismatches between the grid and robot numbers (Fig. 4 as shown later). Although the desired shape is represented as a discrete grid, the robots can move freely across the boundaries of the cells in the grid. More information can be found in Methods and Section 1 in the Supplementary Information.

### Distributed negotiation on the desired shape

Once the grayscale grid has been generated by the human-swarm interface, it is sent to all the robots and stored in each robot’s memory. The memory sizes of the grids used in our work are around tens of kilobytes (details are given in Section 1 in the Supplementary Information), which is affordable for mainstream embedded computers nowadays. The Euclidean parameters of the desired shape including the position or orientation are not assigned manually. Instead, the robots negotiate with each other to autonomously determine them in a distributed manner. In particular, each robot first interprets the desired shape based on its local interest. For example, every robot may initially interpret itself as the center of the desired shape (Fig. 1b), which is locally optimal for each robot in the sense that they do not need to move because they are already inside the desired shape. While the initial interpretations of different robots may conflict, each robot negotiates its interpretation with its neighbors through local wireless communication by a distributed consensus algorithm. With the negotiation algorithm, all the robots would eventually reach a consensus on the translation and orientation of the final desired shape (Fig. 1b). By introducing a small number of informed robots, the swarm can reach a consensus on a user-specified trajectory of the desired shape. More information about shape negotiation is given in Methods and Section 2 and Section 6.1 in the Supplementary Information.

### Distributed control of shape assembly

The aim of each robot is to move into the desired shape. This process is highly dynamic because its goal location is time-varying due to the dynamical convergence process of its interpretation and, more importantly, the avoidance of inter-robot collision or goal location competition. In our method, each robot executes the same control strategy based on its locally sensed information so as to enter the desired shape while avoiding collision. In particular, the velocity command for the *i*th robot is

where \({{{{{{{{\bf{v}}}}}}}}}_{i}^{{{{{{{{\rm{ent}}}}}}}}},\,{{{{{{{{\bf{v}}}}}}}}}_{i}^{\exp }\), and \({{{{{{{{\bf{v}}}}}}}}}_{i}^{{{{{{{{\rm{int}}}}}}}}}\) represent the shape-entering, shape-exploring, and interaction velocity commands, respectively. The roles of the three velocity commands are explained as follows, whereas their mathematical expressions are given in Methods.

The shape-entering command \({{{{{{{{\bf{v}}}}}}}}}_{i}^{{{{{{{{\rm{ent}}}}}}}}}\) aims to steer robot *i* to the desired shape by seeking the darkest grid cells around them (Fig. 1c and Methods). More specifically, when robot *i* is too far away from the desired shape, \({{{{{{{{\bf{v}}}}}}}}}_{i}^{{{{{{{{\rm{ent}}}}}}}}}\) would drive the robot toward the closest gray cell. When robot *i* has entered the area of the gray cells, \({{{{{{{{\bf{v}}}}}}}}}_{i}^{{{{{{{{\rm{ent}}}}}}}}}\) would push the robot toward one of the neighboring gray cells that has the lowest gray level. In this way, the robot can gradually reach a black cell in the desired shape. It is notable that \({{{{{{{{\bf{v}}}}}}}}}_{i}^{{{{{{{{\rm{ent}}}}}}}}}\) would vanish once a robot steps into the desired shape area. In this case, the robot would stop moving and hence block those robots behind. To resolve this problem, the following velocity command \({{{{{{{{\bf{v}}}}}}}}}_{i}^{\exp }\) is designed so that the robots can continue moving into the deep of the desired shape.

The shape-exploring command \({{{{{{{{\bf{v}}}}}}}}}_{i}^{\exp }\), which is realized by an adapted version of the mean-shift algorithm^{27}, aims to push each robot into the desired shape and then explore unoccupied cells inside the desired shape (Fig. 1c and Methods). More specifically, there are two cases. In the first case where robot *i* is near the edge of the desired shape so that the cells within its sensing radius are either black or gray, \({{{{{{{{\bf{v}}}}}}}}}_{i}^{\exp }\) would drive the robot toward the highest density of black cells. In this way, the robot is attracted into the shape. In the second case where robot *i* is already inside the desired shape so that the cells within its sensing radius are all black, \({{{{{{{{\bf{v}}}}}}}}}_{i}^{\exp }\) would drive the robot toward the direction with the highest density of the unoccupied black cells. A cell is defined as unoccupied if the distance between its center and any robot is greater than half of the robot’s collision avoidance radius (F 3B). The shape-exploring command is the key to resolving inter-robot competition. When a robot is surrounded by neighboring robots and unoccupied locations, it would actively give up its current location by exploring the highest density of nearby unoccupied locations.

The interaction command \({{{{{{{{\bf{v}}}}}}}}}_{i}^{{{{{{{{\rm{int}}}}}}}}}\) consists of two sub-terms (Fig. 1c and Methods). The first is a collision-avoidance term that generates repulsive velocity commands when robot *i* is too close to its surroundings. The second is a velocity-alignment term that aligns the velocity of robot *i* with its neighbors. The velocity-alignment term plays two roles. First, it can reduce velocity mismatches among the robots to reduce the chances of inter-robot collision. Second, since the desired shape may maneuver across in the motion space, the velocity-alignment term is necessary for robot *i* to track the velocity of the moving desired shape.

### Platforms and experimental setup

We implemented the proposed strategy on a swarm of 50 holonomic wheeled robots (Fig. 2a, b). Details of the robotic platforms are given in Section 5.1 in the Supplementary Information. Each robot can exhibit its velocity command by using a LED belt inside the robot’s dome. In particular, each LED can show four colors: red, green, blue, and white (Fig. 2a). The numbers of LEDs showing the four colors correspond to the magnitudes of \({{{{{{{{\bf{v}}}}}}}}}_{i}^{{{{{{{{\rm{ent}}}}}}}}},{{{{{{{{\bf{v}}}}}}}}}_{i}^{\exp },{{{{{{{{\bf{v}}}}}}}}}_{i}^{{{{{{{{\rm{int}}}}}}}}}\), and **v**_{i}, respectively (Section 5.1 in the Supplementary Information). The experiments were conducted in an indoor environment with the support of a motion capture system (Fig. 2c, d). To implement the proposed strategy, each robot needs to acquire certain information about its neighbors and sense its surroundings to identify unoccupied cells and obstacles. These functions are realized through a parallel multi-thread system in a distributed manner (Fig. 2e). More specifically, each robot is assigned an independent control thread running on the workstation and communicates with its thread using wireless routers. Although all the information is available in the workstation, the control command for each robot merely uses the local information that each robot is supposed to have. Details of the experimental setup are given in Section 5.2 in the Supplementary Information.

### Complex shape assembly

The proposed strategy is able to assemble nonconvex complex shapes. One representative complex shape is the snowflake as shown in Fig. 3a. The snowflake shape has 6 major branches and 18 minor branches, making the shape highly nonconvex and challenging to assemble. The proposed strategy can assemble the snowflake shape, which is quantitatively verified by the fact that the coverage rate converges to 100% (Fig. 3a). The coverage rate and other metrics to evaluate the assembly performance are defined in Methods. The shape-exploring control term of the strategy plays a key role in this process. In particular, when this term is disabled, the swarm fails to assemble the snowflake shape since many robots get trapped in local minima and the coverage rate drops sharply to 75% (Fig. 3a). The reason for the failure is that a robot would stop moving and hover near the boundary once it has entered the desired shape in the absence of this term. As a result, it may block those robots behind from entering the shape (Fig. 3b). By contrast, with this term, a robot would continue moving toward the deep of the desired shape after passing the boundary so that those robots behind it can enter the shape smoothly (Fig. 3b). In addition to the shape-exploring term, the shape-entering term in the proposed strategy also plays a necessary role to steer all the robots into the desired shape. As shown in Fig. 3b, not all the robots can enter the shape in the absence of this shape-entering term even though the shape can still be fulfilled by a subset of the swarm. The proposed strategy exhibits smooth swarming motion in shape forming and switching tasks. As shown in the experimental results in Fig. 3c, the swarm can assemble different nonconvex shapes and switch from one to another smoothly.

In the following, the efficiency of the proposed method is compared with that of two state-of-the-art methods. The first is the assignment-based method in ref. ^{12} and the second is the assignment-free method in ref. ^{21}. Both of the methods are decentralized. The method in ref. ^{12} and our proposed method require external global positioning, whereas the one in ref. ^{21} does not. The comparison results are presented in Fig. 4 and Supplementary Fig. 8. It is shown that, given the same initial configurations, the three methods have similar convergence times when the number of robots is as small as 20. However, as the number of robots increases, the convergence times of the state-of-the-art methods increase rapidly while that of the proposed one increases slightly. Specifically, when the number of robots is as large as 300, the convergence time of the proposed method is at least 20 times shorter than the others. The reason why the method in ref. ^{12} requires longer convergence time is that it executes local goal swaps constantly and the robots must move along the grid lines. By contrast, the proposed method does not rely on goal assignment due to the mean-shift exploration control. It also allows the robots to move continuously in the working space. The merit of the method in ref. ^{21} is that it does not rely on any global or local positioning system. However, since the robots do not know their positions relative to the desired shape, they make use of random walks to search for the edges of the desired shape, which would lead to random and long trajectories.

### Adaptability to swarm scale variants

The proposed strategy is adaptive to the variant of robot number in a swarm. When some robots leave the swarm, the remaining robots can autonomously assemble the desired shape without any goal re-assignment or fault-tolerant control. A representative example is the shape regeneration behavior as demonstrated in Fig. 5, where the desired geometric shape is a starfish. After one arm of the starfish shape has been removed, the rest of the swarm spontaneously grows a new arm and assembles the starfish shape again. The adaptability is due to the shape-exploring term, by which the robots can actively search unoccupied cells and hence replace the roles of the removed robots.

Notably, the newly generated starfish has less number of robots than the original one, verifying that the proposed strategy can assemble the same shape with different numbers of robots. In fact, the strategy exhibits stable performance in the presence of mismatches between the number of robots, denoted as *n*_{robot}, and the number of black cells in the desired shape, denoted as *n*_{cell} (Fig. 5d). For the starfish example, it is shown by the statistical results in Fig. 5e that the proposed strategy can successfully assemble the desired shape with a wide range of *n*_{cell}/*n*_{robot}. Specifically, the coverage rate of robots for the desired shape is greater than 93% and the entering rate remains 100% when the ratio *n*_{cell}/*n*_{robot} varies from 0.45 to 128.06. As a consequence, the strategy avoids the requirement of the condition *n*_{robot} = *n*_{cell}, a condition widely adopted in shape assembly methods^{11}, and hence makes the strategy more adaptive. In addition, when the swarm scale increases from *n*_{robot} = 16 to 1024, the convergence time of the entire swarm increases mildly as shown in the rightmost subfigure of Fig. 4E, verifying the high motion efficiency of the strategy.

### Cooperative transportation and shape maneuvering

In nature, ants can surround and transport a piece of food that is much larger than their individual body size in a cooperative manner^{4}. The proposed strategy can be applied to cooperative cargo transportation. To do that, we can specify a hollow shape centered at a cargo (Fig. 6a). When the robots assemble the hollow shape, they would encircle the cargo tightly. The consequent task is to steer the swarm to move collectively while maintaining the desired shape. In particular, we introduce a small number of informed robots that know the desired translational and rotational trajectory of the shape. The informed robots have counterparts in ant swarms^{4}, where some leader ants can guide the others to transport food to a goal location. During the inter-robot negotiation process, the informed robots play a stubborn role by insisting on their knowledge of the desired trajectory. The interpretations of the rest uninformed robots would gradually converge to the informed ones so that all the robots reach a consensus on the desired trajectory of the shape (see the details of the algorithm in Section 2 in the Supplementary Information). While the robots attempt to assemble the desired moving shape, they automatically transport the cargo encircled in the center (Fig. 6b). The velocity alignment component in the interaction control term of the proposed strategy is essential for all the robots tracking the maneuvering swarm shape.

The number of informed robots is required to be at least one. The more informed robots there are, the faster the negotiation process can converge. As shown in Supplementary Figure 6, seven informed robots are randomly selected out of 128 ones, which can reach a consensus on the time-varying translation and orientation of the desired shape efficiently. As a consequence, the swarm can maneuver while the overall shape varies. Being able to track moving shapes is an important feature of the proposed strategy. By contrast, the state-of-the-art methods for homogeneous swarms are only applicable to static shapes^{10,12,18,19,30}. Although there is a rich body of control-theoretic methods that can achieve maneuvering formations^{14,31,32}, these methods require goal assignments or unique robot identities.

### Complex environment exploration

The proposed shape assembly strategy can be applied to environment exploration tasks, in which a swarm must evenly fulfill an environment while avoiding obstacles. As a representative example, a swarm can flood into a room through a narrow passage without getting trapped at the entrance (Fig. 7a). This example mimics the process of pedestrians entering a passenger elevator, which well demonstrates the idea of mean-shift exploration of the proposed strategy: The pedestrians that enter the elevator first should move into the deep rather than staying near the entrance to block the people behind.

The proposed strategy can also be used to explore more complex environments such as a maze as shown in Fig. 7b. Although there are many nonconvex corners in the maze, the swarm can successfully fulfill the maze without getting deadlocked at any corner, verifying the strong exploration ability of the proposed strategy. The strategy can well balance the exploration speed and inter-robot connectivity. That is, a robot inside the desired shape would stop exploring when it has no neighbors inside its sensing radius. As a result, the robots would not explore too fast to depart from the group. This feature is useful in practice because robots may have to explore an environment while keeping sufficiently close to others to maintain wireless connections. In addition, the system is resilient to robot additions or removals. For example, if the connectivity of the swarm is broken due to the removal of some robots, the isolated ones that are already in the maze would stop moving. As more and more robots move into the maze, the isolated ones would get connected to the others again.

## Discussion

The distributed negotiation and control strategy proposed in this work is free of both centralized and decentralized goal assignment procedures. This strategy introduces a mean-shift exploration mechanism so that each robot can actively explore unoccupied locations around them to avoid inter-robot competition for the same locations. The mean-shift exploration is achieved through an adapted version of the mean-shift algorithm so that each robot keeps seeking the highest density of unoccupied cells. It is noticed that the mean-shift algorithm is also adopted in ref. ^{25} to achieve shape assembly. However, the mean-shift algorithm therein is used as an alternative to generating attractive and repulsive forces and hence is not able to resolve the inter-robot competition problem.

Experimental results verified that the proposed strategy can assemble highly nonconvex shapes and exhibits strong adaptability against mismatches between the number of robots and the number of cells in the desired shape. The strategy can also be applied to solve some challenging tasks such as cooperative cargo transportation and environment exploration with minimal modifications of the algorithms. It, therefore, provides a promising method for efficient and adaptive shape assembly tasks of robot swarms.

The desired shape in a shape assembly task is a global constraint for all the robots. The global constraint sets a fundamental requirement that certain global information must be used during the process of shape assembly. However, the required global information may exhibit in different forms in different methods. For example, in the proposed strategy and the state-of-the-art ones^{10,12,30}, the required global information is that every robot has a common sense on a global reference frame. With this global information, every robot can evaluate its location relative to the desired shape. In practice, the global reference frame can be obtained by GPS in outdoor open spaces. If the global reference frame is unavailable, there must be alternatives to acquire global information. For example, the method in ref. ^{18} replaces the global reference frame by using a relative positioning system based on four seed robots.

The proposed negotiation and control algorithms are applied to assemble 2D shapes in this work. They can be extended to 3D scenarios since these algorithms are based on vector calculations where the position and velocity vectors are not restricted to 2D. However, a new human-swarm interface that can specify 3D shapes is required. In addition to user-specified geometric shapes, the proposed algorithms may be extended to assemble shapes specified in a natural way. For example, a swarm of robots may be used to clean an area of chemical pollutants in a water body. Here, the area of the densest pollutants can be treated as the desired shape and the density change in the nearby water can be regarded as the gradient. In this case, the shape is “specified” by the environment naturally and the proposed algorithms can be potentially applied.

One limitation of the proposed strategy is that it can only assemble shapes with single connected components. That is because the mean-shift exploration of each robot is based on local sensing of unoccupied locations. Since the sensing range of each robot is limited, if two disconnected shape components are far away from each other, it is difficult for a robot to explore unoccupied locations across different shape components. This problem may be potentially solved in several ways in practice. The first is to globally assign the robots to different components in advance. This way requires a centralized assignment process. A simple distributed solution is to add a line segment to bridge two disconnected components so that the entire shape becomes connected. In this way, each robot can explore the bridge to move from one shape component to another.

Moreover, we achieved the environment exploration tasks by treating them as shape assembly tasks, where the map of the environment is known and specified as the desired shape. While the map of the environment to be explored may not be available in practical search and rescue tasks, the proposed strategy can be applied if a minimal modification is made so that any regions unoccupied by obstacles are treated as desired shape regions to fulfill. Finally, although the algorithms proposed in this work are fully distributed, the physical robots in our experiments were provided data from a centralized motion capture system. It will be a promising research direction to develop robotic hardware systems with onboard sensing units to realize fully distributed shape assembly tasks of large-scale robot swarms.

## Methods

### Specify a desired shape as a black-white grid

The proposed human-swarm interface allows the user to either load a predesigned image or manually draw one in the workspace (Supplementary Figure 1). Either way, we can obtain a binary grid with black and white cells. Each cell of the grid is described by two basic parameters ** ρ** and

*ξ*

_{ρ}. Here,

**= (**

*ρ**ρ*

_{x},

*ρ*

_{y}) is the column and row indexes of the cell and

*ξ*

_{ρ}∈ {0, 1} represents the color of the cell:

*ξ*

_{ρ}= 0 if the cell is black and

*ξ*

_{ρ}= 1 if the cell is white. The desired shape corresponds to the set of black cells. More information is given in Section 1 in the Supplementary Information.

### Convert the desired shape to a grayscale grid

The purpose to convert the black-white grid to a grayscale grid is to expand the influence scope of the desired shape so that the robots can move into the desired shape more smoothly. The gray conversion is based on the distance transformation algorithm^{29}. Specifically, we expand the set of black cells out by *h* cells to generate an *h*-level grayscale grid. The gray level of each cell denoted as *ξ*_{ρ} is calculated based on a local parallel method described in ref. ^{29}. In particular, for any cell ** ρ** in the grid, its gray value is calculated iteratively by

Here, the superscript *k* denotes the *k*-th iteration. In every iteration, we conduct (1) for each cell. We need at most *h* − 1 iterations to obtain the final grayscale grid^{29}. Here, \({{{{{{{{\mathcal{M}}}}}}}}}_{{{{{{{{\boldsymbol{\rho }}}}}}}}}\) is a 3 × 3 mask centered at the cell ** ρ**. Illustrative examples and more information can be found in Section 1 in the Supplementary Information.

### Parameterization of the size of each cell in the grid

The desired shape represented by a grid obtained from the human-swarm interface is merely graphical. Some important parameters of the desired shape are to be determined automatically. The first parameter is the size length of each cell denoted as *ℓ*_{cell}. On the one hand, the total area of all the black cells is \({\ell }_{{{{{{{{\rm{cell}}}}}}}}}^{2}{n}_{{{{{{{{\rm{cell}}}}}}}}}\). On the other hand, let *r*_{avoid} be the collision avoidance distance between the centers of any two robots. It is expected that the distance among each pair of robots in the desired shape is equal to *r*_{avoid} (Supplementary Figure 2c). As a result, the space occupied by each robot could be approximated by a circle with the center at the robot and the radius as *r*_{avoid}/2. Thus, the area occupied by all the *n*_{robot} robots is \(\pi {({r}_{{{{{{{{\rm{avoid}}}}}}}}}/2)}^{2}{n}_{{{{{{{{\rm{robot}}}}}}}}}\). The above two areas are expected to be equal so that the robots could cover the desired shape (Supplementary Figure 2c):

from which *ℓ*_{cell} can be solved as

Equation (2) indicates that, when *n*_{robot} and *r*_{avoid} are given, *ℓ*_{cell} is inversely proportional to \(\sqrt{{n}_{{{{{{{{\rm{cell}}}}}}}}}}\).

It is notable that *n*_{cell} is specified by the user when drawing a shape in the human-swarm interface. If *n*_{cell} = *n*_{robot}, equation (2) becomes \({\ell }_{{{{{{{{\rm{cell}}}}}}}}}=\sqrt{\frac{\pi }{4}}{r}_{{{{{{{{\rm{avoid}}}}}}}}}\,\approx \,{r}_{{{{{{{{\rm{avoid}}}}}}}}}\), which means the side length of each cell is equal to the avoidance distance. In this case, each robot can approximately occupy one cell. However, our method does not necessarily require *n*_{cell} = *n*_{robot}, which is a strict condition. The proposed method allows more general cases where *n*_{cell} and *n*_{robot} may mismatch (see examples in Fig. 5 and Supplementary Figure 5).

### Parameterization of the position and orientation of the desired shape

Another two parameters of the desired shape are the center position and orientation. There is no centralized assignment of the position or orientation. The robots can negotiate the two parameters in a distributed manner.

Before proceeding further, we need to introduce some necessary notations that will be used frequently later. Suppose there are *n*_{robot} mobile robots in \({{\mathbb{R}}}^{2}\) and *n*_{robot} ≥ 2. Each robot is regarded as a circle with the radius as *r*_{body}. Let \({{{{{{{{\bf{p}}}}}}}}}_{i}\in {{\mathbb{R}}}^{2}\) be the position of the center point of robot *i* in a global coordinate frame. The dynamic model of each robot is assumed to be \({\dot{{{{{{{{\bf{p}}}}}}}}}}_{i}={{{{{{{{\bf{v}}}}}}}}}_{i}\) where *i* = 1, …, *n*_{robot}. Here, **v**_{i} is the velocity command to be designed. When the distance between two robots is less than a threshold *r*_{sense}, the two robots could share information with each other. The information network defines an undirected graph \({{{{{{{\mathcal{G}}}}}}}}=({{{{{{{\mathcal{V}}}}}}}},\,{{{{{{{\mathcal{E}}}}}}}})\), which consists of a vertex set \({{{{{{{\mathcal{V}}}}}}}}=\{1,\ldots,\,{n}_{{{{{{{{\rm{robot}}}}}}}}}\}\) and an edge set \({{{{{{{\mathcal{E}}}}}}}}\subseteq {{{{{{{\mathcal{V}}}}}}}}\times {{{{{{{\mathcal{V}}}}}}}}\) such that \({{{{{{{\mathcal{E}}}}}}}}=\{(i,\,j):\parallel {{{{{{{{\bf{p}}}}}}}}}_{i}-{{{{{{{{\bf{p}}}}}}}}}_{j}\parallel < \,{r}_{{{{{{{{\rm{sense}}}}}}}}},\,j\ne i\}\). Here, ∥ ⋅ ∥ is the Euclidean norm. The set of neighbors of robot *i* is \({{{{{{{{\mathcal{N}}}}}}}}}_{i}=\{j\in {{{{{{{\mathcal{V}}}}}}}}:(i,\,j)\in {{{{{{{\mathcal{E}}}}}}}}\}\).

Suppose *ρ*_{o} is the cell located closest to the center of the desired shape. Let \({{{{{{{{\bf{p}}}}}}}}}_{{{{{{{{{\boldsymbol{\rho }}}}}}}}}_{o}}\) and \({{{{{{{{\bf{v}}}}}}}}}_{{{{{{{{{\boldsymbol{\rho }}}}}}}}}_{o}}\) be the position and velocity of the center point of cell *ρ*_{o} in a global reference frame. Let *ϕ* denote the orientation angle of the desired shape. Then, the position and orientation of the desired shape could be represented by \({{{{{{{{\bf{p}}}}}}}}}_{{{{{{{{{\boldsymbol{\rho }}}}}}}}}_{o}}\) and *ϕ*, respectively.

Every robot has its own interpretation of \({{{{{{{{\bf{p}}}}}}}}}_{{{{{{{{{\boldsymbol{\rho }}}}}}}}}_{o}}\) and \({{{{{{{{\bf{v}}}}}}}}}_{{{{{{{{{\boldsymbol{\rho }}}}}}}}}_{o}}\), denoted as \({{{{{{{{\bf{p}}}}}}}}}_{{{{{{{{{\boldsymbol{\rho }}}}}}}}}_{o},i}\) and \({{{{{{{{\bf{v}}}}}}}}}_{{{{{{{{{\boldsymbol{\rho }}}}}}}}}_{o},i}\). Initially, \({{{{{{{{\bf{p}}}}}}}}}_{{{{{{{{{\boldsymbol{\rho }}}}}}}}}_{o},i}({t}_{0})={{{{{{{{\bf{p}}}}}}}}}_{i}({t}_{0})\) and \({{{{{{{{\bf{v}}}}}}}}}_{{{{{{{{{\boldsymbol{\rho }}}}}}}}}_{o},i}({t}_{0})={{{{{{{{\bf{v}}}}}}}}}_{i}({t}_{0})\), which means that each robot initially treats itself as the center of the desired group shape. The interpretations of different robots can gradually reach a consensus by the following distributed negotiation algorithm:

There are two terms in (3). The first term in (3) is the average of the deficiency of the position interpretations between robot *i* and its neighbors. Here, sign( ⋅ ) and ∣ ⋅ ∣ denote the sign and the absolute value of a real number. In addition, *c*_{1} > 0 and 0 < *α* < 1 are two constant gains. The role of the first term is to drive \({{{{{{{{\bf{p}}}}}}}}}_{{{{{{{{{\boldsymbol{\rho }}}}}}}}}_{o},i}\to {{{{{{{{\bf{p}}}}}}}}}_{{{{{{{{{\boldsymbol{\rho }}}}}}}}}_{o},j}\) where \(j\in {{{{{{{{\mathcal{N}}}}}}}}}_{i}\). The second term is the average of the velocities of the neighboring robots. Its role is to drive \({{{{{{{{\bf{v}}}}}}}}}_{{{{{{{{{\boldsymbol{\rho }}}}}}}}}_{o},i}\to {{{{{{{{\bf{v}}}}}}}}}_{{{{{{{{{\boldsymbol{\rho }}}}}}}}}_{o},j}\) where \(j\in {{{{{{{{\mathcal{N}}}}}}}}}_{i}\). Since *α* < 1, consensus can be achieved in a finite time. This is important for speeding up the negotiation process. The convergence analysis is given in Theorem 1 in the Supplementary Information.

Regarding the orientation negotiation, let *ϕ*_{i} be the interpretation of *ϕ* by robot *i*. The initial values of *ϕ*_{i} could be randomly selected or based on task-oriented requirements. The distributed orientation negotiation algorithm is

where *c*_{2} is a positive constant coefficient. This algorithm has the same structure as (3) and can be analyzed analogously.

If the desired shape is required to track a specified trajectory, we can introduce informed robots to achieve that. Details are given in Section 2 in Supplementary Information.

### Shape-entering velocity command

The shape-entering velocity command \({{{{{{{{\bf{v}}}}}}}}}_{i}^{{{{{{{{\rm{ent}}}}}}}}}\) in (5) aims to drive robot *i* into its interpretation of the desired shape, as depicted in Supplementary Figure 3a. In particular, it is designed as

where **p**_{i} is the position of robot *i* and **p**_{T,i} is a target location for robot *i* to move toward. Hence, \(\frac{{{{{{{{{\bf{p}}}}}}}}}_{{{{{{{{\rm{T}}}}}}}},i}-{{{{{{{{\bf{p}}}}}}}}}_{i}}{\parallel {{{{{{{{\bf{p}}}}}}}}}_{{{{{{{{\rm{T}}}}}}}},i}-{{{{{{{{\bf{p}}}}}}}}}_{i}\parallel }\) is a unit vector pointing from **p**_{i} to **p**_{T,i}. Therefore, the first term in (5) drives the robot toward its target location. Here, \({\kappa }_{1}{\xi }_{{{{{{{{{\boldsymbol{\rho }}}}}}}}}_{i}}\) is a control gain where *ρ*_{i} is the index of the cell that is closest to \({{{{{{{{\bf{p}}}}}}}}}_{i},\,{\xi }_{{{{{{{{{\boldsymbol{\rho }}}}}}}}}_{i}}\) is the gray level of that cell, and *κ*_{1} > 0 is a constant gain. The second term \({{{{{{{{\bf{v}}}}}}}}}_{{{{{{{{{\boldsymbol{\rho }}}}}}}}}_{o},i}\) in (5) is the local interpretation of robot *i* on the moving velocity of the entire shape. This term is necessary when the desired shape is moving. The calculation of the cell index *ρ*_{i} and the target location **p**_{T,i} is given in Section 3 in the Supplementary Information.

### Shape-exploration velocity command

The shape-exploration velocity command \({{{{{{{{\bf{v}}}}}}}}}_{i}^{\exp }\) in (6) aims to push robot *i* into the desired shape and explore unoccupied black cells. Here, a cell is defined as occupied if the distance between its center and any robot is less than *r*_{avoid}/2; and unoccupied otherwise.

In particular, we employ the mean-shift concept^{26} to design \({{{{{{{{\bf{v}}}}}}}}}_{i}^{\exp }\) as

which is a normalized weighted average of **p**_{ρ} − **p**_{i}, where **p**_{i} is robot *i*’s position and **p**_{ρ} is a position of a valid cell whose index is ** ρ**. Here,

**belongs to the set \({{{{{{{{\mathcal{M}}}}}}}}}_{i}^{{{{{{{{\rm{neigh}}}}}}}}}\). The factor**

*ρ**κ*

_{2}∈ {

*σ*

_{1},

*σ*

_{2}} with

*σ*

_{1},

*σ*

_{2}> 0 is a positive constant. The weight for

**p**

_{ρ}−

**p**

_{i}is a function

*ψ*defined as

This function is monotonically decreasing from 1 to 0 as *z* increases. As a result, the weight *ψ*(∥**p**_{ρ} − **p**_{i}∥/*r*_{sense}) is large when the distance between **p**_{ρ} and **p**_{i} is small. Hence, more weights are given to the cells that are closer to robot *i*.

The velocity command in (6) encourages robot *i* to explore unoccupied black cells. Here, \({{{{{{{{\mathcal{M}}}}}}}}}_{i}^{{{{{{{{\rm{neigh}}}}}}}}}\) is defined as the set of all the unoccupied black cells that are within the sensing radius *r*_{sense} of the robot (see Supplementary Figure 3b) and *κ*_{2} = *σ*_{2}. In addition, when robot *i* is close to the boundary of the shape so that there are non-black cells within the sensing radius, on top of exploring unoccupied black cells, the velocity command also aims to push robot *i* into the desired shape. In this case, \({{{{{{{{\mathcal{M}}}}}}}}}_{i}^{{{{{{{{\rm{neigh}}}}}}}}}\) includes all the black cells no matter whether they are occupied or not and *κ*_{2} = *σ*_{1}.

### Interaction velocity command

The aim of the interaction velocity command \({{{{{{{{\bf{v}}}}}}}}}_{i}^{{{{{{{{\rm{int}}}}}}}}}\) in (7) is to achieve collision avoidance and velocity alignment. To that end, it is designed as

where *κ*_{3} is a positive control gain.

The first term in (7) is a weighted sum of **p**_{i} − **p**_{j}, where **p**_{i} is the position of robot *i* and **p**_{j} is the position of the neighboring robot *j* or a collision point *j*. Here, \(j\in {{{{{{{{\mathcal{N}}}}}}}}}_{i}\cup {{{{{{{{\mathcal{O}}}}}}}}}_{i}\) where \({{{{{{{{\mathcal{N}}}}}}}}}_{i}\) is the set of neighboring robots and \({{{{{{{{\mathcal{O}}}}}}}}}_{i}\) is the set of collision points (see Supplementary Figure 3c). The weight *μ*(∥**p**_{i} − **p**_{j}∥) is defined as

By definition, when ∥**p**_{i} − **p**_{j}∥ is close to zero, the weight *μ*(∥**p**_{i} − **p**_{j}∥) methods to infinity. When ∥**p**_{i} − **p**_{j}∥ is close to *r*_{avoid}, the weight *μ*(∥**p**_{i} − **p**_{j}∥) monotonically decreases to zero. Since **p**_{i} − **p**_{j} is a vector pointing from **p**_{j} to **p**_{i}, the first term is a repulsive velocity that pushes robot *i* away from its surroundings to avoid a collision. The second term in (7) aims to align robot *i*’s velocity with its neighbors. Velocity alignment can help reduce the chance of inter-robot collision. In the meantime, it is necessary when the entire shape needs to track a moving trajectory.

### Performance metrics

To evaluate the performance of the proposed strategy, we consider the following four metrics.

The first metric, coverage rate, is to evaluate the proportion of the black cells in the desired shape that are occupied by robots. Here, a cell is defined as occupied if the distance between its center and any robot is less than *r*_{avoid}/2. The metric is defined as

where *n*_{occ} is the number of black cells occupied by robots. If all black cells are occupied, that is *n*_{occ} = *n*_{cell}, then *M*_{1} = 100%. If no black cells are occupied, then *M*_{1} = 0.

The second metric, entering rate, is the proportion of the robots that have entered the desired shape. Here, a robot is said to be inside the shape if it occupies a black cell. The metric is defined as

where *n*_{in} is the number of robots inside the desired shape. If all robots are inside the desired shape, then *M*_{2} = 100%, and otherwise *M*_{2} < 100%.

The third metric, distribution uniformity, measures the distribution uniformity of the robots in the desired shape. Denote \({r}_{\min,i}={\min }_{j\in {{{{{{{{\mathcal{N}}}}}}}}}_{i}}\parallel {{{{{{{{\bf{p}}}}}}}}}_{i}-{{{{{{{{\bf{p}}}}}}}}}_{j}\parallel\) as the minimum distance from robot *i* to its neighbors. Then, the metric is defined as

where \({\bar{r}}_{\min,i}=\frac{1}{n}\mathop{\sum }\nolimits_{i=1}^{n}{r}_{\min \!,i}\). If the distance between every pair of robots is the same, then *M*_{3} = 0; otherwise, *M*_{3} > 0.

The fourth metric, velocity polarization, is to measure the polarization of the velocities of the robots. The metric is defined as:

If all robots move with the same velocity, then *M*_{4} = 1. If the robots’ velocities are random, then *M*_{4} is close to zero.

### Statistics and reproducibility

Data analysis was done by using the native functions of Originlab. All the statistical results are displayed in the format of “average ± minimum/maximum”. During the statistical analysis, no statistical method was used to predetermine sample size, and no data were excluded. All statistical trials used random initialization of robots’ locations and their interpretations of the desired shape. In all bar charts, the lower and upper edges of the bar represent the minimum and maximum, and the shaded region represents the average.

We implemented the simulation in Matlab. Details of the simulation setup and parameters used are given in Section 4 in the Supplementary Information. In our experiments, we use 50 holonomic wheeled robots. We implemented the experiments in Visual Studio using C++. Details of the experimental setup, robotic platform, and parameter settings are given in Section 5 in the Supplementary Information.

## Data availability

The data used in this study are available from the corresponding author upon request.

## Code availability

The code of the human-swarm interface used in this study is publicly available at https://github.com/WestlakeAerialRobotics/Human-swarm-interface(DOI: 10.5281/zenodo.7960508).

## References

Anderson, C., Theraulaz, G. & Deneubourg, J. L. Self-assemblages in insect societies.

*Insectes Soc.***49**, 99–110 (2002).Chen, Y. & Kolokolnikov, T. A minimal model of predator-swarm interactions.

*J. R. Soc. Interface***11**, 20131208 (2014).Vicsek, T. & Zafeiris, A. Collective motion.

*Phys. Rep.***517**, 71–140 (2012).Gelblum, A. et al. Ant groups optimally amplify the effect of transiently informed individuals.

*Nat. Commun.***6**, 7729 (2015).Berman, S., Lindsey, Q., Sakar, M. S., Kumar, V. & Pratt, S. C. Experimental study and modeling of group retrieval in ants as an approach to collective transport in swarm robotic systems.

*Proc. IEEE***99**, 1470–1481 (2011).Ren, W., Beard, R. W. & Atkins, E. M. Information consensus in multivehicle cooperative control.

*IEEE Control Syst. Mag.***27**, 71–82 (2007).Brambilla, M., Ferrante, E., Birattari, M. & Dorigo, M. Swarm robotics: a review from the swarm engineering perspective.

*Swarm Intell.***7**, 1–41 (2013).Oh, K. K., Park, M. C. & Ahn, H. S. A survey of multi-agent formation control.

*Automatica***53**, 424–440 (2015).Oh, H., Shirazi, A. R., Sun, C. & Jin, Y. Bio-inspired self-organising multi-robot pattern formation: a review.

*Rob. Autom. Syst.***91**, 83–100 (2017).Alonso-Mora, J., Breitenmoser, A., Rufli, M., Siegwart, R. & Beardsley, P. Image and animation display with multiple mobile robots.

*Int. J. Robot. Res.***31**, 753–773 (2012).Sakurama, K. & Ahn, H. S. Multi-agent coordination over local indexes via clique-based distributed assignment.

*Automatica***112**, 108670 (2020).Wang, H. & Rubenstein, M. Shape formation in homogeneous swarms using local task swapping.

*IEEE Trans. Robot.***36**, 597–612 (2020).Anderson, B. D., Yu, C., Fidan, B. & Hendrickx, J. M. Rigid graph control architectures for autonomous formations.

*IEEE Control Syst. Mag.***28**, 48–63 (2008).Zhao, S. & Zelazo, D. Bearing rigidity theory and its applications for control and estimation of network systems: Life beyond distance rigidity.

*IEEE Control Syst. Mag.***39**, 66–83 (2019).Kuhn, H. W. The Hungarian method for the assignment problem.

*Naval Res. Logistics***2**, 83–97 (1955).Bertsekas, D. P. The auction algorithm: a distributed relaxation method for the assignment problem.

*Ann. Oper. Res.***14**, 105–123 (1988).Kamel, M. A., Yu, X. & Zhang, Y. Formation control and coordination of multiple unmanned ground vehicles in normal and faulty situations: a review.

*Annu. Rev. Control***49**, 128–144 (2020).Rubenstein, M., Cornejo, A. & Nagpal, R. Programmable self-assembly in a thousand-robot swarm.

*Science***345**, 795–799 (2014).Slavkov, I. et al. Morphogenesis in robot swarms.

*Sci. Robot.***3**, eaau9178 (2018).Gauci, M., Nagpal, R. & Rubenstein, M. Programmable self-disassembly for shape formation in large-scale robot collectives, pp. 573–586 (Springer, Cambridge, 2018).

Alhafnawi, M., Hauert, S. & O’Dowd, P. Self-organised saliency detection and representation in robot swarms.

*IEEE Robot. Autom. Lett.***6**, 1487–1494 (2021).Hsieh, M. A., Kumar, V. & Chaimowicz, L. Decentralized controllers for shape generation with robotic swarms.

*Robotica***26**, 691–701 (2008).Sabattini, L., Secchi, C. & Fantuzzi, C. Arbitrarily shaped formations of mobile robots: artificial potential fields and coordinate transformation.

*Auto. Robots***30**, 385 (2011).Hou, S. P. & Cheah, C. C. Dynamic compound shape control of robot swarm.

*IET Control Theory Appl.***6**, 454–460 (2012).Vickery, C. & Salehi, S.A. A mean shift-based pattern formation algorithm for robot swarms. in

*7th Int. Conf. Autom., Robot. Appl*., pp. 16–20 (Prague, Czech Republic, IEEE, 2021).Fukunaga, K. & Hostetler, L. The estimation of the gradient of a density function, with applications in pattern recognition.

*IEEE Trans. Inf. Theory***21**, 32–40 (1975).Cheng, Y. Mean shift, mode seeking, and clustering.

*IEEE Trans. Pattern Anal. Mach. Intell.***17**, 790–799 (1995).Comaniciu, D. & Meer, P. Mean shift: a robust approach toward feature space analysis.

*IEEE Trans. Pattern Anal. Mach. Intell.***24**, 603–619 (2002).Borgefors, G. Distance transformations in digital images.

*Comput. Vis. Graph. Image Process.***34**, 344–371 (1986).Morgan, D., Subramanian, G. P., Chung, S. J. & Hadaegh, F. Y. Swarm assignment and trajectory optimization using variable-swarm, distributed auction assignment and sequential convex programming.

*Int. J. Robot. Res.***35**, 1261–1285 (2016).Zhao, S. Affine formation maneuver control of multiagent systems.

*IEEE Trans. Autom. Control***63**, 4140–4155 (2018).Dong, X., Yu, B., Shi, Z. & Zhong, Y. Time-varying formation control for unmanned aerial vehicles: Theories and applications.

*IEEE Trans. Control Syst. Technol.***23**, 340–348 (2014).Yang, T. et al. A damage-tolerant, dual-scale, single-crystalline microlattice in the knobby starfish, protoreaster nodosus.

*Science***375**, 647–652 (2022).

## Acknowledgements

We thank Fei Chen, Canlun Zheng, Weicong Zhang, Zian Ning, Ye Zheng, and Yin Zhang at Westlake University for their help in the experiments, and Chen Wei at Beihang University for helpful discussions. This work was supported by the STI 2030-Major Projects (Grant No. 2022ZD0208804), the Beihang Zhuoyue Postdoctoral Fellow Program (G.S.), the Hangzhou Key Technology Research and Development Program (Grant No. 20200416A16), and Research Center for Industries of the Future at Westlake University (Grant No. WU2022C027).

## Author information

### Authors and Affiliations

### Contributions

S.Z., G.S., and R.Z. designed research; G.S., Z.M., Y.L., and S.Z. performed research; G.S., S.Z., R.G., and Z.C. analyzed data; S.Z., G.S., R.G., and Z.C. wrote the paper.

### Corresponding author

## Ethics declarations

### Competing interests

The authors declare no competing interests.

## Peer review

### Peer review information

*Nature Communications* thanks Nicola Bastianello, Sabine Hauert, and the other, anonymous, reviewer(s) for their contribution to the peer review of this work.

## Additional information

**Publisher’s note** Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

## Rights and permissions

**Open Access** This article is licensed under a Creative Commons Attribution 4.0 International License, which permits use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons license, and indicate if changes were made. The images or other third party material in this article are included in the article’s Creative Commons license, unless indicated otherwise in a credit line to the material. If material is not included in the article’s Creative Commons license and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder. To view a copy of this license, visit http://creativecommons.org/licenses/by/4.0/.

## About this article

### Cite this article

Sun, G., Zhou, R., Ma, Z. *et al.* Mean-shift exploration in shape assembly of robot swarms.
*Nat Commun* **14**, 3476 (2023). https://doi.org/10.1038/s41467-023-39251-5

Received:

Accepted:

Published:

DOI: https://doi.org/10.1038/s41467-023-39251-5

## Comments

By submitting a comment you agree to abide by our Terms and Community Guidelines. If you find something abusive or that does not comply with our terms or guidelines please flag it as inappropriate.