Networking chemical robots for reaction multitasking

The development of the internet of things has led to an explosion in the number of networked devices capable of control and computing. However, whilst common place in remote sensing, these approaches have not impacted chemistry due to difficulty in developing systems flexible enough for experimental data collection. Herein we present a simple and affordable (<$500) chemistry capable robot built with a standard set of hardware and software protocols that can be networked to coordinate many chemical experiments in real time. We demonstrate how multiple processes can be done with two internet-connected robots collaboratively, exploring a set of azo-coupling reactions in a fraction of time needed for a single robot, as well as encoding and decoding information into a network of oscillating reactions. The system can also be used to assess the reproducibility of chemical reactions and discover new reaction outcomes using game playing to explore a chemical space.


Robot design and concept:
The robot computational core is a pcDuino3 running Linux Ubuntu operating system that executes homebuilt code in python to control a number of pumps and a webcam. Access to internet is achieved via a wired ethernet or WiFi connection. For everyday use the board was connected also to a monitor, mouse and keyboard. Liquid handling is performed by a set of peristaltic pumps, the pumps are turned on for duration of the required addition time. The pumps are connected through tygon® tubing with the reagents and the reaction flask. Generally 5 pumps are dedicated to adding the reagents, one adds water for washing and the last one is used to empty the reaction flask. Data is aquired with a USB webcam able to record images and video from the reactions. The reactions are performed in a standard 14ml glass vial. It is magnetically stirred with a home built stirrer using a small fan. The robot has been designed to be as simple and affordable as possible. Therefore it can be assembled in just few hours.

Peristaltic pumps
The control over the solutions was performed using a set of peristaltic pumps. The pump is driven by a 12V DC and it is connected to the driver board mounted on pcDuino. In this work, we used the model KFS-HB2B06M, where M is either R,B,G,P which refer to pump colour (Red,Blue,Green,Purple). The pumps are designed to have a flow rate of 4ml/min towards a single direction. Since a loss in precision over time was observed the pumps were recalibrated every week and after any manteinance operations.

pcDuino board
The robot runs on a pcDuino3, it is powered by a 5V (2A) power supply fed through a micro USB cable. This board features the following: • CPU: AllWinner A20 SoC 1GHz ARM Cortex A7 Dual Core

Power supply unit
The robot is powered by a 5V (2A) DC power source. However, the Peristaltic pumps are driven by a 12V (1A) DC source. In this work, a 500W ATX power supply unit was used.

Software
The pcDuino3 runs with the Ubuntu operating system. The platform is controlled by a dedicated program written in python. Due to specific experiments each project part has been completed by using a dedicated program. However, the low-level software is the same and is composed by three main parts with respective external libraries: Pump control: This is based on gpio, a common library to control the pins of the pcDuino, and therefore operate the pumps. Since there is no feedback from the pumps a code converts the amount of solution required into a time interval used to run the pump. This time interval is derived from a calibration process where the flow rate of each pump is tested, verified and saved as a variable.
Webcam control: This is based on the OpenCV library. The webcam is accessed by the computer and provides images and videos of the reaction. Further image/video analysis will be discussed in the respective project sections.

S6
Network management: This uses the Twython library and controls the networked part of the platforms. It allows the platform to update its state by sending a tweet on its account and scan other accounts for synchronization and collaboration.
Coordinator: The software core of each project section is a "coordinator" program. It manages all the experiment components: physical reactions, image analysis, network synchronization and search algorithm. Figure 5: Description of the general operational software. At the top the common and lower layer code is reported. Based on that we developed specific programs for each project. They will be discussed in the relative sections.

Colour detection
For the colour determination, the image frames recorded are converted into hsv colour domain. To allow the calibration of each colour, they have to be associated to a specific hsv range value. In each experiment a region of interest is analyzed and the pixel values are compared with the color ranges.
The colour with the highest pixel count is considered the solution colour (red, orange, yellow, blue, colorless, black). Individual colour counts are saved and stored in a csv file for post processing.

Collaborative algorithm
Two identical and physically separated platforms have been used to explore the 117 reaction grid.
They run the same algorithm and the aim was to find a blue reaction using a random search, sharing the results in real time using Twitter to reduce total time. The algorithm starts by selecting a random reaction and sending a Tweet with the reaction parameters. The system then performs the selected reaction and saves 4 frames. These are analyzed on board, the database is updated and an "end" Tweet with the results is sent. If a blue reaction is not present in the database the board will restart with a new random reaction, otherwise it will send a "stop" Tweet and stop. A separated thread in the background checks every 5 minutes the other board's Tweets and update the database with those reactions result. In this way both boards will avoid performing the same reaction twice. This script has been used to look for a blue reaction out of 117 total combinations. After 14 sequences blue has been found on average after 15.1 reactions. The theoretical number of reactions necessary for two platforms sharing results looking for 3 blue reactions out of 117 is 19.5.

Plot the oscillations
In order to observe the oscillation period behaviour over time a script for data processing was created and the output demonstrated in Supplementary Figure 11

Predicting the chemical influence on oscillation period.
In order to predict the behaviour of the oscillation period when small amounts of water and potassium bromate are added we monitored several reactions while constant and regular additions were made.
By processing the results, it has been possible to obtain two functions that correlate the amount of material added with the oscillation period change, within a reasonable time window and error. It will give an estimate of water amount to add in order to reach a specific period. Since it is referred to the reaction start, for real-time additions we need to consider also the current period: It easy to see that the empirical constant k is irrelevant, the function used to predict water additions is: When bromate is added there is a faster period, see Figure 18 and 19. By using the data obtained in multiple addition tests we obtained the first empirical function Num value Current peridod -Goal period S15 4. Inorganic

Stage one-Collaboratively explore a chemical space
At each stage of synthesis and analysis both platforms update shared network files for the other to read and proceed accordingly. When one platform selects a reaction volume at random to explore, the other will acknowledge this and remove it from its own series before continuing with its own choice. Conditions that have produced crystals are stored by both platforms for repetition later. The flow diagram (supplementary Figure 17) describes the collaborative process between two platforms, the leader and follower: Different parameters are tested by two systems sharing the workload. S16

Stage two-Repetition of Successful conditions
The successful reactions conditions from the collaborative stage are compiled and repeated in order to establish the reporducibility of the chemistry/crystallization. One set of conditions is choosen and both platforms perform repeat reactions. Once enough data has been collected to establish an average percentage of reproducibility of obtaining crystals is complete and the next set of reaction conditions are begun, see Supplementary Figures 18 and 19.
Seen below is the outline of stage 2, Assessment of the reproducibilty of crystal producing reaction conditions.

Grid search of reaction conditions
8 reaction series each with varying Mn:W ratio were performed collaboratively by the two platforms by the methods detailed above (Supplementary table 1). Each reaction series varied in acid volume from 1.4-1.54mL HCl (approximately between pH 3-6.5) and each reaction was monitored by web cam for crystal formation within 2 hours of reaction completion. The full grid was repeated 3 times S18 to more thoroughly explore the space. The results of each grid can be seen in below as 2D colour Each of the conditions marked in red produced crystals at least once during these automated runs, all were repeated to assess the likelihood of growing crystals again. After 15 repeat reactions if no crystals had been produced the reaction was abandoned and the next set of conditions were started.
A significant number of these crystalizing conditions never again produced crystals. Others varied from 10-50% in frequency of crystal formation across up to 48 reactions (Supplementary table 2).

Agent based simulation
To demonstrate the importance of information sharing between chemical robots we first developed less advantageous. The individual strategy is useful with a small number of robots but has diminishing returns with increasing numbers of agents. An increase of one robot from one to two yields a 50% improvement while an increase from two robots to three yields a lower improvement of 33.3% and so on. The reason of the performance constantly rising in Supplementary Figure 23-right is in the parallel search: when performing the searches in parallel with multiple robots although only one robot (most likely) reaches the goal the rest of the robots still perform their actions and these excess actions are therefore wasted. The amount of excess increases with the number of robots in use. It is important to note however that this is only an excess of actions so that the waste is one of resources. The time it takes to find the goal is always better with more robots and is unaffected by this latter issue. The simulations show that the collaboration strategy is by far the most efficient and that as the number of available robots increases the benefit of using collaboration increases as well. Supplementary Figure   23-right shows that for all strategies, the total number of searches that had to be conducted decreases.
With the y axis logarithmic, the constant slopes show that the improvement in searching is exponential. The individual strategy will always be better than the random one, no matter the number S22 6. Game

General Overview
Two automated platforms were tasked with playing a game of Hex. New/rare reaction results allowed the player to use the optimum movement (determined algorithmically described later) with uncommon/common results allowing only for sub-optimal/random movements. Losing games trigger a change in strategy for the losing platform, in this case an expansion of the reaction grid the player was allowed to explore. The idea being to show that a game outcome could drive a player to either change or maintain its current strategy in the hope of making more chemical discoveries in future.
The chemistry chosen for this project was the same seen in the Organic section (Methods part I Organic) and results were gathered and analyzed via web-cam.

Decision Making
The goal for players in a Hex game is to connect one side of the board with the opposite side using a continuous line of that player's color. The game cannot end in a draw. From a randomly assigned first board position or the current state of the board the optimal movement was calculated using Monte-Carlo simulations with the goal of completing the game. Once an optimal movement has been calculated, the results of the chemistry determine if the player may use it. Color rarity vs move selection allowance is determined by the following: -Unique/Rare colors observed up to 4 times Optimal movement -Uncommon Colors observed between 5 and 7 times Sub-optimal movement -Common Colors observed more than 7 times Random movement Sub-optimal movements were defined as a position beside, above or below the optimal and was selected based on availability.

Communication Between Platforms
In order to keep both players in sync with one another, a remote server was developed to handle all communications between the platforms. Each platform selects a reaction and processes the information through image analysis and the decision-making algorithm as described previously. The selected move is then sent to the remote server from the platform for processing. All logic for the game, such as updating board movements, is handled by the server. Once an iteration of the game has been completed, the server broadcasts a message to all connected clients detailing who has won the game. The players then adjust their strategies accordingly.

S23
The reasoning behind developing a remote server system for this task was a separation of concerns.
By separating the game logic from the platforms, as opposed to each platform having its own representation of the game, we minimize the risk of each platform falling out of sync with one another leading to inaccurate results. We also prevent potential race conditions with platforms attempting to access a single file at the same time. A single server with file access eliminates this risk. The design of the server allows for multiple concurrent connections and data processing which opens the possibility of increasing the number networked platforms working towards a common goal.

Strategy
Both players begin the first game in the sequence by selecting reactions from an identical chemical space (Supplementary Figure 24 center Given that the game sequence proceeds one platform after another the original reaction space restricts the total reaction number to 81 for each player (9 grids of 3x3 reagent volumes). A typical game sequence can consist of between 2-5 completed games. Seen below in Supplementary Figure 25 is a game sequence showing 4 complete games (5 th game was incomplete) in which the losing strategy was adopted by player 2 after game 1. Against the logical expectation the losing strategy, whilst S24 allowing player 2 to win game 2, did not result in many new unique discoveries. However, when player 1 adopted the losing strategy after game 2, its unique discovery count increased significantly, but did not result in a victory for the remained of the total game sequence. This can be explained simply by the fact a game is still based on probability and a new advantageous strategy will work most, but not all of the time.
Supplementary Figure 25: Example game series. A 4 game sequence showing adoption of a new strategy allows, over time, for an increased number chemical discoveries.