Closed-loop control of zebrafish behaviour in three dimensions using a robotic stimulus

Robotics is continuously being integrated in animal behaviour studies to create customizable, controllable, and repeatable stimuli. However, few systems have capitalized on recent breakthroughs in computer vision and real-time control to enable a two-way interaction between the animal and the robot. Here, we present a “closed-loop control” system to investigate the behaviour of zebrafish, a popular animal model in preclinical studies. The system allows for actuating a biologically-inspired 3D-printed replica in a 3D workspace, in response to the behaviour of a zebrafish. We demonstrate the role of closed-loop control in modulating the response of zebrafish, across a range of behavioural and information-theoretic measures. Our results suggest that closed-loop control could enhance the degree of biomimicry of the replica, by increasing the attraction of live subjects and their interaction with the stimulus. Interactive experiments hold promise to advance our understanding of zebrafish, offering new means for high throughput behavioural phenotyping.

Interpolation process to infer 3D coordinates A critical element in our real-time implementation was to compensate for the distortion associated with the perspective view from each camera. In the description of our interpolation process, all coordinates are referenced to the origin located at the center of the water tank from the top view and at the height of the water level in the front view.
From the tracking software, we obtained the 2D coordinates of the target ( , , ) as functions of time, where the X-and Y-coordinates were taken from the top view, and the Z-coordinate was taken from the front view. The X-coordinate from the front view was disregarded. The 2D coordinates, measured in centimeters, are denoted with a subscript "2D" to emphasize their derivation from the independent 2D views.
Before each experimental trial, we performed a simple calibration using a single frame from the two cameras, and the pixel position of the corners of the near, and far side walls of the water tank (relatively to the front camera) were manually extracted from the frames. The length of the tank, width of the tank, and height of the water level (half of the height of the tank) were measured in pixels for both perspectives using the two frames (see figure S1). The length of the tank, width of the tank, and height of the water level inferred from the near side perspective are labelled as , , and , and these same quantities from the far side perspective are termed , , and , respectively. Measured values for the near and far side perspective lengths are tabulated in table S2.
These quantities were confronted with the physical dimensions of the swimming tank for calibration -we use the notation , , and for the length of the tank, the width of the tank, and the water level, which are 74, 30, and 15 cm, respectively. The coordinates of the target obtained after interpolation are denoted with a subscript "3D" ( , , and ) and were measured in centimeters.
Briefly, the process of interpolation consisted of the following steps. First, we interpolated the 2D Z-coordinate of the target ( ) in the front view. The -coordinate from the top view (ranging from to ) was utilized to determine the value of the interpolated Z-coordinate between the far and near perspectives, based on in the front view. The -coordinate was obtained by scaling by factor between and 1 corresponding to the far and near perspective, respectively (equation 1). (1) After resolving the Z-coordinate of the focal fish, , we interpolated the X-and Ycoordinates between the far and near perspective, based on the and on the top view. Specifically, was scaled by a term ranging between 1 and , corresponding to the near and far perspective, respectively. Similarly, was scaled by a term ranging between 1 and , corresponding to the near and far perspective, respectively. The X-and Ycoordinates of the target after interpolation are shown equation 2 and 3. (2) (3)

Analysis on average speed of focal fish and stimuli
The speed of the focal fish and stimuli were computed from the distance traveled between consecutive frames and averaged across all trials. One-way ANOVA was used to compare the average speed of focal fish and stimuli with the replica conditions as the independent variable. To assess whether the appraisal of the robotic stimulus and the live counterpart by the zebrafish was comparable, we confronted the average speed of the focal fish and of the stimulus in the condition 2-Fish with respect to all the replica conditions using a one-tail twosample t-tests assuming equal variances. All the analyses were conducted with p<0.05, except for the pairwise comparison, in which the statistical significance was determined based on a corrected p-value, which was set to 0.01 based on Bonferroni correction (1).
The average speed of the focal fish was found not to vary across the replica conditions ( figure S2). Pairwise comparisons did not indicate a difference in the focal fish average speed between the 2-Fish and the replica conditions (XYZ-OL: ; X-CL: ; Y-CL: ; and Z-CL: , except for the XYZ-CL condition ( . The average speed of the stimulus was not found to vary across the replica conditions ( figure S3). Pairwise comparisons did not indicate a difference in the stimulus average speed between condition 2-Fish and the replica conditions (XYZ-OL: ; Z-CL: ; and XYZ-CL: ), except for X-CL ), and Y-CL . The increase of the average speed in closed-loop control along the three axes may be related to the specific effect produced by closing the loop in the X-axis. As explained in the manuscript, this might have led the focal fish to perceive that the replica was thrashing against the transparent wall, thereby triggering an escape response in the focal subjects.
Our findings indicate that the replica was generally slower than the live stimulus, with a significant difference attained when closing the loop in the X-or Y-axis. This decrease could be explained by the reduced workspace of the replica with respect to the compartment, which might have resulted in a reduced average speed of the replica.    Video S3. Front/top sample video of one experiment of fish-fish condition. The sample video shows a portion of a trial for the fish-fish (2-Fish) condition.