AI-enabled, implantable, multichannel wireless telemetry for photodynamic therapy

Photodynamic therapy (PDT) offers several advantages for treating cancers, but its efficacy is highly dependent on light delivery to activate a photosensitizer. Advances in wireless technologies enable remote delivery of light to tumors, but suffer from key limitations, including low levels of tissue penetration and photosensitizer activation. Here, we introduce DeepLabCut (DLC)-informed low-power wireless telemetry with an integrated thermal/light simulation platform that overcomes the above constraints. The simulator produces an optimized combination of wavelengths and light sources, and DLC-assisted wireless telemetry uses the parameters from the simulator to enable adequate illumination of tumors through high-throughput (<20 mice) and multi-wavelength operation. Together, they establish a range of guidelines for effective PDT regimen design. In vivo Hypericin and Foscan mediated PDT, using cancer xenograft models, demonstrates substantial suppression of tumor growth, warranting further investigation in research and/or clinical settings.

4. The in vivo experiments compared HYP(+)/LED(+) conditions with controls. However, the multichannel (multiple wavelength) work has only been demonstrated in vitro. Since the multichannel capability is a major contribution of this work, an in vivo demonstration of the multichannel/wavelength activation would be very helpful. Is it realistic to perform in vivo demonstration of the multichannel activation? How might the movement of the implant affect the results?
Minor: 5. On page 5, the authors use the five criteria (1-5) to choose the optimal condition for PDT. Some sentence describing how these 5 criteria relate to the optimum condition should be added. In addition, "Results performed under different conditions (wavelength) revealed that uniform light delivery and minimum heat dissipation for Hypericin are reached with..." Uniform light delivery and minimum heat dissipation only refers to criteria 3 and 5. A more comprehensive description may also include amount/rate of energy absorption, etc.
6. In figure 2b, what is the distance between the LED and the tumor? Is it chosen based on the experimental conditions during the in vitro testing (such as the condition in Fig.5g)?
7. On page 10, "...power budgets as low as tens of μA." Why does the power have a unit of µA? Can the authors estimate the power consumption with the reed switch compared with the power consumption simply using multiple light sources?
Reviewer #3: Remarks to the Author: I am not a domain expert in the main subject matter of this paper. I am, however, an expert in animal pose estimation. Thus, I will only comment on these aspects. The combination of antenna tracking and deeplabcut is a nice contribution (and from a non-expert view, the paper is quite exciting). However, I have questions about the AI method being used.
It appears the authors are using a beta release of DeepLabCut code that has not been published (or no preprint yet). Therefore, performance is quite unclear for this method. There are also not sufficient details in the method to understand what the authors used exactly. In the main text they mention using neural networks for pose estimation in multiple animals, and then in the methods they say deeplabcut 2.2b7--looking at this code it's clear there are many options that are not clearly described in the methods. Moreover, the authors mention bipartite matching but the methods don't detail this fully. It seems unusual, to me, that the authors here are using unpublished code without proper benchmarking and performance metrics. I give more concrete examples beflow.
"Modification to DLC model We utilized, custom-trained, and modified the DLC Python package (Ver. 2.2b7). Specifically, we used the custom-trained DLC model to estimate the locations of the body parts such as snouts and tails of the mice within an image (i.e., video frame). Note that the original DLC python package does not support a real-time processing feature, instead it only runs on video files. Hence, we directly modified the Python package in such a way that it can infer the locations of the body parts of the mice and estimate the optimal coil antenna through the functional modules, illustrated in Fig. 3a, in a real-time manner. We conducted all experiments including training the DLC on a GPU workstation (Lambda workstation with Intel Core i9-9960X, 128 GB RAM, and two GEFORCE RTX 2080 Ti graphics cards)." -It appears the authors are using unpublished DeepLabCut code: 2.2beta7 is a beta release. It would seem appropriate to ask those authors how to appropriately cite this version? Specifically, Figure 3A 2-4 seems to be closely related to the unpublished multi-animal DeepLabCut contribution. DLC has 14 networks available, multi-data augmentation steps, new steps introduced in 2.2.beta, etc. The authors should give sufficient details to be clear to the reader what they used.
-There are also several papers that present real-time options for DeepLabCut; can the authors comment on the differences here from Forys et al, Kane et al, etc? (Technically, DLC could always take in frames, whether in a video or frame format--it is a frame-by-frame pose estimation algorithm that always allowed for batch size 1). Note that in Kane et al, 2020 there is also a novel predictive model to reduce delays in low-latency tracking.
-The authors should be sure to cite the open source code they use, such as Matplotlib, numpy, and importantly the DLC Python package of the code (Nath et al, 2019).

Minor Points:
"The utilization of an advanced AI algorithm for automated video analysis allows for real time tracking of the freely moving animals in a cage to ensure robust activation of animals (implantable devices) in cages" --the authors might consider stating what they used, vs. the flashy and non-informative "AI"; simply stating they used DeepLabCut is sufficient. --this is not a newly "proposed" software (this is DeepLabCut), so please consider revisiting the figure caption and also be sure to include a ™ to the DLC image you have in Figure 3.

3.
One justification is mistaken. It is well known that a moderate increase in temperature (hyperthermia) potentiates the effects of PDT, so major efforts to restrict heat production are misguided.

Response:
The purpose of monitoring temperature was to ensure that there was no increase in temperature during LED treatment. An Increase in LED operating temperatures, even moderate, could potentially lead to adverse toxicity in normal tissue or impact the overall wellbeing of animals, which would severely limit the impact of PDT. In this study, we were interested in purely the phototoxicity effects of PDT when light sources affixing to the tumor only without affecting other healthy tissue.

4.
One of the best justified applications of metronomic PDT is ALA-induced PPIX because ALA can be administered in drinking water. This should have at least been discussed as the blue and red light excitation would have been ideal.
Response: PPIX has previously been shown to be a good photosensitiser for superficial lesions and tumours, where the biggest impact is seen only in the outer layers of cells. This is unlike Foscan, that can penetrate deeper in tumours and studies have shown this drug to exert phototoxic activities in deeper layers of cells and has a more penetrative effect. We added the following sentence in the last paragraph of Discussion section: "In this study, the photosensitizer Foscan was chosen to elicit phototoxicity due to previous studies demonstrating great efficacy in Foscan-mediated PDT. In comparison to 5-aminolevulinic acid, and its downstream intermediary product, protoporphyrin IX which tends to accumulate only in the superficial top layers of cells, Foscan is able to penetrate much deeper into solid tumours and more effectively utilize the photoactivation properties of deeply penetrating >600nm wavelength as well as superficial 400nm.".

5.
In fig 5g "inner and outer" LEDs are confusing.
Response: Reviewer's comment is well taken. We changed the term in Figure 5(g) and add the following comments in Figure 5 legend: "Peripheral denotes outer 4 LEDs of an implantable device and Central indicates inner 4 LEDs of an implantable device; images of an implantable device in Fig4(e).".

6.
Hypercin is not soluble in DPBS so it must have contained some % of ethanol.
Response: Both Hypericin and Foscan stocks in 100% ethanol. Final concentration of ethanol in cell media was 0.2% for in vitro experiments and 6% ethanol per 0.5mg/kg injection for Hypericin and 0.5% for in vitro experiments and 10% ethanol per 0.5mg/kg injection for Foscan. No adverse toxicities was observed in any mice in all treatment groups. We added the following sentences in Methods-Materials for PDT in colorectal cancer models: "Hypericin and Foscan stock solutions were prepared in 100% ethanol. Final concentration of ethanol in cell media for Hypericin was 0.2% for in vitro experiments and 6% ethanol per 0.5mg/kg injection. Final concentration of ethanol in cell media for Foscan was 0.5% for in vitro experiments and 10% ethanol per 0.5mg/kg injection." 7. You need to give the IACUC approval number for the protocol. 8. There are many mistakes, I shall try to list some of them. 9. Singlet oxygen is not an "oxygen free radical".

Response:
Oxygen free radical has been changed to be singlet oxygen. We modified the following sentences in line 68: "For example, tissue oxygenation in tumors is critical for the production of singlet oxygen and successful phototoxicity, with tumor hypoxia being a limiting factor to PDT efficacy PDT 6,7 ." 10. Absorption coefficient not "absorption rate".

Response:
We agree with reviewer's point. We modified the following sentence in line 84-85: "For example, Foscan has two peaks at 406 nm and 652 nm in absorption spectra according to different absorption coefficient, respectively.". 11. "EM waves at UHF ranges (300 MHz ~ 3 GHz) are not transparent to EM waves," is meaningless.

Response:
We removed the sentence from Introduction.

12.
You cannot activate an animal only a device.

Response:
We mentioned a sentence, "simultaneous activation of multiple animals in 4 home cages using…" in line 285. Here, an animal indicates not exactly animal itself, but it means an animal which is implanted a device fully onto the right dorsal flank area already. Like reviewer's comment, it seems to occur misunderstandings, so we modified the following sentence: "Simultaneous activation of implanted devices in multiple animals in 4 home cages using a single RF power source is achieved primarily due to a channel isolation strategy and the use of supercapacitor.".

13.
A tumor cell cannot be 4mm in diameter.

Response:
We correct it as tumor volume in the 2nd paragraph of Results section: "(representing the tumor volume; 4 mm in diameter)", and "Figs. 2c and d summarize performance comparisons in five criteria (degree of light penetration, rate of energy absorbance, level of uniformity of energy absorbance into a tumor volume, the time required for delivery of targeted light energy, and range of temperature variation) and show distributions of light absorption into a tumor volume for Hypericin and Foscan, respectively".

Comments:
In this manuscript, the authors presented AI-informed low-power wireless telemetry with an integrated thermal/light simulation platform for photodynamic therapy (PDT). The AI-assisted wireless telemetry enables adequate illumination of tumors through high-throughput (<20 mice) and multi-wavelength operation. Furthermore, Hypericin and Foscan mediated PDT demonstrates the significant suppression of tumor growth. The AI-assisted wireless telemetry combined with a supercapacitor-enabled switching mechanism allows for continuous and efficient delivery of multichannel light to mice in a cage and to multiple cages, potentially enabling more efficient and high throughput PDT study in animals. This highly multidisciplinary work is novel and can be of great interest to others in the community. However, some major revisions are needed to clarify and strengthen some of the conclusions.

(Major)
The motivation/introduction part is somewhat confusing and should be clarified. For example, in the abstract, the authors mentioned the challenge is "Existing approaches... suffer from poor spatiotemporal resolution due to inabilities to minimize oxygen depletion in a tumor." It is not clear why poor spatiotemporal resolution is caused by oxygen depletion -they seem to be two parallel goals. In addition, in the first paragraph, it was mentioned that the current PDT reached a plateau due to poor spatiotemporal resolution. In the second paragraph, it was pointed out that recent advances in wireless technologies improved the spatiotemporal resolution, but the challenges are compensation for light loss versus tissue heat/damage by high intensity light and UHF EM waves. It is somewhat confusing what the proposed technology is trying to achieve -improve the spatiotemporal resolution, minimize oxygen depletion, or reduce tissue heat/damage by using multichannel light sources?
Response: The reviewer's point is well taken. The technical highlight of this study is an implantable wireless device equipped with a dual-channel function that can be induced by RF power, which significantly improves levels of tissue penetration and PS activation and increases the throughput of PDT experiments. Key limitations in PDT experiments involve 1) low level of light penetration & PS activation and 2) low throughput. First, existing PDT hardware delivers light to tumors via a cable tethered to a light source and such physical tethering would complicate access to targeted tumor cells. In addition, wired or wireless PDT approaches use single light source (wavelength) to activate a PS and would activate a PS at a modes rate if a PS has two absorption peaks. Together, low levels of light penetration and PS activation would significantly degrade PDT efficacy. Second, current PDT methods are low-yield, and cannot be multiplexed. This is a major constraint that prohibits the high-throughput utilization of wireless methods in most laboratory settings. In this paper, we proposed a novel research platform that integrates Monte Carlo analysis and DeepLabCut algorithm into an implantable wireless telemetry system. Here, the thermal/ light Monte Carlo simulator yields a bespoken PDT regimen such as choice of wavelengths, determination of the number of light sources, and its placement onto an implantable device to be delivered. The DeepLabCut algorithm allows for real-time instantaneous pose estimation of the freely moving animals in a cage (up to 4 cages) to ensure reliable activation of the implanted devices. Collectively, the DeepLabCut-assisted low-power wireless telemetry system uses the parameters from the MC simulator to enable the most effective PDT. This would facilitate efficient use of resources.

(Major)
Following the previous comment, if one of the major advantages of this technology is to reduce oxygen depletion, some evidence showing that this new method could reduce oxygen depletion in tumors should be provided to support the claim. If the focus is on the capability of delivery light at multiple wavelengths continuously at a low power and the activation of mice in multiple cages, the abstract and the introduction should be modified accordingly.

Response:
We agree the abstract and introduction should be revised accordingly. To be clear, we modified the Abstract and 1 st and 2 nd paragraphs in Introduction section: "Advances in wireless technologies enable remote delivery of light to tumors, but suffer from key limitations, including low levels of tissue penetration and PS activation." & "Together, they establish a range of guidelines for effective PDT regimen design. In vivo Hypericin and Foscan mediated PDT, using cancer xenograft models, demonstrated substantial suppression of tumor growth, warranting further investigation in research and/or clinical settings." in Abstract, and "Existing hardware PDT, in particular the light source, suffers from key limitations in their ability to: 1) deliver light to deep tumor cells (e.g., physical tethers to light sources would complicate access to targeted tumor cell), 2) control light sources and/or wavelengths in a programmed manner (e.g., light sources with single wavelength would activate a PS at a modest rate if a PS has two absorption peaks), and 3) be multiplexed so that multiple animals can be studied in parallel.", in 2 nd paragraph of Introduction; "Foscan has two peaks at 406 nm and 652 nm in absorption spectra according to different absorption coefficient, respectively. 17 . Thus, 78% of light absorbance at 406 nm, which is not normally used for PDT, does not contribute to the generation of oxygen-free radicals and tumor killing. One way to compensate for this is to increase the light intensity, however, its benefit is marginal. Furthermore, increases in light intensity result in heat generation, which can damage normal healthy tissue 18-20 . Another limitation is the absence of high-throughput pipelines for the analysis of PDT outcomes 21-23 . Most existing wireless approaches utilize an RF power generator for each animal cage. Multiple RF power generators can be used, but they must be operated at least 1m apart to avoid electromagnetic interference 24 . This is a major constraint that prohibits the high-throughput utilization of wireless methods in most laboratory settings." in 3 rd paragraph of Introduction. Figure 2): in this figure, the devices are kept inside a bag filled with saline solution, and the IR camera is measuring the temperature from the top. It seems that the camera would be measuring the average temperature of the solution or the bag which may not be the temperature right at or next to the device.

(Major) Measurement of heat dissipation using IR camera (Supplementary
Response: Thank you for your comments about direct measurement of temperature of devices. A saline solution bag mimics an animal or biological tissues where a device is mounted and justifies the use of it in experiments. IR camera also offers enough spatial resolution (sub mm) to identify a heat source. In addition, we already measured the device directly without solution bag for checking the temperature changes of device itself; this condition represented in Supplementary Figure 2a and the result also can be found in Supplementary Figure 2c. All variation of temperature were below 0.6 ºC.

(Major)
The in vivo experiments compared HYP(+)/LED(+) conditions with controls. However, the multichannel (multiple wavelength) work has only been demonstrated in vitro. Since the multichannel capability is a major contribution of this work, an in vivo demonstration of the multichannel/wavelength activation would be very helpful. Is it realistic to perform in vivo demonstration of the multichannel activation? How might the movement of the implant affect the results?

Response:
We agree with reviewer's comment. We conducted animal experiments of each three group with Foscan: 1) red LEDs only, 2) purple LEDs only, and 3) combined dual red/purple colors in a device. As we expected, the tumor's growth was dramatically suppressed by dual color device (Figure 5g). We added the following sentence in the last paragraph of Result 4 th subheading section: "To further highlight the advantage of dual-wavelength LED PDT, in vivo evaluation was conducted in murine models of HT29 tumor xenografts. After 5 days of continuous PDT treatment, a 0.89-fold (89 %) decrease, a 0.51-fold (51 %) decrease, a 0.64fold (64%) decrease, and 1.94-fold (194 %) increase in tumor volumes (Fig. 5g)". We have included data in Figure 5. To comment on the movement of the implant, we observed that the implanted devices remained stable and tightly affixed adjacent to the tumor xenografts under the skin following surgical implantation. No implant movement was observed.

(Minor)
On page 5, the authors use the five criteria (1-5) to choose the optimal condition for PDT. Some sentence describing how these 5 criteria relate to the optimum condition should be added. In addition, "Results performed under different conditions (wavelength) revealed that uniform light delivery and minimum heat dissipation for Hypericin are reached with..." Uniform light delivery and minimum heat dissipation only refers to criteria 3 and 5. A more comprehensive description may also include amount/rate of energy absorption, etc.

Response:
We agree with reviewer's comment. We added more details about the criteria and revised followings in Result 2 nd subheading section: "Figs. 2c and d summarize performance comparisons in five criteria (degree of light penetration, rate of energy absorbance, level of uniformity of energy absorbance into a tumor volume, the time required for delivery of targeted light energy, and range of temperature variation) and show distributions of light absorption into a tumor volume for Hypericin and Foscan, respectively. Here, we set and calculate these variables from simulation in order to obtain the detailed information on the critical variables for deriving the results: 1) the capability to pass through the type of different medium, such as epidermis, dermis, and tumors, by each light source, 2) the energy absorbance ratio in the tumor according to the light spectra, and 3) the time to reach threshold energy significant for tumor growth suppression. The results, performed under different conditions (wavelength), revealed that the optimum condition for Hypericin to be most effective in PDT is reached with light sources (wavelength of 590 nm) and 25% of duty-cycle operation while that for Foscan is met with a combination of 406 and 652 nm wavelengths and 25% duty cycle operation. de For Foscan the optimal conditions were with a combination of 406 and 652 nm wavelengths and 25% duty cycle operation. ". Response: As the reviewer mentioned, µA is not a unit of power. We changed the term 'power budgets' to 'current consumption' and added the following sentences to Discussion: "In addition, we designed a low-power circuit to control these multichannel light sources: an actuation mechanism triggered by a reed switch enabled efficient activation of a photosensitizer with current consumption as low as tens of µA. Existing multichannel devices are not suitable for wireless power systems because they require a stable rectified current supply of at least several tens of mA since they contain additional IC chips for multichannel control.".

Comments:
I am not a domain expert in the main subject matter of this paper. I am, however, an expert in animal pose estimation. Thus, I will only comment on these aspects. The combination of antenna tracking and deeplabcut is a nice contribution (and from a non-expert view, the paper is quite exciting). However, I have questions about the AI method being used. It appears the authors are using a beta release of DeepLabCut code that has not been published (or no preprint yet). Therefore, performance is quite unclear for this method. There are also not sufficient details in the method to understand what the authors used exactly. In the main text they mention using neural networks for pose estimation in multiple animals, and then in the methods they say deeplabcut 2.2b7--looking at this code it's clear there are many options that are not clearly described in the methods. Moreover, the authors mention bipartite matching but the methods don't detail this fully. It seems unusual, to me, that the authors here are using unpublished code without proper benchmarking and performance metrics. I give more concrete examples below. "Modification to DLC model We utilized, custom-trained, and modified the DLC Python package (Ver. 2.2b7). Specifically, we used the custom-trained DLC model to estimate the locations of the body parts such as snouts and tails of the mice within an image (i.e., video frame). Note that the original DLC python package does not support a real-time processing feature, instead it only runs on video files. Hence, we directly modified the Python package in such a way that it can infer the locations of the body parts of the mice and estimate the optimal coil antenna through the functional modules, illustrated in Fig. 3a, in a real-time manner. We conducted all experiments including training the DLC on a GPU workstation (Lambda workstation with Intel Core i9-9960X, 128 GB RAM, and two GEFORCE RTX 2080 Ti graphics cards)."

(Major)
It appears the authors are using unpublished DeepLabCut code: 2.2beta7 is a beta release. It would seem appropriate to ask those authors how to appropriately cite this version? Specifically, Figure 3A 2-4 seems to be closely related to the unpublished multi-animal DeepLabCut contribution. DLC has 14 networks available, multi-data augmentation steps, new steps introduced in 2.2.beta, etc. The authors should give sufficient details to be clear to the reader what they used.
Response: Thank you for your comment regarding the citation of DeepLabCut.
First of all, we would like to emphasize that Figure 3a.(3)-(6) illustrates the contribution we have made to this work, which can run on the original (previous) version of DeepLabCut as well. Thus, I believe it will suffice to cite the original DLC manuscript. While we have developed the proposed software based on the latest beta version (2.2b7) of the DeepLabCut package, as it is more stable than the original version, we do not utilize or rely on its new features that were not included in the original DLC that we cited in our manuscript.
The primary objective of the software that we developed was to efficiently identify the optimal power transmission antenna based on the "instantaneous poses" of the multiple mice in a real-time manner. In that regard, we believe it may be worthwhile to briefly discuss the characteristics of the new feature in the latest DeepLabCut (referred to as MT) for tracking multiple animals. MT is indeed a multiple animal instance "tracking" post-processing algorithm that requires manual interventions from a user, which cannot be easily converted to a real-time processing setting -which is our main focus. In fact, the MT feature works only for offline processing of recorded video files. Besides, for the purpose of optimal antenna control, our method only requires real-time identification of the "instantaneous poses" of multiple mice without the need for continuously tracking them.
To this aim, we trained the DeepLabCut model, which was already implemented in the original (previous) DeepLabCut package cited in our manuscript and used the model only to detect the locations of the body parts that are not yet associated with the instance label (for individual animal). Our proposed real-time post-processing module, illustrated in Figure 3a.(3)-(6), "intercepts" the raw output from the output layer of the DLC model, based on which it predicts the optimal power transmission coil antenna. To be more specific, our algorithm computes the matching score based on the weighted Euclidean distance between all possible pairs of different body parts (i.e., all possible head-tail pairs). Then, it finds the optimal one-toone mapping through the maximum weighted bipartite matching (MWBM) to identify the instantaneous poses (directions) of the multiple mice. The optimal transmission antenna for each animal is identified according to the direction of each optoelectronic device, and the globally optimal power transmission coil antenna is determined based on the majority rule.
To clearly describe our contribution and avoid any potential confusion, we updated the terminologies in the manuscript accordingly (motion tracking  instantaneous pose estimation). In addition, we changed the title of the corresponding section "AI-enabled real-time motion tracking of multiple animals" to "Real-time identification of the optimal coil antenna based on the instantaneous pose estimation of multiple animals" and updated the content as follows:

Real-time identification of the optimal coil antenna based on the instantaneous pose estimation of multiple animals
We developed software that identifies the optimal power transmission coil antenna in a real-time manner in the sense that the number of optoelectronic devices receiving wireless power is maximized. The proposed software is built on the strength of the state-of-the-art deep learning model, DLC 25 , combined with the proposed real-time post-processing module based on the optimal graph matching algorithm, maximum weighted bipartite matching (MWBM) 26 . To be more specific, we directly customized the DLC Python package such that the custom-trained DLC model runs with a real-time video stream. The proposed real-time post-processing module intercepts the raw estimated locations of the body parts (heads and tails) of the mice, where the estimated locations are not associated with the instance label, from the output layer, and estimates instantaneous poses (directions) of mice. Based on the estimated poses, it identifies the optimal coil antenna. Fig. 3a illustrates the procedures of the proposed software. As shown in Fig. 3a-(1), we assume that there are five freely behaving mice in which an optoelectronic device has been implanted. For the wireless power transmission, four power transmission coil antennas, only one of which activates at a time, surround the case while forming an X-shape to all sides. A webcam on the top of the cage sends a video stream to the custom-trained DLC at the rate of 25 fps. As a frame arrives, the trained DLC model detects the locations of the snouts or tails (Fig. 3a-(2)). Note that the raw output (i.e., the locations of the snouts or tails) from the output layer of DLC are not associated with the instance information. The proposed software intercepts the raw output, filters out the detected body parts of which the confidence score is less than 0.6, and passes it to the proposed real-time post-processing module. Next, the proposed postprocessing module computes the matching score between all possible pairs consisting of different body parts (i.e., head-tail pair) (Fig. 3a-(3)). Based on the matching scores, MWBM finds the optimal one-to-one pairs in such a way that the sum of the matching scores of the pairs in the optimal matching set is maximized (Fig. 3a-(4)). The optimal pairs directly correspond to the instantaneous poses (direction) of the mice in the cage (Fig. 3a-(5)). In turn, for each optimal pair or, equivalently, the direction of each mouse, the optimal coil antenna which is expected to achieve maximum power efficiency to each device is selected (Supplementary Fig. 5). Finally, the global optimal power transmission coil antenna is selected and activated according to the rule of the majority (Fig. 3a-(6)). The software repeats these whole procedures throughout the experiment.
For assessment of the proposed software, we use a metric, defined as the percentage of correct predictions for the data tested 27 . Fig. 3b shows antenna selection accuracy for three different antenna settings: 1) two pairs of X-shaped antenna coils, one pair of X-shaped coil aligning with 2) the x-or 3) y-axis. Results revealed that the proposed software guarantees the accuracy of 80 % or above in every setting that we tested (Fig. 3b). Figs. 3c and d show statistics of the number of frames for two representative cases; how long a selected antenna remains activated (Fig. 3c) and how many frames (or long interval times) it takes between activation of an antenna and reactivation of itself after the first deactivation (Fig. 3d). Likely, some occupants, not all of them, in a cage may not receive enough power for activation of a photosensitizer due to a decision by the software (e.g., when two mice or vectors along the length of their body are aligned with the x-axis and corresponding vectors for the rest three mice are on the y-axis, the software chooses an antenna coil that offers better wireless coverage in the y-direction). Experimental results revealed that discharges of power stored in an embedded supercapacitor can last longer than the time intervals shown in Supplementary Figs. 7 and 8. This suggests that the proposed software paired with the use of a supercapacitor ensures robust activation of devices in a cage. Detailed information on evaluations of the DeepLabCut software is found in Method section.

(Major)
There are also several papers that present real-time options for DeepLabCut; can the authors comment on the differences here from Forys et al, Kane et al, etc? (Technically, DLC could always take in frames, whether in a video or frame format--it is a frame-by-frame pose estimation algorithm that always allowed for batch size 1). Note that in Kane et al, 2020 there is also a novel predictive model to reduce delays in low-latency tracking.
Response: Thank you for your insightful comment on the recent works relevant to DLC. Conceptually, the recent works by Forys et al. and Kane et al. and our proposed algorithm bear some high-level similarities. However, our proposed algorithm has distinctive characteristics that clearly differentiate it from the works you've mentioned. We delineate the important differences in what follows.
The work by Forys et al. is relevant to our proposed algorithm in the sense that they both aim to control the experimental platform in a real-time manner based on the behavior of the animal subjects by modifying the DLC package. Forys et al. developed software that tracks the movement of specific body parts (i.e., paws) of a mouse in a real-time manner and activates the indicative device if the degree of activity exceeds a certain threshold with very low latency. However, despite the outward similarity, there is a critical difference between the work by Forys et al. and our proposed algorithm. For example, the approach by Forys et al. has been restricted to tracking body parts of a single mouse whose head is fixed. On the other hand, our proposed algorithm estimates the instantaneous poses of multiple mice, based on which we identify the optimal power transmission coil antenna that effectively delivers power to the wireless optoelectronic devices. This is clearly beyond the capability of the original DLC as well as the software developed by Forys et al.
Kane et al. developed a lightweight python package called DeepLabCut-live! (DLC-live) based on the DLC, mainly aiming at extending the applicability of the DLC for diverse experimental studies that typically involve monitoring the behaviors of an animal and intervening in the experiment in a real-time manner. However, to date, DLC-live has been mainly validated for tracking a single animal. Furthermore, considering that the multiple animal tracking feature introduced in the latest DLC requires frequent user intervention, it is not straightforward to combine DLC-live with the multiple animal tracking feature in the latest DLC. Also recall that we are only interested in identifying the optimal power transmission coil antenna based on the "instantaneous poses" of the multiple mice, rather than real-time tracking of their temporal trajectories. Our proposed approach effectively accomplishes this primary objective in a realtime setting, which is not possible by the original DLC or DLC-live.

(Major)
The authors should be sure to cite the open source code they use, such as Matplotlib, numpy, and importantly the DLC Python package of the code (Nath et al, 2019).
Response: Thank you for the valuable comment. DLC software provides the Conda environmental file that contains the information of all packages utilized by DLC. In addition to the packages, we utilized the NetworkX package in order to construct a body parts network and to predict the instantaneous poses by predicting the snout-tail pairs using the maximum weighted bipartite matching (MWBM) scheme. The resulting python package information is summarized in the table below. We included the following table in the supplementary information and added the following comments in Method section: "We utilized, custom-trained, and modified the DLC Python package (Ver. 2.2b7); the summary of Python package information is in Supplementary Table 1."

(Minor) "
The utilization of an advanced AI algorithm for automated video analysis allows for real time tracking of the freely moving animals in a cage to ensure robust activation of animals (implantable devices) in cages" --the authors might consider stating what they used, vs. the flashy and noninformative "AI"; simply stating they used DeepLabCut is sufficient.

Response:
We appreciate your thoughtful suggestion. In the revised manuscript, we changed "AI" to "DeepLabCut" according to your comment: for example in title, "DeepLabCut-based, implantable, multichannel wireless telemetry for photodynamic therapy" in abstract, "Here, we introduce DeepLabCut (DLC)-informed low-power wireless telemetry with an integrated thermal/light simulation platform that overcomes the above constraints. The simulator produces an optimized combination of wavelengths and light sources, and DLC-assisted wireless telemetry uses the parameters from the simulator to enable adequate illumination of tumors through high-throughput (<20 mice) and multi-wavelength operation." In introduction, "The utilization of an advanced DLC algorithm for automated video analysis allows for real-time instantaneous pose estimation of the freely moving animals in a cage to ensure robust activation of animals (implantable devices) in cages." ······ in method, "The proposed DLC algorithm yields the following information for each frame". Figure 3. (a) Illustration of step-by-step procedures for the proposed artificial intelligence algorithm. --this is not a newly "proposed" software (this is DeepLabCut), so please consider revisiting the figure caption and also be sure to include a ™ to the DLC image you have in Figure 3.

Response:
We appreciate your suggestion regarding Figure 3. As we answered above (esp., see answers to the first and second comments), our proposed software consists of two core modules: (1) a custom trained DLC and (2) a post-processing module that takes the raw output from the output layer of the DLC for instantaneous pose estimation of multiple animals and the optimal antenna prediction. It should be noted that that there is no instance information in the raw output from the DLC that connects the predicted body parts to individual mice. Our contribution is to find the optimal pairs (i.e., snout-tail pairs) via maximum weighted bipartite matching (MWBM), thereby estimating the directions of up to five mice. Based on the directions of the mice identified by our algorithm, we predict the optimal power transmission antenna that maximizes the power transmission efficiency. For this reason, we prefer to maintain the "proposed" software in Figure 3 as is, since this is an important new capability that cannot be simply achieved by the DLC without significant development and optimization. Nevertheless, we certainly agree with your suggestion that it would be important to clearly show the role of DLC in our proposed software. Following your suggestion, we included ™ in the DLC image and updated Figure 3a.(2) to clearly specify our contribution.
Lastly, I find it usual they changed the title of the paper to be "DeepLabCut..."--the authors are not contributors to the DeepLabCut on GitHub (I checked), nor do they hold TM rights to the name; I would suggest changing this back to avoid any issues.
Missing details example such that the work is not reproducible: "Modification to DLC model 372 We utilized, custom-trained, and modified the DLC Python package (Ver. 2.2b7); the summary 373 of Python package information is in Supplementary Table 1. Specifically, we used the custom374 trained DLC model to estimate the locations of the body parts such as snouts and tails of the 375 mice within an image (i.e., video frame). Note that the original DLC python package does not 376 support a real-time processing feature, instead it only runs on video files. Hence, we directly 377 modified the Python package in such a way that it can infer the locations of the body parts of the 378 mice and estimate the optimal coil antenna through the functional modules, illustrated in Fig.  3a, 379 in a real-time manner. We conducted all experiments including training the DLC on a GPU 380 workstation (Lambda workstation with Intel Core i9-9960X, 128 GB RAM, and two GEFORCE 381 RTX 2080 Ti graphics cards)." There are 14 backbone architectures in DeepLabCut, 3 data augmentation methods, and many hyperparameters. What did they use? How many images are labeled? How long was the model trained? How did they select the snapshot they used? I would, again, highly suggest you take the recommendation to be specific. Here are guidelines for reproducible reporting of DeepLabCutbased models: https://deeplabcut.github.io/DeepLabCut/docs/recipes/DLCMethods.html 1. The authors have addressed a lot of my concerns, and the quality of the manuscript has been improved. However, in the newly added in vivo experiment (Fig.5g), n=2 is not sufficient to draw a conclusion. A minimum of n>=3 should be used in these experiments.
Response: It was a typo and we fixed it (n=5) on the manuscript.

Reviewer #3 (Remarks to the Author):
I thank the authors for their revisions, but I still find the methods unacceptably lacking details and missing citations. This must be fixed prior to any publication.

1.
You use code that is absolutely not the original DLC and you directly use and slightly modify the unpublished multianimal code (I read and compared the code e.g. this line in their code specifically states it is DLC 2.X: https://github.com/parkgroup-tamu/AI-enabled-implantable-multichannel-wireless-telemetry-forphotodynamictherapy/blob/8a0a6f78e6d7f221718530664aa76dee9669dd84/DeepLabCut_Modified/predict_multianimal.py #L7); as someone who is very familiar with the DLC code base. I do not know why these authors do not feel it's necessary to provide proper credit to the authors of DeepLabCut. https://github.com/DeepLabCut/DeepLabCut#references which states: There is no limitation from Nature Communications that prevents this.
Response: Thank you very much for your kind suggestions and the detailed information. We absolutely agree with you that it is critically important to give due credit to the DLC authors that they deserve and acknowledge their work. We regret we were not more careful when referencing the DLC-related publications in our original manuscript. Following your suggestions, in the revised manuscript, we cited all the relevant references that you mentioned in your comments. Specifically, we cited the manuscripts by Mathis et al [25], Nath, T. et al. 2019 [26], and Lauer, J. et al 2021 [27] in our manuscript. Regarding converting the DLC for real-time processing, we would like to emphasize that we already converted and utilized DLC for real-time processing in our previous study [46] (presented in Nov. 2020 @ 2020 54th Asilomar Conference on Signals, Systems, and Computers, https://ieeexplore.ieee.org/abstract/document/9443501) before the DLC-Live (Dec. 2020) and DeepLabStream (2021). However, since our main contribution in this work is less relevant to converting the source code for real-time processing, we believe it will be worth introducing the recent efforts to the readers. In that regard, we cited DLC-Live [47] and DeepLabStream [48] in our manuscript as follows:

DeepLabCut model training
We trained the DLC model provided in the DLC Python package (version 2.2b7 25,26,27 ) such that the trained model is capable of locating snouts and tails of up to five mice within an image (i.e., video frame). To this aim, we extracted 30 frames from a video file recorded for 10 minutes 15 seconds (15,375 frames in total) at the framerate of 25 frames per second via a K-means clustering algorithm to collect representative training frames. We used a ResNet-50 43,44 neural network with default parameters. For example, we optimized the network via ADAM 45 with 200,000 iterations and a gradually decreasing learning rate. As a result, the trained model achieved a validation loss of 0.0013. For all details, see config.yaml on the GitHub repository (https://github.com/parkgroup-tamu/AI-enabled-implantable-multichannel-wireless-telemetry-forphotodynamic-therapy/tree/main/DeepLabCut_Modified). We conducted all experiments including training the DLC on Lambda workstation with Intel Core i9-9960X, 128 GB RAM, and two GEFORCE RTX 2080 Ti graphics cards. All python packages used in this study are summarized in Supplementary Table 1.

Modification of DeepLabCut Python package for identifying the optimal coil antenna
In order to identify the optimal transmission coil antenna that maximizes instantaneous system power transmission efficiency online, we directly modified the DLC Python package (Ver. 2.2b7) 25,26,27 . Specifically, as the original DLC python package does not support a real-time processing feature, for each frame, we intercepted the raw estimation results (i.e., the locations of snouts and tails without instance information) from the trained DLC model and used them as input to the real-time post-processing module we developed as illustrated in Fig. 3a, which was inspired by our previous study 46 . It is worth noting that there have been several attempts to add a real-time processing feature to DLC and platform it 47,48 . Although the software developed in this study was not built on these platforms to minimize potential unnecessary overhead, technically and conceptually, the modified DLC Python package has in common with these works.
As a frame arrives at the trained DLC model (Fig. 3a-(2)), the trained DLC model identifies the snouts and tails with confidence scores. In this study, we assumed that the body part whose score exceeds 0.6 is valid. As shown in Fig. 3a-(3), the proposed real-time post-processing module computes matching score ( ) among all possible pairs of snout and tails to find the optimal pairs as follows: where and are the positions of the th tail and th snout on the -axis, respectively; and and are the positions of the th tail and th snout on theaxis, respectively. ( ) is the number of pixels, whose intensity value is greater than or equal to , on the straight line connecting two points ( ) and ( ) . Next, the proposed module finds the optimal one-to-one mapping set via the MWBM as follows (Fig. 3a-(4)): (∑ ( ) ( ) ).
Construction of the optimal one-to-one mapping set naturally leads to identifying the projection vector ( ) of the orientation vector on the -plane of the th mouse. Under the practically reasonable assumption that a mouse can only change the angle up to 90 degrees in the direction the mouse are looking (i.e., mice do not stand on their hands), the optimal antenna index for th mouse can be found as follows: where , , is the antenna vector of the th coil antenna (See supplementary Fig. 5). Finally, the optimal coil antenna that maximize the instantaneous system power transmission efficiency is determined by the principle of majority vote. video frame). To this aim, we extracted 30 frames from a video file recorded for 10 minutes 15 seconds (15,375 frames in total) at the framerate of 25 frames per second via a K-means clustering algorithm to collect representative training frames. We used a ResNet-50 43,44 neural network with default parameters. For example, we optimized the network via ADAM 45 with 200,000 iterations and a gradually decreasing learning rate. As a result, the trained model achieved a validation loss of 0.0013. For all details, see config.yaml on the GitHub repository (https://github.com/parkgroup-tamu/AI-enabled-implantable-multichannel-wireless-telemetry-forphotodynamic-therapy/tree/main/DeepLabCut_Modified). We conducted all experiments including training the DLC on Lambda workstation with Intel Core i9-9960X, 128 GB RAM, and two GEFORCE RTX 2080 Ti graphics cards. All python packages used in this study are summarized in Supplementary Table 1.

Modification of DeepLabCut Python package for identifying the optimal coil antenna
In order to identify the optimal transmission coil antenna that maximizes instantaneous system power transmission efficiency online, we directly modified the DLC Python package (Ver. 2.2b7) 25,26,27 . Specifically, as the original DLC python package does not support a real-time processing feature, for each frame, we intercepted the raw estimation results (i.e., the locations of snouts and tails without instance information) from the trained DLC model and used them as input to the real-time post-processing module we developed as illustrated in Fig. 3a, which was inspired by our previous study 46 . It is worth noting that there have been several attempts to add a real-time processing feature to DLC and platform it 47,48 . Although the software developed in this study was not built on these platforms to minimize potential unnecessary overhead, technically and conceptually, the modified DLC Python package has in common with these works.
As a frame arrives at the trained DLC model (Fig. 3a-(2)), the trained DLC model identifies the snouts and tails with confidence scores. In this study, we assumed that the body part whose score exceeds 0.6 is valid. As shown in Fig. 3a-(3), the proposed real-time post-processing module computes matching score ( ) among all possible pairs of snout and tails to find the optimal pairs as follows: where { ( ) ∞ ( ) ; and are the positions of the th tail and th snout on the -axis, respectively; and and are the positions of the th tail and th snout on theaxis, respectively. ( ) is the number of pixels, whose intensity value is greater than or equal to , on the straight line connecting two points ( ) and ( ) . Next, the proposed module finds the optimal one-to-one mapping set via the MWBM as follows (Fig. 3a-(4)): ).
Construction of the optimal one-to-one mapping set naturally leads to identifying the projection vector ( ) of the orientation vector on the -plane of the th mouse. Under the practically reasonable assumption that a mouse can only change the angle up to 90 degrees in the direction the mouse are looking (i.e., mice do not stand on their hands), the optimal antenna index for th mouse can be found as follows: where , , is the antenna vector of the th coil antenna (See supplementary Fig. 5). Finally, the optimal coil antenna that maximize the instantaneous system power transmission efficiency is determined by the principle of majority vote.