Abstract
The effects of molecularly targeted drug perturbations on cellular activities and fates are difficult to predict using intuition alone because of the complex behaviors of cellular regulatory networks. An approach to overcoming this problem is to develop mathematical models for predicting drug effects. Such an approach beckons for codevelopment of computational methods for extracting insights useful for guiding therapy selection and optimizing drug scheduling. Here, we present and evaluate a generalizable strategy for identifying drug dosing schedules that minimize the amount of drug needed to achieve sustained suppression or elevation of an important cellular activity/process, the recycling of cytoplasmic contents through (macro)autophagy. Therapeutic targeting of autophagy is currently being evaluated in diverse clinical trials but without the benefit of a control engineering perspective. Using a nonlinear ordinary differential equation (ODE) model that accounts for activating and inhibiting influences among protein and lipid kinases that regulate autophagy (MTORC1, ULK1, AMPK and VPS34) and methods guaranteed to find locally optimal control strategies, we find optimal drug dosing schedules (openloop controllers) for each of six classes of drugs and drug pairs. Our approach is generalizable to designing monotherapy and multi therapy drug schedules that affect different cell signaling networks of interest.
Introduction
Although there is much current interest in using combinations of molecularly targeted drugs to improve outcomes for cancer patients^{1,2}, relatively little work has been done in the area of formal therapy design, meaning therapy selection and/or scheduling driven by insights from mathematical models^{3,4}. Formal approaches to therapy design are potentially useful for at least three reasons. First, all possible combinations of drugs may be difficult, if not impossible, to evaluate experimentally simply because of the large number of possible combinations. Second, an ability to extrapolate accurately beyond wellcharacterized scenarios with the aid of predictive models would be valuable for individualized treatment, especially in cases where molecular causes of disease are diverse and vary from patient to patient, as in many forms of cancer^{5}. Third, it is often nonobvious how the immediate effects of drug perturbations propagate through a cellular regulatory network to affect cellular phenotypes and fates^{6} or how drug combinations might be deployed to avoid or delay the emergence of resistance, a common response of malignant cells to targeted therapies^{7}. Predictive models promise to help identify new robust therapies.
Here, we apply mathematical modeling and optimal control methods to design drug schedules for manipulating autophagy, a stressrelieving/homeostatic cellular recycling process that, when nutrients are in limited supply, generates building blocks for protein synthesis through degradation of cytoplasmic contents^{8}, such as cytotoxic protein aggregates that are too large for proteosomal degradation and damaged organelles (e.g., depolarized mitochondria). Autophagy also plays an important role in immunity^{9,10}; the autophagic degradative machinery can be directed to target intracellular microbes, such as Mycobacterium tuberculosis, for destruction.
Cytoplasmic contents that are targeted for autophagic degradation are first trapped in doublemembrane vesicles, termed autophagosomes or autophagic vesicles (AVs), and then delivered to lysosomes for digestion^{11,12}. The production of AVs is controlled by an intricate regulatory network, in which three protein kinasecontaining complexes are prominent: the heterotrimeric AMPactivated kinase (AMPK), which senses energy (glucose) supply through interactions with adenosine derivatives (AMP and ATP)^{13,14}; MTOR complex 1 (MTORC1), which senses amino acid supply and growth factor signaling through interactions with small GTPases localized to lysosomal surfaces (Rag proteins and RHEB)^{15,16}; and the ULK1 complex, which is activated by AMPK and repressed by MTORC1^{17,18,19}. A fourth complex, which contains a lipid kinase, VPS34, also plays an important role^{20,21}. Interestingly, VPS34 and MTOR are phylogenetically related: they are both members of the phosphoinositide 3kinase (PI3K) family. Drugs with specificity for each of these kinases are available, and because of the relationship between MTOR and VPS34, drugs are also available with dual specifity for this pair of kinases^{22,23,24}.
In cancer, and other contexts, autophagy is a doubleedged sword^{25}. It can protect cancer cells from stresses of the tumor environment (e.g., lack of nutrients because of defective vasculature) or induce cell death if recycling is excessive. Thus, there are potential benefits to be gained by using drugs to either upregulate autophagy (to kill malignant cells through excessive recycling) or downregulate autophagy (to kill cancer cells that rely on autophagy for survival)^{26}.
To investigate how single drugs and drug pairs might be best used for these purposes, we constructed a system of nonlinear ordinary differential equations (ODE) that captures regulatory interactions between MTORC1, ULK1, AMPK, and VPS34, as well as the idealized pharmacokinetics of kinase inhibitors specific for MTORC1, ULK1, AMPK, and VPS34, such as rapamycin^{27}, SBI0206965^{28}, dorsomorphin^{29}, and SAR405^{30}, respectively. We also considered an allosteric activator of AMPK (e.g., PF06409577^{31}) and a kinase inhibitor with dual specificity for MTORC1 and VPS34 (e.g., buparlisib^{32}). Although the model is minimalist by design, it reproduces key behavioral features of earlier, more mechanistically detailed models^{33,34}, such as oscillatory responses to intermediate levels of nutrient or energy stress. We then applied optimization methods implemented in the opensource \({\mathscr{P}}{\mathscr{S}}{\mathscr{O}}{\mathscr{P}}{\mathscr{T}}\) software package^{35} to find locally optimal dosing schedules that minimize the total amount of drug needed to drive the network to a desired, nonattracting operating point (corresponding to low or high AV count/turnover) and maintain it there. The dosing schedules are nonobvious, and synergistic drug pairs were predicted (drug 6 plus drug 1, 2 or 3), such as the combination of a VPS34 inhibitor and a dual specificity PI3K inhibitor, which acts on both VPS34 and MTORC1. This drug pair requires less total drug to achieve the same effect than either of the individual drugs alone and is relatively fast acting, which may be important for preventing or slowing the emergence of resistance.
The approach illustrated here differs from earlier applications of control theory concepts in the area of formal therapy design^{36,37,38,39,40} in that 1) the system being controlled is a cellular regulatory network, 2) the control interventions are injections (i.e., inputs) of (combinations of) molecularly targeted drugs, and 3) the control objective is manipulation of a cellular phenotype, namely the number of AVs per cell, which is related to the rate of AV turnover, with minimization of total drug used and a constraint on the maximum instantaneous drug concentration. The rationale for minimizing drug use is to avoid offtarget effects and associated toxicities. Our work is distinct from earlier studies of (nonbiological) nonlinear network control^{41,42,43,44}, in that our control goal is not to drive the system to an attractor (e.g., a stable steady state or limit cycle), but to an arbitrary point in phase space (i.e., the multidimensional space defined by the state variables of a system) and to then maintain the system there indefinitely. The approach is both flexible and generalizable and provides a means for computationally prioritizing drug dosing schedules for experimental evaluation.
Results
Model for cellular regulation of autophagy and the effects of targeted drug interventions
A prerequisite for formal therapy design is a mathematical model that captures the relevant effects of drugs of interest. Given our interest in using drugs to modify the process of (macro)autophagy, we constructed a model for regulation of the rate of synthesis of autophagic vesicles (AVs) that accounts for the enzymatic activities and interactions of four kinases that play critical roles in regulating autophagy, all of which are potential drug targets. The model further considers the effects of achievable drug interventions and idealized drug pharmacokinetics, meaning instantaneous drug injection according to a timedependent control function and firstorder clearance. The model is illustrated in Fig. 1.
The model was constructed in two steps. First, we constructed a minimalist model for physiological regulation of autophagy consistent with key features of earlier, more mechanistically detailed models^{33,34} (see “Formulation of the Model” in Supplementary Methods for details). These features include the time scale of drugstimulated autophagy induction and the dynamic range of regulation characterized by Martin et al.^{34} and the qualitative system behaviors characterized by Szymańska et al.^{33}, including a steady, low level of autophagy at low stress levels, oscillatory behavior at intermediate stress levels, and a steady, high level of autophagy at high stress levels. Simulations based on the present model—generated through numerical integration of the equations given below—and simulations based on earlier, related models^{33,34} are compared in Supplementary Fig. S1. Simulations of AV dynamics are compared to measured AV dynamics^{33} in Supplementary Fig. S2.
The model of Fig. 1 is intended to provide an idealized representation of regulation of AV synthesis in a single (average) cell in response to changes in the cellular supplies of energy and nutrients, which are treated in the model as external inputs that modulate the serine/threoninespecific protein kinase activities of AMPK and MTORC1, respectively. Thus, the model reflects regulation of AMPK activity by the cellular AMP:ATP ratio, which is affected by glucose availability, for example, and regulation of MTORC1 activity via, for example, the various amino acidsensing regulators of Ragulatorassociated heterodimeric Rag proteins, which recruit MTORC1 to lysosomes for activation in a manner that depends on their regulated guanine nucleotide binding states. The model further accounts for regulatory interactions among AMPK, MTORC1, a third serine/threoninespecific protein kinase ULK1, and a class III phosphoinositide 3kinase (PI3K) VPS34. As noted earlier, these kinases are key regulators of autophagy, and each is a potential drug target.
In the second step of model construction, we added idealized consideration of six distinct drug interventions, which correspond to interventions achievable through use of available smallmolecule compounds, such as rapamycin^{27} (an inhibitor of MTORC1 kinase activity), buparlisib^{32} (an inhibitor of PI3Kfamily kinases that has specificity for both MTORC1 and VPS34), SBI206965^{28} (an inhibitor of ULK1 kinase activity), dorsomorphin^{29} (an inhibitor of AMPK kinase activity), PF06409577^{31} (a direct activator of AMPK kinase activity), and SAR405^{30} (an inhibitor of VPS34 kinase activity). Each drug \(i\in \{1,\ldots ,6\}\) (Fig. 1) is taken to be cleared via a pseudo firstorder process and introduced in accordance with a specified, timedependent injection function u_{i}.
The model was formulated as a coupled system of nonlinear ordinary differential equations (ODEs):
In these equations, t is time (in min) and T is a timescale, which we specify as 1.0 min. The variable x_{1} represents the fraction of MTORC1 that is active, the variable x_{2} represents the fraction of ULK1 that is active, the variable x_{3} represents the fraction of AMPK that is active, the variable x_{4} represents the fraction of VPS34 that is active, and the variable x_{5} represents the AV count or number of AVs per cell (on a continuum scale). Thus, x_{i} always lies somewhere in the interval \([0,1]\) for \(i=1,\ldots ,4\). The AV count is bounded \(0\le {x}_{5}\le {k}_{3}/{k}_{4}\) because \({x}_{5}(t)=0\) implies \({\dot{x}}_{5}(t)\ge 0\) and \({x}_{5}(t)={k}_{3}/{k}_{4}\) implies \({\dot{x}}_{5}\le 0\) (by the previously stated bound on x_{4}(t)). The variables \({w}_{1},\ldots ,{w}_{6}\) represent the dimensionless concentrations of drugs 1–6. Thus, \({w}_{i}\ge 0\) for each i. The nondimensional parameters C_{En} and C_{Nu} are conditiondependent constants that define the supplies of energy and nutrients. An increase in energy supply is taken to positively influence the rate of deactivation of AMPK, and an increase in nutrient supply is taken to positively influence the rate of activation of MTORC1. The nondimensional parameters k_{1} and k_{2} influence the rate of activation of AMPK and the rate of deactivation of VPS34, respectively. The nondimensional parameter k_{3} is the maximal rate of VPS34dependent synthesis of AVs, and the nondimensional parameter k_{4} is the rate constant for clearance of AVs. Taking the rate of AV synthesis to be proportional to VPS34 activity is consistent with the model of Martin et al.^{34}, as is (pseudo) firstorder clearance of AVs. The nondimensional parameters \({\delta }_{1},\ldots ,{\delta }_{6}\) are rate constants for clearance of drugs 1–6. Each h_{ji}(x_{i}) is a nondimensional Hill function that has the following form:
where n_{ji} (the Hill coefficient), r_{b,ji}, r_{m,ji} and θ_{ji} are nonnegative constants. The h functions account for regulatory influences among the four kinases considered in the model; the influences considered are the same as those considered in the model of Szymańska et al.^{33} (cf. Fig. 1 and Figs 1 and 2 in ref.^{33}). Each H(w_{i}) is a nondimensional Hill function that has the following form:
where n (the Hill coefficient), r_{b}, r_{m} and θ are nonnegative constants. The H functions account for drug effects on kinase activities. The parameters b_{i} (\(i=1,\ldots ,6\)) in Eq. (1f) are Boolean variables introduced for convenience, for the purpose of defining allowable drug combinations. Recall that the u_{i} terms represent drug injection/input functions, which will be determined by solving an optimal control problem (described in the following section).
Parameter settings are summarized in Supplementary Tables S1 and S2. Each δ parameter was assigned a value consistent with a known drug halflife^{31,45,46,47,48,49} (Supplementary Table S2). Other parameters were assigned values that allow the model to reproduce the qualitative signaling behaviors of the AMPKMTORC1ULK1 triad characterized in the theoretical study of Szymańska et al.^{33} and to reproduce the timescale of autophagy induction and the range of regulation quantified experimentally in the study of Martin et al.^{34}. According to Szymańska et al.^{33}, at low levels of energy/nutrient stress, ULK1 activity, which can be expected to correlate with autophagic flux and AV count, is steady and low; at intermediate levels of stress, ULK1 activity is oscillatory; and at high levels of stress, ULK1 activity is steady and high. As noted earlier, in Supplementary Fig. S1, we compare simulations based on Eq. (1) with simulations based on models of Szymańska et al.^{33} and Martin et al.^{34}, and in Supplementary Fig. S2, we compare simulations of AV dynamics based on Eq. (1) with experimental measurements of AV dynamics reported by Martin et al.^{34}. Parameter settings are further explained in Supplementary Methods. In Supplementary Methods, we also elaborate on how earlier models^{33,34} guided our formulation of Eq. (1) and how these models differ from Eq. (1).
Modelpredicted physiological regulation of autophagy, by energy and nutrients, is summarized in Fig. 2. Figure 2A shows how qualitative longtime behavior depends on the supplies of energy and nutrients, when these supplies are maintained at constant levels and in the absence of external control inputs \(({u}_{i}=0,\,i=1,\ldots ,6)\). Figure 2B–E show time courses of autophagy induction or repression triggered by different energy/nutrient changes. All together, these plots show that model predictions of responses to physiological perturbations (i.e., changes in C_{En} and C_{Nu}) are consistent with expectations based on the studies of Martin et al.^{34} and Szymańska et al.^{33}.
Doseresponse curves predicted by the model for singledrug, constantconcentration perturbations are shown in Fig. 3. As can be seen, with increasing dosage, drugs 1 and 5 tend to increase the number of AVs per cell, whereas the other drugs tend to decrease the number of AVs per cell. These results are consistent with negative regulation of autophagy by MTORC1 and positive regulation of autophagy by ULK1, AMPK, and VPS34. As is the case for some physiological conditions (Fig. 2), AV count oscillates at some of the drug doses, depending on the supplies of energy and nutrients. All together, the plots shown in Fig. 3 indicate that responses to singledrug, constantconcentration perturbations are consistent with accepted regulatory influences of MTORC1, ULK1, AMPK and VPS34 on autophagy.
As can be seen in Fig. 3, the ability of each drug i to influence x_{5} depends on the supplies of energy and nutrients, meaning the values of C_{En} and C_{Nu} (cf. the left and right panels in each row). In this figure, two energy/nutrient conditions are considered (\({C}_{{\rm{En}}}={C}_{{\rm{Nu}}}=0.1\) and 0.6); additional conditions are considered in Supplementary Figs S3 and S4. Taken together, these results define the conditiondependent ranges over which x_{5} can be feasibly controlled by each drug i.
Therapy design as an optimal control problem
To design optimal therapies, we must first introduce design goals. Below, we introduce a series of goals/constraints that we will require optimal therapies to satisfy. However, let us first introduce notation useful for referring to therapies. We will refer to the set of six available drugs, or more precisely, drug types, as \({\mathscr{D}}=\{1,\ldots ,6\}\), and we will refer to a therapy involving k drugs chosen from \({\mathscr{D}}\) as \({{\mathscr{T}}}_{k}\), where
Thus, for example, we will use \({{\mathscr{T}}}_{1}\) to refer to a monotherapy, and \({{\mathscr{T}}}_{2}\) to refer to a dual therapy. There are six possible monotherapies and, in general, \({C}_{k}^{6}\) distinct therapies that combine k of the six drugs. Here, we will focus on monotherapies and dual therapies, leaving the evaluation of higherorder combination therapies for future work. As a simplification, we will assume that drugs used together in a combination do not interact. Thus, for example, for dual therapy with drugs 2 and 6 (Fig. 1), we consider these drugs to bind/inhibit VPS34 independently (i.e., noncompetitively).
Our first, and most important, therapy design goal can be described (somewhat informally) as follows. Starting from a stationary (or recurrent) state at time \(t=0\), we wish to use drug injections (i.e., drug inputs) according to a schedule defined by \(u(t)=({u}_{1}(t),\ldots ,{u}_{6}(t))\) to eventually maintain, after a transient of duration t_{0}, the number of AVs in an average cell, x_{5}, near (to within a tolerance ε) a specified target level, \({x}_{5}^{f}\), for a period of at least t_{f} − t_{0} (\({t}_{f} > {t}_{0} > 0\)), thereby achieving sustained control of the level of autophagic degradative flux in a cell, which is given by k_{4}x_{5} according to Eq. (1). In our analyses, we will consider t_{0} = 120 min and t_{f} = 240 min because these times are longer than typical transients (Fig. 2B–E).
A second therapy design goal of interest is minimization of the total amount of drug used, which is motivated by a desire to avoid drug toxicity arising from dosedependent offtarget effects. In the optimal control literature, a problem entailing this type of constraint is called a minimum fuel problem^{50,51}. The constraint can be expressed mathematically as follows:
where \({u}_{i}(t)\ge 0\) for \(i=1,\ldots ,6\). As a simplification, we are considering an objective functional J{u_{i}} that treats the different drugs equally, i.e., the sum in Eq. (5) is unweighted. With this approach, we are assuming that the different drugs of interest have equivalent toxicities. If drugs are known to have different toxicities, this assumption can be lifted simply by introducing weights to capture the toxicity differences, with greater weight assigned for greater toxicity. Indeed, arbitrary modifications of the form of the objective functional J{u_{i}} would be feasible if such modifications are needed to capture problemspecific constraints on drug dosing.
A third design goal is to disallow the instantaneous concentration of any drug i, w_{i}(t), from ever rising above a threshold \({w}_{i}^{{\rm{\max }}}\). The rationale for this constraint is again related to a desire to eliminate or minimize dosedependent drug toxicity. In other words, we are assuming that a drug i is tolerable so long as its concentration w_{i} is below a toxicity threshold \({w}_{i}^{{\rm{\max }}}\). In our analyses, we set the toxicity threshold of a drug as a factor (>1) times its EC_{50} dosage, which we define as the concentration of the drug at which its effect on x_{5}, negative or positive, is half maximal (see Eqs (2) and (3)).
We are now prepared to formulate the problem of (combination) therapy design as a constrained, optimal control problem. The problem, for a given \({{\mathscr{T}}}_{k}\) (Eq. (4)), is to find a drug schedule u(t) that minimizes the objective functional defined in Eq. (5) and that also satisfies the following constraints:
Here, X(t) is defined as \([{\bf{x}}(t),{\bf{w}}(t)]\), where \({\bf{x}}(t)=({x}_{1}(t),\ldots ,{x}_{5}(t))\) and \({\bf{w}}(t)=({w}_{1}(t),\ldots ,{w}_{6}(t))\), and \({\bf{f}}({\bf{X}}(t),{\bf{u}}(t))\) is the vector field of Eq. (1). The initial condition \({{\bf{X}}}_{0}={\bf{X}}(0)\) is taken to be a stationary (or recurrent) state of Eq. (1) where supplies of energy and nutrients are constant (i.e., C_{En} and C_{Nu} are fixed) and drugs are absent (i.e., u(t) = 0). With this formulation, it should be noted that we are attempting to drive the system variable x_{5} to a specified final value \({x}_{5}^{f}\) (to within a tolerance ε), but we are making no attempt to control the other system variables x_{1}, x_{2}, x_{3}, and x_{4}. This approach is called target control^{52,53}. In all of our analyses, we set \(\varepsilon =1\).
A useful measure of the amount of ‘fuel’ used to achieve drug control of autophagy is the total dosage of drug i used up to time t during a therapy \({{\mathscr{T}}}_{k}\), which we denote as \({r}_{i,k}^{\ast }(t)\). This quantity is calculated using
where \({u}_{i}^{\ast }(t)\) for \(i\in {{\mathscr{T}}}_{k}\) is the solution of the nonlinear optimal control problem defined by Eqs (5) and (6).
Optimal monotherapies
We will illustrate generic features of solutions to the nonlinear optimal control problem defined by Eqs (5) and (6) by focusing on a particular (severe) energy/nutrient stress condition (i.e., the condition where \({C}_{{\rm{Nu}}}={C}_{{\rm{En}}}=0.1\)). For this condition, the system represented by Eq. (1) has a near maximal, steadystate AV count of approximately 37 per cell (i.e., \({x}_{5}\approx 37\)). Let us focus for the moment on monotherapy with drug 4 (an AMPK inhibitor) to downregulate the number of AVs to a target level of 10 per cell (i.e., \({x}_{5}^{f}=10\)) over the time period between \({t}_{0}=120\,{\rm{\min }}\) and \({t}_{f}=240\,{\rm{\min }}\) from an unperturbed steady state (i.e., dynamics with u_{i} = 0) at t = 0.
We solved the optimal control problem using the approach outlined in the Methods section and described in more detail in “PseudoSpectral Optimal Control” in Supplementary Methods. The solution, represented by the optimal cumulative dosage of drug 4 (i.e., \({r}_{4,1}^{\ast }(t)\)) (Eq. (7)), is presented in Fig. 4A. The optimal solution exhibits several generic features of the system’s dynamics, regardless of its parameterization. First, the computation suggests an optimal earliest time to apply the drug. In this particular example, this time is \(t\lesssim 60\,{\rm{\min }}\). The difference between the target time t_{0} and the earliest time to apply the drug quantitatively measures the speed of action of the drug. Secondly, the function \({r}_{4,1}^{\ast }(t)\) exhibits a staircase behavior, indicating that the optimal strategy of drug administration for this particular problem is to intermittently inject a specific dosage of drug into the system at specific times. Mathematically, this is due to the fact that the objective functional (Eq. (5)) is a linear combination of the L^{1} norm of the injection/input rate u_{i}’s—see Sections 5.5 and 5.6 in Kirk^{50}.
Figure 4B depicts how the drug concentration w_{4}(t) evolves subject to the optimal protocol \({u}_{4}^{\ast }(t)\). We observe surges of w_{4}(t) in response to the drug being applied to the system in large quantities over small time intervals, and slow decays in between applications of the drug (caused by the natural decay of the drug concentration in the absence of external drug inputs dictated by δ_{i}). As a consequence, the optimal solution is to inject a relatively large dose of a drug periodically, and to continuously supply small amounts of that drug to replenish drug cleared from the system to stably maintain autophagic flux (i.e., constant AV count and constant degradative flux, which we take to be proportional to the AV count).
Figure 4C illustrates the time evolution of x_{5} (AV count) subject to the optimal drug administration protocol. As can be seen, for \(t\ge 120\,{\rm{\min }}\), x_{5} is maintained within the desired interval \({x}_{{5}_{f}}\pm \varepsilon =10\pm 1\). The time evolution of the nontarget variables x_{1}, x_{2}, x_{3} and x_{4} (i.e., the activities of the regulatory kinases) are presented in Fig. 4D. Together, Fig. 4C,D provide a full representation of the time evolution of the system represented by Eq. (1) (the target and nontarget variables) under the influence of the optimal drug administration schedule. Because our procedure to find the optimal solution to the nonlinear optimal control problem is numerical, we have verified that the optimal control solution satisfies the necessary conditions that it must satisfy for optimality. See “PseudoSpectral Optimal Control” in Supplementary Methods for details.
Given that cancer cells may be killed by using drugs to either elevate or suppress autophagy^{26}, we will now consider optimal control solutions that either upregulate or downregulate autophagic flux by using a single drug. We will identify the drugs which can perturb and maintain the system near the target AV count. Perhaps more importantly, our analysis will deliver optimal protocols which include the precise times to inject the drugs, whose dosages are also tightly controlled to minimize the total quantities of drugs that are supplied.
Let us consider the case of intermediate energy/nutrient stress before treatment (i.e., the condition corresponding to \({C}_{{\rm{Nu}}}={C}_{{\rm{En}}}=0.6\); see Fig. 2), for which the system exhibits oscillations in the range \([20,27]\) without treatments. For this scenario, our goal is to either downregulate the number of AVs to \({x}_{5}^{f}\approx 9\) (shown in Fig. 4E–H) or to upregulate the AVs to \({x}_{5}^{f}\approx 37\) (shown in Fig. 4I–L). We have performed extensive numerical solutions of the monotherapy optimal control problem with various settings of the parameters \({w}_{i}^{{\rm{\max }}}\), t_{0}, t_{f} and \({x}_{5}^{f}\). We set the control window in the interval between \({t}_{0}=120\,{\rm{\min }}\) and \({t}_{f}=240\,{\rm{\min }}\) and imposed a constraint on each drug concentration w_{i}, requiring it not to exceed \({w}^{{\rm{\max }}}=4\times {{\rm{EC}}}_{50}\).
We found drug 2 to be best suited for downregulation for two reasons. First, drug 2 is able to drive x_{5} nearly to zero (in contrast with the case for drug 3 or 4). See Fig. 2B,H and compare with Fig. 2C,D,I,J. Second, drug 2 (in contrast with drug 6) is able to overcome the autonomous oscillatory behavior in x_{5}. In the analysis summarized in Supplementary Fig. S7, we found that drug 6 cannot eliminate oscillatory behavior; thus, it is incapable of maintaining a low, steady AV level. Drug 6 becomes viable if we remove the lower bound from the constraint of Eq. (6c). Without the lower bound, oscillations in x_{5} are permitted. We choose to keep the constraint of Eq. (6c) as written to avoid oscillatory solutions because, depending on period and amplitude, oscillations in x_{5} may allow for autophagyaddicted cells to survive periods of relatively low autophagy by thriving during periods of relatively high autophagy. In the other direction (i.e., druginduced upregulation of autophagy), it is only possible to use drug 5 to upregulate autophagy to the target value \({x}_{5}^{f}=37\) (Fig. 3). Figures 4E–H and I–L illustrate the optimal solutions using drugs 2 and 5 to downregulate and upregulate autophagy, respectively.
Although the selection of a single drug to achieve a given qualitative change in x_{5} is intuitive, especially given the results of Fig. 3, optimization of drug scheduling (Fig. 4) delivers better solutions in the sense that the total dosage applied to achieve the same effect (compared to constant input) is lower (minimized). Furthermore, the generic staircaselike solutions for \({r}_{i,k}^{\ast }\) illustrated in Fig. 4 persist for all the parameter sets we have tested (see below), indicating that variable, tightly controlled dosages should be injected into the system at controlled times. Given a particular type of drug, the central result of our optimal control analysis is to provide injection/input times and the amounts of drugs to be injected/added.
Optimal combination therapies
Let us now consider dual therapies (\(k=2\)). The motivation is to identify therapies—protocols involving lower quantities of drugs and faster responses—that are even more efficient than optimal monotherapies. We have evaluated all possible dual therapies (\({C}_{2}^{6}=15\)) for each of two energy/nutrient stress conditions: \({C}_{{\rm{En}}}={C}_{{\rm{Nu}}}=0.1\) (corresponding to severe stress) and \({C}_{{\rm{En}}}={C}_{{\rm{Nu}}}=0.6\) (corresponding to moderate stress). With an identical control objective and identical constraints \({w}_{i}^{{\rm{\max }}}=2.0\) \({t}_{0}=120\), \({t}_{f}=240\), \({x}_{5}^{f}=10\), and \(\varepsilon =1\), we found four pairs of drugs that are each more efficient than the optimal monotherapy with either of the two drugs included in the combination. These dual therapies are illustrated in Fig. 5. Additional results from our analyses of dual therapies are presented in the Supplementary Note and Supplementary Figs S3–S10.
We found that when baseline autophagy is high (\({C}_{{\rm{En}}}={C}_{{\rm{Nu}}}=0.1\)), the only combination of drugs that can drive AV count down to the target \({x}_{5}^{f}\) is the combination of drugs 2 and 6. The dynamical response of the system is shown in Fig. 5A–D. For this particular combination, either drug alone cannot lower x_{5} to 10 without violating one or both of the constraints \({w}_{i} < {w}_{i}^{{\rm{\max }}}\) (i = 2, 6). However, with use of drugs 2 and 6 in combination, it is possible to achieve the target AV count because the effects of the drugs are multiplicative (Eq. (1d)) and drug 2 directly affects both MTORC1 (Eq. (1a)) and VPS34 (Eq. (1d)).
Our analysis predicts nontrivial synergistic activities between drugs when the baseline level of autophagy is intermediate (on average) and exhibits oscillatory behavior (\({C}_{{\rm{En}}}={C}_{{\rm{Nu}}}=0.6\)). The results are summarized in Fig. 5E–P. In this scenario, multiple drug combinations (drugs 1 and 6, 2 and 6, and 3 and 6) are able to downregulate and stabilize x_{5}, whereas drug 6 alone cannot do so. Using drug 6 alone results in oscillations in x_{5}, causing a violation of the constraint of Eq. (6c). More interestingly, the optimal application of the drugs reveals a clear sequential protocol: first apply a drug other than drug 6 (1, 2, or 3) to suppress oscillations (see Fig. 5H,L,P), then apply drug 6 to drive AV count down to the desired level. The combination of drugs 1 and 6 is peculiar in that in this case application of drug 1 drives the system out of the oscillatory regime (Fig. 5O) but also upregulates autophagy; subsequent application of drug 6 is effective in downregulating autophagy.
It is important to emphasize that the two drugs acting together in any given combination therapy are, for simplicity, modeled as noninteracting, which may or may not be reasonable, depending on the mechanisms of actions of specific drugs of interest. The drug synergies detected in our analyses arise from the nonlinear dynamics of the regulatory network controlling autophagy. Without the formal framework presented here for therapy design, it would arguably be difficult to identify these synergies.
Discussion
Here, we have taken up the problem of designing targeted therapies to control a cellular phenotype of cancer cells, namely, their commitment to recycling of cytoplasmic contents through the process of autophagy, as measured by cellular autophagic vesicle (AV) count. Autophagy generates building blocks needed for de novo protein synthesis in support of growth (and proliferation). Modulation of autophagy, up or down, in autophagyaddicted cancer cells has the potential to selectively kill these cells^{26}.
Our approach was to first construct a mathematical model for autophagy regulation that captures the effects of key physiological stimuli—changes in the supplies of energy and nutrients—and the idealized effects of six available drug types (Eq. (1), Figs. 1–3) and to then pose the question of therapy design as a constrained, optimal control problem (Eqs (4)–(6)). Numerical solution of this problem, through optimization of a control input accounted for in the model (i.e., an adjustable timedependent drug injection/input rate), yielded monotherapy drug schedules that require a minimum amount of drug, maintain drug concentration below a specified threshold at all times, and that bring about desired effects in the most efficient manner possible (Fig. 4), in a welldefined sense. Furthermore, through the essentially same approach, but with consideration of adjustable timedependent drug injection/input rates for two different drugs, we were able to predict synergistic drug pairs (Fig. 5).
Optimal monotherapies were found to entail intermittent pulses of drug injection/input at irregular, nonobvious intervals and doses (Fig. 4). These features of optimal drug schedules—the pulsatile nature of drug administration and the irregularity of drug administration in terms of both timing and dosage—appear to be generic and each is discussed in further detail below.
The pulsatile nature of optimal monotherapy arises from the optimal control problem that we posed (Eqs (4)–(6)), which can be viewed as a minimumfuel problem, in that our control problem calls for usage of a minimal total amount of drug. The rationale for this control objective is that drugs typically have dosedependent offtarget effects, which may contribute to drug toxicity. Thus, by seeking drug schedules that achieve desired endpoints while using only a minimal total amount of drug, we seek to mitigate the possible negative consequences of offtarget drug effects. Mathematically, our minimumfuel objective function, Eq. (5), leads to pulsatile drug administration because the Hamiltonian of the optimal control problem is linear in the control inputs u_{i}(t), \(i\in {{\mathscr{T}}}_{k}\) (see “PseudoSpectral Optimal Control” in Supplementary Methods for a detailed derivation). Optimal control problems which have Hamiltonians that are linear in the control input are wellknown to have singular arcs, that is, discontinuities jumping between upper and lower bounds of the control input (see Chapter 5 in Kirk^{50} for the derivation of singular arc behavior and the brief overview of this issue in “PseudoSpectral Optimal Control” in Supplementary Methods). Because we do not impose an upper bound on u_{i}(t), the discontinuities we expect to see are Dirac delta type functions, a pulse of infinite magnitude but infinitesimal width. With the use of numerical methods to find solutions of the optimal control problem, we cannot capture the Dirac delta behavior exactly. Instead, we see finite pulses of finite width, which, while likely suboptimal, are more physically realistic.
Although pulses of drug input are consistent with convenient drug delivery modalities, such as oral administration of a drug in pill form or intravenous injection, optimal schedules do not entail uniform drug doses, nor uniform periods of drug administration. This irregular nature of optimal drug administration depends on the structure of the nonlinear cellular network that controls the synthesis of AVs. In particular, in our model, each drug specifically targets individual nodes of the cellular network, and therefore, different drugs play dynamically distinct roles and cannot be treated as equivalent control inputs. Thus, it may be critically important to better understand the interplay between targeted therapies and archetypical cellular regulatory network dynamics if we are to design the best possible therapies for populations of patients. Because network dynamics can be expected to vary between patients, patientspecific variability in network dynamics, which we have not considered in our analyses here, is a factor that likely affects the efficacy of individualized targeted therapy and that therefore should receive attention in future studies. The study of Fey et al.^{54} points to the feasibility of considering patientspecific parameters in mathematical models. In this study, gene expression data available for individual patients were used to set the abundances of gene products in patientspecific models for a cell signaling system. Because mutations can be detected in the tumors of individual patients, effects of oncogenic mutations could also potentially be accounted for in patientspecific models. The study of Rukhlenko et al.^{6} provides an example of a study where the effects of an oncogenic mutation were considered in a mathematical model. In the study of Fröhlich et al.^{55}, gene expression and mutational profiles were both considered in cell linespecific models.
The therapy design approach presented here is flexible and allows for the evaluation of drug combinations. In our analyses, we focused on dual therapies. Somewhat surprisingly, we found several drug pairs that together are more effective than either drug alone (according to our model). These pairs are drug 2 and drug 6 when \({C}_{{\rm{Nu}}}={C}_{{\rm{En}}}=0.1\) (severe energy/nutrient stress) and the combination of drug 6 with drug 1, 2, or 3 when \({C}_{{\rm{Nu}}}={C}_{{\rm{En}}}=0.6\) (moderate energy/nutrient stress). In the latter cases, drug 6 alone is incapable of downregulating autophagy to the desired level, but it sensitizes the network to drugs 1–3 when one of these drugs is used in conjunction with drug 6. According to the model (and its parameterization), the most potent synergistic drug pair is the combination of drugs 2 and 6. With this combination, the total amount of drug 2 used was reduced by more than 5fold (see the Supplementary Note and Supplementary Fig. S5) in comparison to the case where drug 2 is used optimally in isolation. More striking perhaps is that drug 6 when used alone is incapable of achieving the performance objective. Interestingly, our results provide mechanistic insight into the optimal sequence of drug delivery: therapy is optimal when drug 2 is injected about 80 minutes earlier than drug 6. That is, the best outcome was achieved when first inhibiting MTORC1, thus halting the intrinsic oscillations of the network dynamics, and then only inhibiting VPS34 to reduce synthesis of AVs. It should be noted that in our evaluation of this drug pair, we have assumed that there is no interaction between drugs 2 and 6, an idealization that may not be appropriate for specific examples of drugs of these types.
The same optimal control approach that we have demonstrated for 2drug combinations can be applied for combinations involving more than two drugs. Indeed, our approach was presented for the general case of k drugs used in combination. Our expectation is that effective combinations involving more than two drugs may be more likely to exist than effective combinations involving only two drugs, because controllability would presumably increase with the availability of more drugs. However, finding an effective combination may be more computationally expensive because of the larger number of possible combinations, and 2drug combinations may be preferable to higherorder combinations because of drug side effects.
As reported by Palmer and Sorger^{56}, many clinically used drug combinations are effective for reasons other than drug synergy, which is rare. In essence, the majority of clinically available drug combinations are, for all intents and purposes, equivalent to monotherapy at the level of individual patients. The basis for their effectiveness at the population level is simply that tumors in different subpopulations of patients have distinct drug sensitivities. Thus, new methods for predicting promising, nonobvious synergistic drug combinations, such as the approach reported here, could be helpful in developing combination therapies that derive their effectiveness from drug synergy. Synergistic drug combinations would seemingly offer significant benefits over monotherapy, or what is effectively monotherapy, in terms of delaying or perhaps eliminating the emergence of drug resistance. We note that our analysis identified synergies between pairs of drugs that are predicted to manifest without fine tuning of the doses used or the timing of drug administration. We admit that these predictions could perhaps have been found through an ad hoc model analysis. Nevertheless, we see value in leveraging an optimal control framework for model analysis, even if an optimal control strategy is not sought, because with this type of approach it is less likely that interesting behavior will be missed.
There is presently cautious optimism that effective drug combinations will be identified through highthroughput screening experiments^{57}, or through learning from data. However, the sheer number of possible drug combinations poses a barrier to experimental discovery of efficacious drug combinations and it is not clear that the data requirements of machine learning approaches can be met in the near term. Thus, it is important to consider alternatives, such as the approach presented here, which leverages available mechanistic understanding of how regulatory protein/lipid kinases influence the synthesis of AVs, which we have consolidated in the form of a mathematical model (Eq. (1)), designed to be useful for computational characterization of drug combinations. We note that our model was formulated specifically for this purpose, and it was not designed to make predictions outside this limited domain of application. Indeed, to facilitate our computational analyses, the model was handcrafted to be as simple as possible while still reproducing key behaviors of more mechanistically detailed models^{33,34}. This approach was helpful in making calculations feasible. Unfortunately, to our best knowledge, there are no proven approaches for systematically and automatically deriving a suitable surrogate model for therapy design from a more detailed, mechanistic model of a cellular regulatory network. Pursuit of such a capability seems like an important subject of future research.
Our intent at the start of this study was to investigate how control engineering concepts might be introduced into formal therapy design. Thus, we have only attempted to demonstrate that our methodology is capable of generating interesting (and testable) predictions of effective drug schedules and drug combinations. Development of novel therapies will, of course, require experimental validation of candidate combinations, which is beyond the intended scope of the present study. Thus, we caution that our predictions of optimal drug schedules and synergistic drug combinations are only intended to demonstrate methodology. The merit of this methodology is not in reaching final conclusions but in prioritizing experimental efforts and thereby accelerating experimental validation of targeted therapies. Because kinase inhibitors of each type considered in our analysis are available for experimental characterization and autophagy is a cellular phenotype that can be readily assayed, as in the study of Martin et al.^{34} or du Toit et al.^{58}, a logical next step would be to probe for the predicted drug synergies in cell line experiments. It might be especially interesting to evaluate a combination of an ULK1specific inhibitor, such as ULK101^{59}, and a VPS34specific inhibitor, such as VPS34IN1^{60}. We predict that this combination will be synergistic, and the combination targets the two kinases considered in our analysis that are most proximal to the cellular machinery for producing autophagosomes. On the computational side, to increase confidence in predictions, sensitivity analysis techniques tailored for optimal control problems could be applied to characterize the robustness of predictions^{61,62}, and experimental design techniques could be applied to aid in generating data useful for reducing parameter uncertainty^{63,64}. Several studies strongly support the potential value of formal therapy design^{65,66,67}, and the main contribution here is a new approach to this subject. Two important distinguishing features of this approach are 1) the consideration of a mathematical model for a cellular regulatory network that controls a cellular phenotype and 2) application of sophisticated methods from automatic control theory.
Methods
Simulations
Simulations were performed by numerical integration of the model ODEs. The parameter settings used in calculations are provided in the Supplementary Tables S1 and S2.
PseudoSpectral Optimal Control
Optimal control as a field of research combines aspects of dynamical systems, mathematical optimization and the calculus of variations^{50}. Together Eqs (5) and (6) form a constrained optimal control problem, which can generally be written as,
In general, there exists no analytic framework that is able to provide the optimal time traces of the controls and the states in Eq. (8), and so we must resort to numerical techniques.
Pseudospectral optimal control (PSOC) has become a popular tool in recent years^{68,69} that has allowed scientists and engineers solve optimal control problems like that of Eq. (8) reliably and efficiently in applications such as guiding autonomous vehicles and maneuvering the international space station^{69}. The main concepts of PSOC are summarized here but are explained at length in “PseudoSpectral Optimal Control” in Supplementary Methods. See also Supplementary Fig. S11. We define a set of N discrete times \(\{{\tau }_{i}\}\) \(i=0,1,\ldots ,N\) where \({\tau }_{0}=\,1\) and \({\tau }_{N}=1\) with a mapping between \(t\in [{t}_{0},{t}_{f}]\) and \(\tau \in [\,\,1,1]\). The choice of \(\{{\tau }_{i}\}\) is key to the convergence of the full discretized problem and so typically they are chosen as the roots of an N + 1th order orthogonal polynomial such as Legendre or Chebyshev. In fact, the type of PSOC one uses is typically named after the type of polynomial used to generate the discretization points. Let \(\hat{{\bf{x}}}(\tau )={\sum }_{i=0}^{N}\,{\hat{{\bf{x}}}}_{i}{L}_{i}(\tau )\) be an approximation of \({\bf{x}}(\tau )\) where \({L}_{i}(\tau )\) is the ith Lagrange interpolating polynomial. The dynamical system is approximated by differentiating the approximation \(\hat{{\bf{x}}}(\tau )={\sum }_{i=0}^{N}\,{\hat{{\bf{x}}}}_{i}{L}_{i}(\tau )\) with respect to time.
Let \({D}_{k,i}=\frac{d}{d\tau }{L}_{i}({\tau }_{k})\) so that we may rewrite the original dynamical system constraint in Eq. (8) as the following set of algebraic constraints.
With the original timevarying states and control inputs now discretized, the dynamical equations approximated with Lagrange interpolating polynomials, and the cost function approximated by a quadrature, the discretized optimal control problem can be expressed as the following nonlinear programming (NLP) problem.
We used \({\mathscr{P}}{\mathscr{S}}{\mathscr{O}}{\mathscr{P}}{\mathscr{T}}\)^{35}, an opensource PSOC toolbox written in C++, to perform the PSOC discretization procedure.
The NLP problem of Eq. (11) can be solved with a number of different techniques, but here we use an interior point algorithm^{70} as implemented in the opensource C++ software Ipopt^{71}.
Data Availability
Problemspecific software used in this study is provided as Supplementary Data.
References
 1.
Jameson, J. L. & Longo, D. L. Precision medicine—personalized, problematic, and promising. Obstetrical & Gynecological Survey 372, 2229–2234, https://doi.org/10.1097/01.ogx.0000472121.21647.38 (2015).
 2.
Gatzka, M. V. Targeted tumor therapy remixed—an update on the use of smallmolecule drugs in combination therapies. Cancers 10, 155, https://doi.org/10.3390/cancers10060155 (2018).
 3.
Anderson, A. R. A. & Quaranta, V. Integrative mathematical oncology. Nature Reviews Cancer 8, 227–234, https://doi.org/10.1038/nrc2329 (2008).
 4.
Michor, F. & Beal, K. Improving cancer treatment via mathematical modeling: surmounting the challenges is worth the effort. Cell 163, 1059–1063, https://doi.org/10.1016/j.cell.2015.11.002 (2015).
 5.
Vogelstein, B. et al. Cancer genome landscapes. Science 339, 1546–1558, https://doi.org/10.1126/science.1235122 (2013).
 6.
Rukhlenko, O. S. et al. Dissecting RAF inhibitor resistance by structurebased modeling reveals ways to overcome oncogenic RAS signaling. Cell Systems 7, 161–179, https://doi.org/10.1016/j.cels.2018.06.002 (2018).
 7.
Ramos, P. & BentiresAlj, M. Mechanismbased cancer therapy: resistance to therapy, therapy for resistance. Oncogene 34, 3617–3626, https://doi.org/10.1016/j.cell.2017.11.009 (2015).
 8.
Klionsky, D. J. & Emr, S. D. Autophagy as a regulated pathway of cellular degradation. Science 290, 1717–1721, https://doi.org/10.1126/science.290.5497.1717 (2000).
 9.
Deretic, V., Saitoh, T. & Akira, S. Autophagy in infection, inflammation and immunity. Nature Reviews Immunology 13, 722–737, https://doi.org/10.1038/nri3532 (2013).
 10.
Levine, B., Mizushima, N. & Virgin, H. W. Autophagy in immunity and inflammation. Nature 469, 323–335, https://doi.org/10.1038/nature09782 (2011).
 11.
Mizushima, N., Levine, B., Cuervo, A. M. & Klionsky, D. J. Autophagy fights disease through cellular selfdigestion. Nature 451, 1069–1075, https://doi.org/10.1038/nature06639 (2008).
 12.
Nakatogawa, H., Suzuki, K., Kamada, Y. & Ohsumi, Y. Dynamics and diversity in autophagy mechanisms: lessons from yeast. Nature Reviews Molecular Cell Biology 10, 458–467, https://doi.org/10.1038/nrm2708 (2009).
 13.
Kahn, B. B., Alquier, T., Carling, D. & Hardie, D. G. AMPactivated protein kinase: ancient energy gauge provides clues to modern understanding of metabolism. Cell Metabolism 1, 15–25, https://doi.org/10.1016/j.cmet.2004.12.003 (2005).
 14.
Löffler, A. S. et al. Ulk1mediated phosphorylation of AMPK constitutes a negative regulatory feedback loop. Autophagy 7, 696–706, https://doi.org/10.4161/auto.7.7.15451 (2011).
 15.
Zoncu, R., Efeyan, A. & Sabatini, D. M. mTOR: from growth signal integration to cancer, diabetes and ageing. Nature Reviews Molecular Cell Biology 12, 21, https://doi.org/10.1038/nrm3025 (2011).
 16.
Nazio, F. et al. mTOR inhibits autophagy by controlling ULK1 ubiquitylation, selfassociation and function through AMBRA1 and TRAF6. Nature Cell Biology 15, 406, https://doi.org/10.1038/ncb2708 (2013).
 17.
Shang, L. et al. Nutrient starvation elicits an acute autophagic response mediated by Ulk1 dephosphorylation and its subsequent dissociation from AMPK. Proceedings of the National Academy of Sciences 108, 4788–4793, https://doi.org/10.1073/pnas.1100844108 (2011).
 18.
Dunlop, E. A., Hunt, D. K., AcostaJaquez, H. A., Fingar, D. C. & Tee, A. R. ULK1 inhibits mTORC1 signaling, promotes multisite Raptor phosphorylation and hinders substrate binding. Autophagy 7, 737–747, https://doi.org/10.4161/auto.7.7.15491 (2011).
 19.
Kim, J., Kundu, M., Viollet, B. & Guan, K.L. AMPK and mTOR regulate autophagy through direct phosphorylation of Ulk1. Nature Cell Biology 13, 132, https://doi.org/10.1038/ncb2152 (2011).
 20.
Fimia, G. M. et al. Ambra1 regulates autophagy and development of the nervous system. Nature 447, 1121, https://doi.org/10.1038/nature05925 (2007).
 21.
Di Bartolomeo, S. et al. The dynamic interaction of AMBRA1 with the dynein motor complex regulates mammalian autophagy. The Journal of Cell Biology 191, 155–168, https://doi.org/10.1083/jcb.201002100 (2010).
 22.
Galluzzi, L., BravoSan Pedro, J. M., Levine, B., Green, D. R. & Kroemer, G. Pharmacological modulation of autophagy: therapeutic potential and persisting obstacles. Nature Reviews Drug Discovery 16, 487–511, https://doi.org/10.1038/nrd.2017.22 (2017).
 23.
Moschetta, M., Reale, A., Marasco, C., Vacca, A. & Carratù, M. R. Therapeutic targeting of the mTORsignalling pathway in cancer: benefits and limitations. British Journal Pharmacology 171, 3801–3813, https://doi.org/10.1111/bph.12749 (2014).
 24.
Hardie, D. G. AMPK: a target for drugs and natural products with effects on both diabetes and cancer. Diabetes 62, 2164–2172, https://doi.org/10.2337/db130368 (2013).
 25.
Shintani, T. & Klionsky, D. J. Autophagy in health and disease: a doubleedged sword. Science 306, 990–995, https://doi.org/10.1126/science.1099993 (2004).
 26.
Mulcahy Levy, J. M., Towers, C. G. & Thorburn, A. Targeting autophagy in cancer. Nature Reviews Cancer 17, 528–542, https://doi.org/10.1038/nrc.2017.53 (2017).
 27.
Edwards, S. R. & Wandless, T. J. The rapamycinbinding domain of the protein kinase mammalian target of rapamycin is a destabilizing domain. Journal of Biological Chemistry 282, 13395–13401, https://doi.org/10.1074/jbc.M700498200 (2007).
 28.
Egan, D. F. et al. Small molecule inhibition of the autophagy kinase ULK1 and identification of ULK1 substrates. Molecular Cell 59, 285–297, https://doi.org/10.1016/j.molcel.2015.05.031 (2015).
 29.
Meley, D. et al. AMPactivated protein kinase and the regulation of autophagic proteolysis. Journal of Biological Chemistry 281, 34870–34879, https://doi.org/10.1074/jbc.M605488200 (2006).
 30.
Ronan, B. et al. A highly potent and selective Vps34 inhibitor alters vesicle trafficking and autophagy. Nature Chemical Biology 10, 1013, https://doi.org/10.1038/nchembio.1681 (2014).
 31.
Cameron, K. O. et al. Discovery and preclinical characterization of 6chloro5[4(1hydroxycyclobutyl) phenyl]1 Hindole3carboxylic acid (PF06409577), a direct activator of adenosine monophosphateactivated protein kinase (AMPK), for the potential treatment of diabetic nephropathy. Journal of Medicinal Chemistry 59, 8068–8081, https://doi.org/10.1021/acs.jmedchem.6b00866 (2016).
 32.
Burger, M. T. et al. Identification of NVPBKM120 as a potent, selective, orally bioavailable class I PI3 kinase inhibitor for treating cancer. ACS Medicinal Chemistry Letters 2, 774–779, https://doi.org/10.1021/ml200156t (2011).
 33.
Szymańska, P., Martin, K. R., MacKeigan, J. P., Hlavacek, W. S. & Lipniacki, T. Computational analysis of an autophagy/translation switch based on mutual inhibition of MTORC1 and ULK1. PLoS One 10, e0116550, https://doi.org/10.1371/journal.pone.0116550 (2015).
 34.
Martin, K. R. et al. Computational model for autophagic vesicle dynamics in single cells. Autophagy 9, 74–92, https://doi.org/10.4161/auto.22532 (2013).
 35.
Becerra, V. M. Solving complex optimal control problems at no cost with PSOPT. In ComputerAided Control System Design (CACSD), 2010 IEEE International Symposium on, 1391–1396, https://doi.org/10.1109/CACSD.2010.5612676 (IEEE, 2010).
 36.
Martin, R. Optimal control drug scheduling of cancer chemotherapy. Automatica 28, 1113–1123, https://doi.org/10.1016/00051098(92)90054J (1992).
 37.
Świerniak, A., Ledzewicz, U. & Schättler, H. Optimal control for a class of compartmental models in cancer chemotherapy. International Journal of Applied Mathematics and Computer Science 13, 357–368 (2003).
 38.
Ledzewicz, U. & Schättler, H. M. Drug resistance in cancer chemotherapy as an optimal control problem. Discrete and Continuous Dynamical Systems Series B 6, 129, https://doi.org/10.3934/dcdsb.2006.6.129 (2006).
 39.
Ledzewicz, U. & Schättler, H. Optimal controls for a model with pharmacokinetics maximizing bone marrow in cancer chemotherapy. Mathematical Biosciences 206, 320–342, https://doi.org/10.1016/j.mbs.2005.03.013 (2007).
 40.
Joshi, H. R. Optimal control of an HIV immunology model. Optimal Control Applications and Methods 23, 199–213, https://doi.org/10.1002/oca.710 (2002).
 41.
Cornelius, S. P., Kath, W. L. & Motter, A. E. Realistic control of network dynamics. Nature Communications 4, 1942, https://doi.org/10.1038/ncomms2939 (2013).
 42.
Wang, L.Z. et al. A geometrical approach to control and controllability of nonlinear dynamical networks. Nature Communications 7, 11323, https://doi.org/10.1038/ncomms11323 (2016).
 43.
Zañudo, J. G. T., Yang, G. & Albert, R. Structurebased control of complex networks with nonlinear dynamics. Proceedings of the National Academy of Sciences 114, 7234–7239, https://doi.org/10.1073/pnas.1617387114 (2017).
 44.
Klickstein, I., Shirin, A. & Sorrentino, F. Locally optimal control of complex networks. Physical Review Letters 119, 268301, https://doi.org/10.1103/PhysRevLett.119.268301 (2017).
 45.
Sato, E. et al. Temporal decline in sirolimus elimination immediately after pancreatic islet transplantation. Drug Metabolism and Pharmacokinetics 21, 492–500, https://doi.org/10.2133/dmpk.21.492 (2006).
 46.
Baselga, J. et al. Buparlisib plus fulvestrant versus placebo plus fulvestrant in postmenopausal, hormone receptorpositive, HER2negative, advanced breast cancer (belle2): a randomised, doubleblind, placebocontrolled, phase 3 trial. The Lancet Oncology 18, 904–916, https://doi.org/10.1016/S14702045(17)303765 (2017).
 47.
Milkiewicz, K. L. et al. Improvement in oral bioavailability of 2,4diaminopyrimidine cMet inhibitors by incorporation of a 3amidobenzazepin2one group. Bioorganic & Medicinal Chemistry 19, 6274–6284, https://doi.org/10.1016/j.bmc.2011.09.006 (2011).
 48.
Engers, D. W., Frist, A. Y., Lindsley, C. W., Hong, C. C. & Hopkins, C. R. Synthesis and structure–activity relationships of a novel and selective bone morphogenetic protein receptor (BMP) inhibitor derived from the pyrazolo [1.5a] pyrimidine scaffold of dorsomorphin: the discovery of ML347 as an ALK2 versus ALK3 selective MLPCN probe. Bioorganic & Medicinal Chemistry Letters 23, 3248–3252, https://doi.org/10.1016/j.bmcl.2013.03.113 (2013).
 49.
Juric, D. et al. A firstinhuman, phase I, doseescalation study of TAK117, a selective PI3Kα isoform inhibitor, in patients with advanced solid malignancies. Clinical Cancer Research https://doi.org/10.1158/10780432.CCR162888 (2017).
 50.
Kirk, D. E. Optimal Control Theory: an Introduction (Courier Corporation, 2012).
 51.
Lewis, F. L., Vrabie, D. & Syrmos, V. L. Optimal Control (John Wiley & Sons, 2012).
 52.
Klickstein, I., Shirin, A. & Sorrentino, F. Energy scaling of targeted optimal control of complex networks. Nature Communications 8, 15145, https://doi.org/10.1038/ncomms15145 (2017).
 53.
Shirin, A., Klickstein, I. & Sorrentino, F. Optimal control of complex networks: Balancing accuracy and energy of the control action. Chaos: An Interdisciplinary Journal of Nonlinear Science 27, 041103, https://doi.org/10.1063/1.4979647 (2017).
 54.
Fey, D. et al. Signaling pathway models as biomarkers: Patientspecific simulations of JNK activity predict the survival of neuroblastoma patients. Science Signaling 8, ra130–ra130, https://doi.org/10.1126/scisignal.aab0990 (2015).
 55.
Fröhlich, F. et al. Efficient parameter estimation enables the prediction of drug response using a mechanistic pancancer pathway model. Cell Systems 7, 1–13, https://doi.org/10.1016/j.cels.2018.10.013 (2018).
 56.
Palmer, A. C. & Sorger, P. K. Combination cancer therapy can confer benefit via patienttopatient variability without drug additivity or synergy. Cell 171, 1678–1691, https://doi.org/10.1016/j.cell.2017.11.009 (2017).
 57.
Holbeck, S. L. et al. The National Cancer Institute ALMANAC: a comprehensive screening resource for the detection of anticancer drug pairs with enhanced therapeutic activity. Cancer Research 77, 3564–3576, https://doi.org/10.1158/00085472.CAN170489 (2017).
 58.
du Toit, A., Hofmeyr, J.H. S., Gniadek, T. J. & Loos, B. Measuring autophagosome flux. Autophagy 14, 1060–1071, https://doi.org/10.1080/15548627.2018.1469590 (2018).
 59.
Martin, K. R. et al. A potent and selective ULK1 inhibitor suppresses autophagy and sensitizes cancer cells to nutrient stress. iScience 8, 74–84, https://doi.org/10.1016/j.isci.2018.09.012 (2018).
 60.
Bago, R. et al. Characterization of VPS34IN1, a selective inhibitor of Vps34, reveals that the phosphatidylinositol 3phosphatebinding SGK3 protein kinase is a downstream target of class III phosphoinositide 3kinase. Biochemical Journal 463, 413–427, https://doi.org/10.1042/BJ20140889 (2014).
 61.
Castillo, E., Mnguez, R. & Castillo, C. Sensitivity analysis in optimization and reliability problems. Reliability Engineering & System Safety 93, 1788–1800, https://doi.org/10.1016/j.ress.2008.03.010 (2008).
 62.
Malanowski, K. & Maurer, H. Sensitivity analysis for state constrained optimal control problems. Discrete & Continuous Dynamical SystemsA 4, 241–272, https://doi.org/10.3934/dcds.1998.4.241 (1998).
 63.
Hagen, D. R., White, J. K. & Tidor, B. Convergence in parameters and predictions using computational experimental design. Interface Focus 3, 20130008, https://doi.org/10.1098/rsfs.2013.0008 (2013).
 64.
Dehghannasiri, R., Yoon, B.J. & Dougherty, E. R. Efficient experimental design for uncertainty reduction in gene regulatory networks. In BMC Bioinformatics, vol. 16, S2, https://doi.org/10.1186/1471210516S13S2 (BioMed Central, 2015).
 65.
Chmielecki, J. et al. Optimization of dosing for EGFRmutant nonsmall lung cancer with evolutionary cancer modeling. Science Translational Medicine 3, 90ra59, https://doi.org/10.1126/scitranslmed.3002356 (2011).
 66.
Chakrabarti, S. & Michor, F. Pharmacokinetics and drug interactions determine optimum combination strategies in computational models of cancer evolution. Cancer Research 77, 3908–3921, https://doi.org/10.1158/00085472.CAN162871 (2017).
 67.
Stein, S., Zhao, R., Haeno, H., Vivanco, I. & Michor, F. Mathematical modeling identifies optimum lapatinib dosing schedules for the treatment of glioblastoma patients. PLoS Computational Biology 14, e1005924, https://doi.org/10.1371/journal.pcbi.1005924 (2018).
 68.
Rao, A. V. A survey of numerical methods for optimal control. Advances in the Astronautical Sciences 135, 497–528 (2009).
 69.
Ross, I. M. & Karpenko, M. A review of pseudospectral optimal control: From theory to flight. Annual Reviews in Control 36, 182–197, https://doi.org/10.1016/j.arcontrol.2012.09.002 (2012).
 70.
Nocedal, J. & Wright, S. Numerical Optimization (Springer, 2006).
 71.
Wächter, A. & Biegler, L. T. On the implementation of an interiorpoint filter linesearch algorithm for largescale nonlinear programming. Mathematical Programming 106, 25–57, https://doi.org/10.1007/s101070040559y (2006).
Acknowledgements
I.S.K., A.S. and F.S. acknowledge support from the National Science Foundation (CRISP1541148), the Office of Naval Research (N000141612637), and the Defense Threat Reduction Agency (HDTRA11210020). W.S.H. and Y.T.L. acknowledge support from the National Cancer Institute of the National Institutes of Health (R01CA197398). S.F. acknowledges support from the Center for Nonlinear Studies and the LaboratoryDirected Research and Development program at Los Alamos National Laboratory, which is operated by Triad National Security, LLC for the National Nuclear Security Administration of the U.S. Department of Energy (contract no. 89233218CNA000001). W.S.H. performed part of this work at the Aspen Center for Physics, which is supported by the National Science Foundation (PHY1607611). We thank Jeffrey P. MacKeigan and Maria Chiara Maiuri for helpful discussions.
Author information
Affiliations
Contributions
W.S.H. and F.S. designed the research; A.S., I.S.K., S.F. and Y.T.L. performed the research; and all authors contributed to the analyses of results and the writing of the manuscript. Modeling work was the primary responsibility of the Los Alamos National Laboratory authors. Optimal control work was the primary responsibility of the University of New Mexico authors.
Corresponding authors
Ethics declarations
Competing Interests
The authors declare no competing interests.
Additional information
Publisher’s note: Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
Supplementary information
Rights and permissions
Open Access This article is licensed under a Creative Commons Attribution 4.0 International License, which permits use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons license, and indicate if changes were made. The images or other third party material in this article are included in the article’s Creative Commons license, unless indicated otherwise in a credit line to the material. If material is not included in the article’s Creative Commons license and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder. To view a copy of this license, visit http://creativecommons.org/licenses/by/4.0/.
About this article
Cite this article
Shirin, A., Klickstein, I.S., Feng, S. et al. Prediction of Optimal Drug Schedules for Controlling Autophagy. Sci Rep 9, 1428 (2019). https://doi.org/10.1038/s41598019387639
Received:
Accepted:
Published:
Further reading

PyBioNetFit and the Biological Property Specification Language
SSRN Electronic Journal (2019)

PyBioNetFit and the Biological Property Specification Language
iScience (2019)

Opportunities for improving cancer treatment using systems biology
Current Opinion in Systems Biology (2019)
Comments
By submitting a comment you agree to abide by our Terms and Community Guidelines. If you find something abusive or that does not comply with our terms or guidelines please flag it as inappropriate.