# Reciprocal interactions between tumour cell populations enhance growth and reduce radiation sensitivity in prostate cancer

## Abstract

Intratumoural heterogeneity (ITH) contributes to local recurrence following radiotherapy in prostate cancer. Recent studies also show that ecological interactions between heterogeneous tumour cell populations can lead to resistance in chemotherapy. Here, we evaluated whether interactions between heterogenous populations could impact growth and response to radiotherapy in prostate cancer. Using mixed 3D cultures of parental and radioresistant populations from two prostate cancer cell lines and a predator-prey mathematical model to investigate various types of ecological interactions, we show that reciprocal interactions between heterogeneous populations enhance overall growth and reduce radiation sensitivity. The type of interaction influences the time of regrowth after radiation, and, at the population level, alters the survival and cell cycle of each population without eliminating either one. These interactions can arise from oxygen constraints and from cellular cross-talk that alter the tumour microenvironment. These findings suggest that ecological-type interactions are important in radiation response and could be targeted to reduce local recurrence.

## Introduction

Local recurrence remains a challenge in the management of localised prostate cancer with external beam radiotherapy1. Intratumoral heterogeneity (ITH) is one factor that likely contributes to recurrence. This heterogeneity stems in part from genetic variations: primary lesions contain multiple genetically and clonally distinct foci that are often maintained in recurrent and metastatic prostate tumours2,3,4,5. Phenotypic variations, such as changes in metabolism, also affect therapeutic response among prostate tumour clones1,6,7,8,9. Given this genotypic and phenotypic diversity, a population-based approach in targeting ITH with radiotherapy may help to reduce local recurrence.

One recent strategy to target ITH has been to model and treat tumours as complex ecosystems in which treatment-sensitive and resistant cell populations interact with each other10,11. Similar to interactions between species in other ecological systems12, those between tumour cell populations can be competitive, antagonistic (i.e., predatorial), or mutualistic (i.e., cooperative), depending on the microenvironmental pressures or communication between cell populations11. The interactions may alter the survival of each tumour cell population and, consequently, the entire tumour’s response to treatment13,14,15,16,17. By mapping the interactions and survival of sensitive and resistant populations under different treatment regimens, alternative treatment regimens that reduce local recurrence can be identified15,18.

Although well-studied with regard to chemotherapy, little is known about whether ecological interactions could contribute to prostate cancer recurrence to radiotherapy. Conventional approaches to modelling radiotherapy outcomes typically neglect ITH in two important ways. Firstly, the linear-quadratic model used to predict tumour control probability contains single, fixed values for the radiation sensitivity parameters (i.e., α and β, the constants for cell killing in the linear and quadratic phases, respectively)19. Simulations of tumour response to radiotherapy have shown, however, that modulating the radiation dose to account for populations with different radiation sensitivities reduces recurrence in silico20. Secondly, cell kill events are modelled as a Poisson process, with subsequent events occurring independently of one another19,21. Although the discovery of ecological-type interactions in tumours suggests that cell kill events may not occur independently, simulations and experimental evidence demonstrating the impact of these interactions on radiation response are lacking.

Characterising ecological-type interactions between tumour populations with differing radiation sensitivities may lead to new approaches that account for ITH at the radiotherapy treatment planning stage. Here, we evaluate whether such interactions occur between tumour cell populations with different radiation sensitivities and whether these interactions affect tumour growth kinetics and radiation response. Since prostate tumours develop resistance through multiple genetic mechanisms and often contain multiple clones22, we employed polyclonal tumour populations from two prostate cell lines to better simulate clinical scenarios of radio-recurrence. Given the complexity of ecological interactions, we developed an interdisciplinary approach to: (1) quantify the changes in tumour growth (before and after radiation) that result from ecological interactions between tumour cell populations, (2) determine the type of interaction (e.g., competition vs. cooperation), and (3) identify whether microenvironmental pressures and/or cell–cell communication mediate these interactions. Our results show that different types of ecological interactions enhance the growth of untreated tumours before and after radiation and that these interactions may arise in part from microenvironmental pressures and cellular cross-talk.

## Results

### Mixed cultures of tumour populations have enhanced growth and altered survival in the absence of radiation

To assess whether mixed cultures had altered growth characteristics, we measured the growth of unirradiated 3D tumour spheroids comprised of homogeneous or mixed cultures of parental and radioresistant populations (seeded 1:1). This in vitro model was selected because it mimics important features of solid tumours25 such as hypoxia, which renders tumours 2–3 times more resistant to radiation. Compared to homogeneous parental spheroids, volumes of mixed PC3 and DU145 spheroids increased by >30% (Padj < 0.001, day 10) and by >22% (Padj < 0.001, day 15, Fig. 2A), respectively. Since the change in growth could result from the radioresistant population outgrowing the parental one, we quantified the proportions of each population isolated from mixed spheroids over time using flow cytometry (Supplementary Fig. 2). On day 15 of growth, radioresistant cells isolated from mixed PC3 spheroids comprised 60.9 ± 9.6% of total cells, while those isolated from mixed DU145 spheroids comprised 22.7 ± 12.2% of total cells (Fig. 2B).

Analysis of each population’s survival revealed that different dynamics were at play in the two cell lines. In mixed PC3 spheroids, cell death was reduced by ≥14% (parental, Padj = 0.038; RR, Padj = 0.045) for both populations on day 15 (Fig. 2C). However, in mixed DU145 spheroids, cell death was enhanced by 26% for the radioresistant population on day 5 (Padj = 0.01, Fig. 2C), which could have led to an early survival advantage for the parental population. Thus, despite the contrasting population dynamics between the two cell lines, these results indicate that overall tumour growth, and survival of parental populations, in particular, are enhanced in unirradiated tumours with mixed populations.

### Increased growth of unirradiated mixed tumours results from ecological-type interactions

We used mathematical modelling to determine whether enhanced growth of mixed spheroids could have resulted from changes in proliferation capacity or from cellular interactions between the two populations. Baseline characteristics of each population were calculated by fitting the logistic growth model26 to data from homogeneous spheroids to estimate the growth rate r, the carrying capacity K, and the initial volume V0 (Fig. 3A). In PC3 spheroids, the radioresistant population had higher values of r and K than the parental population, while in DU145 spheroids, it had lower r and higher K values; values of V0 were not different between cell lines (Table 1). The growth rate and carrying capacity were held fixed at these values in subsequent in silico experiments.

To characterise the growth dynamics and type of ecological interactions between the populations in mixed spheroids, we fit the Lotka–Volterra (or predator–prey) model to data from the mixed spheroids (Fig. 3A). In this model, the interaction parameters, λP and λRR, describe the effect that parental cells have on radioresistant cells, and vice versa; the interactions may be competitive (λP > 0 and λRR > 0), mutualistic (λP < 0 and λRR < 0) or antagonistic (λP < 0 < λRR or λRR < 0 < λP)11. To confirm our ability to reliably infer the interaction parameters in our experimental data, we performed in silico studies in which the Lotka–Volterra model was used to generate synthetic growth curves of mixed spheroids with known interaction parameters (λP = 0.5 and λRR = 0.5) and three levels of experimental noise (5, 10, and 20%). Interaction parameters estimated by fitting only total volume measurements to the Lotka–Volterra model resulted in large uncertainties, whereas interaction parameters estimated using volume measurements supplemented with measurements of the ratio of the mixed populations decreased uncertainty in the parameter estimates (Supplementary Fig. 3). Thus, using both volume measurements and cellular proportions from our experimental data (Fig. 2A, B), we predicted competitive interactions in PC3 spheroids (λP > 0 and λRR > 0) and antagonistic interactions in DU145 spheroids (λP > 0 > λRR; Fig. 3B).

### The type of ecological interaction dictates regrowth time after radiation

We then investigated whether the type of ecological interaction that occurs could impact the regrowth time after radiotherapy. Using the Lotka–Volterra model, we simulated regrowth of mixed spheroids (seeded 1:1 in silico) after exposure to 6 Gy for a range of values of the interaction parameters λP and λRR. As expected, competitive cell populations (λP < 0 and λRR < 0) have the slowest regrowth times (50–200 days), while those with mutualistic interactions (λP > 0 and λRR > 0) have the fastest regrowth times (14–32 days) (Fig. 5A–D). Interestingly, within a specific interaction type, the values of the interaction parameters λP and λRR impacted the regrowth time. For example, for competitive interactions, combinations of values of λP and λRR between 0.5 and 0.75 day−1 mm−1 generate regrowth in 100–200 days (sharp peaks), while those ranging between 0 and 0.5 day−1 mm−1 generate regrowth in about 50 days (Fig. 5B). Furthermore, for antagonistic interactions, when λP < 0 < λRR regrowth is slower than when λP > 0 > λRR by a factor of almost 2 (Fig. 5A, D). These results suggest that the type of ecological interaction influences the time of regrowth after radiotherapy and that it could contribute to variability in tumour responses to radiotherapy.

### Oxygen constraints and cell–cell communication mediate enhanced growth and reduced radiation sensitivity of mixed PC3 spheroids

Having found that ecological interactions enhance the growth and decrease radiation sensitivity of mixed tumour populations, we subsequently tried to determine whether microenvironmental pressures and/or cell–cell communication could mediate the observed phenotypes. We focused in the first instance on mixed PC3 spheroids, which showed enhanced survival and reduced radiation sensitivity despite having competitive interactions. Since microenvironmental pressures, such as spatial constraints, affect cellular interactions in chemotherapy13, we evaluated the spatial distribution of each population in homogeneous and mixed spheroids using microscopy. Unlike those isolated on day 5, sections of mixed PC3 spheroids isolated on day 11 showed parental cells located predominantly in the centre, a region that overlapped with staining for hypoxia, but not for proliferation. Spheroids comprising mixed and radioresistant cells were visually less compact (Fig. 6A) than those comprising parental cells. The localisation of parental cells in mixed spheroids, which was detected in ~70% of spheroids (Supplementary Fig. 6), was also coupled with a decrease in radiation sensitivity. The clonogenic survival of the parental population isolated from untreated mixed spheroids on day 11 was 25% higher (Padj = 0.04) than that of parentals isolated from homogeneous spheroids; this effect was not seen in populations isolated from day 5 spheroids (Fig. 6B).

Based on these results, we hypothesised that the spatial structure may result from competition for oxygen or differences in oxygen consumption rate (OCR) between the two cell populations. To test this hypothesis, we developed a cellular automaton (CA) model describing the changes in the size and structure of a 2D cross-section through a 3D tumour spheroid in which cells from parental and radioresistant populations divide and die in response to the local oxygen concentration. Parameter values for proliferation were derived from our flow cytometry data; measured OCR values were 17% lower for radioresistant than for parental cells (P = 0.001, Fig. 6C). The model was initialised by seeding equal numbers of parental and radioresistant cells at day 0, based on the estimated values of VP(0) and VRR(0) from the logistic model. On day 10 of our in silico simulations, the radioresistant cells were located close to the spheroid periphery, while parental cells were located close to the centre (Fig. 6D). We then used the CA model to determine whether the growth curves of mixed spheroids could be explained by competition for oxygen. Although the CA model predicted the growth curves of the homogeneous spheroids, the model did not accurately capture the growth curves of the mixed spheroids under the assumption that the two cell populations compete for oxygen (Fig. 6E). Thus, although competition for oxygen may explain the spatial structure of cell populations and the decreased radiation sensitivity of mixed spheroids, OCR differences alone are not sufficient to explain their growth phenotype.

We subsequently looked for evidence of cell–cell communication between parental and radioresistant cells in hypoxia, given that hypoxia appeared to be an important factor for enhanced survival. Cells were co-cultured in hypoxia for 120 h using Transwell inserts; in this system, cells can share factors (e.g., exosomes, microRNA and proteins) but are not in direct contact with each other. The number of parental cells co-cultured with radioresistant cells was 48% higher than the number measured from homogeneous cultures in hypoxia but not in normoxia (Padj < 0.001; Fig. 6F). Altogether, our results indicate that both microenvironmental pressures (through oxygen constraints), and cell-cell communication (through the transfer of factor(s)) mediate ecological interactions between cell populations that can result in enhanced growth and reduced radiation sensitivity of mixed PC3 spheroids.

## Discussion

We evaluated whether ecological interactions between tumour cell populations with different radiation sensitivities alter tumour growth kinetics and radiation response. Using biological experiments and ecological mathematical models, we found that prostate cancer spheroids comprising mixed tumour cell populations have enhanced growth and reduced radiation sensitivity. Numerical simulations predicted that these phenotypic changes result from competitive and antagonistic interactions between the cell populations in spheroids, and result in variable regrowth times after radiation. By demonstrating that ecological-type interactions impact growth and radiation response, our study presents a new aspect that could be considered in radiobiological models to target ITH and reduce local recurrence in prostate cancer.

Ecological interactions between tumour cell populations, irrespective of the type of interaction, appear to have evolutionary advantages for cancer. We found that both competitive and antagonistic interactions increase bulk tumour growth and resistance to radiation. Our results are consistent with findings from previous studies investigating the impact of ecological interactions in chemotherapy: although the type of interaction (e.g., competitive vs. mutualism) can vary, ecological interactions appear to increase resistance to cancer therapy14,17,27,28,29. Furthermore, in our study, interactions increase the growth and post-radiation survival for the parental population in mixed spheroids in both cell lines. The interactions also did not result in the elimination of either population, recapitulating the multiclonality observed in human prostate cancer3,4,22,30,31. Current radiotherapy methods used to target more “resistant” populations, such as dose escalation, may therefore not be sufficient to target ITH if interactions between tumour cell populations reduce overall tumour sensitivity to radiation while maintaining ITH. Future studies could test whether accounting for these interactions in radiobiological models improves tumour response to radiation.

We also found that interactions mediated by oxygen constraints alter the spatial distribution of prostate cell populations in mixed PC3 spheroids. Both our biological and spatially explicit mathematical models showed radioresistant cells in the periphery and parental cells in hypoxic regions. These findings are consistent with previous reports showing that the metabolic demands of drug-resistant cancer cells affect their spatial localisation7. The observed spatial heterogeneity in our study is important because radioresistant cells located in the periphery could metastasise16, while parental cells in hypoxia would have reduced sensitivity to radiation. Since the efficacy of radiotherapy may also depend on the spatial distribution of tumour populations, these results suggest that spatially-explicit mathematical models may be needed to further optimise radiation treatments. For example, the delivery of non-uniform doses of radiation based on the spatial expression of prostate cancer antigen has been shown to improve the tumour control probability in prostate cancer32.

Since experimental approaches to studying ecological interactions can be complex, time-consuming, and/or have limited data, interdisciplinary approaches that treat tumours as evolving ecosystems may be necessary to target ITH10 and reduce local recurrence in prostate cancer. Mathematical simulations, with input data from experiments, can help evaluate the outcome of different treatment strategies and test biological hypotheses. The success of this approach was recently demonstrated in a pilot clinical trial for patients with drug-resistant prostate cancer: tumour population dynamics from each patient were used to inform a Lotka–Volterra model and optimise a chemotherapy regimen that significantly delayed recurrence than standard-of-care18. In our study, we used mathematical models to guide the experimental design of flow cytometry experiments, as well as to test whether oxygen constraints could predict enhanced growth of mixed spheroids. Conversely, we used experimental data to infer parameters with tighter confidence intervals in theoretical models, which refined the ability of our mathematical models to predict ecological interactions. Our approach can be applied to evaluate the impact of interactions during and following radiotherapy in other cancer types and systems.

Although relatively simple, our modelling approach has three advantages over previous approaches. First, unlike other ecological models used to study cellular interactions that only model competition between sensitive and resistant populations15,18,33,34, our Lotka–Volterra model describes a range of interactions, including competition, mutualism, and antagonism. Second, although the model does not account for different cell states (e.g., hypoxic, proliferative and necrotic) or for the nature of the cell-cell interactions to change over time, it accurately predicted the dominant interactions occurring within spheroids. Third, the small number of parameters in the mathematical model can be easily estimated from and validated by experimental data. One limitation is that our Lotka–Volterra model did not account for possible changes in the cellular interactions over time, which might occur due to changes in the tumour microenvironment. Since tumour populations can evolve over time, future studies are needed to investigate whether changing interactions (especially after treatment) could affect regrowth. While our spatially-resolved hybrid CA model was sufficient to study the impact of the microenvironment (i.e., oxygen constraints) on tumour development in a 2D cross-section of a tumour spheroid, the model in its current form also does not account for direct interactions between cell populations. To generate more realistic predictions, the model could be extended to three-dimensional space, along with rules governing cell behaviour that are modified to account for interactions between different cell populations. Since cells can interact directly and indirectly via secretion of molecules, our model could be further modified to explicitly track factors released by coexisting cell populations. These modifications, whilst adding complexity to the model, would allow us to test various hypotheses pertaining to the impact of ITH on tumour growth and dynamics. Finally, while neither model accounts for angiogenesis, which would occur in vivo, both models could be extended to account for the role of the vasculature in tumour response to radiotherapy, by, for example, exploiting existing models of vascular tumour growth35.

## Methods

### Biological experiments

#### Cell culture

We obtained unlabelled parental (sensitive) and radiation-resistant populations from two prostate cell lines, PC3 and DU145, from the Liu laboratory (University of Toronto, Canada). Radioresistant cell populations comprised pooled cells from the parental line that survived a clinically-relevant course of radiotherapy23,24. To produce stable, fluorescent cell lines, we transduced cells using lentiviral particles containing the vectors, pCDH1-CMV-GFP-EF1-Hygro or pCHD1-CMV-DsRed-EF1-Hygro (Systems Biosciences), collected the top 30% of brightest cells by flow cytometry, and used hygromycin B (50 mg/mL, Gibco) for selection (200 µg/mL for PC3 and 250 µg/mL for DU145). Cell lines were cultured in DMEM medium (low glucose, pyruvate, GlutaMAX, Gibco) supplemented with 25 mM HEPES (Gibco), 10% foetal bovine serum (Sigma or Pan-Biotech), and 1% penicillin/streptomycin. Authentication was performed using STR profiling (Promega PowerPlex 21 PCR kit, Eurofins), and mycoplasma checks were performed routinely using MycoAlert Mycoplasma Detection Kit (Lonza). All cells were maintained in an incubator (37 °C, 5% CO2). Unlabelled cells were cultured for up to 10 passages (~6 weeks) for transduction; labelled cells were cultured for up to 10 passages (~6 weeks). No additional courses of radiation were used to maintain resistance. We measured clonogenic survival at 2 Gy (SF2) to verify that the labelled cells maintained the resistance phenotype for up to 12 passages (~8 weeks) in culture.

#### Monolayer growth experiments

Single cells (1 × 103 cells/well, 200 μL medium) were seeded in triplicate in flat-bottom 96-well plates, allowed to attach overnight, irradiated (0, 2, or 6 Gy), and imaged daily using brightfield (Incucyte Live Cell Imaging System, Sartorius). The medium was changed every 2 days. Cell confluence was determined using Incucyte Base Software (Sartorius).

#### Clonogenic assays

Survival of cell lines after radiation was measured using a clonogenic assay36. Briefly, cells were seeded in triplicate in six-well plates and irradiated using a Cs-137 (dose rate of 0.89 Gy/min) or an X-ray irradiator (195 kV, 10 mA). Surviving colonies were stained after 10 days with crystal violet and counted. The surviving fraction was calculated as (number of colonies/number of seeded cells) × plating efficiency.

#### Response to cisplatin

To check whether the RR cells had altered DNA damage response, we measured the cell viability of each population in response to a range of concentrations of cisplatin using a modified cytotoxicity assay37. Briefly, single cells (2 × 103 cells in 100 μL/well) were seeded as monolayers in triplicate in a 96-well plate and allowed to attach for 36 h before treatment. Increasing concentrations of cisplatin (made in 100 μL/well) were added to each well resulting in a final volume of 200 μL/well; cisplatin (Sigma) was prepared fresh for each treatment by dissolving the powder in 0.9% sterile-filtered saline to a stock concentration of 3.3 mM. Treated cells were then cultured for 72 h in cisplatin before they were fixed with 10% formalin. Cell confluence was determined using Incucyte Base Software (Sartorius) and reformatted to a concentration–response curve by normalising cell confluence values to the untreated well.

#### Spheroid generation and culture

Homogeneous and mixed spheroids were generated in 96-well, ultra-low attachment plates (7007, Corning) by seeding different ratios of parental and radioresistant cell populations (2 × 103 total cells/well) using Matrigel (5% v/v, Corning) to promote spheroid formation38. For all spheroid experiments, after a formation phase of 3 days, spheroids were fed every 2 days by replacing 50% of the medium in each well with fresh medium (200 μL total/well). Culture medium and incubation conditions were as described under ‘Cell culture’. Spheroid volumes were calculated using SpheroidSizer39.

Spheroids were generated as described in ‘Spheroid generation and culture’ and monitored for growth by brightfield imaging (Leica DM IRBE, Hamamatsu).

#### Flow cytometry

The proportions, survival, and cell cycle of each population from spheroids were measured by flow cytometry. Mixed unirradiated or irradiated spheroids (seeded 1:1 parental:RR, 6–8 pooled/group) were incubated with EdU (10 µM final concentration) 12 h prior to dissociation, dissociated (100 μL Accumax, Millipore) for 20 min at 37 °C, washed with phosphate-buffered saline (PBS), centrifuged (300 × g, 5 min), and incubated with efluor-780 (1 μL/mL PBS; ThermoFisher Scientific) for 30 min on ice in the dark to distinguish live/dead cells. After washing in PBS, samples were fixed for 10 min in IC Fixation Buffer (ThermoFisher Scientific), and permeabilized and stained with Click-iT Plus EdU Alexa Fluor 647 (ThermoFisher Scientific) according to manufacturer’s instructions. Following a wash in 1× saponin, cells were incubated 30 min with FxCycle Violet Stain (1:1000, 300 µL of 1× saponin; ThermoFisher Scientific) before being run on the BD LSR Fortessa X-20 Cytometer or the Attune NxT Flow Cytometer using the 405, 488, 561, and 633 lasers. Data were analysed using FlowJo (Treestar, Inc.) as described in Supplementary Figs. 2 and 4.

#### Spheroid growth experiments after radiation

To determine bulk radiation response of spheroids, PC3 cells were seeded as spheroids (n = 15 per dose per group) with 4 groups as described in ‘Spheroid generation and culture’: parental, 9:1 parental:RR, 1:1 parental:RR, and RR. After formation, spheroids were irradiated (0, 2.5, 5, 7.5, 10, 15, and 20 Gy) on day 4 and imaged for up to 48 days to monitor regrowth using brightfield (Celigo Imaging Cytometer, Nexelcom). After log-transforming the volume data, we calculated the radiation-induced growth delay (days) relative to untreated spheroids as the time for each irradiated spheroid to reach a volume endpoint (2.5 times the starting volume right before irradiation); we selected the lowest endpoint that was still within the exponential growth phase of all spheroids in the experiment. The average time for untreated spheroids to reach endpoint was estimated in R by local regression using the loess function with “direct” surface estimation to allow extrapolation for the parental spheroids (R project, v. 3.6.2).

Regrowth experiments were repeated by irradiating day 3 spheroids from three groups (parental, mixed and RR) of PC3 cells (6 Gy, n = 17–18/ group) and of DU145 cells (6 Gy, n = 12/group; 10 Gy, n = 20/group). Spheroids were imaged using brightfield (Leica DM IRBE, Hamamatsu) for up to 27 days (PC3) and up to 23 days (DU145). The radiation-induced growth delay (delays) was calculated as above, but with different endpoints (3.5 times starting volume for PC3 and 4 times starting volume for DU145) to ensure the endpoint was within the exponential growth phase. Data from Fig. 2 were used to estimate average time of untreated spheroids; the ‘span’ parameter of the loess function was reduced from the default of 0.75–0.5 for the unirradiated DU145 spheroids to better estimate the average time of reaching the endpoint.

To measure changes in the radiation response of PC3 cell populations within spheroids, untreated homogeneous and mixed spheroids were grown until day 5 or 11, dissociated using Accumax, seeded as single cells for clonogenic experiments, and allowed to attach for 6 h prior to radiation. Fluorescent colonies were counted using the Celigo Cytometer.

#### Immunofluorescence

For immunofluorescence and H&E experiments, spheroids were treated and fixed prior to staining40. To investigate the spatial distribution of fluorescent populations, sections were hydrated in PBS, stained for 10 min with Hoechst (1 μg/mL in PBS, Sigma) to visualise nuclei, and mounted using ProLong Diamond Antifade Mountant (ThermoFisher). For hypoxia, spheroids were pre-treated with 300 μM of the hypoxia drug EF5 (gift from Dr. Cameron Koch, University of Pennsylvania) prior to fixation. They were then permeabilized (PBS containing 0.3% Tween-20, 10 min), blocked (5% goat serum in PBS containing 0.1% Tween-20, 30 min), stained using anti-EF5 antibody (75 μg/mL; from Dr. Cameron Koch) overnight at 4 °C, washed (ice-cold PBS containing 0.3% Tween-20, 2 × 45 min)40, stained for nuclei as above, and mounted. For Ki67, spheroid sections were permeabilized (PBS containing 0.3% Tween-20, 10 min), blocked (5% goat serum in PBS containing 0.1% Tween-20, 30 min), and incubated overnight at 4 °C with primary antibody (clone SP6, 1:100, Vector Laboratories). After washing in PBS, sections were incubated for 1 h with goat anti-rabbit Alexa Fluor 647 (4 μg/mL, ThermoFisher), washed, and stained with Hoechst 33342 (5 μg/mL, Sigma) for 10 min. Slides were mounted and imaged using epifluorescence microscopy (20× objective; 0.30 NA; 0.64 μm resolution; excitation lasers: 395, 470, 555, and 640; Nikon Ti-E). Sections were stained using H&E and imaged using a Bright Field Slide Scanner (Aperio CS2, Leica) to visualise necrosis. To quantify the ratio of parental to radioresistant populations in spheroid cross-sections (n = 16 spheroids from 4 batches), we measured the number of pixels from each population (i.e., signal) by applying a threshold value 5 times higher than the median value of the background (i.e., noise) (Octave 4.4.1).

#### Oxygen consumption measurements

OCR was measured from each population using the Seahorse assay. Cells (1.2 × 104/well) were seeded in triplicate using the normal culture medium in a Seahorse XF 96-well microplate (Agilent) and allowed to attach overnight. Prior to the assay, cells were washed with and incubated in assay medium (DMEM basal medium containing 5 mM glucose, 4 mM glutamine, 5 mM pyruvate, pH 7.4; 200 μL/well) for 2 h at 37 °C without CO2 to degas the medium. Calibrant buffer (200 μL/well) was added to wells of the probe plate and also left at 37 °C without CO2 to degas. After OCR was measured on the Seahorse XF Analyser (Agilent Biosciences), cells were fixed using 4% paraformaldehyde, stained using Hoechst 33342, and counted (Celigo Cytometer, Nexelcom).

#### Transwell experiments

Co-culture experiments were performed to measure whether transferred factors between cell populations enhanced survival under hypoxia. Cells were seeded in triplicate (3.0 × 104/bottom well and 1.0 × 104/insert) in 12-well plates and in Transwell inserts, and allowed to attach overnight. Once the medium was changed, the plates were placed into normoxia or hypoxia (0.1% O2) for 24 and 120 h. Cells were fixed using 4% paraformaldehyde, stained with Hoechst (5 μg/mL), and counted (Celigo Cytometer, Nexelcom).

#### Statistics and reproducibility

Data were evaluated for equal variance using homoscedasticity plots (absolute value of residual vs predicted value) and for normality using QQ plots (Prism 8.0, GraphPad). Unless otherwise indicated, statistical significance was evaluated using one-way ANOVA, two-way ANOVA, or a mixed-effects model followed by multiple testing correction (α = 0.05). For clonogenic assays, the radiation protection factor was calculated as the area under the dose–response curve (AUC) for the RR cell populations divided by that of the parentals; AUC values were analysed for significance using a Student’s t test (unpaired, one-tailed, α = 0.05). For cisplatin cytotoxicity assays, IC50 values were calculated using a normalised response, variable slope, dose–response model (Prism, 8.0, GraphPad) and evaluated for statistical significance using extra sum-of-squares F-test. For post-radiation growth experiments, survival curves were analysed using the Mantel–Cox (log-rank) test and adjusted for multiple testing using Holm’s correction; spheroids that did not reach the endpoint during the timeframe of the experiment were marked as ‘censored’ on the final day of the experiment (please see Supplementary Methods section 3 for further details). Due to heteroscedasticity, cell counts from flow cytometry experiments involving cell cycle and death, and from co-culture Transwell assays were analysed using overdispersed Poisson or binomial regression models (please see Supplementary Methods section 3 for further details). For quantification of population proportions in microscopy images, pixel numbers were analysed using a two-tailed, Wilcoxon matched-pairs signed-rank test. Adjusted P values (Padj) are reported in the main text for experiments where multiple comparisons were performed.

Data points represent biological replicates; experiments were performed using at least two separate batches of cells. We note the following data exclusions: missing data from some time points due to technical failures in imaging (Fig. 1), one excluded mixed DU145 spheroid because its growth did not resemble that of the other 35 spheroids (Fig. 2a), and one excluded plate of PC3 spheroids from survival analysis (Fig. 4) because of irregular growth that did not match the other nine plates. Sample sizes were approximated using effect sizes from pilot studies to ensure power (approximate β = 0.8); randomisation and blinding were not possible.

### Mathematical experiments

#### Non-spatial mathematical models

We used the logistic growth model to describe the growth of homogeneous tumour spheroids26. Thus, the rate of change of spheroid volume V at time t is given by

$$\frac{{dV}}{{dt}} = rV \left(1 - \frac{V}{K}\right),$$
(1)

where r > 0 represents the growth rate, K > 0 is the carrying capacity (the limiting volume of the spheroid) and V(t = 0) = V0 denotes the spheroid volume at t = 0. The analytical solution to the logistic model is given by

$$V\left( t \right) = \frac{{V_0Ke^{rt}}}{{K + V_0(e^{rt} - 1)}}.$$
(2)

The Lotka–Volterra model was used to describe the growth of mixtures of parental and RR cell populations

$$\left. {\begin{array}{*{20}{c}} {\frac{{dV_P}}{{dt}} = r_PK_P\left( {1 - \frac{{V_P}}{{K_P}} - \lambda _{RR}\frac{{V_{RR}}}{{K_P}}} \right)} \\ {\frac{{dV_{RR}}}{{dt}} = r_{RR}K_{RR}\left( {1 - \frac{{V_{RR}}}{{K_{RR}}} - \lambda _P\frac{{V_P}}{{K_{RR}}}} \right)} \end{array}} \right\},$$
(3)

with VP(t = 0) = VP0 and VRR(t = 0) = VRR0. In these equations, VP and VRR represent respectively the volumes of parental and RR populations, rP and rRR their initial growth rates, KP and KRR their carrying capacities, and VP0 and VRR0 their initial volumes. The parameters λP and λRR describe the effect that parental cells have on RR cells, and vice versa. These type of interactions, found in ecology12, may be competitive (λP > 0 and λRR > 0), mutualistic (λP < 0 and λRR < 0) or antagonistic (λP < 0 < λRR or λRR < 0 < λP)41. Further details, describing how the values of the model parameters were estimated, are included in Supplementary Methods section 1.

#### Spatially resolved computational models

To investigate the growth of heterogeneous spheroids we developed a spatially resolved CA model. Our CA model replicated the changes in the size and structure of a 2D cross-section through a 3D tumour spheroid suspended in the culture medium. The model couples a set of automaton elements arranged on a regular 2D grid to a partial differential equation describing the distribution of a growth-rate-limiting nutrient (here, oxygen) which is supplied from the culture medium surrounding the spheroid. Model simulations are initialised by placing a circular cluster of cells in the centre of the grid: this imitates seeding a spheroid in a Petri dish. Both cell populations consume oxygen that diffuses from the medium, and they proliferate and die at rates that depend on the local oxygen concentration and cell density. The parental and RR cells differ in cell cycle times, OCRs, sensitivity to changing oxygen concentration and rates of lysis. Further details about the implementation of the CA model are included in Supplementary Methods section 2.

#### Modelling regrowth times after radiation

Cell kill due to radiotherapy was incorporated into the models using the linear quadratic model42, with α and β values derived for each cell population (PC3 parental and RR) from clonogenic survival curves. For the spatially resolved CA model, oxygen dependence of α and β was included; for the spatially averaged logistic and Lotka–Volterra models, oxygen dependence of α and β was neglected since oxygen levels are not included in these models.

### Reporting summary

Further information on research design is available in the Nature Research Reporting Summary linked to this article.

## Data availability

The datasets that support the findings of this study are available in Zenodo with DOI: 10.5281/zenodo.413069243. Any other relevant data are available upon reasonable request from the corresponding authors.

## Code availability

Computer code used to generate the surface plots in Fig. 5 (interactions_time_to_regrow), the cellular automaton model results in Fig. 6 (ca model), and the statistical analyses of cell cycle, post-radiation regrowth, and transwell data are available in Zenodo with DOI 10.5281/zenodo.413069243. The source code implementing the cellular automaton model (ca_model) includes a demo on how to use it and images representing examples of expected outcomes. The computer code for mathematical modelling was written in MATLAB R2018b (v. 9.5) with Statistics and Machine Learning Toolbox (v.11.4), while the code for statistical analyses was written in R project (v. 3.6.2).

## References

1. 1.

Lalonde, E. et al. Tumour genomic and microenvironmental heterogeneity for integrated prediction of 5-year biochemical recurrence of prostate cancer: a retrospective cohort study. Lancet Oncol. 15, 1521–1532 (2014).

2. 2.

Mcgranahan, N. & Swanton, C. Clonal heterogeneity and tumor evolution: past, present, and the future. Cell 168, 613–628 (2017).

3. 3.

Hong, M. K. H. H. et al. Tracking the origins and drivers of subclonal metastatic expansion in prostate cancer. Nat. Commun. 6, 1–12 (2015).

4. 4.

Boutros, P. C. et al. Spatial genomic heterogeneity within localized, multifocal prostate cancer. Nat. Genet. 47, 736–745 (2015).

5. 5.

Wei, L. et al. Intratumoral and intertumoral genomic heterogeneity of multifocal localized prostate cancer impacts molecular classifications and genomic prognosticators. Eur. Urol. 1, 183–192 (2017).

6. 6.

Chang, L. et al. Identification of protein biomarkers and signaling pathways associated with prostate cancer radioresistance using label-free LC-MS/MS proteomic approach. Sci. Rep. 7, 41834 (2017).

7. 7.

Ibrahim-Hashim, A. et al. Defining cancer subpopulations by adaptive strategies rather than molecular properties provides novel insights into intratumoral evolution. Cancer Res 77, 2242–2254 (2017).

8. 8.

Pellacani, D. et al. Phenotype-independent DNA methylation changes in prostate cancer. Br. J. Cancer 119, 1133–1143 (2018).

9. 9.

Kelly, R. S. et al. The role of tumor metabolism as a driver of prostate cancer progression and lethal disease: results from a nested case-control study. Cancer Metab. 4, 22 (2016).

10. 10.

Staňková, K., Brown, J. S., Dalton, W. S. & Gatenby, R. A. Optimizing Cancer Treatment Using Game Theory: A Review. JAMA Oncol. 5, 96–103 (2019).

11. 11.

Tabassum, D. P. & Polyak, K. Tumorigenesis: it takes a village. Nat. Rev. Cancer 15, 473–483 (2015).

12. 12.

Zélé, F., Magalhães, S., Kéfi, S. & Duncan, A. B. Ecology and evolution of facilitation among symbionts. Nat. Commun. 9, 4869 (2018).

13. 13.

Bacevic, K. et al. Spatial competition constrains resistance to targeted cancer therapy. Nat. Commun. 8, 1995 (2017).

14. 14.

Cleary, A. S., Leonard, T. L., Gestl, S. A. & Gunther, E. J. Tumour cell heterogeneity maintained by cooperating subclones in Wnt-driven mammary cancers. Nature 508, 113–118 (2014).

15. 15.

Gallaher, J. A., Enriquez-Navas, P. M., Luddy, K. A., Gatenby, R. A. & Anderson, A. R. A. Spatial heterogeneity and evolutionary dynamics modulate time to recurrence in continuous and adaptive cancer therapies. Cancer Res. 78, 2127–2139 (2018).

16. 16.

Lloyd, M. C. et al. Darwinian dynamics of intratumoral heterogeneity: not solely random mutations but also variable environmental selection forces. Cancer Res 76, 3136–3144 (2016).

17. 17.

Marusyk, A. et al. Non-cell-autonomous driving of tumour growth supports sub-clonal heterogeneity. Nature 514, 54–58 (2014).

18. 18.

Zhang, J., Cunningham, J. J., Brown, J. S. & Gatenby, R. A. Integrating evolutionary dynamics into treatment of metastatic castrate-resistant prostate cancer. Nat. Commun. 8, 1–9 (2017).

19. 19.

Gong, J., Dos Santos, M. M., Finlay, C. & Hillen, T. Are more complicated tumour control probability models better? Math. Med. Biol. 30, 1–19 (2013).

20. 20.

Alfonso, J. C. L., Jagiella, N., Núñez, L., Herrero, M. A. & Drasdo, D. Estimating dose painting effects in radiotherapy: a mathematical model. PLoS ONE 9, e89380 (2014).

21. 21.

Horas, J. A., Olguín, O. R. & Rizzotto, M. G. Examining the validity of Poissonian models against the birth and death TCP model for various radiotherapy fractionation schemes. Int. J. Radiat. Biol. 86, 711–717 (2010).

22. 22.

Carreira, S. et al. Tumor clone dynamics in lethal prostate cancer. Sci. Transl. Med. 6, 1–10 (2014).

23. 23.

Huang, X. et al. miRNA-95 mediates radioresistance in tumors by targeting the sphingolipid phosphatase SGPP1. Cancer Res. 73, 6972–6986 (2013).

24. 24.

Ghiam, A. F. et al. Long non-coding RNA urothelial carcinoma associated 1 (UCA1) mediates radiation response in prostate cancer. Oncotarget 8, 4668–4689 (2017).

25. 25.

Kunz-Schughart, L. A., Kreutz, M. & Knuechel, R. Multicellular spheroids: a three-dimensional in vitro culture system to study tumour biology. Int. J. Exp. Pathol. 79, 1–23 (1998).

26. 26.

Gerlee, P. The model muddle: in search of tumor growth laws. Cancer Res. 73, 2407–2411 (2013).

27. 27.

Chapman, A. et al. Heterogeneous tumor subpopulations cooperate to drive invasion. Cell Rep. 8, 688–695 (2014).

28. 28.

Wagenblast, E. et al. A model of breast cancer heterogeneity reveals vascular mimicry as a driver of metastasis. Nature 520, 358–362 (2015).

29. 29.

Craig, M. et al. Cooperative adaptation to therapy (CAT) confers resistance in heterogeneous non-small cell lung cancer. PLoS Comput. Biol. 15, 1–19 (2019).

30. 30.

Waghmare, I., Roebke, A., Minata, M., Kango-Singh, M. & Nakano, I. Intercellular cooperation and competition in brain cancers: lessons from drosophila and human studies. Stem Cells Transl. Med. 3, 1262–1268 (2014).

31. 31.

Berglund, E. et al. Spatial maps of prostate cancer transcriptomes reveal an unexplored landscape of heterogeneity. Nat. Commun. 9, 2419–2432 (2018).

32. 32.

Zamboglou, C. et al. Evaluation of intensity modulated radiation therapy dose painting for localized prostate cancer using 68Ga-HBED-CC PSMA-PET/CT: a planning study based on histopathology reference. Radiother. Oncol. 123, 472–477 (2017).

33. 33.

Kaznatcheev, A., Peacock, J., Basanta, D., Marusyk, A. & Scott, J. G. Fibroblasts and Alectinib switch the evolutionary games played by non-small cell lung cancer. Nat. Ecol. Evol. 3, 450–456 (2019).

34. 34.

Harris, L. A., Beik, S., Ozawa, P. M. M., Jimenez, L. & Weaver, A. M. Modeling heterogeneous tumor growth dynamics and cell–cell interactions at single-cell and cell-population resolution. Curr. Opin. Syst. Biol. 17, 24–34 (2019).

35. 35.

Grogan, J. A. et al. Predicting the influence of microvascular structure on tumor response to radiotherapy. IEEE Trans. Biomed. Eng. 64, 504–511 (2017).

36. 36.

Franken, N. A. P., Rodermond, H. M., Stap, J., Haveman, J. & van Bree, C. Clonogenic assay of cells in vitro. Nat. Protoc. 1, 2315–2319 (2006).

37. 37.

Brimacombe, K. et al. A dual-fluorescence high-throughput cell line system for probing multidrug resistance. Assay Drug Dev. Tech. 7, 233–249 (2009).

38. 38.

Ivascu, A. & Kubbies, M. Rapid generation of single-tumor spheroids for high-throughput cell function and toxicity analysis. J. Biomol. Screen. 11, 922–932 (2006).

39. 39.

Chen, W. et al. High-throughput image analysis of tumor spheroids: a user-friendly software application to measure the size of spheroids automatically and accurately. J. Vis. Exp. 89, 51639 (2014).

40. 40.

Grimes, D. R., Kelly, C., Bloch, K. & Partridge, M. A method for estimating the oxygen consumption rate in multicellular tumour spheroids. J. R. Soc. Interface 11, 20131124 (2014).

41. 41.

Murray, J. D. Interdisciplinary Applied Mathematics. Mathematical Biology: I: An Introduction. Third Edition. (2004).

42. 42.

Chadwick, K. & Leenhouts, H. A molecular theory of cell survival. Phys. Med. Biol. 18, 78–87 (1973).

43. 43.

Paczkowski, M. et al. Data and code for paper ‘Reciprocal interactions between tumour cell populations enhance growth and reduce radiation sensitivity in prostate cancer.’ Zenodo. https://doi.org/10.5281/zenodo.4130692 (2020).

## Acknowledgements

We thank Dr. David Robert Grimes for helpful discussions, Dr. Yunhong Cao for technical assistance, and Dr. Graham Brown and Dr. Rhodri Wilson for assistance with microscopy (Microscopy Core Facility, Department of Oncology, University of Oxford). Access to the microscopes (Leica, Ti-E, and Celigo) and flow cytometer (Attune) were provided by the Microscopy and Flow Cytometry Core Facilities (Department of Oncology, University of Oxford). This research was supported by Cancer Research UK and the Engineering Physical Sciences Research Council (grant numbers: C5255/A12678, C2522/A10339, C53469/A19834, and C56606/A21440). BM, M Partridge, and PK were funded by the CRUK/EPSRC Oxford Cancer Imaging Centre. ALH was funded by Breast Cancer Now (grant number: 2015MayPR479). WWK was funded by KTH Royal Institute of Technology. M Paczkowski gratefully acknowledges the EPSRC for funding through a studentship at the Systems Biology programme of the University of Oxford’s Doctoral Training Centre.

## Funding

Open Access funding provided by Karolinska Institute.

## Author information

Authors

### Contributions

Conception: A.L.H., M. Partridge, H.M.B., and P.K. Design: M. Paczkowski, L.K.S., A.L.H., M. Partridge, H.M.B., and P.K. Generation of cell lines: S.K.L. Experiments: M. Paczkowski, B.M., and P.K. Mathematical modelling: M. Paczkowski. Data analysis: M. Paczkowski, W.W.K., and P.K. Project supervision: A.L.H., M. Partridge, H.M.B., and P.K. Paper writing: M. Paczkowski, W.W.K., A.L.H., H.M.B., and P.K. All authors edited/reviewed the paper.

### Corresponding authors

Correspondence to Helen M. Byrne or Pavitra Kannan.

## Ethics declarations

### Competing interests

The authors declare no competing interests.

Publisher’s note Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

## Rights and permissions

Reprints and Permissions

Paczkowski, M., Kretzschmar, W.W., Markelc, B. et al. Reciprocal interactions between tumour cell populations enhance growth and reduce radiation sensitivity in prostate cancer. Commun Biol 4, 6 (2021). https://doi.org/10.1038/s42003-020-01529-5

• Accepted:

• Published: