Article | Open | Published:

PKC activation sensitizes basal-like breast cancer cell lines to Smac mimetics

Cell Death Discovery volume 2, Article number: 16002 (2016) | Download Citation

Subjects

Abstract

There is a need for novel strategies to initiate cancer cell death. One approach is the use of Smac mimetics, which antagonize inhibitor of apoptosis proteins (IAPs). Recent studies have shown that combinations of Smac mimetics such as LBW242 or LCL161 in combination with chemotherapeutic agents increase cancer cell death. Here we show that the protein kinase C (PKC) activator TPA together with the Smac mimetic LBW242 induces cell death in two basal breast cancer cell lines (MDA-MB-468 and BT-549) that are resistant to Smac mimetic as single agent. Ten other LBW242-insensitive cancer cell lines were not influenced by the TPA+LBW242 combination. The TPA+LBW242 effect was suppressed by the PKC inhibitor GF109203X, indicating dependence on PKC enzymatic activity. The PKC effect was mediated via increased synthesis and release of TNFα, which can induce death in the presence of Smac mimetics. The cell death, coinciding with caspase-3 cleavage, was suppressed by caspase inhibition and preceded by the association of RIP1 with caspase-8, as seen in complex II formation. Smac mimetics, but not TPA, induced the non-canonical NF-κB pathway in both MDA-MB-231 and MDA-MB-468 cells. Blocking the canonical NF-κB pathway suppressed TPA induction of TNFα in MDA-MB-468 cells whereas isolated downregulation of either the canonical or non-canonical pathways did not abolish the Smac mimetic induction of the NF-κB driven genes TNFα and BIRC3 in MDA-MB-231 cells although the absolute levels were suppressed. A combined downregulation of the canonical and non-canonical pathways further suppressed TNFα levels and inhibited Smac mimetic-mediated cell death. Our data suggest that in certain basal breast cancer cell lines co-treatment of TPA with a Smac mimetic induces cell death highlighting the potential of using these pathways as molecular targets for basal-like breast cancers.

Introduction

Evasion of cell death is one important hallmark of cancer.1,2 Cell death comprises different subroutines3,4 with two main apoptotic pathways, the extrinsic and the intrinsic, as important examples.5 The extrinsic pathway is induced by death receptors (DRs) leading to the activation of caspase-8 whereas the intrinsic apoptotic pathway is initiated by cellular stress resulting in release of cytochrome c and second mitochondria-derived activator of caspase (Smac) from the mitochondria leading to activation of caspase-9. Both pathways converge in the activation of executioner caspases-3 and 7.6,7

One way to facilitate apoptosis induction and thereby circumvent the evasion of cell death by cancer cells is to mimic the function of Smac. Several small molecules mimicking Smac have been developed and some are under investigation in clinical trials.8

A Smac mimetic (SM) is thought to facilitate cell death by mimicking the antagonizing effect of Smac on inhibitor of apoptosis proteins (IAPs).8 Two IAPs, cellular IAP1 (cIAP1) and cIAP2, regulate tumor necrosis factor receptor 1 (TNFR1) signaling.9 TNFR1 activation can lead to extrinsic apoptotic signaling pathway. However, TNFR1 also induces NF-κB signaling associated with survival and inflammation.10 The effect of the TNFR1 signal is determined by proteins that are available and recruited to the receptor.11,12 After ligand binding, cIAP1 and cIAP2 are recruited to the receptor and induce ubiquitination of other proteins, including receptor-interacting protein-1 (RIP1). Ubiquitination of RIP1 is necessary for the formation of complex I, an essential step for induction of NF-κB signaling. SMs induce downregulation of cIAP1/2 thereby preventing complex I formation and NF-κB signaling. Instead complex II, is formed. Complex II contains FADD, RIP1 and caspase-8 and leads to the activation of the latter and apoptosis.11,12 In addition, SM can facilitate cell death by blocking XIAP, a well-established caspase inhibitor.13

Some cancer cell lines die by SM treatment as single agent.14,​15,​16,​17 The effect has been reported to be due to autocrine TNFα production, which induces cell death in the presence of SM.16,17 The TNFα production can be mediated by accumulation of NF-κB-inducing kinase (NIK), followed by non-canonical NF-κB signaling and TNFα transcription, which occur when cIAPs no longer ubiquitinate and target NIK for degradation.17,​18,​19 However, it is not completely clear what determines if a cell responds to a SM with TNFα production. It also raises the possibility that local induction of TNFα may be a way to make cancer cells susceptible to SM.

We previously found that the pro-apoptotic protein Smac and the protein kinase C (PKC) isoform PKCδ form a complex that is dissociated during cell death induction.20 Here we continue the investigation of Smac and PKC. We found that activation of PKC with subsequent synthesis and release of TNFα can overcome SM insensitivity in breast cancer cell lines of basal phenotype. The effect of TPA is dependent on the canonical NF-κB pathway, whereas both the canonical and non-canonical pathways contribute to sensitivity to SM.

Results

PKC activation by TPA overcomes resistance to SM in MDA-MB-468 and BT-549 cells

To investigate a putative modulating effect of PKC on SM-induced death, we initially co-treated SM-sensitive MDA-MB-23116,21 and MDA-MB-468 cells, which are insensitive to SM treatment (Figure 1a), with the PKC activator TPA and the SM LBW242. TPA made MDA-MB-468 cells sensitive to SM and increased the effect of SM on MDA-MB-231 cells (Figure 1a). Only one more of nine additional cell lines tested, the BT-549 breast cancer cell line, showed a similar response (Figure 1a), except for LNCaP cells, which die upon treatment of TPA alone.22 The effects on MDA-MB-231 and MDA-MB-468 cells were the same with LCL161, which is another SM (Supplementary Figure 1). Because of limited availability of LBW242, LCL161 was used in some subsequent experiments.

Figure 1
Figure 1

TPA overcomes resistance to LBW242 in MDA-MB-468 and BT-549 cells. (a) Different cancer cell lines were treated with 16 nM TPA, 20 μM LBW242 or the combination of both (TL) for 30 h. (b) MDA-MB-468 cells were treated for 30 h with indicated combinations of 16 nM TPA (T), 20 μM LBW242 (L), 100 μM etoposide (E), 100 nM paclitaxel (P) or with corresponding volume of DMSO (Ctl) as control. (c) MDA-MB-468 cells were treated for 30 h with indicated combinations of 16 nM TPA, 20 μM LBW242, 2 μM GF109203X or with corresponding volume of DMSO as control. (d) MDA-MB-468 or (e) BT-549 cells were treated with 16 nM TPA and/or 20 μM LBW242 for 16 h. (f) MDA-MB-468 cells were treated with 16 nM TPA and 20 μM LBW242 for 8, 16, 24 and 32 h. Cell viability data in (ac) were obtained with a WST-1 assay and are expressed as percentage of control cells. For (df) cell death was determined by flow cytometry using APC-conjugated annexin V. Data are mean±S.E.M. of n≥3 independent experiments; **P<0.01 and ***P<0.001.

We next evaluated the effects of TPA and SM together with the chemotherapeutic agents, etoposide and paclitaxel (Figure 1b). Both paclitaxel and etoposide alone suppressed cell viability and the effect was further enhanced by the addition of TPA and SM, indicating that the death-inducing effect can be even more pronounced with multiple agents. The TPA effect was blocked by the pan-PKC inhibitor GF109203X (Figure 1c), indicating that PKC activity is necessary for the effect.

Both MDA-MB-468 and BT-549 cells showed a significantly increased cell death, measured by an annexin V assay, upon treatment with the combination of TPA and SM (Figures 1d and e). Thus, the decreased cell viability is increased conceivably due to cell death.

After 16 h of treatment, an increase in cell death can be observed (Figure 1f), which is further increased with time.

TPA in combination with SM induces complex II formation and caspase activation

Next, we sought to investigate mechanisms mediating cell death by TPA+SM.16,17 Caspase inhibition resulted in a marked reduction in cell death induced by TPA+SM and by etoposide but had limited effects on paclitaxel-mediated death (Figure 2a). It has been proposed that SM enables the formation of complex II upon TNFα stimulation with subsequent activation of caspase-8.16,17 To evaluate the formation of complex II, we used an approach previously described11,21 where caspase-8, one of the constituents of complex II, is immunoprecipitated. When treating cells with TPA alone caspase-8 did not co-immunoprecipitate with RIP1. However, SM treatment led to co-immunoprecipitation of RIP1 and caspase-8, which was further strengthened by simultaneous incubation with TPA (Figure 2b). Neither etoposide nor paclitaxel induced a caspase-8-RIP1 complex (Figure 2c).

Figure 2
Figure 2

Combined treatment with TPA and LBW242 leads to caspase activation and complex II formation. (a) MDA-MB-468 cells were treated with indicated combinations of 16 nM TPA (T), 20 μM LBW242 (L), 100 μM etoposide (E), 100 nM paclitaxel (P) with or without 20 μM of the pan-caspase inhibitor zVADfmk (V) or with corresponding volume of DMSO (Ctl) as control for 30 h. Cell viability was determined with a WST-1 assay and is expressed as percentage values obtained for DMSO-treated control cells. (b) MDA-MB-468 cells were treated with 16 nM TPA and/or 20 μM LBW242 or (c) 16 nM TPA+20 μM LBW242 (TL), 100 μM etoposide or 100 nM paclitaxel for 16 h. All treatments were done in the presence of 20 μM zVADfmk. Cell lysates were immunoprecipitated with caspase-8 antibodies and analyzed with western blot, using antibodies against RIP1 and caspase-8. (d) MDA-MB-468 cells treated with indicated combinations of 16 nM TPA, 20 μM LBW242, 100 μM etoposide, 100 nM paclitaxel or a DMSO control for 16 h were analyzed with western blot using antibodies against caspase-3. Data in (a) represent the mean±S.E.M. of n≥3 independent experiments; **P<0.01 and ***P<0.001. Blots in (bd) are representatives of three independent experiments.

The combination of TPA and SM also resulted in the cleavage of caspase-3 whereas neither TPA nor SM alone had this ability (Figure 2d). TPA together with paclitaxel or etoposide only showed a weak capacity to affect caspase-3.

TPA and SM-mediated cell death is TNFα dependent

Autocrine TNFα production has been reported to be important for SM-mediated cell death.16,17 We therefore examined if the cell death induced by TPA+SM is TNFα dependent as well. A TNFα-blocking antibody suppressed cell death induction both for MDA-MB-468 cells treated with TPA and SM as well as for MDA-MB-231 cells treated with SM alone (Figures 3a and b).

Figure 3
Figure 3

The cell death mediated by TPA and LBW242 is dependent on TNFα. Anti-human TNFα antibodies (2 μg/ml) were added to both (a) MDA-MB-468 cells treated with 16 nM TPA and 20 μM LBW242 as well as (b) MDA-MB-231 cells treated with 20 μM LBW242 alone, for 16 h. (c) MDA-MB-468 and (d) MDA-MB-231 cells were treated for 16 h with 20 μM LBW242 and/or 2 ng/ml TNFα. Cell death was determined by flow cytometry using APC-conjugated annexin V. All data are mean±S.E.M. of n≥3 independent experiments; *P<0.05, **P<0.01 and ***P<0.001.

We next investigated whether TNFα is sufficient to induce cell death in combination with SM in MDA-MB-468 cells. TNFα alone had no effect but together with LBW242 a pronounced induction of cell death was seen (Figure 3c). For the SM-sensitive MDA-MB-231 cells no potentiating effect of TNFα could be seen (Figure 3d).

TPA treatment leads to increased levels of TNFα

To investigate if TPA treatment results in TNFα production, we investigated TNFα levels in cell culture medium. TPA induced higher TNFα protein concentrations in the cell culture medium of MDA-MB-468 cells whereas SM had no effect, neither in the absence nor presence of TPA (Figure 4a). GF109203X abolished the effect of TPA. Contrasting MDA-MB-468 cells, SM alone resulted in increased TNFα levels in MDA-MB-231 cells (Figure 4a).

Figure 4
Figure 4

TNFα levels increase upon TPA treatment. (a) MDA-MB-468 and MDA-MB-231 cells were treated for 16 h with indicated combinations of 16 nM TPA (T), 20 μM LBW242 (L) or 2 μM GF109203X (GX). The concentration of TNFα protein in the cell culture medium was determined by ELISA. (b) TNFα mRNA levels were determined with qPCR and relative mRNA levels were normalized to control (Ctl) MDA-MB-468 cells. (c and d) MDA-MB-468 cells were treated with 16 nM TPA for 0, 1, 2, 4, 8 and 16 h, thereafter levels of TNFα protein in the cell culture medium (c) or mRNA of total cell lysate (d) was determined. (e) T47D, MCF-7 and MDA-MB-468 cells were seeded on a 96-well plate and treated with 2 ng/ml TNFα, 10 μM LCL161 or the combination of both for 30 h. Cell viability data are obtained with a WST-1 assay and are expressed as percentage of control cells. (f) MCF-7 cells were treated with 16 nM TPA for 0, 4 or 8 h. Thereafter TNFa mRNA levels were determined with qPCR. All data are mean±S.E.M. of n≥3 independent experiments; *P<0.05 and ***P<0.001.

In MDA-MB-468 cells, TPA treatment also caused a modest (69%) increase in TNFα mRNA levels and SM had no effect. On the other hand, TNFα mRNA levels in SM-treated MDA-MB-231 cells were markedly increased compared to basal levels (Figure 4b).

TPA led to a more than 19-fold elevation of TNFα protein concentration but only a 69% increase in TNFα mRNA levels in MDA-MB-468 cells. To analyze this discrepancy, the effect of TPA stimulation at different time points was investigated. TNFα protein levels gradually increased during the first 16 h (Figure 4c), whereas TNFα mRNA levels preceded protein changes and peaked after 4 h of TPA treatment (Figure 4d). This conceivably explains the discrepancies between protein and mRNA levels after 16 h of treatment.

According to cell viability results (Figure 1a), most cell lines do not respond with decreased cell viability when combining a SM with TPA. This could potentially be due to an inability of TPA to induce TNFα or a resistance to the combination of SM with TNFα. To assess these hypotheses, T47D and MCF-7 cells were treated with SM and TNFα followed by analysis of cell viability (Figure 4e). T47D cells were insensitive to this combination conceivably explaining the insensitivity to TPA and SM treatment. Although some cell death was induced in MCF-7 cells by the treatment the effect was not as pronounced as for MDA-MB-468 cells. Furthermore, TPA treatment of MCF-7 cells resulted in TNFα mRNA production but with lower and faster declining levels than in MDA-MB-468 cells (Figures 4d and f). Thus, the insensitivity of MCF-7 cells to the combination of TPA and SM may depend on both a lower sensitivity to the combination of TNFα and SM, and a smaller induction of TNFα by TPA.

Global gene expression of SM-treated sensitive and insensitive cell lines

To further delineate mechanisms mediating TNFα expression, we turned to comparison of the response to SM and analyzed changes in global gene expression following SM treatment of MDA-MB-231 and MDA-MB-468 cells. The most apparent finding was that SM-treated MDA-MB-231 cells had a substantially higher number of differentially expressed genes (108 genes) compared to the SM-treated MDA-MB-468 cells (nine genes), of which four were affected in both cell lines, when applying a q-value of 0% as a cutoff. All differentially expressed genes, except for OSR1 in MDA-MB-231 cells were expressed at higher levels following SM treatment (Figures 5a and b). Many of the SM-induced genes are NF-κB target genes, such as BIRC3, CXCL1, IRAK2, TNF and TRAF123 (Supplementary Table 1), which is further supported by a pathway analysis (Supplementary Figure 3).

Figure 5
Figure 5

(a) Heatmap visualizing SM-induced fold change in gene expression of genes with a q-value of 0% for MDA-MB-231 (left) and MDA-MB-468 (right). The color code indicates log2 fold change induced by LBW242. (b) Venn diagram displaying the significantly differentially expressed genes, retrieved from the microarray and the subsequent SAM analysis. The genes indicated in the diagram represents all (MDA-MB-468) genes or the top ten (MDA-MB-231) with the highest log2 fold change, in descending order, induced by LBW242 and with a q-value=0% (Supplementary Table 1), in MDA-MB-231 (gene symbol in blue), MDA-MB-468 (gene symbol in red) or in both cell lines (gene symbol in purple).

Effects of SM on NF-κB signaling in sensitive and insensitive cells

To investigate why SM treatment leads to a more limited effect on gene expression in MDA-MB-468 than in MDA-MB-231 cells, molecular pathways known to be influenced by SM were analyzed. Exposure to SM resulted in an initial degradation of both cIAPs in MDA-MB-468 cells that was sustained for cIAP1 but transient for cIAP2 (Figure 6a). The same pattern has been described in MDA-MB-231 cells.21 Thus, this effect is similar in the two cell lines.

Figure 6
Figure 6

LBW242 activates the non-canonical NF-κB pathway in both MDA-MB-231 and MDA-MB-468 cells. (a) MDA-MB-468 cells were treated with 20 μM LBW242 for indicated time periods. Cell lysates were analyzed with western blot using antibodies targeting cIAP1 and cIAP2. (b) MDA-MB-231 and MDA-MB-468 cells were treated with 16 nM TPA or 20 μM LBW242 for 3 or 8 h. Levels of p100 and p52 were analyzed with western blot. (c) MDA-MB-231 and MDA-MB-468 cells were treated with 16 nM TPA or 20 μM LBW242 for 4 h. Levels of IκBα were analyzed with western blot. In (ac) actin was used as a loading control. (d) Cytosolic- and nuclear fractions from MDA-MB-231 and MDA-MB-468 cells treated with 16 nM TPA or 20 μM LBW242 for 3 h were analyzed for Rel B, p100 and p52 with western blot. Lamin B and α-tubulin are included as markers of nuclear and cytosolic fractions, respectively. All blots are representatives of three independent experiments.

One possible outcome of cIAP1/2 degradation is the stabilization of NIK and subsequent processing of p100 to p52, which is indicative of non-canonical NF-κB signaling. SM treatment led to processing of p100 to p52 in both cell lines after 3 h (Figure 6b). TPA did not result in p100 processing in either cell line. Instead we found decreased levels of IκBα protein following TPA treatment (Figure 6c), whereas SM had no effect here. This suggests that primarily the canonical NF-κB pathway is involved in TPA-mediated TNFα production.

To assess whether the different sensitivity to SM may be related to a differential ability for the processed p52 fragment to enter the nucleus, nuclear and cytosolic fractions of TPA or SM-treated MDA-MB-231 and MDA-MB-468 cells were prepared. After 3 h of LBW242 treatment, p52 as well as the heterodimer Rel B appeared in the nuclear fraction of both cell lines (Figure 6d). Corroborating the results seen in Figure 6b, TPA did not evoke nuclear localization of p52 or Rel B (Figure 6d).

Effects of suppressing non-canonical or canonical NF-κB signaling

Both MDA-MB-231 and MDA-MB-468 cell lines respond to SM with p100 processing and nuclear p52 translocation (Figures 6b and d), which indicates activation of the non-canonical NF-κB pathway in both cell lines. However, SM evokes substantial changes in global gene expression including TNFα only in MDA-MB-231 cells (Figures 4a and b; Supplementary Table 1). These facts raise the question if non-canonical NF-κB signaling is the actual or sole mediator of TNFα induction. Therefore, we sought to investigate SM-induced TNFα expression when blocking either the non-canonical or the canonical pathway. NIK, an initiator of the non-canonical pathway, was downregulated with three different siRNA oligonucleotides. SM treatment induced NIK accumulation in MDA-MB-231 cells transfected with control siRNA, but this was suppressed in cells transfected with siRNA targeting NIK (Figure 7a). The same was seen for p100 processing to p52, which was essentially abolished in SM-treated cells transfected with NIK siRNA indicating a functional termination of the non-canonical NF-κB pathway. The effects were similar using both the LBW242 and the LCL161 SM (Figure 7a; Supplementary Figure 2a). Similar results were obtained in MDA-MB-468 cells (Figure 7b).

Figure 7
Figure 7

Individual suppression of non-canonical and canonical NF-κB signaling. (a) MDA-MB-231 cells were transfected with three different siRNA oligos (b) and MDA-MB-468 were transfected with one of the siRNA oligos targeting NIK. Both cell lines were treated with 20 μM LBW242 for 3 h. NIK expression and processing of p100 to p52 was evaluated by western blot, one representative blot out of three independent experiments is shown. Actin is used as a loading control. (c and d) The mRNA levels of TNFα and BIRC3 were determined by qPCR. (e) MDA-MB-231 and MDA-MB-468 cells were transfected with three different siRNA oligos targeting IKKβ, protein expression was evaluated with western blot. (f) Transfected MDA-MB-231 cells were treated with 16 nM TPA for 3 h and TNFα mRNA levels were analyzed with qPCR. (g and h) Transfected cells were treated with 10 μM LCL161 for 3 h. TNFα and BIRC3 mRNA levels were evaluated with qPCR. The numbers above each bar indicate the fold change in mRNA levels between treated and untreated samples for each siRNA. All qPCR results are mean±S.E.M. of n=3 independent experiments.

With siNIK as a tool to block the non-canonical NF-κB pathway, we investigated the SM-induced expression of BIRC3, the most upregulated gene in the microarray analysis, and TNFα (Figure 7c). In siNIK-transfected and SM-treated MDA-MB-231 cells, a decrease in SM-stimulated absolute levels of TNFα and BIRC3 mRNA was observed except for one siRNA in the case of TNFα. However, the fold change in TNFα and BIRC3 mRNA induced by SM was not as influenced by siRNA treatment (Figure 7c). This contrasts MDA-MB-468 cells, where downregulation of NIK completely suppressed SM-induced increases in BIRC3 mRNA levels (Figure 7d).

Thus, SM induction of TNFα expression can proceed to some extent in MDA-MB-231 cells even if the non-canonical NF-κB pathway is blocked (Figure 7c). This opens up for a possible role of other pathways. Therefore, we investigated the involvement of the canonical NF-κB pathway using siRNA targeting IKKβ (Figure 7e). Downregulating IKKβ with three different siRNAs, all resulted in a strong reduction of TPA-induced TNFα mRNA expression (Figure 7f), indicating a functional suppression of the canonical NF-κB signal. However, downregulation of IKKβ did not have substantial effects on the SM-induced fold changes in TNFα or BIRC3 mRNA levels in MDA-MB-231 cells (Figure 7g) although the absolute levels were suppressed in most cases. The pattern was the same for BIRC3 mRNA expression in MDA-MB-468 cells (Figure 7h).

A combined role for the canonical and non-canonical NF-κB pathways

To investigate the possibility of compensatory mechanisms, we inhibited both the canonical and the non-canonical NF-κB pathways with siRNAs targeting NIK and IKKβ. The absolute levels of SM-induced TNFα and BIRC3 mRNA expression in MDA-MB-231 cells were suppressed upon downregulation of each protein alone with an even stronger effect after downregulation of both proteins in combination (Figures 8a–c). However, the SM-induced fold change still persisted following combined knockdown of NIK and IKKβ even though the absolute TNFα mRNA levels were low, resembling levels of control cells (Figure 8b). The pattern was the same for BIRC3 (Figure 8c). In line with effects on TNFα, the SM-induced cell death was suppressed following treatment with siRNA targeting IKKβ or NIK (Figure 8d). Combined knockdown of both IKKβ and NIK resulted in a significant decrease in cell death whereas knockdown of only one pathway led to a smaller effect. The results indicate that both the canonical and the non-canonical NF-κB pathways contribute to SM-induced TNFα levels and cell death.

Figure 8
Figure 8

Combined suppression of the canonical and non-canonical NF-κB pathways. MDA-MB-231 cells were transfected with siRNA oligos targeting NIK, IKKβ or the combination of both for 48 h and treated with 10 μM LCL161 for 3 h. (a) NIK, IKKβ, p100 and p52 protein expression was evaluated with western blot, and actin was used as a loading control. (b, c) TNFα and BIRC3 mRNA levels were analyzed with qPCR. The numbers above each bar indicate the fold change of mRNA levels between treated and untreated sample for each siRNA. (d) Cell death was determined by flow cytometry using APC-conjugated annexin V. All results represent the mean±S.E.M. of n=4 independent experiments; **P<0.01.

Discussion

Deregulation of cell death is an important contribution to cancer development and may also confer resistance to chemotherapy and radiotherapy. Therefore, promoting cell death induction is one potential strategy for cancer therapy.

One way to promote cell death is by mimicking the function of the pro-apoptotic protein Smac with SM. However, although some cells respond with cell death, most cell types are resistant to SM as single agents. In this report, we show that the PKC activator TPA induces sensitivity to SM in certain basal breast cancer cells and shed light on the differences in SM-induced signaling between sensitive and insensitive cells.

Current understanding indicates that SM-induced cell death involves autocrine TNFα production,16,17 although SM-induced cell death independently of TNFα has been reported.13,14,24 Our data indicate that TPA promotes sensitivity to SM by initiating the production of TNFα, since TPA induced a de novo production of TNFα with increased TNFα mRNA and protein levels (Figure 4). Furthermore, the TPA effect was blocked by TNFα antibodies, and could be mimicked by addition of TNFα (Figures 3a–d). Thus, induction of local production of TNFα may be one way to render cancer cells sensitive to SM, a notion in line with the report that pathogen mimetics generating a cytokine milieu consisting of IFNβ, TNFα and/or TRAIL sensitizes cancer cells to SM.25

The fact that PKC activation can induce TNFα production in many cell lines26,​27,​28,​29 and the proposed dependency on TNFα for SM-induced death,16,17 raise the question why most cell lines were insensitive to the TPA+SM combination (Figure 1a). In fact, only triple-negative breast cancer cell lines were sensitive to this treatment. There are several putative explanations for this, one being that TPA may not lead to sufficiently high TNFα levels in all cell types. One example of this is MCF-7 cells where TPA induces TNFα expression but with shorter duration and at lower levels than in MDA-MB-468 cells (Figure 4f). Another possibility is that the cell does not respond to TNFα. This was the case for T47D cells, which were insensitive to the combination of SM and TNFα (Figure 4e). Insensitivity to the combination of SM and TNFα has also been reported for mesothelioma cells. This was reported to be due to the caspase inhibitory actions exerted by the caspase-8 inhibitor FLICE-like inhibitory protein (cFLIP).30

TNFα is a target gene of the NF-κB signaling pathway31 and TPA has been shown to induce NF-κB activation in several cell lines.26,28 More recent studies indicate PKCδ and/or PKCε isoforms as mediators of TPA-induced NF-κB activation.27,29 Our data show that TPA treatment of MDA-MB-231 and MDA-MB-468 cells results in decreased IκBα levels, indicative of canonical NF-κB signaling (Figure 5c), and that downregulation of the canonical pathway (Figure 7f) suppresses the TNFα induction. It is therefore conceivable that de novo synthesis of TNFα, via the canonical NF-κB pathway, is the primary effect of TPA treatment in MDA-MB-468 breast cancer cells.

Contrasting the effects of TPA, it has been implied that SM induces TNFα production via the non-canonical NF-κB pathway.16,​17,​18 This pathway is mediated through activation of NIK. SM treatment can lead to NIK stabilization and activation due to decreased levels of cIAP1 and cIAP2 that ubiquitinate NIK, causing its degradation.16,​17,​18 Here we show that SM leads to decreased levels of both cIAP1 and cIAP2 in MDA-MB-468 cells (Figure 5a) as has previously been described for MDA-MB-231 cells.21 The subsequent processing of p100 to p52 was seen for both cell lines (Figure 5b). Thus, the non-canonical NF-κB signaling pathway is activated by SM in both cell lines but results in TNFα expression only in MDA-MB-231 cells. This may suggest that non-canonical NF-κB signaling is not sufficient for TNFα induction. Our microarray data further indicate that non-canonical signaling does not elicit major changes in gene expression in MDA-MB-468 cells since few genes were upregulated upon SM treatment contrasting the effect in MDA-MB-231 cells. However, both MDA-MB-468 and MDA-MB-231 cell lines showed a substantial increase in BIRC3 expression following SM treatment indicating similar responses to non-canonical NF-κB signaling for this gene.

The fact that the non-canonical pathway is not sufficient for SM-mediated TNFα production in MDA-MB-468 raises the question whether other pathways are needed. There is substantial support for a role of NF-κB signaling in SM-induced TNFα expression.32,33 Here we used siRNA to inhibit the canonical and the non-canonical NF-κB pathways separately or in combination. Blocking either pathway in the sensitive MDA-MB-231 cell line resulted in lower absolute levels of the NF-κB target genes, TNFα and BIRC3 (Figures 7c and g). However, the ability of SM to induce a relative increase still remained. To suppress SM-stimulated TNFα and BIRC3 levels down to those of untreated control and to mediate a significant decrease in cell death, inhibition of both pathways were required (Figures 8b–d). However, even when both NF-κB pathways are suppressed there is still a relative increase in TNFα mRNA upon SM treatment suggesting that additional IAP-regulated mechanisms may be involved. There are several examples of NF-κB-independent roles for IAPs, such as facilitation of Myc-promoted proliferation by ubiquitinating the Myc-negative regulator Max-dimerization protein-1 (Mad1),34 control of cyclin transcription by regulation of E2F135 and the findings that cIAP1 and XIAP affect morphology of cells by acting on Rho GTPases.36,37

Other potential mechanisms are suggested in studies identifying mechanisms of SM resistance, such as in vivo-selected SM-resistant MDA-MB-231 sublines where lower levels of the leucine-rich repeats and immunoglobulin-like domains 1 (LRIG1) were found. LRIG1 depletion also attenuated production of and influenced the sensitivity to TNFα.38 Furthermore, the interferon regulatory factor 1 (IRF1) was shown to be important for SM-mediated TNFα induction and SM sensitivity.39 Another report showed that overexpression of USP11 led to stabilization of cIAP2 and resistance to SM.40

Taken together, this paper further strengthens the importance of TNFα in SM-mediated cell death and demonstrates that a simultaneous induction of TNFα may be one strategy to increase SM sensitivity. At least for basal-like breast cancer cells, this can potentially be achieved by PKC activation. We also provide indications that induction of the non-canonical NF-κB pathway may not be sufficient and not the sole mediator of SM-induced TNFα production. It is conceivable that other mechanisms are necessary to occur.

Materials and Methods

Materials and cell culture

All cell lines were from American Type Culture Collection (ATCC, Manassas, VA, USA). All media and supplements were from Thermo Scientific (Waltham, MA, USA) unless stated otherwise. Cells were grown in RPMI-1640 except for HeLa and Cama-1 grown in DMEM low glucose, and KCN-69 and SK-N-BE2C grown in MEM/EBSS. All media were supplemented with 10% fetal bovine serum (Biosera, Boussens, France), 100 IU/ml penicillin, and 100 μg/ml streptomycin. In addition, MDA-MB-468 and MDA-MB-321 media were supplemented with 1 mM sodium pyruvate; BT-549 medium was supplemented with 0.023 IU/ml Actrapid Penfill insulin (Novo Nordisk AS, Bagsvaerd, Denmark); and Cama-1 medium was supplemented with 1% MEM non-essential amino acids.

When indicated, cells were treated with etoposide, paclitaxel, TNF-α, TPA (all Sigma-Aldrich, St. Louis, MO, USA), anti-hTNF-α-hIgG2 (InvivoGen, Toulouse, France), GF109203X (Calbiochem, San Diego, CA, USA) and zVADfmk (Enzo Life Sciences, Farmingdale, NY, USA). LBW242 and LCL161 were kindly provided by Novartis (Basel, Switzerland).

Transfections

For siRNA transfections, cells were seeded at a density of 1 000 000–1 500 000 cells per 100-mm cell culture dish in 8 ml complete medium without antibiotics. After 24 h, cells were incubated for 48 h with 2 μl/ml Lipofectamine 2000 (Life Technologies—Invitrogen, Waltham, MA, USA) and 40 nM siRNA in Opti-MEM (Life Technologies—Gibco, Waltham, MA, USA) and complete medium without antibiotics according to the supplier’s protocol.

Annexin V-staining

Cells were seeded at a density of 2 000 000 cells per 100-mm cell culture dish (1 000 000 cells for siRNA-transfected samples) in 8 ml complete medium. Twenty-four hours later the medium was changed and cells were treated with indicated compounds. Cells were stained with annexin V-APC (BD Pharmingen, Franklin Lakes, NJ, USA) and analyzed with FACSCalibur or BD FACSVerse (both BD Biosciences, Franklin Lakes, NJ, USA). In total, 10 000 events were acquired on the FL-4 channel for the annexin V-APC signal. Sample acquisition and analyses were performed with CellQuest software or BD FACSuite software (both BD Biosciences).

Analysis of cell viability

Cells were seeded in 100 μl complete medium at a density of, depending on cell line, 6000–12 000 cells per well in 96-well culture plates. After 24 h, the medium was changed and cells were treated with indicated compounds for 30 h of which the last 4 h were in combination with 10 μl WST-1 (Roche, Penzberg, Germany). The amount of viable cells was thereafter assessed by measuring the conversion of WST-1 to a water-soluble formazan dye. Absorbance (450 nm) was measured by the Synergy 2 Microplate reader (BioTek, Winooski, VT, USA) and analyzed with the Gen5 Reader Control and Data Analysis software (BioTek).

Analysis of complex II formation

For complex II immunoprecipitations, cells were treated with indicated agents in the presence of 20 μM zVADfmk to enable uncleaved caspase-8 immunoprecipitation as previously described.21 Cells were washed once and collected in ice-cold PBS and centrifuged before adding 500 μl kit lysis buffer (Miltenyi Biotec, Bergisch Gladbach, Germany) containing 40 μl/ml complete protease inhibitors (Roche Applied Science, Penzberg, Germany). Lysates were cleared by centrifugation at 14 000×g for 10 min. Immunoprecipitations were performed using caspase-8 antibodies (1 : 75, Cell Signaling 9746, Danvers, MA, USA) together with μMACS Protein G MicroBeads (Miltenyi Biotec).

Immunoprecipitates were added to MACS Separation Columns (Miltenyi Biotec), washed in kit buffers and proteins were eluted with heated sample buffer containing the reducing agent dithiothreitol.

Western blot

Cells were washed in ice-cold PBS and lysed in RIPA buffer (10 mM Tris-HCl (pH 7.2), 160 mM NaCl, 1% Triton X-100 (Sigma-Aldrich), 1% sodium deoxycholate, 0.1% sodium dodecyl sulfate, 1 mM EDTA and 1 mM EGTA) containing 40 μl/ml complete protease inhibitors (Roche Applied Science). Lysates were cleared by centrifugation at 14 000×g for 10 min. Equal amounts of proteins were electrophoretically separated on 10% NuPAGE Novex Bis-Tris gels (Life Technologies—Invitrogen) and transferred to polyvinylidene difluoride membranes (Millipore, Billerica, MA, USA). Membranes were blocked with 5% non-fat milk in PBS, and probed with antibodies toward actin (1 : 2000, MP Biomedicals 691001, Santa Ana, CA, USA and 1 : 1000 sc-81178), caspase-3 (1 : 500, Pharmingen 67341A), caspase-8 (1 : 500, Cell Signaling 97465), cIAP1 (1 : 200, R&D AF8181, Minneapolis, MN, USA), cIAP2 (1 : 300 NB110-57030), p52/p100 (1 : 300, Cell Signaling #4882), Rel B (1 : 300 sc-48366), p65 (1 : 400 sc-372), α-tubulin (1 : 400 sc-5286), lamin B (1 : 400 sc-6216) and RIP1 (1 : 300 BD Trans Lab 610458, Franklin Lakes, NJ, USA), followed by incubation with a horseradish peroxidase-labeled secondary antibody (1 : 5000 GE Healthcare, Little Chalfont, UK). The chemiluminescence was detected with a charge-coupled device camera (Fujifilm, Minato, Japan).

Analysis of TNFα mRNA levels

Total RNA was extracted with the RNeasy kit (Qiagen, Hilden, Germany) and potential DNA contamination was eliminated with the RQ1 RNase-free DNase (Promega, Madison, WI, USA). In total, 2 μg of RNA was used for cDNA synthesis with MultiScribe Reverse Transcriptase (Applied Biosystems, Foster City, CA, USA). The cDNA was thereafter amplified by qPCR for evaluation of relative mRNA expression levels in an Applied Biosystems 7300 real-time quantitative PCR system using the SYBR Green PCR Master Mix (Applied Biosystems). All primer pairs were from Invitrogen Life Sciences (Waltham, MA, USA), designed with the Primer Express software (Applied Biosystems; forward TNF: 5′-GCAGGTCTACTTTGGGATCATTG-3′, reverse TNF: 5′-GCGTTTGGGAAGGTTGGA-3′). The mRNA expression data were normalized to three reference genes (SDHA, UBC and YWHAZ) as described before.41 For relative quantification of gene expression, the comparative Ct method was applied.42

Enzyme-linked immunosorbent assay (ELISA)

TNFα protein levels in the cell culture media were determined using the human TNF alpha ELISA kit (Thermo Scientific) according to the manufacturer’s protocol. Sample diluent was added to an anti-human TNFα precoated 96-well strip plate along with cell culture media, from cells with indicated treatment, or with standard and incubated for 1 h. After rinsing the plate with washing buffer, biotinylated antibodies were added, incubated for 1 h and washed. Streptavidin-HRP reagent was added for 30 min, whereafter the plate was washed before incubation with TMB substrate solution for 30 min in the dark. Following addition of stop solution, absorbance (at 450 and 550 nm) was measured in a Synergy 2 Microplate reader (BioTeck) and analyzed with the Gen5 Reader Control and Data Analysis software (BioTek).

Nuclear fractionation

Cell fractionation protocol was adapted after Kasperczyk et al.43 Briefly, the collected and pelleted cells on ice were resuspended in a low-salt buffer (10 mM HEPES-OH (pH 7.9), 1.5 mM MgCl2 and 10 mM KCl) and incubated at 4 °C for 10 min. Thereafter, NP-40 with a final concentration of 1% was added and the sample was vigorously mixed before centrifugation at 1000×g for 5 min at 4 °C. The supernatant was centrifuged at 12 500×g for 10 min at 4 °C and the resulting supernatant was collected as the cytosol fraction. To retrieve the nuclear fraction, the first pellet was dissolved in a high-salt buffer (20 mM HEPES-OH (pH 7.9), 420 mM NaCl, 1.5 mM MgCl2, 0.2 mM EDTA and 25% glycerol) incubated for 15 min at 4 °C during which time the pellet was vortexed periodically before centrifugation at 12 500×g for 10 min at 4 °C. The resulting supernatant was collected as the nuclear fraction. Both low- and high-salt buffers were supplemented with a protease- and phosphatase inhibitor cocktail: 40 μl/ml complete protease inhibitors (Roche Applied Science), 1 mM sodium-orthovanadate (Na3VO4) and 50 mM β-glycerophosphate (both from Sigma-Aldrich) before use.

Microarray analysis

RNA was extracted from MDA-MB-231 and MDA-MB-468 cells after 4 h of treatment with LBW242 or DMSO using the RNeasy mini kit (Qiagen). Total gene expression analysis was performed by the SCIBLU facility in Lund using the DirectHyb HumanHT-12 v4.0 chip (Illumina, San Diego, CA, USA). Results were normalized using quantile normalization. The significance analysis of microarrays (SAM)44 analysis was performed to identify significantly differentially expressed genes in LBW242-treated cells versus control cells. Pathway analysis was performed using the MetaCore web-based program (MetaCore pathway analysis software) by SCIBLU.

Statistical analysis

Statistical analyses were performed using IBM SPSS Statistics 22 and one-way ANOVA followed by Duncan’s multiple range test.

References

  1. 1.

    , . The hallmarks of cancer. Cell 2000; 100: 57–70.

  2. 2.

    , . Hallmarks of cancer: the next generation. Cell 2011; 144: 646–674.

  3. 3.

    , , , , , et al. Molecular definitions of cell death subroutines: recommendations of the Nomenclature Committee on Cell Death 2012. Cell Death Differ 2012; 19: 107–120.

  4. 4.

    , , . Organelle-specific initiation of cell death. Nat Cell Biol 2014; 16: 728–736.

  5. 5.

    , . Extrinsic versus intrinsic apoptosis pathways in anticancer chemotherapy. Oncogene 2006; 25: 4798–4811.

  6. 6.

    , , . Regulation of TNFR1 and CD95 signalling by receptor compartmentalization. Nat Rev Mol Cell Biol 2008; 9: 655–662.

  7. 7.

    , . Mitochondria and cell death: outer membrane permeabilization and beyond. Nat Rev Mol Cell Biol 2010; 11: 621–632.

  8. 8.

    , . Targeting IAP proteins for therapeutic intervention in cancer. Nat Rev Drug Discov 2012; 11: 109–124.

  9. 9.

    , , , , , et al. Both cIAP1 and cIAP2 regulate TNFalpha-mediated NF-kappaB activation. Proc Natl Acad Sci USA 2008; 105: 11778–11783.

  10. 10.

    . Nuclear factor-kappaB in cancer development and progression. Nature 2006; 441: 431–436.

  11. 11.

    , . Induction of TNF receptor I-mediated apoptosis via two sequential signaling complexes. Cell 2003; 114: 181–190.

  12. 12.

    , , . RIP kinases at the crossroads of cell death and survival. Cell 2009; 138: 229–232.

  13. 13.

    , , , , . Smac mimetic LBW242 sensitizes XIAP-overexpressing neuroblastoma cells for TNF-alpha-independent apoptosis. Cancer Res 2012; 72: 2645–2656.

  14. 14.

    , , , , . Smac mimetic Birinapant induces apoptosis and enhances TRAIL potency in inflammatory breast cancer cells in an IAP-dependent and TNF-alpha-independent mechanism. Breast Cancer Res Treat 2013; 137: 359–371.

  15. 15.

    , , , , , et al. Autocrine TNFalpha signaling renders human cancer cells susceptible to Smac-mimetic-induced apoptosis. Cancer Cell 2007; 12: 445–456.

  16. 16.

    , , , , , et al. IAP antagonists induce autoubiquitination of c-IAPs, NF-kappaB activation, and TNFalpha-dependent apoptosis. Cell 2007; 131: 669–681.

  17. 17.

    , , , , , et al. IAP antagonists target cIAP1 to induce TNFalpha-dependent apoptosis. Cell 2007; 131: 682–693.

  18. 18.

    , , , , , et al. Noncanonical NF-kappaB activation requires coordinated assembly of a regulatory complex of the adaptors cIAP1, cIAP2, TRAF2 and TRAF3 and the kinase NIK. Nat Immunol 2008; 9: 1371–1378.

  19. 19.

    , , , , , et al. Nonredundant and complementary functions of TRAF2 and TRAF3 in a ubiquitination cascade that activates NIK-dependent alternative NF-kappaB signaling. Nat Immunol 2008; 9: 1364–1370.

  20. 20.

    , , , , . Identification of a novel protein kinase Cdelta-Smac complex that dissociates during paclitaxel-induced cell death. FEBS Lett 2012; 586: 1166–1172.

  21. 21.

    , , , , , et al. The Ripoptosome, a signaling platform that assembles in response to genotoxic stress and loss of IAPs. Mol Cell 2011; 43: 432–448.

  22. 22.

    , , . Tumor-promoting phorbol ester-induced cell death and gene expression in a human prostate adenocarcinoma cell line. Oncol Res 1994; 6: 203–210.

  23. 23.

    , , , , , et al. Gene expression profiling of breast cancer cells in response to gemcitabine: NF-kappaB pathway activation as a potential mechanism of resistance. Breast Cancer Res Treat 2007; 102: 157–172.

  24. 24.

    , , , , , et al. SMAC mimetic (JP1201) sensitizes non-small cell lung cancers to multiple chemotherapy agents in an IAP-dependent but TNF-alpha-independent manner. Cancer Res 2011; 71: 7640–7648.

  25. 25.

    , , , , , et al. Smac mimetics and innate immune stimuli synergize to promote tumor death. Nat Biotechnol 2014; 32: 182–190.

  26. 26.

    , . Activation of DNA-binding activity in an apparently cytoplasmic precursor of the NF-kappa B transcription factor. Cell 1988; 53: 211–217.

  27. 27.

    , , , , , . Phorbol ester-stimulated NF-kappaB-dependent transcription: roles for isoforms of novel protein kinase C. Cell Signal 2008; 20: 1338–1348.

  28. 28.

    , . Inducibility of kappa immunoglobulin enhancer-binding protein Nf-kappa B by a posttranslational mechanism. Cell 1986; 47: 921–928.

  29. 29.

    , , , , , . Activation of nuclear factor kappaB (NF-kappaB) in prostate cancer is mediated by protein kinase C epsilon (PKCepsilon). J Biol Chem 2012; 287: 37570–37582.

  30. 30.

    , , , , , et al. SAHA overcomes FLIP-mediated inhibition of SMAC mimetic-induced apoptosis in mesothelioma. Cell Death Dis 2013; 4: e733.

  31. 31.

    . Activators and target genes of Rel/NF-kappaB transcription factors. Oncogene 1999; 18: 6853–6866.

  32. 32.

    , , , , . Identification of non-canonical NF-kappaB signaling as a critical mediator of Smac mimetic-stimulated migration and invasion of glioblastoma cells. Cell Death Dis 2013; 4: e564.

  33. 33.

    , , , , , et al. Smac mimetic sensitizes glioblastoma cells to temozolomide-induced apoptosis in a RIP1- and NF-kappaB-dependent manner. Oncogene 2013; 32: 988–997.

  34. 34.

    , , , , , et al. c-IAP1 cooperates with Myc by acting as a ubiquitin ligase for Mad1. Mol Cell 2007; 28: 914–922.

  35. 35.

    , , , , , et al. Cellular inhibitor of apoptosis protein-1 (cIAP1) can regulate E2F1 transcription factor-mediated control of cyclin transcription. J Biol Chem 2011; 286: 26406–26417.

  36. 36.

    , , , , , et al. IAPs regulate the plasticity of cell migration by directly targeting Rac1 for degradation. EMBO J 2012; 31: 14–28.

  37. 37.

    , , , , , et al. cIAP1 regulates TNF-mediated cdc42 activation and filopodia formation. Oncogene 2014; 33: 5534–5545.

  38. 38.

    , , , , , . LRIG1 modulates cancer cell sensitivity to Smac mimetics by regulating TNFalpha expression and receptor tyrosine kinase signaling. Cancer Res 2012; 72: 1229–1238.

  39. 39.

    , , . Identification of IRF1 as critical dual regulator of Smac mimetic-induced apoptosis and inflammatory cytokine response. Cell Death Dis 2014; 5: e1562.

  40. 40.

    , , , , , . USP11-dependent selective cIAP2 deubiquitylation and stabilization determine sensitivity to Smac mimetics. Cell Death Differ 2015; 22: 1463–1476.

  41. 41.

    , , , , , et al. Induction of ID2 expression by hypoxia-inducible factor-1: a role in dedifferentiation of hypoxic neuroblastoma cells. J Biol Chem 2004; 279: 39223–39231.

  42. 42.

    , , , , , et al. Quantification of MYCN, DDX1, and NAG gene copy number in neuroblastoma using a real-time quantitative PCR assay. Mod Pathol 2002; 15: 159–166.

  43. 43.

    , , , , , et al. Betulinic acid as new activator of NF-kappaB: molecular mechanisms and implications for cancer therapy. Oncogene 2005; 24: 6945–6956.

  44. 44.

    , , . Significance analysis of microarrays applied to the ionizing radiation response. Proc Natl Acad Sci USA 2001; 98: 5116–5121.

Download references

Acknowledgements

The authors thank Novartis for generously providing us with the low-molecular-weight SM LBW242 and LCL161. The authors acknowledge the help with the microarray analysis from SCIBLU, Microarray Resource Centre at Lund University and the Wallenberg Stiftelse (KAW), with professional assistance from Srinivas Veerla. Financial support was obtained from the Swedish Research Council, the Swedish Cancer Society and the Gunnar Nilsson foundations.

Author information

Affiliations

  1. Department of Laboratory Medicine, Translational Cancer Research, Lund University, Lund, Sweden

    • L Cornmark
    • , C Holmgren
    • , K Masoumi
    •  & C Larsson

Authors

  1. Search for L Cornmark in:

  2. Search for C Holmgren in:

  3. Search for K Masoumi in:

  4. Search for C Larsson in:

Competing interests

The authors declare no conflict of interest.

Corresponding author

Correspondence to C Larsson.

Supplementary information

Glossary

PKC

protein kinase c

Smac

second mitochondria-derived activator of caspases

IAP

inhibitor of apoptosis protein

TPA

12-O-tetradecanoylphorbol-13-acetate

TNFα

tumor necrosis factor alpha

RIP1

receptor-interacting protein kinase-1

NF-κB

nuclear factor kappa-light-chain enhancer of activated B cells

BIRC3

baculoviral IAP repeat-containing 3

DR

death receptor

SM

Smac mimetic

cIAP

cellular IAP

TNFR1

tumor necrosis factor receptor-1

FADD

fas-associated protein with death domain

XIAP

X-linked IAP

NIK

NF-κB-inducing kinase

IKKβ

I-kappa-B kinase-β

IFNβ

interferon beta

TRAIL

TNF-related apoptosis-inducing ligand

cFLIP

cellular FLICE-like inhibitory protein

IκBα

inhibitor of kappa B alpha

Mad

max-dimerization protein-1

E2F1

E2 promoter-binding factor-1

LRIG1

leucine-rich repeats and immunoglobulin-like domain-1

IRF1

interferon regulatory factor-1

USP11

ubiquitin specific peptidase 11.

About this article

Publication history

Received

Accepted

Published

DOI

https://doi.org/10.1038/cddiscovery.2016.2

Further reading Further reading