Introduction

Prostate cancer (PCa) alone accounts for 26% of the most common cancers diagnosed in men and leads to 11% of cancer-related death globally in 2021 [1]. Suppression of androgen receptor (AR) signaling has been recognized to be central to the treatment of PCa since androgen has been identified as the driving hormone in PCa progression in 1941 [2]. Primary treatment for PCa is androgen deprivation therapy (ADT), including surgical bilateral orchiectomy or medical castration with a gonadotropin-releasing hormone (GnRH) agonist alone or in combination with an AR antagonist [3]. However, the remission of ADT-responsive treatment only lasts for 2–3 years, and most patients would inevitably progress to ADT-refractory castration-resistant PCa (CRPC), which depends on AR but exhibits reduced requirement for androgens and is resistant to antagonists [4, 5].

Multiple mechanisms of resistance contribute to the development of CRPC, including AR amplification, AR splice variation, AR point mutations and AR bypass facilitated via glucocorticoid receptor (GR) [6]. Previous study on CRPC has focused primarily on the functional blockade of AR. First-generation AR antagonists such as flutamide, nilutamide, and bicalutamide (BIC) show effectiveness on treatment-naive PCa but cannot significantly improve the overall survival of CRPC [6]. Second-generation AR antagonists, including enzalutamide (ENZ) (2012), apalutamide (ARN-509) (2018), and darolutamide (DARO) (2019), had improved potency on AR blockade and have been approved for CRPC treatment [7]. However, AR antagonists remain susceptible to acquired resistance. AR mutations are rare in early-stage PCa but common in CRPC. Point mutations in the AR ligand binding pocket (LBP), such as W741C, H874Y, T877A and F876L, could make the LBP more specious and broaden the ligand binding specificity of AR [8], allowing other steroid hormones and even antiandrogens to activate AR, which limits the durability of the agents and necessitates further research. Furthermore, GR and AR can commonly regulate up to 50% of their target genes in PCa cells [9]. Accordingly, suppression of AR signaling induces high-level expression of GR as an alternative and compensation for AR transcriptional program, thereby bypassing AR blockade [10]. In addition, enhanced GR expression was observed after long-term ADT with abiraterone or enzalutamide, and GR activation diminished the effects of AR antagonists and increased prostate-specific antigen (PSA) secretion, indicating that GR upregulation promotes resistance to enzalutamide and progression of CRPC [11]. In addition, it has been proved that differential GR and AR expressions, autonomous GR and AR activation, and ligand-independent AR expression and activation are related to the development and progression of salivary duct carcinoma [12].

Therefore, some researchers turned their attention to the application of dual AR-GR antagonists in CRPC. According to the study reported by Yemelyanov et al. [13], a stable analog of an aziridine precursor from the African shrub (CpdA) could significantly repress R1881-induced AR activation (50% inhibition at 10 μM) and exhibit anti-inflammatory effects without transactivation of GR [14]. In addition, CpdA could inhibit tumor proliferation and induce caspase-dependent apoptosis in metastatic CRPC cells. CpdA represents a unique dual-target steroid receptor modulator that has a high potential for PCa therapy. Then in 2019, Wu et al. identified Z19 [15] as a dual AR-GR antagonist by pharmacophore-based virtual screening for AR and receptor-based virtual screening for GR. Z19 inhibits the transcription activity towards both AR (about 70% inhibition at 10 μM) and GR (about 40% inhibition at 10 μM), reducing both the protein and mRNA levels of the downstream genes regulated by GR and AR signaling. However, the mechanisms of CpdA and Z19 remain unclear, as there is no evidence that these compounds exert their transcriptional activity by competing with the LBP nor any specific binding pocket. Recently, Rosette et al. [16] discovered a synthetic steroidal compound, CB-03-10, as an dual antagonist that binds to the AR and GR LBP sites with IC50 values of 3.370 μM and 0.219 μM, respectively. Oral administration of CB-03-10 inhibited the tumor growth of both AR-positive LNCaP xenograft model and AR-negative/GR-positive MDA-MB-231 xenograft model, and the mechanism of tumor proliferation inhibition was proved to be related with AR-GR antagonistic activity. However, high dosages of steroidal medicine are largely at risk of inducing severe adverse events, including osteoporosis, hyperkalemia and deterioration of renal function. Accordingly, the demand for novel, potent, and non-steroidal AR-GR dual antagonists that specifically target the LBP has spawned our study.

We first identified compound H18 as an AR and GR binder by combining structure-based virtual screening (SBVS) and biological evaluation. Then we conducted structural analysis and optimization of H18 to further improve its activity. Therein, we discovered the analogs HD51 and HD57 with better LBP competitive ability and antagonistic activity towards both AR and GR. What’s more, HD51 and HD57 also block the function of the tested AR mutants arising in response to antiandrogen therapies, providing a novel non-steroidal scaffold for the development of AR-GR dual antagonists.

Materials and methods

Chemistry

Compounds H1-H18 (Table S1) and HA1-HD58 (Tables 14) were purchased from the commercial vendor of Chemdiv which provides 100% quality control for all compounds and guarantees more than 95% purity (±5%) confirmed by 1H NMR and LC-MS spectra. The compounds with more than 50% inhibition at 10 μM in our study were further determined to be >95% pure by high-performance liquid chromatography (HPLC) analysis. The HPLC traces for the representative compounds H18, HD51, and HD57 are shown in Figs. S3S5.

Protein preparation for SBVS

SBVS based on the AR LBD and GR LBD was expected to discover novel dual AR-GR antagonist. The crystal structure of the AR LBD complex with R-bicalutamide (PDB ID: 1Z95) was selected for SBVS. However, the GR passive antagonist conformation (PDB ID:3H52) reported by Schoch et al. [17] has a missing region: residues 741–753), and its N-terminal end (helix 1 (H1) and H1-H3 loop) is unfolded from the LBD body. In a previous study, we collected the conformational ensemble of the GR LBD upon the RU486 binding [18]. Therefore, to obtain a completely folded passive antagonist conformation of RU486, the root-mean-square deviations (RMSDs) of the residues 581–777 Cα atoms were used as the criterion to compare the similarity between the conformation from the 10 µs MD simulation trajectories and the passive antagonist conformation reported by Schoch et al. The conformation with the smallest RMSD (RMSD = 1.78 Å) was selected for SBVS.

SBVS workflow

The AR-BIC and GR-RU486 complexes were prepared using the Protein Preparation Wizard in Schrödinger 2018. The water was deleted. The native ligand in the AR LBD/GR LBD was retained and used for grid generation. The small molecules in the ChemDiv chemical library were prepared using the LigPrep module and docked into the AR LBD using the Glide module. The binding energies were scored and ranked by the Glide SP scoring mode. The 2000 top-ranked compounds were clustered based on the 2D similarities (Tanimoto coefficients) of the MACCS fingerprints. Then, the binding poses of the clustered compounds were carefully checked. Finally, 18 potential compounds were purchased for the subsequent bioassays.

MD simulations

The structures of the AR LBD/GR LBD bound with the compounds predicted by molecular docking were used as the initial conformations for the MD simulations. The AM1-BCC atomic partial charges for each ligand were assigned with antechamber in the AMBER 18 package [19]. The FF14SB [20] and GAFF2 [21] force fields were assigned to the protein and ligands, respectively. The protein–ligand complex was solvated into a TIP3P water box with a distance of 10 Å extended from any solute atom.

Three phases of minimization were performed to remove unfavorable contacts for the prepared systems using the previously reported procedures [18, 22]. Next, each system was gradually heated from 100 to 300 K over a period of 30 ps in the NVT ensemble, and 110 ps equilibration was performed at 300 K. Finally, 500 ns MD simulations were carried out in the NPT (T = 300 K and P = 1 atm) ensemble with the PMEMD program [23]. The SHAKE algorithm [24] was used to constrain the covalent bonds involving hydrogen atoms and the time step was set to 2 fs. The snapshots were saved every 10 ps. The RMSD and distance analyses were carried out by the cpptraj module in AmberTools18.

MM/GBSA free energy decomposition

The interactions between each residue in the AR LBD/GR LBD and each ligand were analyzed using the MM/GBSA free energy decomposition analysis [25,26,27,28] applied in the MMPBSA.py script as implemented in AmberTools18. The binding interaction for each residue-inhibitor pair includes three terms: van der Waals contribution (ΔGvdw), electrostatic contribution (ΔGele) and desolvation contribution (ΔGsol) (Eq. 1). The electrostatic desolvation energy (ΔGGB) was estimated by using the GB model based on the parameters developed by Onufriev et al. [29]. The non-polar contribution of desolvation (ΔGSA) was measured by the solvent-accessible surface area (SASA).

$$\Delta G_{{{{{{{{\mathrm{residue}}}}}}}} - {{{{{{{\mathrm{ligand}}}}}}}}} = \Delta G_{vdw} + \Delta G_{ele} + \Delta G_{sol}\\ = \Delta G_{vdw} + \Delta G_{ele} + \Delta G_{GB} + \Delta G_{SA}$$
(1)

All the energy components were calculated using the 100 snapshots extracted from the last 50 ns MD trajectory.

Plasmids, cell lines, and cell culture

Plasmids pGL4-MMTV-hygro and pGL4-ARR2PB-luc were constructed by cloning MMTV and ARR2PB fragment into the pGL4.36[luc2P Hyg] (#E6731, Promega), respectively. Plasmid pCMV-hAR (#89078, Addgene) was gift from Elizabeth Wilson. Plasmids hARF876L, hARF876L/T877A, hARH874Y/T877A, hART877G, and hARW741C were constructed by site-directed mutagenesis of pCMV-hAR through overlap extension PCR. LNCaP, C4-2, DU145, 22RV1, PC3, Chang, and GES-1 were propagated in RPMI-1640 (#MA0215, Meilunbio) supplemented with 10% fetal bovine serum (FBS, #F8318, Sigma-Aldrich). Hela-MMTV, 293T, and NIH3T3 cells were cultured in DMEM (#MA0212, Meilunbio) with 10% FBS. All growth media were supplemented with 1% Penicillin–Streptomycin–Glutamine (#10378016, Gibco). Cell cultures were maintained in culture flasks in 5% CO2 atmosphere at 37 °C. To avoid the disturbance of relevant endogenous factors in FBS, FBS were stripped with dextran-coated charcoal (DCC) to remove most of its hormone, growth factors and cytokines. Phenol red-free media along with 5% DCC-stripped serum-starvation (CSS) were used for treatment studies.

eGFP and secreted PSA assays

LNCaP-ARR2PB-eGFP cell was constructed as previously described [30]. The cell line contains an androgen-responsive probasin-derived promoter (ARR2PB) and thus can be used for AR transcriptional assay. Briefly, LNCaP-ARR2PB-eGFP cells were starved for 5 days and then seeded into 96-well plates at a density of 3.5 × 104 cells/well. After 24 h, 5 nM DHT (S4757, Selleck) with the expected concentrations of compounds were added. Cells were incubated for 3 days before the fluorescence was measured using Synergy H1 microplate reader (BioTek. Excitation, 485 nm; emission, 535 nm). Subsequently, the supernatant of each sample was collected with a volume of 300 μL and sent to Cancer Hospital of University of Chinese Academy of Sciences, Zhejiang Cancer Hospital (Hangzhou, Zhejiang, China), and the secreted PSA level was determined by the IMMULITE® 2000 XPi Immunoassay System (Siemens Ltd., Erlangen, Germany).

BLI assay

The direct reversible interaction between the small molecules and the AR/GR LBD was quantified by BLI using an Octet K2 apparatus (Sartorius AG, Göttingen, Germany). AR LBD protein (#A15676, Thermo Fisher) and GR LBD protein (#A15668, Thermo Fisher) were biotinylated by a Genemore Biotinylation Kit (#G-MM-IGT) and then immobilized on the Octet® SSA biosensors (#18-5057, Lot Number: 2107014411, Sartorius) at a concentration of 0.2 mg/mL for 900 s. Duplicated set of sensors was incubated in assay buffer without protein as a background binding control. Interactions with ligands were measured at gradient concentrations in 0.02% PBST, pH 7.4, at 30 °C for 60 s. Baseline and dissociation were conducted in protein- and ligand- free 0.02% PBST, pH 7.4, at 30 °C for 60 s. Data analysis on the FortéBio Octet K2 apparatus was performed using a double reference subtraction in the FortéBio Data Analysis 7.0 software (Pall FortéBio) and further processed with GraphPad PRISM 8.0 software.

Luciferase reporter assays

The in vitro assay for GR transcriptional activity was quantitated using Hela cells stably transfected with pGL4-MMTV-hygro as previously described [22]. Briefly, Hela-MMTV cells were seeded into 96-well plates at a density of 1 × 104 cells/well. For GR agonistic assays, compounds were serially diluted and added to the cells. For GR antagonistic assay, compounds were serially diluted and added to the cells with a final dexamethasone (DEX) concentration of 10 nM. Mifepristone (MIF) was used as the positive control of GR antagonist. Following 24 h incubation, firefly luciferase activity was immediately measured with a One-Lumi Firefly luciferase Assay Kit (#RG055M, Beyotime) as per the manufacture. Luminescence was measured with Synergy H1 (BioTek, USA).

AR competitor assay

The competitive binding of the tested compounds towards AR was assessed with the PolarScreen Androgen Receptor Competitor Assay Kit, Green (#A15880, Thermo Scientific) as the instruction of the manufacturer.

GR competitor assay

The competitive binding of the tested compounds towards GR was assessed with a LanthaScreen® TR-FRET GR Competitive Binding Assay (#A15901, Thermo Scientific) as per the instruction of the manufacturer.

Cell viability and cell proliferation assays

The MTT colorimetric assay was utilized to evaluate the cell viability and cell proliferation of the compounds. 3 × 103 cells/well NIH3T3, Chang and GES-1, 2 × 103 cells/well DU145, PC3 cells, 8 × 103 cells/well 22Rv1, and 5 × 103 cells/well LNCaP cells were seeded in 96-well plates with RPMI-1640. After incubation at 37 °C for 24 h, cells were treated with gradient concentrations of indicated compounds for 72 h. The MTT colorimetric assay was performed as previously described [31].

Western blot assay

Starved LNCaP cells were cultured in six-well plates with a density of 4 × 105 cells per well. After incubation at 37 °C for 24 h, compounds were added at the designated concentration. After 2 days, the cells were collected and lysed in cell lysis buffer (#P0013, Beyotime). Subsequently, the samples were separated with 12.5% SDS-PAGE (#P0465S, Beyotime), and then electrotransfered to PVDF membrane (#IPFL00005, Millipore). The following antibodies were used for detecting the proteins: AR rabbit mAb (1:2000, #5153, CST), PSA/KLK3 rabbit mAb (1:1000, #5356, CST) and GAPDH (1:1000, #AF1186, Beyotime). Proteins were visualized using anti-mouse (1:5000, #D110087, Sangon) or anti-rabbit HRP-conjugated secondary antibodies (1:5000, #D110058, Sangon) and ECL-Plus (#P0018S, Beyotime).

qRT-PCR

Starved LNCaP cells cultured in medium containing 5% CSS were seeded onto six-well plates at a density of 4 × 105 cells/well. After 24 h, the cells were treated with ENZ or H18 (0.1, 1, or 10 μM) with addition of 5 nM DHT. PSA, TMPRSS2, FKBP5 and AR transcripts were measured using the total RNA isolated (#B618133, Sangon) from the cells already treated by compounds for 48 h by qRT-PCR (QuantStudio 3, Applied Biosystems). All the procedures followed the operation manuals. The fold change in expression of the gene was calculated using the 2−ΔΔCt method with GAPDH as the internal control.

AR nuclear translocation assay

HEK293T cells were seeded in 12-well plates containing coverslips and then transfected with pCMV-hAR. After incubation at 37 °C for 24 h, compounds H18 and Enz were individually added at a final concentration of 10 μM and then further incubated for 12 h, and then DHT (10 nM) was added and incubated for 4 h. Then the procedure of immunofluorescence was performed as previously described [31].

Extraction of nuclear and cytoplasmic fractions

Fractionation and extraction of nuclear and cytoplasmic proteins from LNCaP cells were performed using NE-PER Nuclear and Cytoplasmic Extraction Reagents (#78835, Thermo Fisher) according to manufacturer’s protocol. Starved LNCaP cells were pretreated with 10 μM ENZ or H18 for 2 h before 5 nM DHT treatment for 16 h. The AR protein levels in nuclear and cytoplasmic extracts were analyzed by Western blot analysis using anti-AR antibody (#5153, CST) and normalized to the level of β-tubulin (#2128, CST) or Lamin B1(#AF1408, Beyotime).

Study on AR mutants

PC3 cells were starved in RPMI media with 5% DCC-stripped serum for 2 days before seeding. The PC3 cells (1 × 104 cells/well, 96 well) were then transfected with hARF876L, hARF876L, hARF876L/T877A, hARH874Y/T877A, hART877G, or hARW741C mutant plasmid (50 ng), pGL4-ARR2PB-Luc (20 ng) and 5 ng Renilla (16 h). Cells were treated with increasing concentrations of the test compounds in the absence or presence of 1 nM DHT inducing a submaximal reporter gene activation in steroid-free assay medium and incubated for 24 h. Luciferase activity was measured using a Dual-Lumi™ II Luciferase Assay Kit (#RG089M, Beyotime) as per the manufacture. Luminescence was measured with Synergy H1 (BioTek, USA).

Construction of the structures of AR mutants

The complex of AR LBD_DHT_SSRFESLFAG coactivator peptide (PDB entry: 1T7R) was utilized as the initial structure for the following MD simulation. The binding structures of the AR antagonists HD51 and HD57 were obtained from molecular docking. Then, the structures of AR LBD_HD51, AR LBD_HD57 were superimposed onto that of 1T7R, and HD51 and HD57 were extracted and merged into 1T7R to construct the structures of the WT AR LBD in complex with HD51 (AR_HD51) and HD57 (AR_HD57). The Protein Preparation Wizard in Schrödinger was used to minimize the structures of AR_HD51 and AR_HD57 until the RMSD reached a maximum value of 0.3 Å. The T877G and W741C mutants of the AR LBD were established through the Mutate Panel in Schrödinger, and optimized by Protein Preparation Wizard. Finally, six systems (i.e., AR_HD51, AR_HD57, ART877G_HD51, ART877G_HD57, ARW741C_HD51, and ARW741C_HD57) were prepared for the MD simulations.

Statistical analysis

Data were analyzed, and dose–response curves generated, using GraphPad Prism 8 software (GraphPad Software, La Jolla, CA, USA). A P value < 0.05 was considered significant. Pooled means of triplicate experiments are plotted plus or minus the standard error of the mean.

Results and discussion

Discovery of novel hit compound H18 by SBVS

The general architecture of the AR LBD (Fig. 1a) is similar to that of the GR LBD (Fig. 1b), with a general fold consisting of a three-layered, antiparallel, α-helical sandwich and a β-sheet. To discover novel AR and GR antagonists, a SBVS was performed against the ChemDiv database. The workflow of the SBVS protocol used in this study is presented in Fig. 1c. By applying molecular docking, structural clustering, and druglike analysis, a total of 18 potential AR and GR antagonists (Table S1) were screened out and submitted for bioassays.

Fig. 1: SBVS of AR and GR antagonists.
figure 1

a Overview of the AR LBD (PDB ID: 1Z95) and R-bicalutamide is colored light blue. b Overview of the GR LBD sampled by the MD simulations. The ligand binding site is colored light cyan, and RU486 is colored magenta. c Schematic workflow of SBVS.

At first, the 18 compounds were evaluated by in vitro competitor assays to analyze their relative binding affinities. The active compounds that competed for ligand binding to either AR or GR by at least 50% at 10 μM were selected for constructing dose–response curves. Among these, H18 could displace the fluorophore from the AR LBP in a dose-dependent manner with an IC50 value of 19.42 μM, and it would be a moderate AR binder compared with ENZ (IC50 = 2.424 μM) (Fig. 2a). With respect to the binding affinity towards GR, H18 could displace fluorescently labeled tracer from the LBP pocket of GR and decrease the TR-FRET signal in a dose-dependent fashion with an IC50 value of 1.588 μM (Fig. 2d), ~200-fold higher than that of MIF. To confirm the direct binding of the compounds to AR or GR, biolayer interferometry (BLI) experiments were performed using the biotinylated AR LBD or GR LBD protein. The compounds ENZ and MIF were set as the positive control for AR and GR binding assay, respectively. According to the curve-fitting analysis, the BLI measurement of the binding of H18 to AR showed a binding affinity of 341.4 μM (Fig. 2b), and the binding affinity between ENZ and AR was 8.3 μM (Fig. 2c). Similarly, the BLI measurement of the H18 binding to GR showed a binding affinity of 142.9 μM (Fig. 2e), and the binding affinity between MIF and GR was 45.1 μM (Fig. 2f). Taken together, these data suggested that H18 could directly bind to AR and GR, and it exhibited slightly stronger affinity towards GR than AR.

Fig. 2: H18 directly binds to AR and GR.
figure 2

a H18 could displace a potent fluorescent ligand from the AR-LBP. BLI dose–response curves reflect the direct binding of the compounds b H18 and c ENZ to the AR LBD (n = 4). d H18 could competitively bind to the GR LBP as confirmed by GR TR-FRET assay (n = 4). BLI dose–response curves reflecting the direct binding of the compounds e H18 and f MIF to the GR LBD. All results were presented as means ± SEM.

Biological evaluation of compound H18

H18 showed antagonistic activity toward AR, but not GR

Both AR and GR act as transcription factors in the cell nucleus. Binding of hormones to AR or GR triggers their transcriptional activity, and the hormone-receptor complex binds to the promoter of a specific gene and regulates its expression. Small molecules that can simultaneously block AR and GR transcriptional activity could be used as potential CRPC therapeutic drugs. Thus, H18 was subjected to reporter gene assays to assess its antagonistic activity towards AR and GR. As shown in Fig. 3a, H18 demonstrated inhibition of AR transcriptional activity in a dose-dependent manner, with an IC50 value of 5.858 μM, while the IC50 of ENZ in this assay was 0.038 μM. Furthermore, the inhibitory activity of H18 on secreted PSA, the biomarker of PCa, was measured with the LNCaP-ARR2PB-eGFP cell culture in response to compound treatment. It turned out that H18 also decreased the secreted PSA level in a dose-dependent manner and the IC50 value was 1.033 μM while that for ENZ was 0.078 μM (Fig. 3b). However, H18 showed faint antagonism against GR (Fig. 3c). Antagonism to AR transcriptional activity was further tested using qRT-PCR to quantitate the effect of the compounds on the expression of three typical AR-regulated genes, including TMPRSS2 (a gene that is a frequent target for chromosomal translocation in PCa), KLK3 (PSA) and FKBP5. As is shown in Fig. 3e, H18 did not make significant impact on the AR mRNA level. And relative to the 100% in DHT- treated cells, H18 decreased DHT-induced KLK3 mRNA to 22% at 10 μM and 59% at 1 μM, respectively. The mRNA expression of two other AR-regulated genes FKBP5 and TMPRSS2 was decreased to 53% and 42% by H18 (10 μM), individually. The inhibition of H18 on PSA was also observed at the protein level (Fig. 3f). These results were all consistent with the AR transcriptional activity assay. What’s more, H18 showed essentially no AR agonistic activity or GR agonistic activity (Fig. S1).

Fig. 3: Biological evaluation of the compound H18 identified by SBVS.
figure 3

a H18 exhibited antagonistic activity towards AR. b H18 could suppress PSA secreted in LNCaP-ARR2PB-eGFP cells (n = 3). c H18 exhibited no antagonistic activity towards GR (n = 3). d The antiproliferative effects of H18 on three normal cell lines, including Chang, GES, and 3T3 (n = 3). e H18 significantly decreased the mRNA levels of FKBP5, TMPRSS2, and PSA at 10 μM in the presence of 10 nM DHT in LNCaP cells, but had no effect on the AR mRNA level. P value < 0.05 was considered significant (*) compared with DHT control (n = 3). f H18 could decrease the protein levels of PSA and had no effect on the AR protein expression in LNCaP cells as observed by Western blot analysis. g The antiproliferative effects of H18 on four PCa cell lines, including LNCaP, PC3, DU145, and 22Rv1 (n = 3). h Nuclear and cytoplasmic extracts collected from LNCaP cells were subjected to Western blot analysis. β-tubulin was used as the control for the cytoplasmic fraction and Lamin B was used as the control for the nuclear fraction. i Fluorescence microscopy on 293T cells transfected with AR that were pretreated with DMSO, ENZ, and H18, respectively. Green stain is AR and blue stain is DAPI (for nucleus). Scale bar, 25 μm. All results were presented as means ± SEM.

Cell proliferation inhibitory activity of H18

To exclude inherent cytotoxicity, the influence of H18 on the proliferation of normal cells was evaluated. As shown in Fig. 3d, H18 exhibited minor effect on mouse fibroblasts cell line NIH3T3, human liver cells (Chang), and human gastric epithelial cell (GES-1), demonstrating its safety profile. To confirm the antagonistic activity of H18, we examined its effects on the proliferation of prostate cancer cells including AR-dependent prostate cancer cell line (LNCaP) that requires androgens for growth, androgen-independent/androgen-responsive cell line (22Rv1) that does not require androgens but exhibits growth dependence in the presence of androgens and express high level of AR variants, and AR-independent prostate cancer cell lines (PC3 and DU145) that do not require androgens for growth and their growth is not affected by androgens. As a result, H18 inhibited LNCaP in a dose-dependent manner, but had inconsiderable effects on the proliferation of 22Rv1, PC3, and DU145 (Fig. 3g), indicating its selective inhibition on AR signaling pathway.

H18 prevented DHT-induced AR nuclear translocation

An important feature of the second-generation antagonists represented by ENZ is their ability to affect AR nuclear translocation, in addition to their enhanced antagonistic activity. Whether H18 could affect the DHT-induced translocation of AR from the cytoplasm to the nucleus was evaluated by two distinct approaches. At first, Western blot analysis of the endogenous AR levels in biochemically enriched nuclear and cytoplasmic extracts revealed an increase in AR nuclear/cytoplasmic (N/C) ratio upon the addition of DHT in LNCaP cells (Fig. 3h). In contrast, the DMSO and ENZ groups showed little sign of AR in the nucleus, and H18 was also found to reduce the AR N/C ratio in the presence of DHT. Therefore, we reasoned that H18 could inhibit endogenous AR translocation like most second-generation antagonists. Secondly, the real-time nuclear localization of exogenously transfected AR in 293T cells evaluated using fluorescence microscopy showed that the immunofluorescence staining of transfected AR was mainly located in the cytoplasm in the DMSO group (Fig. 3i). Treatment with DHT induced AR translocation from the cytoplasm to the nucleus, which however was effectively blocked by ENZ. As similar with the ENZ group, relatively more fluorescent signals were observed in the cytoplasm rather than the nucleus in the H18-treated group, indicating that both ENZ and H18 could block DHT-induced exogenous AR translocation to the nucleus. In summary, H18 can reduce DHT-mediated translocation of both exogenous and endogenous AR into the nucleus.

Computationally guided structural optimization of H18

To guide further structural optimization of H18, molecular dynamics (MD) simulations were performed to predict the dynamic binding behavior of H18 to the AR/GR LBD. The RMSDs of the heavy atoms of H18 were calculated to monitor the stability of the complexes during the MD simulations. As shown in Fig. 4a, the RMSD evolutions for the heavy atoms of H18 in the AR LBD and GR LBD tended to converge after ~100 ns and ~250 ns, respectively, with the fluctuations within 1 Å, suggesting that the complexes reached stability via 500 ns MD simulations. Subsequently, the snapshots from 450 to 500 ns were extracted from the MD trajectories for the subsequent structural and energetic analyses. The key residues for the binding of H18 to the AR LBD were highlighted by the MM/GBSA free energy decomposition, and the top-ranked 10 residues are Met745, Leu704, Asn705, Met742, Thr877, Met749, Phe764, Met895, Val746, and Gly708 (Fig. 4b). According to structural analysis of H18 (Fig. 4c), the 5,6-dihydro-4H-pyrrolo[3,2,1-ij]quinolin-2(1H)-one moiety (Group A) is located at the hydrophobic pocket formed by Met742, Met745, Val746, Met749, and Phe764. Group A connected to the trifluoromethyl-benzene moiety (Group B) via a sulfonamide linker. The sulfonamide formed hydrogen bonds with Asn705 and Thr877 of the AR LBD to stabilize the flexible hinge region. Then the binding poses of R-bicalutamide and H18 were aligned to further characterize the binding difference between R-bicalutamide and H18. As is shown in Fig. 4d, although R-bicalutamide and H18 share most key residues in the binding pocket, R-bicalutamide could form stronger interactions with the AR LBD than H18 due to the extra hydrogen bonds with the residue Arg752. It can be speculated that the antagonism of H18 mainly depends on the hydrophobic interaction, and the antagonism gap between H18 and R-bicalutamide is probably due to the lack of important hydrogen bond of H18 with Arg752. In search of the essential residues responsible for the binding of H18 to the GR LBD, the per-residue decomposition based on MM/GBSA was carried out as shown in Fig. 4e. It can be found that the major contributors were Asn564, Tyr735, Gln642, Met646, Gly567, Cys736, Phe623, Met604, Leu608, and Leu566. Most of them were hydrophobic residues around H18. The predicted binding mode of H18 in the LBP site of the GR LBD was shown in Fig. 4f. The binding modes for H18 with AR/GR suggested that H18 binds to the AR or GR LBP site in a similar way with its “Group A” buried in the hydrophobic subsite, and the sulfonamide linker formed a hydrogen bond with the target. Considering that Group A is close to most of the key residues, especially Arg752, we need to investigate whether the introduction of different substituents in Group A could make it closer to Arg752 to increase the probability of the formation of a hydrogen bond and whether the hydrogen bond could enhance the antagonistic potency as expected.

Fig. 4: MD simulations and key residues in the AR LBD and GR LBD for the binding of H18.
figure 4

a RMSDs of the heavy atoms of H18 as a function of the simulation time. b The ten top-ranked residues in the AR LBD are responsible for the binding of H18 predicted by MM/GBSA. c Structural analysis of the ten top-ranked residues in the AR LBD to the binding of H18. d Alignment of the structures of H18 (green) and R-bicalutamide (blue) bound to the AR LBD. The residues that form hydrogen bond interactions were highlighted. e The ten top-ranked residues in the GR LBD are responsible for the binding of H18 predicted by MM/GBSA. f Structural analysis of the ten top-ranked residues in the GR LBD to the binding of H18. g Rational design of the H18 analogs.

To validate the hypothesis and explore the structure–activity relationships of H18, rational design based on the novel scaffold was conducted as illustrated in Fig. 4g. First, Series A was identified by switching the positions of sulfur and nitrogen atoms in the necessary sulfonamide to explore the more favorable linker. Then, Series B–D were identified to confirm the necessity of Group A and its interaction with Arg752. A total of 58 analogs (Table S2) were identified from the ChemDiv library with similarity- and substructure-based searching and their bioactivities were evaluated. As shown in Table 1, antagonism and competitive binding affinity for AR and GR are lost in most compounds of Series A. Compared with Series A, Series B and C showed no significant improvement in comparison with H18 despite several compounds retained some activity (Tables 2 and 3). However, Series D with an extended ring of Group A exhibited significant increase of bioactivity, especially AR antagonistic activity. Of the 11 compounds in Series D (Table 4), six compounds suppressed AR transcriptional activity with an average inhibition rate of 53.65%. Among these, HD51 and HD57 showed prominent binding affinities in both AR and GR competitor assays (Fig. 5a) and then were selected for further analysis.

Table 1 Biological evaluation of the Series A analogs of H18.
Table 2 Biological evaluation of the Series B analogs of H18.
Table 3 Biological evaluation of the Series C analogs of H18.
Table 4 Biological evaluation of the Series D analogs of H18.
Fig. 5: Biological evaluation of H18 analogs.
figure 5

a The competitive abilities of H18 analogs to the AR and GR LBP (n = 4). b Competitive binding affinities of compound HD51 and HD57 to the AR LBP (n = 4). c HD51 and HD57 exhibited antagonistic activity towards AR (n = 3). d HD51 and HD57 could suppress PSA secreted in LNCaP-ARR2PB-eGFP cells (n = 6). e Competitive binding affinities of compounds HD51 and HD57 to the GR LBP (n = 4). f HD51 and HD57 exhibited antagonistic activity towards GR (n = 3). g HD51 and HD57 showed no transactivation activity towards GR (n = 3). The antiproliferative effects of h ENZ, i HD51, and j HD57 on four PCa cell lines, including LNCaP, PC3, DU145, and 22Rv1 (n = 3). All results were presented as means ± SEM.

Biological evaluation of HD51 and HD57

The relative binding affinities of HD51 and HD57 to AR and GR were evaluated using in vitro competitor assay. Both compounds competed for ligand binding to AR with significantly improved IC50 values of 3.596 μM and 5.176 μM, respectively (Fig. 5b). The affinities are at the same level as enzalutamide (IC50 = 2.424 μM). With respect to the affinity toward GR, HD51 and HD57 were relatively stronger binders with IC50 values of 1.017 μM and 0.554 μM, respectively (Fig. 5e). The potential for HD51 and HD57 to exhibit AR and GR antagonism or agonism were evaluated using in vitro reporter gene assays. In LNCaP-ARR2PB-eGFP cell-based eGFP assay, HD51 and HD57 exhibited improved AR antagonistic activity with IC50 values of 0.841 and 0.394 μM, respectively (Fig. 5c). In particular, the antagonism of HD57 towards AR is 15-fold higher than that of H18 and only eightfold lower than that of ENZ. HD51 and HD57 were also proved to be more effective in decreasing the secreted PSA level compared to H18 (Fig. 5d). As for GR antagonism, H18 showed no antagonistic activity against GR despite its strong binding affinity to GR as discussed above. However, HD51 and HD57 at 10 μM could decrease the luciferase signal induced by GR to 25% and 39%, respectively, compared with 100% of MIF (Table 4). Therein, HD57 exhibited modest GR antagonistic activity with an IC50 value of 17.81 μM (Fig. 5f). Compared with DEX, HD51 and HD57 did not show agonistic potency associated with the main side effects of GR modulators (Fig. 5g). As is shown in Fig. 5h-j, HD51 and HD57 showed no considerable antiproliferation effects on PC3, DU145, and 22Rv1, while the dose-response curves for ENZ, HD51 and HD57 on LNCaP cells were quite similar, which is consistent with their AR antagonistic activities.

Binding mode analysis of HD57 with AR/GR LBD

To understand the molecular basis of HD57, MD simulations were performed to investigate the interaction between the AR/GR LBD and HD57. The snapshots of the last 50 ns were extracted from the MD trajectories for the subsequent structural and energetic analyses. As shown in Fig. 6a, the ten top-ranked residues for the binding of HD57 to the AR LBD were similar to those of H18. Structural analysis suggested HD57 could form hydrophobic interactions with Met745, Met742, Phe764, Ile899, and Met749. In addition, the polar interaction with Thr877, Leu704, and Arg752 effectively stabilize the conformation of HD57 in the binding pocket (Fig. 6b). To further characterize the binding difference between H18 and HD57, the binding spectra of H18 and HD57 were compared (∆∆G = ∆GHD57 – ∆GH18). The negative values indicate that these residues form stronger interactions with HD57 than H18. It can be observed that residues of Leu704, Arg752, Phe764, Thr877, and Ile899 contribute more to HD57 than H18 (Fig. 6c), indicating the improved potency of HD57 compared with H18 is mainly dependent on the hydrophobic interactions surrounded by Group B of HD57 in addition to the polar interactions including the carbonyl group bonding with Arg752 and sulfonamide linker bonding with Leu704. Similar structural and energetic analyses were also performed to investigate the interaction between the GR LBD and HD57 (Fig. 6d). The predicted binding mode of HD57 and the GR LBD reveals that the hydrophobic interactions and the hydrogen bond interactions (residues Asn564 and Gln738) play crucial roles in stabilizing the binding of HD57 to the GR LBD (Fig. 6e). The key residues including Phe737, Asn564, Gln738, Leu563, Gly567, Met604, Phe623, Leu608, Leu566, and Val571, among which Lys563, Met604, Phe737, Gln738, and Lys743 contribute more to HD57 than H18 (Fig. 6f). This may be one of the reasons why HD57 has better antagonism toward GR than H18.

Fig. 6: MD simulations and key residues in the AR LBD and GR LBD for the binding of HD57.
figure 6

a The ten top-ranked residues in the AR LBD are responsible for the binding of HD57 predicted by MM/GBSA. b Structural analysis of the ten top-ranked residues in the AR LBD to the binding of HD57. c Energetic differences between HD57 and H18 in the AR LBD. d The ten top-ranked residues in the GR LBD are responsible for the binding of HD57 predicted by MM/GBSA. e Structural analysis of the ten top-ranked residues in the GR LBD to the binding of HD57. f Energetic differences between HD57 and H18 in the GR LBD.

HD51 and HD57 suppressed the activity of clinically relevant AR mutants

Gain of function mutations has been observed in the LBD of AR in CRPC. A panel of mutated AR variants was previously reported in the literature or clinically detected in CRPC patients after treatment with ENZ, abiraterone, BIC, or docetaxel. One particular AR mutation is T877A, which has been linked to the development of resistance to first-generation antiandrogen hydroxyflutamide and reportedly promote prostate cancer cell growth and cell survival [32]. It has been predicted that the T877G mutation would confer resistance to enzalutamide [33], although the mutation has not been identified clinically yet. And the mutations W741C/L have been found in LNCaP cells and they have acquired resistance to BIC [34]. The H874Y mutation along with the T877A mutation was detected in circulating cell-free DNA of 7 out of 29 patients resistant to abiraterone and allow AR to be activated by cortisol. The F876L mutation has been detected in plasma DNA from men with CRPC treated with ENZ [35] and ARN509 [36].

To test whether these AR mutants induced under the selective pressure of AR pathway inhibition are cross-resistant to compound HD51 or HD57, we investigated the transcriptional activity of ENZ, DARO, BIC, HD51, and HD57 against the five typical mutants including ARF876L, ARF876L/T877A, ARH874Y/T877A, ARW741C, and ART877G using a luciferase reporter assay in AR-negative PC3 cells. As shown in Fig. 7a, ENZ showed explicit transcriptional agonism to both ARF876L and ARF876L/T877A, partial agonism towards the ARH874Y/T877A and ART877G mutants and full antagonism to ARW741C. Darolutamide demonstrated generally very potent transcriptional inhibition on all the tested variants. The first-generation BIC leads to enhanced AR transcriptional activity for the ARW741C mutant and behaves as a partial antagonist in the AR variant with H874Y/T877A. Interestingly, compounds HD51 and HD57 behaved similarly to darolutamide in inhibiting most AR variants, except that HD57 exhibited no antagonism towards the ART877G mutant.

Fig. 7: Study on AR mutants.
figure 7

Transcriptional activity of ENZ, DARO, BIC, HD51, and HD57 towards the a ARF876L, b ARF876L/T877A, c ARH874Y/T877A, d ART877G, and e ARW741C mutants (n = 3). All results were presented as means ± SEM.

Considering that compounds HD51 and HD57 showed distinct agonistic and antagonistic properties on the ART877G mutant, MD simulations were performed to gain more atomic insights into the modes of action of HD51 and HD57. The ARW741C mutant was taken as a control, as both HD51 and HD57 maintained antagonistic activity against ARW741C. Six systems (i.e., AR_HD51, AR_HD57, ART877G_HD51, ART877G_HD57, ARW741C_HD51, and ARW741C_HD57) were constructed and submitted to 500 ns MD simulations for each system (Fig. S2). It was reported that the binding free energies between the coactivator and agonistic ARs are lower than those between the coactivator and antagonistic ARs, suggesting that the energetic variance between AR and coactivator could be used to predict the agonistic/antagonistic propensities of small molecules bound to the AR LBD [37]. As shown in Fig. S2, the RMSD of the coactivators fluctuated greatly, especially in the AR_HD51 and AR_HD57 systems. The snapshots from 450 to 500 ns were extracted for the subsequent structural and energetic analyses. The binding free energies of coactivators (∆GTif) predicted by MM/GBSA in ART877G_HD51 and ARW741C_HD51 were −48.04 and −46.97 kcal/mol, respectively, which were higher than that of AR_HD51 (−54.96 kcal/mol), indicating that HD51 still inhibited the AR-coactivator interactions for the ART877G and ARW741C mutants (Table S3). While a strong AR-coactivator interaction was observed in the ART877G_HD57 system (∆GTif = −62.28 kcal/mol), which was significantly lower than those of AR_HD57(∆GTif = −49.43 kcal/mol) and ARW741C_HD57 (∆GTif = −39.39 kcal/mol). This was consistent with the loss of antagonistic activity of HD57 against AR with the T877G mutation.

The binding free energy of HD51 was increased in ART877G than of AR (Table S3). And the binding modes of HD51 were similar between AR and ART877G (Fig. 8a). The amino group of HD51 interacts with the side chain of Leu873 in both AR and ART877G, and HD51 established an extra hydrogen bond with Arg752 in the T877G AR complex, explaining why HD51 maintained its antagonism towards ART877G. In contrast, HD57 adopted a different binding pose in the ligand binding site of the ART877G mutant, and the ligand binding free energy (∆Glig) was decreased (Table S3). A large group of residues including Leu704, Asn705, Trp741, Met745, Met749, Phe764, Met895, and Ile898 contributes more to the HD51 binding to AR than ART877G (Fig. 8a). In addition, HD57 formed a key hydrogen bond with Asn705 of the AR backbone, while the sulfonamide of HD57 did not form hydrogen bonds with ART877G, only the carbonyl group formed a hydrogen bond with Gln711 (Fig. 8b). The mutant protein-ligand complexes demonstrate the altered interactions with antiandrogens in contrast to the wild-type AR that may correlate with the agonist feedback. Instead, HD51 (Fig. 8c) and HD57 (Fig. 8d) maintained their binding conformations in the W741C-mutated LBP through hydrogen and van der Waals interactions.

Fig. 8: Regulation of AR LBD-ligand interactions by T877G and W741C mutations.
figure 8

Energy differences and structural analyses of the AR LBD (light pink) and T877G-mutated AR LBD (green) in complex with a HD51 and b HD57, respectively. Energy differences and structural analyses of the AR LBD (light pink) and W741C-mutated AR LBD (mint green) in complex with c HD51 and d HD57, respectively.

HD51 and HD57 could combat representative AR mutations resulting from the long-term clinical use of first- and second-generation antiandrogens. More importantly, our data suggested that none of the tested AR mutants induced under the selective pressures of AR pathway inhibition is cross-resistant to compound HD51. Our data support further preclinical studies of HD51 and HD57 on enzalutamide-resistant CRPC patients [38].

Conclusions

AR and GR are similar in many aspects, including their sequences, protein structures, activation modes, and target genes, etc. It is theoretically feasible to design AR-GR dual antagonists considering that the LBPs of AR and GR are highly conserved. And the expression level of AR and GR are highly correlated in the development and progression of Pca. In the present study, a SBVS was performed to identify dual antagonists towards AR and GR. A total of 18 potential hits were selected and subjected to bioassays. Among these compounds, H18 showed optimal binding affinity with both AR and GR, but only exhibited antagonistic activity against AR. In addition, H18 could also influence the nuclear localization of exogenous and endogenous AR. According to the structural and energetic analyses of H18, we designed the A-D series of analogs to further improve the activity of H18. Therein Series D showed improved binding affinities and antagonisms to both AR and GR. Analogs HD51 and HD57 not only exhibited highly improved binding affinity and antagonistic activity towards AR but also effectively suppressed the transcriptional activity of various clinically relevant mutants that confer resistance to current antiandrogens. What’s more, the activity of both HD51 and HD57 on GR was distinctly improved compared to the originally designed H18. Although the activities against AR and GR are unbalanced, HD51 and HD57 are the first group of effective non-steroidal AR-GR dual antagonist reported, are suitable lead compounds for further optimization. Our study is a representative and successful case of identifying dual antagonists targeting the LBP of both AR and GR by combining computation and biological evaluation, and may offer alternative therapeutic strategies for CRPC treatment.