To plan the syntheses of small organic molecules, chemists use retrosynthesis, a problem-solving technique in which target molecules are recursively transformed into increasingly simpler precursors. Computer-aided retrosynthesis would be a valuable tool but at present it is slow and provides results of unsatisfactory quality. Here we use Monte Carlo tree search and symbolic artificial intelligence (AI) to discover retrosynthetic routes. We combined Monte Carlo tree search with an expansion policy network that guides the search, and a filter network to pre-select the most promising retrosynthetic steps. These deep neural networks were trained on essentially all reactions ever published in organic chemistry. Our system solves for almost twice as many molecules, thirty times faster than the traditional computer-aided search method, which is based on extracted rules and hand-designed heuristics. In a double-blind AB test, chemists on average considered our computer-generated routes to be equivalent to reported literature routes.
Subscribe to Journal
Get full journal access for 1 year
only $3.90 per issue
All prices are NET prices.
VAT will be added later in the checkout.
Rent or Buy article
Get time limited or full article access on ReadCube.
All prices are NET prices.
Clayden, J., Greeves, N., Warren, S. & Wothers, P. Organic Chemistry 2nd edn (Oxford Univ. Press, 2008)
Brückner, R. Reaktionsmechanismen: Osganische Reaktionen, Stereochemie, Moderne Synthesemethoden (Springer, 2014)
Robinson, R. LXIII. A synthesis of tropinone. J. Chem. Soc. Trans. 111, 762–768 (1917)
Corey, E. & Cheng, X. The Logic of Chemical Synthesis (Wiley, 1989)
Kurti, L. & Czakó, B. Strategic Applications of Named Reactions in Organic Synthesis (Elsevier, 2005)
Evans, J. in The Oxford Handbook of Thinking and Reasoning (eds Holyoak, K. J. & Morrison, R. G.) 115–133 (Oxford Univ. Press, 2012)
Collins, K. D. & Glorius, F. A robustness screen for the rapid assessment of chemical reactions. Nat. Chem. 5, 597–601 (2013)
Ley, S. V., Fitzpatrick, D. E., Ingham, R. & Myers, R. M. Organic synthesis: march of the machines. Angew. Chem. Int. Ed. 54, 3449–3464 (2015)
Schneider, P. & Schneider, G. De novo design at the edge of chaos: miniperspective. J. Med. Chem. 59, 4077–4086 (2016)
Segler, M. H., Kogej, T., Tyrchan, C. & Waller, M. P. Generating focussed molecule libraries for drug discovery with recurrent neural networks. ACS Cent. Sci. 4, 120–131 (2018)
Vléduts, G. Concerning one system of classification and codification of organic reactions. Inform. Storage Retrieval 1, 117–146 (1963)
Todd, M. H. Computer-aided organic synthesis. Chem. Soc. Rev. 34, 247–266 (2005)
Szymkuć, S. et al. Computer-assisted synthetic planning: the end of the beginning. Angew. Chem. Int. Ed. 55, 5904–5937 (2016)
Cook, A. et al. Computer-aided synthesis design: 40 years on. Wiley Interdiscip. Rev. Comput. Mol. Sci. 2, 79–107 (2012)
Ihlenfeldt, W.-D. & Gasteiger, J. Computer-assisted planning of organic syntheses: the second generation of programs. Angew. Chem. Int. Edn Engl. 34, 2613–2633 (1996)
Fick, R. Konzepte zur Syntheseplanung: Strukturelle Ähnlichkeit und Strategische Bindungen. PhD thesis, Friedrich-Alexander-Universität (1996)
Ugi, I. et al. Models, concepts, theories, and formal languages in chemistry and their use as a basis for computer assistance in chemistry. J. Chem. Inf. Comput. Sci. 34, 3–16 (1994)
Kayala, M. A., Azencott, C.-A., Chen, J. H. & Baldi, P. Learning to predict chemical reactions. J. Chem. Inf. Model. 51, 2209–2222 (2011)
Minsky, M. A Framework for Representing Knowledge. Technical Report (Massachusetts Institute of Technology, 1974)
Bøgevig, A. et al. Route design in the 21st century: the ICSYNTH software tool as an idea generator for synthesis prediction. Org. Process Res. Dev. 19, 357–368 (2015)
Law, J. et al. Route designer: a retrosynthetic analysis tool utilizing automated retrosynthetic rule generation. J. Chem. Inf. Model. 49, 593–602 (2009)
Christ, C. D., Zentgraf, M. & Kriegl, J. M. Mining electronic laboratory notebooks: analysis, retrosynthesis, and reaction based enumeration. J. Chem. Inf. Model. 52, 1745–1756 (2012)
Segler, M. H. & Waller, M. P. Neural-symbolic machine learning for retrosynthesis and reaction prediction. Chemistry 23, 5966–5971 (2017)
Boda, K., Seidel, T. & Gasteiger, J. Structure and reaction based evaluation of synthetic accessibility. J. Comput. Aided Mol. Des. 21, 311–325 (2007)
Ertl, P. & Schuffenhauer, A. Estimation of synthetic accessibility score of drug-like molecules based on molecular complexity and fragment contributions. J. Cheminform. 1, 8 (2009)
Coulom, R. Efficient selectivity and backup operators in Monte-Carlo tree search. In Int. Conf. on Computers and Games 72–83 (Springer, 2006)
Kocsis, L. & Szepesvári, C. Bandit based Monte-Carlo planning. In 17th Eur. Conf. on Machine Learning 282–293 (Springer, 2006)
Browne, C. B. et al. A survey of Monte Carlo tree search methods. IEEE Trans. Comput. Intell. AI Games 4, 1–43 (2012)
Sutton, R. S. & Barto, A. G. Reinforcement Learning: An Introduction 2nd edn (MIT Press, in the press)
Coulom, R. Computing “elo ratings” of move patterns in the game of go. ICGA J. 30, 198–208 (2007)
Stern, D., Herbrich, R. & Graepel, T. Bayesian pattern ranking for move prediction in the game of Go. In Int. Conf. on Machine Learning 873–880 (Omni Press, 2006)
Maddison, C. J., Huang, A., Sutskever, I. & Silver, D. Move evaluation in Go using deep convolutional neural networks. In 3rd Int. Conf. on Learning Representations (2015); preprint at https://arxiv.org/abs/1412.6564
Clark, C. & Storkey, A. Training deep convolutional neural networks to play Go. In 32nd Int. Conf. on Machine Learning 1766–1774 (PMLR, 2015); http://proceedings.mlr.press/v37/clark15.html
Winands, M. Neural networks for video game AI. In Artificial and Computational Intelligence in Games: Integration (Dagstuhl Seminar 15051) Vol. 5 (eds Lucas, S. M. et al.) 224 (2015)
Silver, D. et al. Mastering the game of Go with deep neural networks and tree search. Nature 529, 484–489 (2016)
Reaxyshttp://www.reaxys.com (Elsevier Life Sciences, 2017)
Srivastava, R. K ., Greff, K. & Schmidhuber, J. Training very deep networks. In Advances in Neural Information Processing Systems 2377–2385 (MIT Press, 2015); preprint at https://arxiv.org/abs/1507.06228
Clevert, D.-A., Unterthiner, T. & Hochreiter, S. Fast and accurate deep network learning by exponential linear units (ELUs). In 4th Int. Conf. on Learning Representations (2016); preprint at https://arxiv.org/abs/1511.07289
Sheridan, R. P. Time-split cross-validation as a method for estimating the goodness of prospective prediction. J. Chem. Inf. Model. 53, 783–790 (2013)
Marcou, G. et al. Expert system for predicting reaction conditions: the Michael reaction case. J. Chem. Inf. Model. 55, 239–250 (2015)
Segler, M. H. & Waller, M. P. Modelling chemical reasoning to predict and invent reactions. Chemistry 23, 6118–6128 (2017)
Coley, C. W., Barzilay, R., Jaakkola, T. S., Green, W. H. & Jensen, K. F. Prediction of organic reaction outcomes using machine learning. ACS Cent. Sci. 3, 434–443 (2017)
Murphy, K. P. Machine Learning: a Probabilistic Perspective (MIT Press, 2012)
Nirogi, R. V., Badange, R., Reballi, V. & Khagga, M. Design, synthesis and biological evaluation of novel benzopyran sulfonamide derivatives as 5-HT6 receptor ligands. Asian J. Chem. 27, 2117–2124 (2015)
Lake, B. M., Ullman, T. D., Tenenbaum, J. B. & Gershman, S. J. Building machines that learn and think like people. Behav. Brain Sci. 40, 1–101 (2016)
Sierra, M. A. & de la Torre, M. C. Dead ends and detours en route to total syntheses of the 1990s. Angew. Chem. Int. Ed. 39, 1538–1559 (2000)
Rocktäschel, T. & Riedel, S. End-to-end differentiable proving. In Advances of Neural Information Processing Systems (eds Guyon, I. et al.) 3788–3800 (Curran Associates, 2017); https://papers.nips.cc/paper/6969-end-to-end-differentiable-proving
Peng, Q., Duarte, F. & Paton, R. S. Computing organic stereoselectivity—from concepts to quantitative calculations and predictions. Chem. Soc. Rev. 45, 6093–6107 (2016)
Lin, A. I. et al. Automatized assessment of protective group reactivity: a step toward big reaction data analysis. J. Chem. Inf. Model. 56, 2140–2148 (2016)
Gini, A., Segler, M., Kellner, D. & Garcia Mancheno, O. Dehydrogenative tempo-mediated formation of unstable nitrones: easy access to n-carbamoyl isoxazolines. Chemistry 21, 12053–12060 (2015)
Andersen, J. L., Flamm, C., Merkle, D. & Stadler, P. F. Generic strategies for chemical space exploration. Int. J. Comput. Biol. Drug Des. 7, 225–258 (2014)
Steinbeck, C. et al. Recent developments of the chemistry development kit (CDK)-an open-source Java library for chemo- and bioinformatics. Curr. Pharm. Des. 12, 2111–2120 (2006)
Landrum, G. RDKit: Open-Source Cheminformaticshttp://www.rdkit.org
Silver, D. Reinforcement Learning and Simulation-Based Search. PhD thesis, Univ. Alberta (2009)
Reymond, J.-L., Ruddigkeit, L., Blum, L. & van Deursen, R. The enumeration of chemical space. Wiley Interdiscip. Rev. Comput. Mol. Sci. 2, 717–733 (2012)
Färber, M ., Kaliszyk, C. & Urban, J. Monte Carlo connection prover. Preprint at https://arxiv.org/abs/1611.05990 (2016)
Rosin, C. D. Multi-armed bandits with episode context. Ann. Math. Artif. Intell. 61, 203–230 (2011)
Winands, M. H., Björnsson, Y. & Saito, J.-T. Monte-Carlo tree search solver. In Int. Conf. on Computers and Games 25–36 (Springer, 2008)
Schneider, N., Lowe, D. M., Sayle, R. A. & Landrum, G. A. Development of a novel fingerprint for chemical reactions and its application to large-scale reaction classification and similarity. J. Chem. Inf. Mod. 55, (2015)
Coley, C. W., Rogers, L., Green, W. H. & Jensen, K. F. Computer-assisted retrosynthesis based on molecular similarity. ACS Cent. Sci. 3, 1237–1245 (2017)
Gelernter, H., Rose, J. R. & Chen, C. Building and refining a knowledge base for synthetic organic chemistry via the methodology of inductive and deductive machine learning. J. Chem. Inf. Comput. Sci. 30, 492–504 (1990)
Rose, J. R. & Gasteiger, J. Horace: an automatic system for the hierarchical classification of chemical reactions. J. Chem. Inf. Comput. Sci. 34, 74–90 (1994)
Liu, B. et al. Retrosynthetic reaction prediction using neural sequence-to-sequence models. ACS Cent. Sci. 3, 1103–1113 (2017)
Kingma, D.P. & Ba, J. ADAM: a method for stochastic optimization. In 3rd Int. Conf. for Learning Representations; preprint at https://arxiv.org/abs/1412.6980 (2015)
Chollet, F. et al. Kerashttps://github.com/fchollet/keras (2015)
The Theano Development Team Theano: a Python framework for fast computation of mathematical expressions. Preprint at https://arxiv.org/abs/1605.02688 (2016)
Rogers, D. & Hahn, M. Extended-connectivity fingerprints. J. Chem. Inf. Model. 50, 742–754 (2010)
Wei, J. N., Duvenaud, D. & Aspuru-Guzik, A. Neural networks for the prediction of organic chemistry reactions. ACS Cent. Sci. 2, 725–732 (2016)
Socorro, I. M. & Goodman, J. M. The ROBIA program for predicting organic reactivity. J. Chem. Inf. Model. 46, 606–614 (2006)
Satoh, H. & Funatsu, K. Sophia, a knowledge base-guided reaction prediction system—utilization of a knowledge base derived from a reaction database. J. Chem. Inf. Comput. Sci. 35, 34–44 (1995)
Patel, H., Bodkin, M. J., Chen, B. & Gillet, V. J. Knowledge-based approach to de novo design using reaction vectors. J. Chem. Inf. Model. 49, 1163–1184 (2009)
Zhang, Q.-Y. & Aires-de Sousa, J. Structure-based classification of chemical reactions without assignment of reaction centers. J. Chem. Inf. Model. 45, 1775–1783 (2005)
Polishchuk, P. et al. Structure–reactivity modeling using mixture-based representation of chemical reactions. J. Comput. Aided Mol. Des. 31, 829–839 (2017)
Carrera, G. V., Gupta, S. & Aires-de Sousa, J. Machine learning of chemical reactivity from databases of organic reactions. J. Comput. Aided Mol. Des. 23, 419–429 (2009)
Neese, F. The ORCA program system. WIREs Comput. Mol. Sci. 2, 73–78 (2012)
Butina, D. Unsupervised data base clustering based on Daylight’s fingerprint and Tanimoto similarity: a fast and automated way to cluster small and large data sets. J. Chem. Inf. Comput. Sci. 39, 747–750 (1999)
Parsy, C. C. et al. Discovery and structural diversity of the hepatitis C virus NS3/4a serine protease inhibitor series leading to clinical candidate IDX320. Bioorg. Med. Chem. Lett. 25, 5427–5436 (2015)
M.H.S.S. and M.P.W. thank the Deutsche Forschungsgemeinschaft (SFB858) for funding. M.H.S.S. and M.P.W. also thank D. Evans (RELX Intellectual Properties) and J. Swienty-Busch (Elsevier Information Systems) for the reaction dataset. We thank all AB-test participants in Shanghai and Münster, and J. Guo for assistance in AB testing. M.H.S.S. thanks M. Wiesenfeldt, the Studer group, D. Barton, S. McAnanama-Brereton, R. Vidyadharan and T. Kogej for discussions. M.P. thanks M. Winands and J. Togelius for insights.
The authors declare no competing financial interests.
Reviewer Information thanks D. Duvenaud, W. H. Green and the other anonymous reviewer(s) for their contribution to the peer review of this work.
Publisher's note: Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
Extended data figures and tables
The area under the curve is 0.99.
Extended Data Figure 2 An exemplary 10-step synthesis route for a complex intermediate in a drug synthesis.
It resembles the published route77 (with intermediates A and B) and was found by our algorithm autonomously within 30?s. The target was not contained in the training set.
Extended Data Figure 3 Example of reaction used in the AB testing, where the MCTS-derived route was less favoured.
In this task, the participants preferred the literature solution, as its key step was presumably perceived to be more convergent.
The output logit score of the neural network correlates surprisingly well with calculated quantum-mechanical properties (LUMO energies, in Hartree) in Diels–Alder reactions (r2?=?0.74) (a) and with empirically measured Hammond parameters in electrophilic brominations (r2?=?0.78) (b), even though the input features (ECFP4 fingerprints) do not contain electronic information.
This file contains the DOE, route diversity analysis, and failed molecules (including Supplementary Figures 1-7, Supplementary Table 1 and Supplementary References). Available on figshare (DOI 10.6084/m9.figshare.5832054) are 2 files, mcts_examples.pdf which contains routes found by the 3N-MCTS algorithm and heuristicBFS_examples.pdf which contains routes found by heuristic best first search without policy network and in-scope filter. (PDF 692 kb)
This file contains the AB test. (PDF 5648 kb)
This file contains the experiment for correlating the in-scope filter to physicochemical properties. (XLSX 45 kb)
About this article
Cite this article
Segler, M., Preuss, M. & Waller, M. Planning chemical syntheses with deep neural networks and symbolic AI. Nature 555, 604–610 (2018). https://doi.org/10.1038/nature25978
Journal of Chemical Information and Modeling (2020)
IEEE Journal on Emerging and Selected Topics in Circuits and Systems (2020)
Active learning of many-body configuration space: Application to the Cs+–water MB-nrg potential energy function as a case study
The Journal of Chemical Physics (2020)
Advanced Therapeutics (2020)
Journal of Medicinal Chemistry (2020)