A standard database for drug repositioning

Drug repositioning, the process of discovering, validating, and marketing previously approved drugs for new indications, is of growing interest to academia and industry due to reduced time and costs associated with repositioned drugs. Computational methods for repositioning are appealing because they putatively nominate the most promising candidate drugs for a given indication. Comparing the wide array of computational repositioning methods, however, is a challenge due to inconsistencies in method validation in the field. Furthermore, a common simplifying assumption, that all novel predictions are false, is intellectually unsatisfying and hinders reproducibility. We address this assumption by providing a gold standard database, repoDB, that consists of both true positives (approved drugs), and true negatives (failed drugs). We have made the full database and all code used to prepare it publicly available, and have developed a web application that allows users to browse subsets of the data (http://apps.chiragjpgroup.org/repoDB/).


Background & Summary
Drug repositioning is the process of discovering, validating, and marketing previously approved drugs for new indications. The drug repositioning field is growing rapidly, due to the promise of reduced costs and expedited approval schedules 1 . Unsurprisingly, the number of publications in PubMed with the text 'drug repositioning' in their abstracts has ballooned from only 11 articles per year in 2007 to 274 in 2015. Prevalent among repositioning publications are computational methods, which perform in silico experiments to determine the most promising repositioning candidates for further preclinical testing 2,3 . Computational repositioning methods have been developed that use a variety of direct and indirect evidence for hypothesis generation, including molecular 4-7 , literature-derived [8][9][10][11] , and clinical 12,13 data. In many computational repositioning methods papers, authors claim that their methods are analytically validated in some way. Authors typically present either case studies, in which they describe a single wellsupported example, or sensitivity-and specificity-based analyses to support their claims. Sensitivity-and specificity-based methods rely on comparing the full spectrum of predictions made by a repositioning method to currently approved or investigational drug-indication pairs 14 .
It is difficult, however, to directly compare and/or independently assess computational methods or reported new repositioning candidates due to the variety of analytic validation methodologies favored by different groups (or impossible if only case studies are provided). Furthermore, studies that claim to use unbiased validation methods like predictive 'area under the receiver-operator curve' (AUROC) 15 rely on true, approved drug-indication pairs only, and typically assume that all other drug-indication pairs are false. This assumption is unsatisfying because it relies on a database of approved drugs (the choice of which varies widely in the repositioning literature 14 ), and suggests that all novel repositioning predictions are false.
To address these concerns, we present repoDB, a database of approved and failed drugs and their indications. repoDB approved indications were drawn from DrugCentral, which contains United Medical Language System (UMLS) indications mapped from free-text mentions in drug labels 16,17 . The UMLS is a large biomedical thesaurus that contains information about a wide variety of medical concepts 16 . Failed indications were drawn from the American Association of Clinical Trials Database (the 'AACT Database', Clinical Trials Transformation Initiative, 2016), which contains structured records from the National Library of Medicine's ClinicalTrials.gov database service. Indications in the AACT database are again annotated using medical subject heading (MeSH) terms (a subset of UMLS terms), and represent a mix of investigator-submitted and automatically extracted annotations (see Table 1 for database characteristics and Fig. 1a for an overview of our methodology) 18 . repoDB spans 1,571 drugs and 2,051 UMLS disease concepts, accounting for 6,677 approved and 4,123 failed drug-indication pairs (see Table 2 and Fig. 1b for trial status breakdown). To further assist investigators, we provide a web application (http://apps. chiragjpgroup.org/repoDB/) that enables browsing of the full repoDB database and allows users to download either the full database (or portions relevant to their work). repoDB will enable investigators to not only benchmark their computational repositioning methods, but also gain insight into trends in the drug discovery field and avenues that have not yet been explored.

Approved indication retrieval
As our source of information on currently approved drugs and their indications, we downloaded the full DrugCentral PostgreSQL database, and extracted the tables containing DrugBank identifiers, synonyms for all drugs, and UMLS-mapped indication terms (http://drugcentral.org/, DrugCentral [Full PostgreSQL Database], Data Citation 1) 17 . DrugCentral provides comprehensive information about approved and investigational drugs, including UMLS-mapped approved indication(s) and, important for the construction of repoDB, all synonyms for a given drug. DrugCentral uses the OMOP annotation pipeline to map free text drug labels to UMLS terms, which achieves high annotation accuracy (F1 measures around 0.98) 19 . We retrieved all DrugCentral synonyms for all Food and Drug Administration of the United States (FDA) approved drugs. A list FDA approved drugs was derived from DrugBank, a large drug database that is commonly used by computational drug repositioning methods and is frequently updated with new information (see Fig. 1a) 20 . contains structured clinical trial records from the National Library of Medicine's ClinicalTrials.gov service, and includes information about current trial status and interventions (e.g., drugs, life-style changes) studied in each trial. We chose to use AACT/ClinicalTrials.gov as our source for trial information because the sponsors of most failed trials do not publish their results in the scientific literature (around 78% fail to publish) 21 . We loaded and parsed the full database in R statistical programming environment 22 , and took only those trials that included: 1) an annotated phase between phase 0 and phase 3, 2) a current, overall status of suspended, terminated, or withdrawn, and 3) a MeSH  term-mapped intervention (provided by AACT), and 4) a UMLS term-mapped indication (provided by investigators and/or MetaMap analysis of free-text trial descriptions). While the majority of terms are derived from investigator supplied UMLS terms, ClinicalTrials.gov supplements these using the NLM Medical Text Indexer (MTI) to map text to high confidence MeSH/UMLS (F1 measure around 0.55) 23 . We mapped all annotated interventions to DrugCentral synonyms and excluded trials that were not mappable to at least one approved drug (Fig. 1a). Indication information was mapped to UMLS identifiers using the UMLS REST API 16 .

Final database compilation
As the final step in creating the repoDB database, we reconciled the approved and failed indication information. We removed all failed trial information for drug-indication pairs that were currently approved: for example, metformin is an FDA-approved drug for diabetes mellitus; there are, however, trials marked as terminated with metformin as a primary intervention (e.g., metformin combination therapies, see NCT00762957) and these trials were removed. After combining the approved and failed indications, we kept only those drug-indication pairs for which the indication fell within a UMLS semantic type related to disease ('Disease or Syndrome', 'Neoplastic Process', 'Pathologic Function', 'Finding', 'Mental or Behavioral Dysfunction', 'Sign or Symptom', 'Injury or Poisoning', 'Congenital Abnormality', 'Acquired Abnormality', and 'Cell or Molecular Dysfunction'). Semantic types describe broad categories of disease as well as other medicine-related concepts; it is therefore necessary to filter out non-disease terms, including those with semantic types such as, 'Health Care Related Organization.' See Supplementary Table 1 for the highest frequency terms by semantic type. The final database (see Fig. 2) was used to create an interactive R/Shiny application (http://apps.chiragjpgroup.org/repodb) whose contents are available for download ( Code availability R code used to (1) pre-process DrugCentral and AACT, (2) compile the final repoDB database, and (3) deploy the repoDB R shiny application is available from figshare (repoDB Production Code, Data Citation 1) and from GitHub (https://github.com/adam-sam-brown/repoDB).

Data Records
We

Technical Validation
The drug-indication pairs provided herein are derived from automated annotations of FDA-approved drug labels (in DrugCentral) and investigator-submitted clinical trial records (in AACT). More information about the accuracy of the annotations can be found in the methods, as well as in publications describing the Medical Text Indexer and the Observational Medical Outcomes Partnership (F1 measures of 0.55 and 0.98, used in AACT and DrugCentral respectively) [17][18][19]23 . We note here that the methods upon which repoDB relies are not the only tools available for named entity recognition in the medical field. Other databases of drug information may therefore vary widely in terms of the granularity (e.g., 'diabetes' versus 'maturity onset diabetes of the young, type I,' among others) of the indication information contained therein.  By using the repoDB database, users agree to cite both our work, as well as both AACT and DrugBank for their role in data curation. This data is available under a Creative Commons Attribution 4.0 International License (see https://creativecommons.org/licenses/by/4.0/ for details).