Genomic Data Resources: Challenges and Promises

Figure 1: The phenomenal growth of sequence data in GenBank is challenging to manage, and continues unabated.

The number of database resources that organize and display the data are also increasing rapidly. The rise is concurrent. This figure shows the number of base pairs deposited in GenBank from December 1982 to August 2008 (on left axis). The right axis shows the number of resources listed from 1996 to 2008 in the Nucleic Acids Research database issues. In 1999, databases and resources listed in current and previous database issues were compiled in one online directory. (Obtained from GenBank release notes: ftp://ftp.ncbi.nih.gov/genbank/gbrel.txt and list size obtained from NAR archives: http://nar.oxfordjournals.org/archive/).

© 2008 Nature Education All rights reserved.

Computer databases are an increasingly necessary tool for organizing the vast amounts of biological data currently available and for making it easier for researchers to locate relevant information. In 1979, the Los Alamos Sequence Database was established as a repository for biological sequences. In 1982, this database was renamed GenBank and, later the same year, moved to the newly instituted National Center for Biotechnology Information (NCBI), where it lives today. By the end of 1983, more than 2,000 sequences were stored in GenBank, with a total of just under 1 million base pairs (Cooper & Patterson, 2008).

At about the same time, a joint effort between NCBI, the European Molecular Biology Laboratory (EMBL), and the DNA Databank of Japan (DDBJ) created the International Nucleotide Sequence Database Collaboration (INSDC) to collect and disseminate the burgeoning amount of nucleotide and amino acid sequence data that was becoming available. Since then, the INSDC databases have grown to contain over 95 billion base pairs, reflecting an exponential growth rate in which the amount of stored data has doubled every 18 months (Figure 1). The advent of next-generation sequencing technologies, metagenomics, genome-wide association studies (GWAS), and endeavors such as the 1000 Genomes Project will only increase the tremendous volume and complexity of this and other sequence data collections (Siva, 2008).

The sheer volume of the raw sequence data in these repositories has led to attempts to reorganize this information into various kinds of smaller, specialized databases. Such databases include various genome browsers, model organism databases, molecule- or process-specific databases, and others. To get an understanding of the growth of these resources, one need only look at the annual database issue of the journal Nucleic Acids Research. In one of the first database issues—the one in which GenBank is described—only a few dozen databases are listed (Bilofsky et al., 1986). In contrast, the latest database issue describes over 1,000 genomics databases and tools (Galperin, 2008). However, even this list of resources is only part of the overall picture. Today, it appears that there are upwards of 3,000 distinct genomic resources, tools, and databases publicly available on the Internet. This article outlines the current state of such databases, reviewing some of the databases available for use, various issues concerning the growth of these databases, and the number and sheer size of these resources.

As previously mentioned, the INSDC is a collaboration of NCBI's GenBank in the U.S., EMBL in Europe, and the DDBJ in Japan. Each of these databases accepts direct submissions of biological sequences from individual researchers, from sequencing projects, and from patent applications from around the world. Sequences are entered into the database and given a unique identification or accession number. These submitted entries are stored in a "library" of records, and each entry is "owned" by—and can only be updated by—its submitter. The data integrated in these entries include the submitter's name, the originating organism, the definition, the actual sequence, related references, and more. (Examples of IDs and and entries are accessible through these links.) The submitted entries are then shared across the three repositories on a daily basis, and releases of the data are made regularly. This has been a boon to the research community, facilitating the sharing of sequence data and allowing the advancement of research. These sequence repositories have become the universal, comprehensive, and authoritative resources for the exponentially growing amount of sequence data currently available to researchers.

However, there are challenges associated with these repositories. For one, exponential growth makes it difficult to maintain accuracy and accessibility across the three databases. Although researchers are able to update sequences they have submitted to GenBank and other repositories, a large portion of the stored data may be incorrect or incomplete due to the volume of the submitted information and the nature of research (e.g., researchers move on to other projects, mistakes in the original data go unnoticed, etc.). There are also issues of duplication with minor variations and redundancy. Because of this, several programs and efforts have been developed to help correct and curate sequence data. (Pruitt et al., 2005). RefSeq is a database developed and maintained at NCBI that aims to provide a scientist-curated nonredundant set of biological sequences. Here, the data are propagated from sequences deposited within the GenBank database and then carefully annotated using community collaboration, automated computer annotation, and NCBI staff curation. This provides a reference set of well-curated sequences. Related efforts include (a repository for high quality, frequently updated, manual annotation of vertebrate finished genome sequences), , "a collaborative effort to identify a core set of human and mouse protein coding regions that are consistently annotated" [National Center for Biotechnology Information, 2008]), and other projects that utilize the INSDC archival database sequences and build on them to create curated, accurate sequence data. Note, however, that these projects cannot tackle every species and sequence in each repository, especially because sequence data is produced faster than researchers can annotate and organize it.

Another challenge associated with genome repositories is that repository data rarely have context or annotation, nor are they organized in a manner that facilitates research. For example, a gene record might be linked to references for what the gene does, but images showing localization in the cell or primary data showing potential protein modifications might only be accessible in research papers and not databases. This means that much of the data are difficult to access and utilize. Given this situation, database developers have taken the data from GenBank and repurposed it to their specific aesthetic, project history, annotation, and community needs. As a result, there are currently many databases and strategies for presenting and providing access to genomic data. The main categories of such databases are described in the sections that follow.

Figure 2

Figure Detail

One highly useful answer to the needs of many researchers is the development of general genome browsers. Three successful examples include the University of California, Santa Cruz (UCSC) Genome Browser, EBI's Ensembl, and NCBI's MapViewer (Karolchik et al., 2008; Flicek et al., 2008; Wheeler et al., 2008b). These browsers repackage genome and gene annotation data sets from GenBank and other subject-specific databases to provide a genomic context for individual genome features, such as genes or disease loci. Furthermore, not only do these browsers provide genomic context, but they also allow users to see common formats between diverse species so that information is more easily viewable and extractable. For example, a user can search for a specific region of a genome, such as a disease gene, and the sequence and attendant annotation will be displayed visually. This display will then link out to additional data and databases for further study, as well as back to repository sources for the original data (Figure 2).

This framework, based on the official genomic sequence, also permits the addition of new data and data types organized according to the sequence. For instance, new ENCODE data that maps regulatory elements and many other DNA elements can be easily placed on this existing structure to enhance researchers' understanding of any given genomic region (ENCODE Project Consortium, 2007). These genome browsers also offer advanced searching capabilities, such as Table Browser (UCSC) and BioMart (Ensembl), which allow researchers to obtain and analyze data in a highly customized fashion (Karolchik et al., 2005; Fernandez-Suarez & Birney, 2008). These customized searches and data downloads can be used as input for further bioinformatics analyses or experiments to verify computational predictions in the lab.

Because of historical, biological, and practical reasons, data are not completely consistent between species genomes and research projects. Model organism databases often have unique schemas that are not easily comparable to those used in databases for other species. Indeed, the terminology, analysis techniques, and importance attached to different sequence elements and annotations can be quite different across databases. The genome browsers mentioned above are one solution to this problem. However, another option has emerged to provide deeper and broader data for individual species' genomes, as well as increased standardization that allows for better cross-species comparisons and greater ease of use.

In particular, the consortium has worked on the development of an open-source standard database and set of visualization tools to make querying, browsing, and using genome databases similar for all species. GMOD has collaboratively developed a set of tools and database schema that include an annotation editor (Apollo), a genome browser (GBrowse), pathway tools, an advanced search capability (BioMart), a biological database schema (Chado), and additional resources that allow species research communities to develop databases that are standard and compatible across genomes. The goal of these efforts is to facilitate research and comparative studies.

Many species- and taxa-specific genome databases have made use of this standard, open-source set of database tools. The RGD, TAIR, Gramene, FlyBase, MGI, SGD, and WormBase databases are just a few. Sometimes, these tools are the main foundation of a database; in other cases, they supplement existing databases. Organisms with smaller research communities can also use these handy tools to create the annotation, visualization, and query options they need. As the rate of genome sequencing continues to increase thanks to new technologies, the GMOD tools may prove to be a boon for researchers who need to better explore their sequences of interest. Note that these tools can support many data types. In fact, there are many diverse and creative examples of ways in which the GMOD tools can be used, such as the (HGSDD), human variation data, and even personal genomics in the form of Watson's genome (Cheung & Estivill, 2003; International HapMap Consortium, 2003; Wheeler et al., 2008a). You can find a large and varied list of resources that use these tools by accessing the GMOD website.

In addition to species- or genome-oriented databases, there are also databases organized by almost any biological data category one can imagine. For example, there are databases specifically for protein domain information (Pfam) and protein structure information (PDB). There are also repositories and databases of expression data, such as NCBI's Gene Expression Omnibus (GEO) and EBI's ArrayExpress (Berman & Westbrook, 2000; Barrett et al., 2006; Parkinson et al., 2007). In addition, GWAS databases such as dbGaP and HuGE Navigator are emerging (Yu et al., 2008). The list of subject-specific databases is quite large—as mentioned earlier, there are over 3,000 such resources—and the variety of these "focused" databases is as unlimited in scope as the data they contain. Nonetheless, this large number of species- and subject-specific databases, though extremely useful, can lead to its own issues of redundancy and lack of integration.

All of the aforementioned resources, from the respositories to the genome databases and subject-specific databases, are increasingly faced with the challenge of ensuring accurate data and efficiently managing and curating that data. Recently, several solutions have been proposed (Waldrop, 2008; Howe et al., 2008). These solutions range from a greater focus on the education of database biocurators in learning institutions and the standardized inclusion of sequence data and references in publications to "community curation." One community curation solution envisions a sort of "wikification" of data update and curation, in which research communities curate their databases themselves. This has been proposed for repositories, specifically GenBank, as well as for focused resources, such as model organism databases (Pennisi, 2008; Salzberg, 2007).

GenBank has resisted this "wikification" proposal for various reasons, feeling that the current system allows an authoritative repository and a database of record and that community editing might diminish this strength. Additionally, there are already programs and efforts aimed at correcting and curating the sequence data, such as RefSeq (mentioned earlier in this article). As for model organism databases, there are currently several relatively successful efforts at community curation and annotation, including the Daphnia Genomics Consortium wiki and several other extensive undertakings. However, these efforts are hampered by factors such as the reliability of curation, the lack of incentives for researchers to contribute, and more. As the authors of a recent paper in Nature suggest, "To date, not much of the research community is rolling up its sleeves to annotate" (Howe et al., 2008).

Various efforts at building archives, databases, and analysis tools have proven successful at facilitating a better understanding of the genomes of multiple species. They have offered researchers authoritative repositories, contextual information, and curated data as a method of handling the exponentially growing amount of sequence data. Although these resources have been useful and have solved many issues, they will continue to face new types and ever-growing amounts of data that will exacerbate the challenges with which the research community is already faced. For example, genome-wide association studies will generate an enormous amount of data that will provide insight into the multifactorial genetic origins of disease, evolution, and more.

These data have also created new challenges related to the development of methods for visualizing and searching information. Recently, unforeseen privacy issues have required that large datasets be removed from public databases (Couzin, 2008; Zerhouni & Nabel, 2008) because personal genetic information could be associated to individuals. Metagenome sequencing projects, which analyze communities of genomes instead of individual genomes, are also creating large, complicated data sets that require unique tools and databases (Markowitz et al., 2008a; National Research Council Committee on Metagenomics, 2007).

Lastly, but importantly, the growing number of genome databases, analysis tools, and other resources available on the web has made it daunting for researchers to use these resources effectively. Even with efforts toward standardization and documentation, researchers continue to find it difficult to locate and learn to use these resources (Collins & Green, 2003). Solutions involving advanced, life-long training on the use and access of specific resources must be found. Next-generation sequencing and personal genomics will further burden efforts in this arena.

In spite of the challenges that have arisen with the growth of data and databases, the rewards and opportunities provided by this information have proven fruitful. Today, there is a wealth of data that was undreamed of just a couple of decades ago, enabling new discoveries and uncovering new relationships between different disciplines. The authors often joke to their students that if these resources had been available when they were in graduate school just 15 years ago, it would have taken them months—not years—to complete their degrees. With the growth of available data and resources in the next few years, amazing discoveries will continue to be made if the scientific community can meet the challenge.