Main

The burgeoning of public repositories of experimentally derived genomic1,2 and transcriptomic3,4,5 data and the concomitant increases in protein sequence6 and integrated7,8 databases are well documented, providing to the scientific community a rich set of resources that has enabled more rapid advances in the understanding of gene function than would otherwise have been possible. Practitioners of proteomics—the direct study of sets of proteins occurring together in particular parts or states of biological entities (that is, proteomes)—are now beginning to share data through communal resources9,10,11,12,13,14. This raises challenging issues: for example, should datasets contain a level of description beyond what is found in the 'average' published paper; how can experimentalists be supported in collecting and transmitting such information; and what might the benefits of more systematic reporting be, for individuals and the wider community?

The role of experimental metadata

There are many different subsets of the 'total' (that is, all parts and all states) proteome of an organism, just as there are many related patterns of gene transcription, each distinguished both by cell type and by condition. Furthermore, protein identifications obtained by mass spectrometry, for example, are dependent on the separation technologies employed, the particular mass spectrometer, and the protein identification tool and database with which identities were assigned. To understand an analysis, perform comparisons between datasets, or derive statistics from their aggregation, it is crucial to understand both the biological and the methodological contexts. Inadequate description can allow inappropriate experimental design and random or systematic errors to go undetected. Conversely, confidence in data and data analysis can be increased by, for example, reporting the performance of appropriate calibration runs15 or using power analyses to support the particular study design16.

Proteomics data should therefore ideally be accompanied by contextualizing 'metadata' (essentially 'data about the data'), making explicit both where samples came from and how analyses were performed. To that end, the Proteomics Standards Initiative (PSI; http://www.psidev.info)17,18 develops guidance documents specifying the data and metadata that should be collected from various proteomics workflows (Box 1), known collectively as the “minimum information about a proteomics experiment” (MIAPE) guidelines.

There are several precedents in the biomedical sciences for such reporting prescriptions. The “minimum information about a microarray experiment” (MIAME) guidelines19, which deal specifically with transcriptomics data, has become an accepted community standard; the original paper had been cited in >1,100 published papers (source: Google Scholar), many of which describe MIAME-compliant software development. Several major journals now require that papers reporting transcriptomics experiments are accompanied by the MIAME-prescribed set of metadata, either as supplementary information supplied to the journal or in the form of a database submission, as a prerequisite for publication20,21,22. MIAME demonstrably facilitates the reuse of data from 'compliant' repositories such as ArrayExpress4 in new research23,24, although there are still issues with respect to the willingness25 or abili-ty26,27 of authors to submit fully compliant datasets. Other examples of established reporting requirements include CONSORT28 (randomized controlled trials), QUORUM29 (meta-analyses of randomized, controlled trials), STARD30 (assessment of the accuracy of diagnostic tests), REMARK31 (tumor-marker prognostic studies) and the Clinical Data Interchange Standards Consortium's SEND32 (non-clinical toxicology). Studies of some of these standards33,34 have found that they increase the quality of reports without incurring a disproportionate cost for users. Other sets of reporting requirements are under development across the biosciences. For example, some fifteen ongoing projects (including MIAPE) are now listed on the website of a central registry of reporting guidelines, the “minimal information for biological and biomedical investigations” (MIBBI)35 (http://mibbi.sf.net)—a clear demonstration of the appetite in the bioscience community for the regularization of reporting.

MIAPE: principles, process and product

Proteomics workflows frequently consist of sample collection and processing, separation by liquid chromatography or some form of electrophoresis, examination of separands by mass spectrometry, identity assignment, and absolute or relative quantification of proteins through bioinformatic analysis of the mass spectra generated36,37. However, there are many technologies available, each enabling the analysis of different (although frequently overlapping) 'subproteomes' differentiated by mass, surface charge or localization in time or space. These various technologies, many of which are still evolving, produce a dauntingly diverse array of data. Additionally, the future holds the promise of many new technologies: improved prefractionation and depletion techniques, evolving array- and bead-based technologies, and new types of mass spectrometer components. The volume and complexity of (meta)data potentially available from most proteomics experiments makes the issue of what to keep and what to discard an important one. To guide specific decisions on the data and metadata that should be required by each MIAPE module, we employ two very general criteria:

  1. 1

    Sufficiency. The MIAPE guidelines should require sufficient information about a dataset and its experimental context to allow a reader to understand and critically evaluate the interpretation and conclusions, and to support their experimental corroboration.

  2. 2

    Practicability. Achieving compliance with MIAPE should not be so burdensome as to prohibit its widespread use.

The resulting guidelines, about which some general information is provided in Box 2, are reminiscent of the schema underlying a laboratory information management system (LIMS) in that they require not only data but also metadata that are relevant to the discovery and interpretation of the results.

The development of MIAPE documents

This document exists to make explicit the scope, purpose and manner of use of the modular MIAPE guidelines that accompany it, and to lay out the principles underlying module production. This document should be stable, as the principles described herein are technique independent and should therefore remain valid for the foreseeable future. The associated modules will be more labile; they will both evolve over time and increase in number, to track changes in experimental techniques. We recognize, however, that it is just as desirable to have stable reporting guidelines as it is to have stable data formats. There will, therefore, be a period of at least one year between successive versions of any one module.

Initial versions of all modules are generated through the PSI, drawing on community expertise gathered at PSI and other meetings, through the discussion lists, and through direct interactions with experts in the relevant techniques. These 'candidate' documents then enter the PSI document process, where they are subject to public comment and (anonymous) review, including tests by experimentalists in the lab of the feasibility of collecting the required information, and are finally submitted for publication in a peer-reviewed journal. Throughout their development and after their release, modules can be discussed in a number of ways: (i) on e-mail discussion lists (details on the website, http://www.psidev.info); (ii) on the scheduled open teleconferences held by the working groups; and (iii) by attending one of the free annual meetings. Conflicts arising in relation to extant or emerging modules should be resolved through the discussion lists and teleconferences where possible. If consensus cannot be achieved informally then issues will be resolved at the next annual meeting, by vote if necessary.

The MIAPE modules

Each MIAPE module relates to a particular technology or group of technologies. The first modules that will be deployed are briefly described in Box 1. The most recent versions of all modules are available from the MIAPE home page on the PSI website (http://www.psidev.info/miape). Where a particular (newly developed or legacy) technique is not yet covered by MIAPE, authors should attempt to match existing modules in terms of depth of coverage; ideally the expansion of an existing module or the development of a new module for that technique should then be raised through the available discussion lists (as described at http://www.psidev.info).

'Shared' modules

The number of ongoing reporting requirements projects raises a concern about overlap between independently generated standards. For example, were MIAPE and MIAME to offer differing prescriptions for describing study organisms, the subsequent integration of proteome and transcriptome datasets would be unnecessarily complicated. We understood this early in the development process; thus, the PSI has focused on techniques that are more or less specific to proteomics rather than very general areas, such as the description of the biological material under study (which has relevance for many areas of bioscience). However, there are two complicating factors. First, proteomics, metabolomics and the like are not completely discrete entities; for example, mass spectrometry is as much a tool of metabolomics as it is of proteomics. Second, different kinds of study often require different levels of detail; for example, with respect to the husbandry of animal study subjects, dietary information is key for metabolomics but relatively unimportant for genomic sequencing. Reporting requirements for all technologies, protocols or entities that have relevance for many kinds of bioscience should therefore be developed in common between the relevant standards bodies (or by way of representative collaborations if no official standards body exists). In many cases, a 'tiered' solution should be sought (for example, for genomic sequencing, identify the source of the organism only; for proteomics or metabolomics, also give feeding schedule; and so forth). To address all of these concerns, the PSI has become an active participant in the MIBBI project35, which aims to anticipate or remedy such overlaps between sets of requirements.

Cui bono?

We now address the question most famously asked by the Roman orator Cicero: “For whose benefit?” The cost of acceding to requests for richer annotation of proteomics data falls almost completely on the shoulders of the experimentalists generating the data, so what is the justification for their increased expenditure of time and resources?

Several groups stand to benefit from the acceptance of MIAPE by the proteomics community: users of 'MIAPE-compliant' public repositories ('data consumers'), public-sector data producers, and the private sector. Researchers acting as data consumers (whether in the public or private sector) stand to benefit because:

  • Data sets generated by specific techniques can be easily identified and retrieved (or excluded).

  • Data can be used for purposes different from those for which the data were generated.

  • Data and analyses can be assessed in the light of the methods deployed.

  • Protocols associated with high-quality data can be more easily retrieved.

  • Sufficient information will be available to enable parallel or orthogonal studies to confirm or refute a given result.

However, experimentalists are often under severe time, budget and productivity constraints. They must be assured that they too will reap direct benefits, not just the kudos of enhancing the publicly available corpus of biological data. For public-sector data producers the direct benefits accruing to the routine capture of MIAPE-specified (meta)data include:

  • Straightforward, regularized promotion of new protocols and best practice to others.

  • Obviation of the need to repeatedly construct sets of appropriate contextualizing information for a project:

    • Facilitates the sharing of data with collaborators.

    • Avoids the risk of loss of information through staff turnover.

    • Enables time-efficient handover of projects from one researcher to another.

  • Support for the assessment of results that may have been generated months or even years ago (perhaps in answer to referees' comments during the process of publishing work).

  • The performance of better-informed comparisons of datasets, increasing the likelihood of discovering the factors (both controlled and uncontrolled) that might differentiate them. Importantly, this includes the discovery of sources of systematic or random error by correlating data with metadata features such as the equipment used or the date or operator.

  • The ability to aggregate proteomics datasets in an informed manner, and to combine proteomics data with data from other kinds of study (for example, correlating changes in mRNA and protein expression levels in response to a stimulus).

All of the above cases are affected by decisions on what information to capture and keep from an experiment and what to ignore or dispose of; MIAPE is relevant to such decisions, because it incorporates the views of many experimentalists considering diverse issues.

Private sector interest in MIAPE has slightly different drivers. The pharmaceutical industry, for example, is accustomed to archiving all the data and metadata generated in the course of an experiment under the US Food and Drug Administration's 21 CFR Part 11 requirements. So arguments based on the benefits of capturing particular kinds of (meta)data do not apply. For that sector the issue is mainly one of efficiency: capturing a reduced set of metadata in a rigorous way (in addition to archiving the full set) facilitates more efficient retrieval, reanalysis and integration of data. The validity of this argument is supported by the abundance of sophisticated software designed to distill all-encompassing datasets into useful summary reports.

Support from funding agencies and journals

At present, projected benefits for data consumers may not directly justify the use of data producers' time, largely because of the mechanisms by which publicly funded researchers are assessed—usually through publications. However, this situation is changing. Funders around the world see that potential value is being squandered through failures to maximize the utility of datasets, which are frequently expensive to produce. Many already have data sharing policies that direct those receiving funds to make their data publicly available. Some go further; for example, the UK Biotechnology and Biological Sciences Research Council (BBSRC) has now finalized a policy statement38 that not only requires plans to be established for the provision of access to datasets that were generated in the course of BBSRC-funded work (as many other funders, such as the US National Institutes of Health and the National Science Foundation, also do), but also require adherence to agreed community standards, where they exist. This stated aim of having data made publicly available in a manner compliant with community standards will enable researchers to seek support for time spent annotating and depositing their data.

Journals already support and in some cases drive initiatives to establish reporting guidelines, the aim being to ensure that reviewers and readers alike have access to sufficient information to make informed judgments about the content of papers. As stated above, several journals20,21,22 already require that authors of transcriptomics papers satisfy the MIAME19 requirements. Other journals, such as Molecular and Cellular Proteomics39,40 and more recently Proteomics41, have themselves driven processes to generate appropriate guidelines. However, both those journals' requirements incorporate guidance on appropriate experimental processes, whereas MIAPE simply requires the provision of sufficient information to allow quality to be independently assessed (for example, as a part of the review process). It has always been a matter of policy that the PSI should neither attempt to produce standard operating procedures specifying how particular techniques should be performed nor attempt to establish quality assessment benchmarks. We do not believe it is the job of this body to dictate to the proteomics community how it should perform experiments or analyses.

Ultimately, we anticipate that the publication of a proteomics experiment will involve (i) the submission of a 'classical' paper for the print or electronic version of the journal; (ii) the submission of optional supplementary data to the journal, providing further analysis; and (iii) the submission of a supporting dataset, annotated to the level specified by MIAPE, to a MIAPE-compliant database. This scenario has two benefits: first, the foundations of conclusions presented in papers would routinely be made available for inspection; second, journals would retain the right to determine the level of detail appropriate for the paper while still requiring compliance with MIAPE. For this situation to obtain, it is clear that MIAPE should endeavor to include all the (non-quality-related) components of guidelines such as those produced by Molecular and Cellular Proteomics and Proteomics. This is already the case for developed MIAPE modules, as analyses conducted by the PSI have established.

File formats, controlled vocabularies, tools and databases

The information requested by MIAPE modules could be captured and transmitted in any number of ways; a typical workflow is outlined in Figure 1. MIAPE is an implementation-independent description of information that should be transmitted along with the core data from an experiment. MIAPE is not a data format (such as an XML or a tab-delimited text file), nor is it a source of 'controlled vocabulary' terms (well characterized, consensus terms for use predominantly in data files). Furthermore, neither the MIAPE modules nor this document make the use of any particular file format or any other resource a condition of MIAPE compliance; this is important, as to require the use of a particular informatics framework could, for example, exclude those who would wish to comply with MIAPE in a purely commercial setting where a LIMS stores data using a proprietary format. However, the PSI also develops data formats and vocabularies (see http://www.psidev.info); a specific 'use case' for these is the transmission of MIAPE-specified information. Note also that although there is no requirement in MIAPE that data must be made publicly accessible, we endorse public access to data as being of benefit to science.

Figure 1: An example of MIAPE-compliant data management.
figure 1

(1) Data and metadata are generated by an experiment; (2) some form of software collects the data and metadata, either by importing from computer-controlled instruments or from manual data entry; (3) MIAPE specifies the data and metadata to be requested by the software tool; (4) a controlled vocabulary supplies classifiers via the software; (5) the software uses a data format specification when exporting a MIAPE-compliant dataset; (6) the dataset is stored in a MIAPE-compliant database and assigned an accession number; (7) a paper, including the appropriate accession number, is published in a journal.

Full size image

Neither journals nor funders will move to require MIAPE compliance until tools and databases are in place to ease data capture and support data sharing. The prevalence of computer technology in the lab is a boon here; instrument manufacturers and LIMS and analysis software vendors have shown continued interest in MIAPE (evidenced by the regular attendance of representatives of such companies at the PSI's public meetings). Support from such commercial vendors will greatly simplify the reporting process.

Tool support is important for the success of guidelines such as MIAPE. The policy initiatives of various funders and the position taken by many journals has now created a market for commercial software that, first, supports the capture of the required data either directly from instruments or by acting as an 'electronic lab book' and, second, supports the export of MIAPE-compliant reports (that is, data files of some description containing everything required by the guidelines). Ideally such software will be able to export standard data formats, such as those developed by the PSI.

Of course the funds required to purchase commercial software may not be available to all; only once appropriate free tools become available will it be reasonable to expect that experimentalists comply fully with MIAPE and its kin. It is not unrealistic to expect such tools to appear. Substantial tool development can be achieved by the public sector, as shown by projects such as CPAS42, and public funders look ever more favorably on projects that aim to develop appropriate tools to support data sharing. The PSI will provide, as a base solution, specially designed Microsoft Excel spreadsheets similar to the ProteomeHarvest tool that has proved so successful as a submission route for the PRIDE database12, one per MIAPE module, to assist in capturing the MIAPE-specified set of data and metadata. These Excel spreadsheets are intended to be used only for low volumes of data, such as the data supporting a paper. For situations where legacy software or large data volumes make MIAPE compliance challenging, the PSI will encourage the development of appropriate tools. We anticipate that both free and commercial software will track standards as they are produced.

Having collected the data and metadata from an experiment and encoded it in a PSI format using the appropriate controlled vocabulary, experimentalists will need a place to deposit those data. As stated above, there are now a number of broadly appropriate databases9,10,11,12,13,14, although at present only PRIDE12 can store the level of description MIAPE requires in a structured manner. MIAPE should no more require the use of any particular repository than it should require the use of any one format, although the PRIDE repository will endeavor to support PSI standards as they emerge.

Finally, we consider the issue of validating whether an allegedly MIAPE-compliant dataset actually contains all that MIAPE specifies. Were a journal to require MIAPE compliance, reviewers would benefit from assistance in checking that datasets submitted in support of publications contain the required information. Repositories can help in this; as a likely destination for datasets supporting unpublished work, they can put mechanisms in place to check for compliance upon submission of a dataset and approve that dataset on the reviewers' behalf. This kind of service has recently been offered27,43 in the context of transcriptomics by the ArrayExpress4 and GEO5 repositories.

Conclusion

One of the main objectives of the MIAPE process is to increase the value derived by the scientific community from ongoing experimentation in proteomics, through community processes that support sharing, dissemination and reanalysis of datasets, and that assist in establishing and promoting best practice in specific technical areas. Guidelines such as these promote transparency in experimental reporting, enhance accessibility to data and support effective quality assessment, thereby increasing the general value of a body of work (and by extension the competitiveness of the originators of that work).

The MIAPE guidelines require a fairly rich description without being overly burdensome: much of the required data should be readily available in electronic form and therefore amenable to export, especially as vendors of instruments, analysis software and LIMS implement standards-compliant export facilities. Additionally, a substantial portion of the captured metadata will be common to many experiments, permitting some economies of scale through reuse. MIAPE therefore provides a sound base for developers of repositories and tools to work from, and a rational framework for journals and funders to consider enforcing.

Several benefits will arise from the widespread acceptance of MIAPE. Compliant datasets will contain sufficient information to quickly establish the provenance and relevance (to the researcher) of a dataset. Additionally, tools will be developed that afford easy access to, and analysis of, large numbers of such datasets. Tool development will be facilitated by standardized XML-based data transport formats and controlled vocabulary terms generated by the PSI. The MIAPE modules constitute a set of 'requirements documents' for such development. Details of all PSI standards development projects can be found on the project website (http://www.psidev.info).

In this age of genome- and proteome-scale experiments, the need to standardize the content of reports of biological experiments is evident if we are to extract full value from our activities44,45. It is our hope that this document and the modules accompanying it will begin to fulfill this need for proteomics researchers and for the proteomics community as a whole, increasing the value of both individual pieces of work and of the general, diverse corpus to which so many contribute.