info:doi/10.1038/440413a 2006-03-23T00:00:00Z 2006-03-23T00:00:00Z Alexander Szalay Jim Gray © 2006 Nature Publishing Group Nature 440 7083 413 414 0028-0836 1476-4679 (?-mix:\s+\W+|\W+\s+|\s+|\/) data will can not analysis they experiments so sets their But one scientists instruments scientific more new have today but there them archives also growth these analyse several do such tools others science become results reproduce large databases sharing paper up computers standards way increasingly process steps all important provide complex most how across year time must facilities years many There volumes network example Large notebooks no both documenting becoming see difficult Scientists few arduous did GenBank procedures generation trends Internet exponential find Matlab into Many believe measurements database organize another Synoptic just achieve big public 20 Fully than One Telescope vocabulary replaced scientist standard at petabytes terabytes ago could record patterns ways cheaper created 1,000 expect millions without soil analysed which gigabytes environmental petabyte electronic should management http: model group format access cubes archive essential themselves each system has controlled some Survey combine Standards yet disciplines publish scaling when discoveries grow longer workflows raises astronomy easily smaller need obsolete used about 2020 place computations questions sensors include If Some pattern only continue collaborations areas recognize records done software capture often may other long using same Data small none biodiversity work current automatically match change document world. Virtual review ultimately annually limits International measurement occurs leapfrogged draw innovations processes proprietary However roughly once quickly Exponential capacity amounts copying bandwidth programs integrates text revisions helping experts supposed understood looking would entered converting distributed explosion fastest free sort part been straightforward processing regional integral rewarded picture us 402 preprint primitive ones reproduced 100 although doubling scaling-out offers issues formatting Experiments concepts summary performing biogeochemistry world's www.IVOA.net understanding Observatory out Feature recording literature Procedures Galileo times 1 examine super-computers sending approach statistics Excel repeat systematic compute worked read explicit local semantics per encourage address students US$300 astronomy's Instead ten deploy books So repository know our every sensor modern 10 Increasingly old equivalent linear credit step desktop scales cross-correlated enduring traditional experimental points departments filtered affect DNA-sequencing international observations specified Mendel able two kept distorted scale later publishers collect previous use fulfil parameter available improvements reduce billion itself environment criteria develop gene Such hold future Today even go disappear data-mining physicists who written challenge disseminate collaborators training universal executed graduate progress ideal involving visualize form experience ever provenance might computed chips similar billion-dollar require collected data-collection teams objects acquire response perform switched multibillion-dollar exchange copy translated reach Hadron better back Smaller measuring experiment exceeding its beyond useful hydrology e-mail affordable laboratory technological publishing technology take solution always attempting formal picking via websites central individual handling published changes becomes query including let parts predict Multidisciplinary petascale today's implications imagine show medium challenges crucial techniques dictionary fine well chance involve sub-disciplines page hypothesis computer someone trained link gold-mine deposit generating horizon Fortunately yield build nearly chemists coming then arXiv.org now protocols dramatic point numbers wireless rather statistical These extract happening discipline manage argue sequence suspect News portal decades different data-rich integrating early Mathematica speed cannot ref eventually analysing hundreds terms Public cost terabyte language faster Internet-connected long-term building exact knowledge does single deluge Collider Too enormously machine produce appendices realize made dimensions ecosystems Databases first exploit Likely Astronomers needed description Similar piece Current archiving Paper labour working being before multiple DNA required amount over settings information climate merely term correlated publication your procedure survive skill details collaboration reason forms replicate mine popular dollar mean much already myriad computational tenfold lab old-fashioned careful rich output certain Alliance possible array semantic date taken biological vast role data-visualization next typically impossible www.lifeunderyourfeet.org leapfrogs effort usable pace Darwin occurred powerful repeatable reviewer needs security handle commodity creators sense levels parameters 2011 notebook properly fit wasted isolated given little requires meaning groups problem systems Few workstation pipette common variables reaching enhanced load soon storage computer-performance Desktop keep own faces phenomenon plateau combined producing zones lead Analysing alongside sequencers statisticians 30 biologists (?-mix:\.\s+(?=[A-Z])) A collaboration involving hundreds of Internet-connected scientists raises questions about standards for data sharing A similar phenomenon occurred with DNA chips and gene sequencers Analysing terabytes of data (one terabyte is 1,000 gigabytes) is a challenge; but petabyte data sets (of more than 1,000 terabytes) are on the horizon And once these petascale experiments are switched on they will produce roughly the same amount of data each year — merely linear growth And so although some sub-disciplines may reach a plateau in data generation, other technological innovations will take their place And so large data archives are becoming increasingly "isolated" in the network sense —one can copy gigabytes across the Internet today, but not petabytes Are we reaching the limits of what one scientist, or one lab, can expect to achieve in data handling and analysis? If so, this will have implications for how we review and publish our work As data volumes grow, it is increasingly arduous to extract knowledge As experiments yield more data, and analysis becomes more complex, data become increasingly difficult to document and reproduce Astronomers created the International Virtual Observatory Alliance ( http://www.IVOA.net ), integrating most of the world's medium and large astronomy archives But data volumes are doubling every year in most areas of modern science and the analysis is becoming more and more complex, exceeding the capacity of the paper notebook But in the scaling-out model, experiments that deploy an array of small instruments can exploit the coming explosion in cheaper commodity technology But none of these programs scale up to handle millions of data records — and they are primitive by most standards But the management of distributed computations raises new questions of security, free access to public data and cost But there are few tools to properly visualize data across multiple scales and data sets But there is, as yet, no standard for publishing large volumes of data But this data-rich approach to science faces challenges But we believe that with current trends it is nearly impossible to reproduce experiments Current databases have a long way to go to achieve this ideal Data gold-mine Multidisciplinary databases also provide a rich environment for performing science; that is, a scientist may collect new data, combine them with data from other archives, and ultimately deposit the summary data back into a common archive Databases provide tools to organize large data sets, find objects that match certain criteria, compute statistics about the data, and analyse them to find patterns Exponential growth occurs when a new generation of instruments leapfrogs the previous generation, which become obsolete Few data archives address these issues today For example, a data-mining paper needs to include the explicit description (database query) of how the data that were analysed in the paper were collected and filtered, but not the data themselves For example, imagine a measurement taken using a DNA-sequencing machine For how long will this exponential growth in scientific data continue? Desktop computers today are as powerful as the super-computers of 10 years ago For the analysis to be repeatable in 20 years' time requires archiving both data and tools For these data to be useful to others, they must be published using a controlled vocabulary and in standard forms, and the instruments and measurements must be well specified Fully documenting the sensors and data-collection process is arduous, and there are few standards for us to draw on Fully documenting these steps would be arduous, and there is little chance that someone could repeat the exact procedure 20 years from now; both Matlab and GenBank will change enormously in that time Galileo did it, Mendel did it, Darwin did it, and we are supposed to do it If we can no longer examine all the data on a single piece of paper, how can we "see" a new pattern or find a data point that does not fit a hypothesis? Fortunately there are database tools, such as data cubes, that we believe can fulfil this role (see "Data cubes" ) In response to this data deluge, the systematic use of databases has become an integral part of the scientific process In the future, working with large data sets will typically mean sending computations to the data, rather than copying the data to your workstation In this way, a reviewer with access to public data could reproduce the data sets and analysis procedures Increasingly, scientists are analysing complex systems that require data to be combined from several groups and even several disciplines Instead they "mine" available databases, looking for new patterns and discoveries, without ever picking up a pipette It is difficult to capture all the model numbers, software revisions, parameter settings and process steps in an enduring format It is important to realize that today's graduate students need formal training in areas beyond their central discipline: they need to know some data management, computational concepts and statistical techniques It is important to recognize this pattern; it is universal Likely computer-performance improvements by 2011 include tenfold more processing, storage and network bandwidth per dollar Many experiments today load their data into databases before attempting to analyse them Many predict dramatic changes to the way science is done, and suspect that few traditional processes will survive in their current form by 2020 (ref. 1 ) Many scientists no longer "do" experiments the old-fashioned way One might argue that complex biological experiments have always been difficult to reproduce, as there are so many variables One petabyte is equivalent to the text in one billion books, yet many scientific instruments, including the Large Synoptic Survey Telescope, will soon be generating several petabytes annually Paper appendices cannot hold all the data needed to reproduce the results Procedures already involve instruments and software with myriad parameters Public data creators and publishers should be given credit, and archives must be able to automatically provide provenance details Scientists are trained early to keep careful records in their laboratory notebooks — recording both experimental procedures and observations, so that they can analyse their results and so that others can replicate what they have done Scientists in 2020 will continue to work in an exponential world. Scientists must labour to organize, sort and reduce the data, with each analysis step producing smaller data sets that eventually lead to the big picture Similar progress is happening with scientific instruments — they quickly become obsolete and are replaced by better and often cheaper ones Smaller is faster However, not all experiments will experience exponential growth So we can expect ten times more data Some disciplines have created their own data archives, such as GenBank; others just let data show up, and then disappear, on individual scientists' websites Some scientists are building billion-dollar facilities, such as astronomy's Large Synoptic Survey Telescope or the Large Hadron Collider, which are only affordable as international collaborations Standards are essential at several levels: in formatting, so that data written by one group can be easily read and understood by others; in semantics, so that a term used by one group can be translated (often automatically) by another without its meaning being distorted; and in workflows, so that analysis steps can be executed across the Internet and reproduced by others at a later date Standards for sharing data are crucial, for example, in understanding soil ecosystems Such facilities are not easily leapfrogged The output is cross-correlated with a sequence archive (GenBank) and the results are analysed with Matlab The publication process itself is increasingly electronic, with new ways to disseminate scientific information (such as the preprint repository arXiv.org) The same language Experiments are themselves becoming electronic as computers become essential parts of scientific instruments; they are used not only to manage and analyse vast data sets, but also to acquire them in the first place The speed of the Internet has not kept pace with the growth of scientific data sets The wireless sensors that were US$300 a year ago are $100 today, and will be $30 next year There are collaborations sharing data across departments and time zones, and important discoveries are made by scientists and teams who combine different skill sets — not just biologists, physicists and chemists, but also computer scientists, statisticians and data-visualization experts There are two trends in science today, scaling up and scaling out There is reason to believe that it will be the smaller experiments, not the big multibillion-dollar facilities, that will grow the fastest These computers also perform data analysis; Matlab, Mathematica and Excel are popular analysis tools This required new standards for data exchange, and a semantic dictionary that offers a controlled vocabulary of astronomy terms This system integrates local environmental data from a sensor network with regional data on hydrology, climate, biodiversity and biogeochemistry This worked fine when small amounts of data were entered into notebooks and the analysis was computed alongside them To encourage data sharing, it should be rewarded Today, most scientists have replaced or enhanced their notebooks with desktop computers that record their results, provide a portal to the scientific literature, and link them to collaborators via e-mail Too much effort is wasted on converting from one proprietary data format to another We are helping to build a system for measuring long-term environmental trends that affect soil biodiversity ( http://www.lifeunderyourfeet.org ; see also News Feature, page 402 ) We do not have a solution for this problem, but it is important to recognize it as such, and to do what is possible to capture the workflows and to develop protocols for documenting instruments, procedures and measurements in ways that will be usable in several decades' time With data correlated over many dimensions and millions of points, none of the old steps — do experiment, record results, analyse and publish — is straightforward 1