Impact beyond citations

    The translational achievements, commercialization prospects and eventual clinical and societal impacts of biomedical work accrue over a long time. They should be better captured and publicized.

    When assessing published work, focusing on citations is easy, yet short-sighted — even more so if the work has impact prospects that extend well beyond academia. Such impacts can be myriad. For example, better understanding of disease can lead to changes in health policy; a more accurate diagnostic assay may be incorporated into clinical workflows and ultimately extend the lives of patients; a cheaper and easy-to-use medical device can make healthcare more accessible to patients in low-resource communities; and the commercialization of faster clinical imaging technologies coupled with artificial intelligence could make health systems more efficient and save taxpayer money. None of these outcomes are properly captured by citations to academic papers.

    Citations to published work provide a rough measure of academic interest. Yet there is considerable academic impact that cannot be directly accounted for by citation statistics — for example, the career prospects of students and postdoctoral researchers that participated in the work, the forging of new collaborations, invitations to lecture at prestigious meetings, recognition through academic awards, the securing of additional research funding, and the inception of new ideas. Because these outcomes are difficult to measure, they are less widely visible. It is unfortunate (particularly for junior researchers) that what is measured and rewarded is often what can be easily counted rather than what matters most.

    The impacts of academic work should be systematically collected and analysed. A pioneering example is the 2014 Research Excellence Framework of the UK, which represented the first large-scale collection of case studies of the socioeconomic impact of publicly funded university research. A synthetic analysis of this effort highlighted the diversity of the impacts, and the high degree of multidisciplinarity of the research underpinning the case studies. Although evaluating wider research impacts, especially across disciplines, is not an easy task, such case studies can inspire and encourage researchers to carry out high-quality research that maximizes the likelihood of social and economic outcomes.

    To this end, we asked the authors of the 13 research articles published in the first and second issues of Nature Biomedical Engineering (the January and February 2017 issues) for feedback on any early direct impact of the published work. The responses revealed that the work in three of the papers has helped raise funding for clinical trials, that technologies reported in six of the papers have been licensed to university spin-offs or led to the launch or commercial growth of start-up companies, and that patents have been filed for most of the studies. Half of the teams have told us that medical or biotechnology companies have expressed interest in the published findings. Notably, nearly all authors mentioned that the publication of their work has triggered new academic collaborations, and that media attention after publication has been helpful in many ways.

    Naturally, more media attention goes to reports that tap into human ingenuity, to surprising findings, and to research areas that are ‘in vogue’. For example, three papers published in Nature Biomedical Engineering in early 2017 attracted unusually broad attention (in terms of their Altmetric scores): a paper-and-string centrifuge, a man/machine interface for prosthetics and stimulated Raman histology for the rapid examination of fresh tissue in the operating room. Video highlights of the do-it-yourself centrifuge rapidly surpassed 1.5 million views on social media, and Manu Prakash, the principal investigator of the study, highlighted the device in a TED talk later in the year. This level of public attention, although rare, will probably be dwarfed by the educational and health impacts of the device, especially in low-income countries where access to electricity remains a luxury. Published work can also attract rapid attention from academia and business. For instance, the authors of the paper describing an interface for prosthetics have been invited to discuss the technology in over 30 seminars at universities and in industry, and the stimulated Raman histology technique is being assessed by seven different institutions for potential incorporation into clinical practice.

    Such early signs of impact are promising, and speak to the value of applied biomedical work beyond traditional academic boundaries. Differently from citations, which can be attributed to a specific piece of work, broader impacts cannot usually be associated with a single paper, and they involve a wider range of experts, such as technology-transfer professionals, investors and health-intervention specialists. Still, journals, funders and academic institutions should not be deterred from directing more efforts into highlighting the impact of the research that they have published, funded or produced. In this regard, we invite researchers to describe tangible outcomes of their work on In Translation , our community site, and are exploring ways of analysing and synthesizing evidence of the broader impact of research published in the journal.

    As stated by Jinming Gao when referring to his team’s work on transistor-like pH nanoprobes for tumour detection (published in this journal's first issue) “at this time of technology development, the paper has brought to the research team meaning and rewards that are bigger than the sum of the parts (we could have reported the same work across a few papers in specialty journals)”. Impact matters far beyond what can be counted, indeed.

    Rights and permissions

    Reprints and Permissions

    About this article

    Verify currency and authenticity via CrossMark

    Cite this article

    Impact beyond citations. Nat Biomed Eng 2, 1 (2018). https://doi.org/10.1038/s41551-018-0189-y

    Download citation

    Search

    Sign up for the Nature Briefing newsletter for a daily update on COVID-19 science.
    Get the most important science stories of the day, free in your inbox. Sign up for Nature Briefing