For many involved in the United Kingdom’s mammoth national assessment of university research, the release of the results late last year marked the end of the process, or at least a welcome rest from it. But for a hard core of research-policy wonks, assessment never sleeps. So begins a new phase: assessment of the assessment. A meta-assessment? This is not as insular as it might sound. Policy-makers have to decide whether the original assessment was worthwhile, how to develop it if so, and whether it should be repeated.

University-funding agencies have commissioned independent experts to pick over the results. The first such analysis is published this week, and it deserves a broad audience.

The focus of the report is the collection of innovative (and, to some, controversial) accounts of the societal impact claimed for research. These impacts were reported as part of universities’ submissions to the 2014 Research Excellence Framework (REF), which will be used to help to allocate public funds on the basis of merit (

One university research group, for example, developed a model and database to quantify pollution of urban water sources. It also analysed peatland drainage systems to work out how to reduce water discolouration by dissolved organic carbon. The outputs of this research helped to improve water quality and the performance of the water industry.

Another group experimented on interventions that rewarded staff in primary health-care centres for their performance, to assess the effects on the quality of care delivery. The work delivered demonstrable impacts both nationally and internationally.

This week’s report, The Nature, Scale and Beneficiaries of Research Impact, was produced by King’s College London and Digital Science, a sister company of Nature’s publishers. It features attractive diagrams of flows between research and impacts, but is heaped with qualifications as to its limitations: the analysis is based on text-mining of the roughly 7,000 submitted impact statements, which, at the time, were not searchable or classified with metadata.

Nevertheless, some of the report’s broad conclusions are credible. UK academics make contributions that are felt in every country around the world, and have provided a rich variety of narratives about how universities affect policy, cultural development or research commercialization. Multiple fields have impacts on the same broad topics, yet each research field also splashes its impacts over multiple topics. Small institutions have larger societal impacts than is generally recognized.

Applied research can gain its own professional merit and public recognition.

The analysts freely admit that their report is a work in progress: the free-form nature of the REF meant that few impact statements presented information in standardized ways or used the same approaches to evaluate impact numerically. More consensus on approaches and standardization would probably be required to identify the qualitative and quantitative outcomes of research — yet a more prescriptive approach to gathering information might discourage academics from displaying their very diverse and heterogeneous impacts.

At this point, it is worth stepping back and noting the fundamental point. This exercise has shown that societal impacts can be documented and independently assessed.

Each of the assessment panels across academic disciplines has documented the particular steps that it took to assess impact statements, and (assisted by a pilot study) to develop criteria for doing so (see

There will be justified debates about the cost of the exercise, how its outputs are geared to draw university funding, and the incentives to which such assessments will inevitably give rise. But there is a long-term perspective that should be considered.

In a country the size of the United Kingdom, no one should doubt that a significant proportion of the research base should be dedicated to research that tries to understand how the Universe and our planet function, and what makes humans tick. And outstanding performers in such basic research fully deserve the prestige given by prizes and academies.

But there are also outstanding performers in research addresses quite different goals: to directly improve the health, sustainability and resilience of individuals and the state of the world. Some of their work is published in highly specialized journals, but has a relatively broad societal impact.

By showing that standards of impact can be set and performance can be assessed, the REF has demonstrated in principle that applied research can gain its own professional merit and public recognition. Funders and institutions should celebrate, support and reward such achievements just as much as they celebrate prizes and academy memberships for fundamental research. Through such positive incentives and prestige, society at large will obtain a more appropriate balance of returns on its investments in university research.