Published online 23 September 2009 | Nature | doi:10.1038/news.2009.933


Plans for UK research assessment revealed

Peer review remains key for determining the distribution of university cash.

Scientists with pipettePlans for assessing the quality of university research in the UK have been released.Punchstock

Proposals for a new system for auditing research quality in UK institutions have received a cautious welcome from the nation's university and research leaders.

The new system — called the Research Excellence Framework (REF) — will use peer review to judge the quality of work, like the Research Assessment Exercise (RAE) that it replaces. But now, to help assessors make their judgements, they can also use data on the extent to which the work being assessed has been cited by research publications. The Higher Education Funding Council for England (HEFCE), which runs the auditing exercise, published the proposals for consultation today.

The UK government had initially proposed that bibliometrics, such as citations of published work, would play an even larger part in the new auditing system when it announced in December 2006 that the RAE would be overhauled. But the idea that peer review would take a back seat in assessing the quality of university research proved controversial with scientists and national academic bodies such as the Royal Society and the Institute of Physics. In the new proposals, HEFCE says that its inquiries found that bibliometrics would not provide a "sufficiently robust" assessment of research quality.

Science for society?

Another key change is that the REF will include an assessment of the economic and social benefits of research — the 'research impact'. Universities will need to submit case studies detailing specific examples of research-driven societal benefits that have become evident during the period 2008–12 — although the research on which these are based could have been carried out 10–15 years previously. Institutes must also submit statements describing their range of partnerships with research users, such as industry and charities. HEFCE is running a pilot exercise during 2010 to further develop and refine the details of the impact assessment.

David Sweeney, the director of research at HEFCE, told a press conference that the REF is "not the same" as the RAE, the last of which ran in 2008. "There is a greater use of metrics and there will be an assessment of the impact arising from excellent research," he said.

Research quality will contribute to 60% of the overall judgement of work submitted for assessment. Research impact and the research environment — which includes evaluations of how universities support career progression and research infrastructure — will contribute 25% and 15% to the total assessment, respectively.

Measurement challenge

Rama Thirunamachandran, deputy vice-chancellor at Keele University in Staffordshire, UK, and former director of research at HEFCE, told Nature that the "significant new dimension is the assessment of impact". He agrees that universities should demonstrate the impact of the research they carry out and show how it represents value for money. But he warns that, as they stand, the proposals for collecting information on impacts "may be over-elaborate".


Meanwhile, Bahram Bekhradnia, director of the Higher Education Policy Institute, a think tank based in Oxford, UK, welcomed the retention of peer review as the backbone of the REF assessment, saying it is an "incomparable improvement" on the government's original plans. But he warns that establishing credible measures of research impact will be challenging, and that it is "unwise" to rely on impact assessments "for as much as 25% of total research funding".

Wendy Piatt, director-general of the Russell Group of 20 leading UK universities, agrees that the links between research and its wider impact are "not straightforward". "There are particular challenges in terms of defining the length of time between research and its impacts, how to attribute impacts to research and providing corroborating case studies," she says.

The first REF exercise is due to run in 2013. From 2014, the results will be used to carve up the £1.76-billion funding pot for UK university research. HEFCE says that it will run the assessment "periodically", possibly every four to six years. 


If you find something abusive or inappropriate or which does not otherwise comply with our Terms or Community Guidelines, please select the relevant 'Report this comment' link.

Comments on this thread are vetted after posting.

  • #60143

    Assessment is hugely important to me. When I first meet a patient, I immediately observe how they get out of the chair in the waiting room, how they walk when we're heading to the treatment room, and how they sit when I'm doing my initial interview. Everything I do more or less follows a HOPMNRS formula: History, observation, palpation, movement, neurological s&s, referred pain, special tests.

Commenting is now closed.