Imperial College London was confirmed as one of the top-rated universities in the United Kingdom. Credit: Construction Photography/Photoshot

Academics across the United Kingdom awoke this morning to face two documents that will determine the future of their university departments in the coming years.

Europe proposes joint Moon trips with Russia Microsoft billionaire takes on cell biology Watson's Nobel medal sells for US$4.1 million

The long-awaited and highly controversial Research Excellence Framework, or REF, the latest iteration of an assessment that rates the quality of academic departments in every UK university every five to six years, is claimed to be the largest assessment exercise of its type in the world. Overall, the report released on 18 December found that the country's academic institutions have performed well, rating three-quarters of the research it evaluated as either 'world leading' or 'internationally excellent'.

The REF will ultimately determine how big a slice each university receives of the annual £2 billion (US$3.1 billion) of core research funding administered by regional funding bodies in England, Wales, Scotland and Northern Ireland. Universities use the money to cover operational and maintenance costs for laboratories, but also to fund research grants internally.

The funding allocated on the basis of the REF is separate from the research grants awarded for specific projects by the seven UK research councils. In addition to the REF, the government sprung a science strategy outlining how it will spend £5.9 billion in infrastructure cash, in part through the research councils, from 2016 to 2021.

To claim a share of the REF cash, universities prepared exhaustive reports on selected staff members and their work. The four regional funding bodies then appointed panels of academics to peer-review each submission. The review produced a profile for each university department submitted for assessment, which gave a percentage of the submission rated as 4* (world leading), 3* (internationally excellent), 2* (recognized internationally) and 1* (recognized nationally).

The REF was the first such UK assessment to include a weighting of the ‘impact’ of university research — defined as its effect on life outside academia, including the economy, culture, policy and environment. This counted for 20% of each department's total rating, with another 65% coming from ‘research outputs’ such as peer-reviewed papers and 15% from ‘research environment’, including departments’ infrastructure and strategies.

The grades are out

The latest assessment — which covers the years 2008 to 2013 — rated 30% of submitted work as 4*, and another 46% as 3*.

Graeme Rosenberg, the manager of the REF, told reporters at a 17 December press conference in London that the scores are not directly comparable with those in the 2008 report, but he added that overall the data showed that UK universities had substantially improved their quality.

The REF does not produce an overall ranking of universities, but some indication of an institution's performance can be gleaned from the percentage of their submissions rated 4* and 3*. For instance, in the biological-sciences category, the Institute of Cancer Research in London had 58% of its submission rated as 4*, as did the University of Dundee. However they submitted only 34 and 73 staff members respectively, compared with the University of Oxford, which submitted 224 staff members in biological sciences, and received a 47% 4* rating in this category.

Imperial College London released a statement claiming that it had been rated “the UK’s top research university” after 91% of its research was classed as 4* or 3*. University College London (UCL) also claimed success, pointing out that by some methods of analysing the data it rated higher than the universities of Oxford and Cambridge.

Exactly how these scores translate into funding for universities is not yet settled. England, Scotland, Wales and Northern Ireland will each produce its own formula to allocate money to departments on the basis of the number of researchers submitted and the quality of the submission. In the past, these formulae have heavily favoured 4* research; departments with profiles heavily skewed towards 1* are likely to see vastly less money.

Previously, some researchers have voiced concerns that the money may be increasingly concentrated in the small number of institutions with the highest ratings, to the detriment of universities that produce less research (see 'Research assessments: Judgement day'). David Sweeney, a senior director at the Higher Education Funding Council for England (HEFCE), which distributes the REF cash in England, said that the formulae had not yet been finalized this time around. But, he noted, there was nothing in the results to make the case for funding to be increasingly concentrated on a smaller number of institutions. “There will be changes,” he said, “but the order of things will not be turned upside down.”

Researchers have also long complained about how much effort has been required to assemble submissions for the REF and its predecessors, and have suggested that it is potentially open to manipulation: universities can select who among their staff is submitted for assessment, and can poach highly rated research teams from other institutions to boost their own scores.

But James Wilsdon, a University of Sussex science-policy researcher who was present at the London press conference, says that the account given by the REF panel members “really does demonstrate the rigour, the quality, the robustness of the exercise”. This, he says, contrasts with the general mood in the academic community, which is “very REF sceptic”. Wilsdon has been commissioned by HEFCE to review how to improve future REFs using additional metrics of research quality, such as the impact factors of journals in which academics publish their work.

Being strategic

The government science strategy also announced on 18 December set up overarching priorities for UK research for the five-year period from 2016 to 2021. These also covered the country's other major funding scheme, the seven research councils.

The £5.9-billion plan includes a pledge of £2.9 billion towards large-scale, 'grand challenge' projects. Some £1.8 billion of this is already allocated, but new spending includes £30 million towards the European XFEL project, a high-powered X-ray laser that will probe matter once it is built in Hamburg, Germany.

The strategy also outlines how the remaining £3 billion in infrastructure funding will be directed through the seven councils. This pot covers existing and planned UK facilities, as well as annual contributions to international projects and institutions, and was slashed in a previous government spending review in 2010. The government made no commitment on spending for research grants, which will come in a spending review following the general election in May.

The government also announced that Paul Nurse, president of the Royal Society, will lead a review into the research councils, reporting in the middle of next year. The review will “look at how Councils can evolve to support research in the most effective ways by drawing on a range of evidence, including international comparisons and the views of the scientific and business communities”.