European nations have long cast envious glances across the Atlantic at the industrial spin-offs — and the associated wealth — created by US universities. But how hard should the continent's governments push when it comes to persuading their universities to become more focused on application? A report released last week shows that Britain is pushing pretty hard, and critics say the country risks undermining fundamental research in order to subsidize industry-funded projects.

Scientists might have been expected to welcome the proposed changes, which involve reform of the much-criticized Research Assessment Exercise (RAE). The roughly five-yearly process, in which peer review is used to assess and reward individual university departments, is widely acknowledged to have boosted the quality of British science since it was introduced in 1992. But given that the assessment has paid off, and the preparing and processing of the submissions is so time-consuming, the RAE is felt to have had its day.

Yet proposals to make the 2008 RAE the last of its kind are attracting as much criticism as the current mechanism does. In a consultation document released on 13 June, the government suggested that money be allocated to universities based on how much income they already attract from competitive grants, charities and industrial contracts. The system was chosen because, across the whole university sector, this income correlates well with the results of RAE peer review.

But the correlation doesn't always hold at the level of individual universities. Five different models are proposed, all of which have roughly the same effect of boosting universities that are good at bringing in money, such as those with the research capacity to attract significant funding from medical charities (for example, University College London) or industry (such as Cranfield University), and taking money from those for whom such activity is a smaller component of their total income (see ‘Who would the scheme most affect?’).

Table 1 Who would the scheme most affect?
Full size table

Poorly rated

Other countries have made use of metrics, but the British initiative is the boldest and most application-focused. Germany, for example, uses income measures to allocate around a quarter of local-government funding for universities, but a range of other metrics, including the number of graduates and PhDs, are also considered.

If the British system goes ahead, it will mean that “universities are there to do research for other people”, says Tom Sastry, a Bristol-based senior researcher at the Higher Education Policy Institute, an independent think-tank. He and other critics fear that this is the intention of the Treasury, which wants to push universities towards application-driven projects; the Treasury first outlined the metrics plans in the budget this March. Fundamental research, which does not attract so much external funding, could then suffer. “Many theoretical fields are going to find it hard,” agrees Anthony Van Raan, an expert in research assessment methods at Leiden University in the Netherlands.

The University of Oxford stands to gain from Britain's plan to reward universities good at winning grants and contracts. Credit: N. HAMID

Van Raan and others say that the Treasury is right to focus on metrics, but only if a broad range of measures is used. These could include numbers of PhD students trained, together with some form of bibliometric analysis and other output measures, such as patents gained. “The scientific community will be disappointed if the final models are not more sophisticated,” says Jonathan Adam, director of Evidence, a Leeds-based consultancy that analyses the quality of international research.

The Treasury seems, however, to have limited interest in expanding the range of metrics used. Neither of the authors behind the consultation document — David Eastwood, vice-chancellor of the University of East Anglia in Norwich, and Alan Wilson, director-general for higher education in the government's Department for Education and Skills — would comment. But a Treasury spokesman, while noting that the consultation asked for views on the possibility of including other metrics, said that “bibliometrics are not sufficiently developed to have been incorporated into the modelling” and described PhD numbers as a “less robust measure”.

Such points are disputed by policy experts. Crude bibliometrics measures, such as the number of citations earned by a department, are accepted as being inadequate because some fields, such as mathematics, tend to use fewer references than others, such as genetics. But Van Raan and Adam have independently developed more sophisticated impact measures that place citation numbers in context within disciplines and between countries, and both say that such metrics should form a part of any university assessment system. Unless the range is expanded, says Van Raan, universities could become overly focused on short-term pay-offs: “This might stop research that could be interesting in ten years or so.”

The proposals are open for comment until 13 October. A test drive of the agreed scheme will run alongside the existing RAE scheme in 2008, before being phased in as its replacement.