london

University departments whose ‘structures’ do not match those followed by Britain's Research Assessment Exercise (RAE), the regular initiative used as a basis for calculating funding allocations to universities, appear to get systematically lower research ratings.

According to an independent report on interdisciplinary research, nearly a quarter of university departments in the last Research Assessment Exercise either split their researchers between assessment panels, or sought cross-referral between panels.

The report found that such ‘boundary critical’ submissions received on average a 0.5-point lower rating than non-critical submissions. The RAE rates departments on a 5-point scale, and there are serious financial consequences for those that receive a reduced rating, particularly at the top of the scale.

The report, by consultancy Evaluation Associates, was commissioned by the four regional higher education funding bodies in preparation for the next Research Assessment Exercise, due to take place in 2001. It says of submissions that were cross-referred and split between panels that “our concern is that assessment in these circumstances is inherently difficult for panels. They are not considering entire departments but potentially arbitrary subsets of departments”.

But the report also says that it found no evidence that RAE inhibits interdisciplinary research, despite a widely held conviction by researchers that it does. It did find a lack of consistency in the treatment of interdisciplinary research by individual panels, however.

“I've never had any problem with [the RAE] and I don't think it is a problem,” says one interdisciplinary researcher involved with the next RAE. “In the area I operate, there is no disadvantage and I am extremely involved in interdisciplinary research.”

Bahram Bekhradnia, director of policy at the Higher Education Funding Council for England (HEFCE) — which was responsible for the last exercise — says that interdisciplinary work “has always been one of the more difficult aspects of the exercise which is a discipline-based process. This was one of the reasons for the review.”

He said that the report gave general comfort to interdisciplinary researchers, and that the funding council “expected to implement” the specific recommendations. The report recommends new mechanisms for boundary critical submissions, and monitoring mechanisms to “ensure the effectiveness of cross-referral”.

Anita Jackson, a member of the RAE 2001 team, says that the assessment process is “evolutionary”, and that the next exercise will take on board the recommendations of the report. “We will ask departments to say if their units do not fit. We have put in mechanisms where anything cross-referred will happen straight away — which was not what happened in the past.”

It remains to be seen whether this will impress departments that feel they were badly rated in the last exercise. “We are aware that if anyone feels aggrieved they can have a judicial review,” says David Pilsbury of HEFCE. But he points out that “it happened once before, and the outcome was in our favour”.

The report finds that the RAE has encouraged researchers to concentrate on the quality of their research, but that they are doing more research that produces results in the short rather than long term.