Museum volunteering among new impact indicators for UK universities

How to compare apples with apples.

11 September 2019

mark phillips/Alamy Stock Photo

A strategy to measure the transfer of knowledge between universities, industry and the community is coming to fruition.

Research England, the organization that funds English higher education, will release its first Knowledge Exchange Framework (KEF) draft in 2020.

This annual assessment will provide public information on university performance alongside the existing Research Excellence Framework (REF) and the Teaching Excellence Framework (TEF).

Initiated in 2017, the project takes place against a background of UK government targets to increase university research and development funding to 2.4% of GDP by 2027, up from 1.7% in 2016.

One assumption underpinning the KEF is that “knowledge exchange performance”, which includes everything from intellectual property (IP) commercialization to voluntary museum work by academics, will eventually influence funding.

An existing dataset available through the annual Higher Education-Business and Community Interaction Survey (HEBCIS) will be used to inform the KEF so universities will not be further burdened with major data-gathering requirements.

The HEBCIS survey includes information not only on technology transfer but also categories such as social and cultural engagement and research partnerships, providing a basic comparative resource across a broad spectrum of activity.

The KEF aims is also to convey a subtler and more well-rounded picture of university performance than HEBCIS by clustering institutions into eight peer groups, or ‘bands’, to compare ‘like for like’.

For example, the University of Oxford might previously have been compared to an institution with less research funding, and would have outshone it accordingly.

The KEF aims to remove such distortions by clustering together institutions with a significant portfolio in clinical medicine, for instance, and institutions specializing in the arts or STEM subjects.

“Knowledge exchange is affected by underlying institutional capabilities that are difficult to compare,” says Alice Frost, director of knowledge exchange at Research England.

“Previously, we had been comparing only on the basis that the institutions are different from each other. Now we will be capable of much more robust comparisons by isolating performance factors.”

The world moves to measure impact

Experts at Research England claim the KEF is the most comprehensive and advanced measurement of its kind, though it is not the first.

Several countries have been attempting to formulate knowledge exchange assessments, but in 2018 to 2019 an Australian Research Council initiative, the Engagement and Impact Assessment (EIA), led the way.

Ksenia Sawczal, director of research services at the University of Canberra, said the exercise had been unpopular due to the amount of work it demanded from university staff, and had raised doubts on its ability to generate meaningful findings. Sawczal made the comments in 2016 post on the London School of Economics Impact blog.

Meanwhile, critics of the KEF in the UK argue that it adds little to existing assessments of research impact available through the REF, which integrate some aspects of knowledge exchange.

As a more thorough exercise, however, the KEF is being presented as a step towards greater transparency and accountability while helping to improve knowledge exchange assessment practice in the UK generally.

It’s hoped that it will not just raise the profile of knowledge exchange among organizations concerned with funding and research, but will also help to improve the understanding of universities among prospective students and the general public.

Refining knowledge exchange definitions also protects higher education institutions from crude comparisons on IP. Existing indicators from elsewhere, such as the US, have been limited either to IP transfer or simple measures of input, such as research funding per spinout company or technology.

“All the political debate on knowledge exchange is flawed unless you treat the data in a more sophisticated way,” says Frost. “A lot of countries have difficulties finding the right suite of measurements, but there’s a lot of interest in developing KEFs for the next generation.”

Read next

Steps to a top-scoring impact case study

The allure of the journal impact factor holds firm, despite its flaws

Government funding will be tied to uni performance from 2020