Nodes represent OECD subregions where universities are based, and weights are obtained from the pairwise correlation between their territorial indicators. Colors of nodes and intra-community edges are determined by the territorial community to which they belong. Credit: Loredana Bellantuono
Scientists in Italy have used an analysis based on complex networks to remove what they describe as territorial bias from university rankings. Drawing on detailed social, economic and technological data, they found a strong correlation between a region’s wealth and the performance of its academic institutions in the most commonly used rankings of universities, across Italy, and globally. Published in Scientific Reports, their study proposes a method for recalibrating league tables, so that individual universities are rewarded if they do better than other institutes with similar socio-economic conditions1.
Publications such as Times Higher Education (THE) at the international level, or CENSIS (Centro Studi Investimenti Sociali) at the Italian one, regularly rank universities according to their research, teaching and industrial relevance, among other metrics. The resulting tables are used to attract students as well as public and private funding, but have been criticized for fostering academic conformism in order to best meet the assessment criteria and favouring universities in richer countries or regions – thus entrenching disparities.
Loredana Bellantuono and colleagues at the University of Bari, working with researchers at the United Nations and Bank of England, set out to quantify the extent of such regional bias by turning to network theory. Their analysis involved setting up ‘complex networks’, sets of interconnected nodes that can be used to represent relationships inside real systems, such as brains and online social networks.
The researchers performed one global analysis, and one focused on Italy. The former relied on a dataset consisting of 1088 universities in 343 sub-regions of OECD countries, using the 2021 rankings from the Times Higher Education. The latter looked at 92 universities in Italy's 53 provinces that were ranked by CENSIS for the 2019-2020 academic year.
The study used more than 100 indicators of socio-economic prosperity — from employment rates to changes in tree cover — to create networks in which each node was a region. Any two nodes in a network were connected by a link if their respective values of the indicators were correlated, with the link being weighted according to the strength of that correlation. The researchers then replaced the regions with universities, such that each node became an individual institution – any two nodes being linked if the corresponding regions had been, and sharing the accompanying weights.
The idea was to establish whether there was a correlation between the similarity in nodes' socio-economic conditions and a similarity in the universities' performance scores. Both the global and the Italian analysis found what the authors describe as “significant evidence of territorial bias”.
Bellantuono and colleagues then used this territorial bias to establish the bias experienced by each individual institution. They first sorted regions into groups according to their wealth and then assessed whether each university significantly over- or under-performed compared to its peers by comparing its performance score against the average of its region’s grouping. Finally, they used these results to calculate revised league tables and to quantify the impact of territorial bias on each institution’s score.
Internationally, a revised version of the THE ranking showed two new entries, both established universities – Imperial College London in the United Kingdom, and ETH Zurich in Switzerland. In contrast, the Italian revised ranking saw four new institutions in the top ten, with two from less well-off areas – the University of Sassari in Sardinia, which gained five places compared to the ‘official’ ranking, and the University of Calabria in the South of the country, which gained seven. According to Bellantuono, the University of Calabria's upgrading is due to its student facilities, including generous accommodation and digital services. The researchers also calculated a ‘territorial dragging effect’ that measures how each university’s performance in standard rankings is negatively affected by its geographical position. This bias is highest for some Russian, Argentinian and Brazilian institutes at the international level. In Italy, it mostly affects southern institutes such as Politecnico di Bari, and the Universities of Calabria, Sassari, Salerno, Salento and Messina.
Bellantuono says it would be good to have additional data, such as the amount of public and private funding obtained by each institution, to better pinpoint the reasons for their performance relative to their neighbors.
According to Gerardo Iñiguez, a network and data scientist at the Central European University in Vienna, who was not involved in the research, the study illuminates the ‘non-trivial interplay’ between university networks and academic rankings. “It supports the notion that rankings do not measure academic excellence exclusively,” he says.
