Indian university rankings inherently inconsistent

Anomalies in the third national assessment of Indian institutions intensify criticisms of linking the exercise to funding.

9 April 2018

Hindustan Times/Getty Images

Indian Minister of Human Resource Development Prakash Javadekar.

The latest annual ranking of Indian universities reveals few consistent performers below the top ten. The Indian Institute of Science (IISc) in Bangalore ranked first for the third consecutive year, followed by many of the Indian Institutes of Technology (IITs). But disparities among the lower ranks have drawn attention to inherent flaws in the exercise.

The annual assessment, known as the National Institutional Ranking Framework (NIRF), was launched in 2016 as an attempt to link funding with quality.

Its creators proposed that an institution’s performance in the national ranking would decide its level of government funding. While this hasn’t been implemented yet, the government recently granted greater autonomy to a handful of universities based on their accreditation score.

Scientists have questioned the strategy. Madhusudan Raman, a PhD scholar at the Institute of Mathematical Sciences, calls the proposal “an easy way of getting out of making difficult decisions”. He says that institutions should be funded based on a qualitative assessment of their needs and plans, not a quantitative valuation of rank.

Nirmalya Kajuri, a post-doctoral fellow at the Chennai Mathematical Institute, says that linking ranking to funding would sustain existing inequalities in the distribution of funds. “Punishing a college for bad performance does not cut off some luxuries, it cuts off some necessities,” he says.

Ups and downs

Two consecutive editions of the ranking have exposed the many flaws in the system including problems with comparing very different types of institutions, incomplete data, bias towards technical disciplines, and arbitrary weighting of parameters. These discrepancies have cast doubt on the usefulness of the NIRF for students and administrators.

But Kajuri argues that some data is better than no data. By making the information publicly available, students can “sift through it and figure it out,” he says.

India’s Minister of Human Resource Development Prakash Javadekar released the results of the 2018 rankings at an event in New Delhi earlier this month. Some 4,220 institutions submitted data related to five parameters of teaching, research, graduate outcomes, inclusivity and reputation.

As in 2017, the overall ranking, which compares institutions across categories and disciplines, bears little resemblance to the previous year, raising questions about the rigour of the analysis. The discipline lists also have inexplicable gaps arising from the absence of certain institutions for unknown reasons.

Comparing the 2017 and 2018 overall rankings, 26 of the top 100 ranked institutions are new entrants. Among those that have dropped out of the rankings is the country’s foremost business school, the Indian Institute of Management Ahmedabad, which was ranked 17th overall last year.

In another anachronism, the old and venerable King George’s Medical University debuted this year in the 23rd position. This makes it the highest-ranked medical institution in the overall category, despite being ranked 5th in the medical category. The overall category has different weightings than the disciplinary lists, and institutions can seemingly opt out of the former.

×

Research skew

The research section of the NIRF assessment combines publication and citation data from the Web of Science and Scopus databases, as well as patent numbers and funds raised for research projects. This is in contrast to the Nature Index, which ranks institutions based on their contribution to the authorship of papers in 68 high-quality journals.

The Indian research ecosystem looks more skewed in the index than in the NIRF. While the NIRF’s research scores gradually decrease further down the rankings, there is a much steeper drop-off in terms of weighted fractional counts in the index. This indicates that when it comes to high-quality research, there’s a wider gap between top Indian institutions and the rest.

Uncoupling variables

An analysis of an institution’s rank in the NIRF and its total expenditure finds no correlation between the two variables. Most institutions in India have budgets of less than US$50 million a year for both capital and revenue expenses. The top-ranking institute, IISc, spent about US$80 million in 2017. This is miniscule compared to, for example, the US$1.3 billion that University of Pennsylvania spent in 2016 on research and development alone.

The biggest spender among the top 100 institutions in the NIRF was Manipal Academy of Higher Education, one of the largest private universities offering courses in everything from medicine to philosophy. It had a budget of around USD200 million and ranked 18th overall in the NIRF 2018.