India and the Rankings
There is a sense of crisis in Indian higher education. Everywhere, there are accusations of fraud, a vast quantity of papers has been retracted, the national rankings have been plagued by corruption, and there is concern over the mindless pursuit of citations and the proliferation of worthless paper mills.
Many observers believe that these problems are linked to the influence of the global rankings, especially the Times Higher Education (THE) World University Rankings and its various progeny.
THE has achieved massive success among the media, governments and university administrators in India, as elsewhere, at the cost of the disdain and anger of working academics and researchers. It has, however, had a malign influence on education and research and has brought the whole idea of ranking, or any kind of rating or external assessment, into disrepute.
Indian higher education has been unhappy with THE for some time. Back in 2020, seven leading Indian Institutes of Technology (IITs), Delhi, Bombay, Guwahati, Kanpur, Kharagpur, Madras, and Roorkee, announced that they would boycott the THE World University Rankings. They complained about the lack of transparency of the complex methodology and the poor showing of leading Indian institutions compared to their performance in the QS world rankings. In the current rankings, IIT Delhi (IITD) and IIT Bombay (IITB) were in the 401-500 band, and the Indian Institute of Science (IISc) was 301-350. These three all did much better in the QS world rankings, where IIT Delhi was 182nd, IIT Bombay 152nd, and IISc 184th.
It is only fair to point out that the THE rankings were not the only ones to give Indian schools a poor grade. the Shanghai Rankings, which are concerned only with research, had IITD and IITB in the 701-800 band, and IISc at 401-500. In the US News Best Global Universities IITD was 654th, ITB 513th, and IISc 530th.
THE rankings had seen several strange movements. In the 2013-2014 world rankings, Panjab University soared into first place in India ahead of four IITS, but then in the 2015-2016 rankings, it fell to 501-600. This spectacular rise was the result solely of the university's fractional participation in the Large Hadron Collider project, which typically has thousands of authors per paper and thousands of citations. Panjab University's spectacular decline was the result of a methodological tweak by THE, which discounted papers with over a thousand authors.
In 2017 THE had a narrow escape. In that year, Vel Tech University was announced in the THE Asian rankings as the 3rd best university in India, 43rd in Asia, and first for citations, supposedly a measure of the quality of research. This was done by industrial scale self-citation, nothing else, by a single researcher who claimed to have done it unintentionally. THE dealt with the situation before the world rankings were released.
THE responded to the boycott by hinting that it would hurt the institution's academic reputation. In fact, that did not happen. The boycotters thrived outside the WUR and did well in comparable QS and US News indicators. Frankly, it looks like staying in the THE rankings is not always a good idea. Some of the original remaining IITs suffered serious declines.
Recently, disquiet with the THE rankings has returned. There is now concern about the treatment of the Indian Institution of Science (IISC), which many regard as India's premier research institution. In the current THE world rankings, it is ranked as 201-250 in the world and first in India. Looking at THE's five "pillars", IISc is first for teaching research environment, and industry in India, 30th for International Outlook and fiftieth for research quality. That is in India, not Asia or the world. That, frankly, is quite implausible for anyone with even a glancing knowledge of Indian higher education.
There is a radical disconnection between the Research Environment and Research Quality indicators in the world rankings. There is only one university in the Indian top ten for both, Anna University. Research Quality is led by Chitkara University, Saveetha Institute of Medical And Technical Sciences, Shoolini University of Biotechnology and Management Sciences, Lovely Professional University, and Thapar Institute of Engineering & Technology. I am not knocking them, but I doubt that anyone would seriously consider them the equal, let alone the superior, of IISc.
Here is the description of the THE Research Quality indicator from their methodology page.
"research quality: 30%
Citation impact: 15%
Research strength: 5%
Research excellence: 5%
Research influence: 5%
Our research quality pillar looks at universities' role in spreading new knowledge and ideas.
We examine citation impact by capturing the average number of times a university's published work is cited by scholars globally. This year, our bibliometric data supplier Elsevier provided more than 157 million citations to 18 million journal articles, article reviews, conference proceedings, books, and book chapters published over five years. The data include more than 30,000 active peer-reviewed journals indexed by Elsevier's Scopus database and all indexed publications between 2019 and 2023. Citations to these publications made in the six years from 2019 to 2024 are also collected. These data are now analysed by THE's data team, rather than Elsevier.
The citations help to show us how much each university is contributing to the sum of human knowledge: they tell us whose research has stood out, has been picked up and built on by other scholars and, most importantly, has been shared around the global scholarly community to expand the boundaries of our understanding, irrespective of discipline.
The data are normalized to reflect variations in citation volume between different subject areas. This means that institutions with high levels of research activity in subjects with traditionally high citation counts do not gain an unfair advantage.
We have blended equal measures of a country-adjusted and non-country-adjusted raw measure of citations scores.
Three new research quality measures were added in 2023. Research strength calculates the 75th percentile of field-weighted citation impact – a very robust guide to how strong typical research is.
Research excellence looks at the number of research publications in the top 10 per cent for field-weighted citation impact worldwide – a guide to the amount of world-leading research at an institution. It is normalised by year, subject and staff numbers.
Research influence helps us to understand when research is recognised in turn by the most influential research in the world – a broader look at excellence. The idea behind the metric is that the value of citations is not equal: a citation from an "important" paper is more significant than a citation from an "unimportant" one. We use an iterative method to measure the importance of a paper by not only counting the number of citations but also taking into account the importance of the citing papers. We also consider the subject of the research, as different disciplines have different citation patterns."
I hope that's clear.
To simplify a bit, the THE citations metric revolves around the concept of normalisation. What counts is not the absolute number of citations but the number compared to the world average for specific disciplines. I understand that there are now over three hundred.
One consequence of this is that a small contribution to a big project like the Global Burden of Disease Study can produce massive dividends compared to a large number of citations for many papers. Another perversity of this indicator is that it pays to keep output as low as possible without falling below the publication threshold. Consequently, universities who have been unrecognised by other rankings can, if they choose their research fields carefully, do surprisingly well in the THE Research Quality metric, even if they rank at the bottom of the other indicators.
Whether or not to participate in the THE rankings is up to Indian universities. But if they do, they should be prepared for the fate of Panjab or Vel Tech University.