The business of international university rankings is getting bigger and bigger. The US News has announced the publication of its Best Global Universities rankings to compete with the Shanghai rankings and the Quacquarelli Symonds (QS) and Times Higher Education (THE) World University Rankings, bringing the number of big brand name contenders to four. At the same time, the established rankings have been spinning off regional and specialised by products. We now have Asian, Latin American, Middle East and North Africa, Greater China and many subject-specific rankings.
Added to this, QS and THE have both come out with rankings of universities in developing or emerging countries. QS has published its BRICS rankings of universities in Brazil, Russia, India, China and South Africa. THE has produced a university ranking for the BRICS plus emerging economies, as defined by FTSE. This is a very varied collection including countries as prosperous as Taiwan and as poor as Egypt and as prominent in research as Brazil and as limited as Pakistan.
It seems that there is a huge market for university rankings in some developing countries both as a guide to breaking into the inner circle of world class universities and to validate places in the national hierarchy.
The publication of the latest rankings has been greeted with excitement in countries like China, Russia, Taiwan and Turkey, where some institutions appear to have done well, and with predictable gloom in India where higher education appears to be making little progress. However, a closer look suggests that a degree of caution is needed when drawing conclusions.
In the first year of publication in 2013 the THE BRICS Plus rankings simply used the same methodology as the THE World University Rankings. The difference was that the BRICS Plus rankings included several additional universities. But this year THE introduced some methodological changes that, although probably necessary, revealed some problems in the overall structure.
First, here are the top ten of this year’s rankings:
Peking University, China
Tsinghua University, China
Middle East Technical University (METU), Turkey
University of Cape Town, South Africa
Lomonosov Moscow State University, Russia
National Taiwan University
Boǧaziҫi University, Turkey
Istanbul Technical University, Turkey
Fudan University, China
University of São Paulo, Brazil.
Chinese universities continue to dominate the developing world. The first two universities on the list are Chinese as are three out of the top ten, five out of the top twenty and twenty-seven out of the top 100. This is certainly a much better performance than India with only 11 institutions in the top 100, the best of which is the Indian Institute of Science Bangalore in 25th place.
However, we should not forget that China has a huge population and that when we put population into the equation, the achievement of Mainland China looks less impressive.
Although China has 27 universities in the top 100, Taiwan with a much smaller population has 19. If we had to declare an overall winner in these rankings based on the number in the top one hundred per population then it would be Taiwan. Countries such as Brazil, South Africa, Thailand and the United Arab Emirates also did creditably.
At first glance, the star performer was Turkey. The number of Turkish universities has increased from seven to eight and it has as many institutions in the top ten as China. There have been some notable rises between 2013 and this year. Middle East Technical University in Ankara has gone from 9th to 3rd place and the University of Istanbul from 73rd to 51st.
On the other hand, some Turkish universities have declined. Boǧaziҫi University has fallen from 5th place to 7th, Bilkent University from 12th to 19th, and Koҫ University from 20th to 29th.
What happened to Turkish universities largely depended on just one indicator in the rankings, namely the Citations: Research Influence indicator.
Some Turkish universities have very high scores for citations -- METU scored 92 out of a maximum 100, Boǧaziҫi 96.8, Sabanci 88.5 – and they are much higher than those for any other indicator, including Research.
This is because, firstly, they have been involved in high profile physics projects that have a huge number of authors and are cited many times. The THE system used by data providers Thomson Reuters gives a great advantage to universities that get lots of citations in certain fields especially if they are within one or two years after publication. Then there is a “regional modification” which helps universities in countries with a low overall research impact. Also, these scores refer to impact per paper so that a few highly cited papers in physics can have a dramatic effect if there are not many uncited or infrequently cited papers to drag down the average.
So the prominence of some Turkish and some other universities is due largely to their participation in the Large Hadron Collider project or contributions to the Review of Particle Physics combined with either being small or not productive and being located in a country whose universities generally get few citations. Those Turkish universities that were not active in physics research did not do so well.
The Turkish achievement could be very transitory. A change in THE‘s methodology and it might evaporate. In fact, this year there has already been a change and it has had some impact.
In this year’s World University Rankings and in last year’s BRICS Plus rankings the citation indicator had a weighting of 30%. But in this year’s BRICS Plus it gets only 20% while the International Outlook indicators are boosted from 7.5 % to 10% and the Industry Income: Innovation indicator from 2.5% to 10%. The effect of this is to boost the scores of universities with a strong international presence or ties with industry while reducing those of universities taking part in citation-rich projects especially in particle physics.
Lomonosov Moscow State University did very well from this change. A score of 46 in the world rankings became 50, and enabled it move ahead of Boǧaziҫi, Sabanci and Istanbul Technical University.
The achievements of Chinese universities and some from Russia and Turkey may not be so brilliant after all.