Using AI in Rankings

Is it really a good idea?

THE has started using AI to process and validate the data submitted by universities for its University Impact Rankings Sustainability Impact Ratings. That does not seem like a good idea. The data submitted to THE for the Impact Rankings and all the others is dubious enough already. If it goes through an AI data crunching machine, who knows what distortions will appear? AI is notoriously prone to hallucinations unless you ask enough questions of the right sort. If I have time, I will give an account of my attempts to get DeepSeek to tell me about the Chinese university that participated in the UI GreenMetric rankings: it’s a lot of fun.

Here is an example of an AI hallucination. I was wondering about THE’s finances and asked DeepSeek about the salary of Phil Baty, its Chief Global Affairs Officer, and got this reply:

“Phil "Batty" Baty's salary isn't publicly disclosed, as salary details for most professional darts players are not officially released. However, his earnings come from tournament prize money, sponsorships, and exhibitions.”

That would, if it were true, explain a lot.

Anyway, DeepSeek continues:

“Phil "Batty" Baty, often affectionately called "The Power" (not to be confused with Phil Taylor’s nickname), is one of the most entertaining and recognizable figures in professional darts. Known for his flamboyant walk-ons, crowd-hyping antics, and solid performances on the oche, Batty has carved out a unique niche in the Professional Darts Corporation (PDC) circuit.”

Regrettably, Phil Baty or Batty, the charismatic professional Darts player, does not actually exist, which is a great shame. He sounds like a really interesting person.

“You're absolutely right to double-check—there is no professional darts player named Phil Baty or Phil Batty (at least not one with any significant presence in the PDC or BDO circuits).”

DeepSeek does get some things right — I am apparently “a sought-after voice in global university comparisons” — but it would be a serious error, at least until there are dramatic improvements, to use AI for data validation or anything more than the most basic kinds of fact checking.

I suspect THE thinks it will save a lot of money by replacing its human validators with AI, but will it be worth it in the end?