University Rankings....

Earlier this month, the Times Higher Education Supplment came out with its latest university ranking. And if you live in Hong Kong, you would have had to turn off all news media to have failed to learn that my employer, HKU, ranked 18th worldwide! That's an impressive performance by any means, and the University is rightly trumpeting its performance.

But university rankings are inherently problematic. In this case, the THES takes a series of relevant indicators (peer review, employer review, international staff, international students, staff/student ratio, and citations), combines them with a formula, and comes out with a number that can be compared between universities.

In general, these indicators are proxies for latent variables that cannot be accurately measured, and may be poor proxies at that. For example, I suspect the "international" variables are trying to capture the extent to which the university encourages its students to think outside the box, and take into account other viewpoints and perspectives. But just having international students and international staff is neither necessary nor sufficient for this to occur.

There are likely to be significant measurement issues as well. Who exactly are the peers and employers who are asked about their views of different universities? Is the sample biased towards one country, or one demographic? How do we count international students, and international staff? Are mainland students in HK domestic or international? Are exchange students, who come to HKU for one semester or one year, treated identically to international students who come for their complete degree program? Is "international" determined by passport, or birthplace? In the staff count for the staff-student ratio, who is included? Only permanent, tenured staff? Gardeners and cleaners? I could go on....

There are two reasonable responses to these problems. One would be to dismiss these rankings as noisy indicators of very little. The other is to treat the noise in the ranking as sampling error, and try to increase the sampling size, by considering more university rankings.

Unfortunately there's only one other well-known global university ranking, by Shanghai Jiao Tong University. And where does HKU rank here? Somewhere between 203rd and 304th.

Oops.... I guess we need a lot more rankings than two to get any accurate idea of where HKU really stands!