SOM administrator questions Bloomberg’s MBA program rankings
Anjani Jain, the deputy dean for academic programs at the SOM, published two articles calling the rankings into question.
Kai Nip, Senior Photographer
After Bloomberg Businessweek ranked the Yale School of Management as the 12th-best MBA program in the country, one senior administrator at SOM is calling the ranking methodology into question.
Anjani Jain, the Deputy Dean for Academic Programs at the SOM, released two articles criticizing Businessweek’s annual ranking of MBA programs. Jain’s articles were published on Poets & Quants, a website dedicated to coverage of MBA programs. In his first article, released on Oct. 8, Jain argued that Businessweek’s rankings do not align with its published index scores and weights. On Oct. 14, Jain received a statement from Businessweek that claimed his analysis was inaccurate, to which he responded in an open letter published on his LinkedIn page.
“Bloomberg Businessweek’s response is disingenuous and nonsensical,” Jain wrote in his LinkedIn post. “In the two weeks since I brought the error to the editors’ attention, they have offered no explanation and have refused to answer simple questions I asked repeatedly, which could have identified the source of the error.”
According to Businessweek, MBA program rankings are determined by five criteria: compensation, learning, networking, entrepreneurship and diversity. Each of these criteria are assigned a number and then, using stated weights, calculated into a general score.
However, according to Jain, if the stated weights were applied to the given criteria, the ranking would have been drastically different. For example, Yale, which Businessweek ranked as 12th place, would by Jain’s calculations rise to eighth. Meanwhile, MIT Sloan and the University of Pennsylvania Wharton would fall from seventh and ninth to 21st and 28th respectively.
“So what is the root of the error? I don’t know and would rather not speculate,” Jain wrote in his LinkedIn post. “Media rankings of b-schools influence tens of thousands of prospective students each year, and participating schools should indeed compile the requested data with unimpeachable integrity and diligence. A parallel expectation of methodological rigor, transparency, and data integrity rests upon media organizations producing the rankings and seeking public trust.”
Jain has since also investigated Businessweek’s 2018 and 2019 rankings of MBA programs and found similar alleged issues. According to Jain’s calculations, across all three years, compensation is given nearly twice as much weight in reality than BBW claims, while learning actually contributes much less to a school’s rank. Jain noted that he found it “an odd coincidence” that Businessweek’s published rankings were “more plausible” than what would have resulted from a proper application of its methodology.
Following Jain’s article, a spokesperson for Businessweek criticized both Poets & Quants as well as Jain, saying that neither Yale nor the publication had access to the raw scores used in the calculations. The raw scores, Businessweek told Poets & Quants, differ from those given to each criteria online.
“By design, our proprietary ranking cannot be replicated or gamed using published data,” the Businessweek statement said. “Disclosing our raw data would create the possibility that the rankings could be reverse-engineered or gamed by a school for an unfair advantage. In addition, our methodology is repeatedly and carefully vetted by multiple data scientists. We fully stand by our ranking and our methodology.”
However, Jain said that the use of the released normalized scores should be enough to gain sufficient understanding of the methodology. For him, the possibility is either that Businessweek has committed a statistical error or something else “too troubling to contemplate.”
Two students at the School of Management agreed with Jain’s assessment.
“While Dean Jain’s findings are significant given their resulting changes to school rankings, what’s more stunning is their implications for integrity in business journalism,” said Penelope Williams SOM ’23. “BBW has either made a mistake they’re bafflingly unwilling to own up to, or concocted an intentional warping of data to produce the desired result. Whether or not readers use these rankings to make personal or professional judgements, such a warping would be an egregious abuse of trust.”
Seffi Kogen SOM ’23 agreed that Jain’s argument was persuasive, adding that he found Bloomberg’s rebuttals unable to truly address Jain’s main critiques.
Kogen also said that he was less concerned by SOM’s placement in the ranking than he was by UT Dallas Naveen Jindal School of Management’s, as it ranked 31st on Bloomberg’s list despite Jain’s calculations placing it in ninth. For Kogen, it appeared as though Jindal had been “cheated” of a ranking that would be more beneficial for a public school than a “top-tier business school.
In the long run, however, Jain believes that these rankings are not as significant as they may seem.
“To reduce the multidimensional complexity of business schools (or universities in general) to a simple linear order is an exercise in oversimplification (but people like 1-2-3 …!),” Jain wrote in an email to the News. “Rankings can influence the perceptions of prospective students and other stakeholders in the short run, but the long term reputation of an academic institution comes principally from the reach and impact of the knowledge created by scholars at the institution, and the reflected glory of accomplishments of its graduates.”
Jain has been associated with the School of Management since June 2012.