US News solicits input on rankings

US News & World Report, publisher of an oft-cited yet highly controversial set of annual college rankings, is trying to improve its image by diversifying the rank criteria it uses, but students, administrators and experts agree: The potential for progress is minimal.

In response to criticism from parents and educators that the US News ranking method fosters superficial perceptions of colleges, the magazine announced last week that it is distributing surveys to the guidance departments of all 1,600 high schools that placed in the 2008 US News’ America’s Best High Schools rankings. Counselors at those schools will be asked to evaluate the undergraduate programs of both national universities and liberal-arts colleges, although US News has not revealed how or whether the survey results will factor into a new iteration of its rankings.

Yale’s Law School ranked first in the recently-released US News report on graduate school programs.
Aileen Agricola
Yale’s Law School ranked first in the recently-released US News report on graduate school programs.

“We think it’s responsible to wait until we analyze the results before we commit to how to utilize this information,” Robert Morse, director of data research for US News & World Report, said on his blog, “Morse Code,” on the magazine’s Web site. “Nevertheless, our goal is to publish the findings in some form in the upcoming 2009 edition of America’s Best Colleges.”

US News & World Report has come under scrutiny from educators and parents in recent months for its ranking criteria and practice of comparing, on an absolute scale, schools with different missions and educational philosophies. Despite its consistent top-three ranking, Yale has added to the criticism, donating $30,000 to the Education Conservancy, an organization working to create an online alternative to the US News rankings.

“The real problem is that U.S. News both decides on what criteria will count and how each of those criteria is weighted, then puts schools on an ordinal scale based on that quite rigid formula,” Dean of Undergraduate Admissions Jeff Brenzel wrote in an e-mail. “My basic view is that the rankings are a highly artificial way of comparing highly diverse colleges.”

Brenzel said he is not convinced that the inclusion of high-school counselors’ opinions would increase the accuracy of the rankings, as it is yet not clear how the magazine will use the evaluations.

In an article published by Inside Higher Ed, Lloyd Thacker, president of the Education Conservancy, said the high-school-counselor surveys prove US News is taking such criticism to heart. The magazine is expanding its input data to give the appearance of improvement, he said.

“These changes reveal that they are acknowledging the lack of precision and the impropriety attached to this instrument,” Thacker said in the article.

Yet Rahul Dalal ’11, who conducted independent research on the US News ranking system for a class this year, said input from high-school counselors could even make the accuracy of the rankings suffer.

“It increases the methodology’s subjectivity,” Dalal said. “The rankings will be more representative of the public perception of schools, but this is not always accurate because it is not based on hard data.”

But these additional data could be beneficial for Yale, Dalal said, as public perception is often based on prestige.

“The input of high-school counselors may improve Yale’s ranking because Yale generally follows Harvard in name recognition,” he said. “High-school counselors … do not necessarily have the proper experience and qualifications to evaluate colleges, and their input may be largely based on an institution’s reputation.”

Corrina Li ’11 said including the opinions of high-school counselors will not improve the US News rankings because they are subjective to begin with.

“The strength of a school is not measured by a formula, and I think the way the rankings are done is very formulaic,” Li said. “That’s why we have things like Bulldogs Days — so that students can experience Yale.”

But Alexa Chu ’11 said she thinks the data collected from the counselors could be a worthwhile addition to the US News’ methodology. The rankings are compiled by professional statisticians, she said, and the counselors have the insider’s perspective that is missing.

“A lot of the college counselors go to visit colleges, they hear back from alums,” Chu said. “They know what students are looking for.”

A current component of the US News score formulation is college administrators’ reviews of their peer institutions.

Both Brenzel and Thacker said they think US News should completely abandon numerical college rankings because it is difficult to quantify a student’s college experience. Brenzel said he would be in favor of an online database in which students and families could choose which criteria are important to them, weigh those criteria as they think best and then use that data to compare schools.

The Education Conservancy project, which features a similar methodology, is currently under development. Since May 2007, 65 university presidents and chancellors have signed the Education Conservancy’s “Beyond the Ranking” letter, which commits the undersigned parties to work towards a better alternative to the US News rankings.

The 2009 edition of America’s Best Colleges will be released by US News & World Report in August 2008.

Comments

  • David Bowie's Ch-Ch-Ch-Changes

    I support any change to US News' ranking methodology which moves Yale ahead of Princeton or, at the least, ahead of Harvard for #2.