National Research Council ‘rankings’ take new methodology

After a 15-year wait, university administrators across the country were supposed to be able to compare their graduate programs using new rankings released by the National Research Council Tuesday. But the new rankings may raise more questions than they answer.

In a memo to University President Richard Levin, Provost Peter Salovey, School of Medicine Dean Robert Alpern and faculty of the Graduate School Monday, Graduate School Dean Thomas Pollard said that while the NRC’s new report — posted on the National Academies Press website Tuesday afternoon — ranked 4,838 doctoral programs in 62 fields at 222 American universities, the word “rank” may be misleading. Yale’s Office of Institutional Research will release its own formal analysis of the rankings, and the Graduate School hopes to eventually use the data for departmental self-evaluations.

“The analysis did not assign a unique rank to each doctoral program, making it difficult to compare the outcome with previous studies, which rank-ordered programs,” Pollard said in the memo.

The NRC tries to release an exhaustive report on American doctoral programs once per decade, according to its website. In an attempt to make the rankings more quantitative and less subjective than the set released in 1995, the NRC redesigned its research methodology to incorporate more hard numbers.

Each program is now ranked according to R-rankings ­— regression-based rankings reflecting programs’ reputations among faculty in the field — and S-rankings, based on data from a 2005-’06 survey that covered 20 subject areas and 19 for the humanities, including average length of time needed to complete a doctorate. Programs are also ranked by three “illustrative rankings,” which are intended to help individual readers use the data to compare programs based on the characteristics most important to them. The illustrative rankings include research activity, student financial support and outcomes and the diversity of the academic environment.

Though the NRC organized doctoral programs into ranked lists, the NRC declined to assign programs hard and fast scores, opting instead for ranges. For example, Yale’s American Studies program is ranked 1-2 in R-rankings and 1-4 in S-rankings, but 12-16 in diversity in the academic environment. Not all of the scores were given as a range, however. The NRC gave Yale’s American Studies program a 100 percent score for first-year financial support, meaning all first-year doctoral students in the program were given full financial support.

In an interview with the Chronicle of Higher Education Monday, Jeremiah P. Ostriker, the chairman of the NRC project committee and a professor of astrophysics at Princeton University, said the range rankings were used because the data itself contains “many sources of uncertainty.”

“That means that we can’t say that this is the 10th-best program by such-and-such criteria,” Ostriker said. “Instead, we can say that it’s between fifth and 20th, where that range includes a 90 percent confidence level. It’s a little unsatisfactory, but at least it’s honest.”

Though the OIR is still working on its own analysis of the NRC data, Pollard said the Graduate School will hold a meeting for people interested in the report later this week or next week, depending on the level of interest in the community.

“We are hoping to have programs take a look at the data as part of a self-evaluation,” Pollard said, adding that he will consult with department chairs about the project.

In his memo, Pollard said that 742 members of the graduate Faculty of Arts and Sciences in 55 of Yale’s doctoral programs participated in the survey, though 959 received the survey. For the NRC’s purposes, Pollard said, the faculty were assigned to graduate programs according to their service on thesis committees. The NRC also surveyed students in English, economics, physics, neuroscience and neurobiology and chemical engineering, Pollard said, and 113 Yale graduate students responded of the 163 who received the survey.

Comments

  • Prof3

    This article misses the point. Yes, they give ranges, and different kinds of scores. But a number of important departments at Yale did very badly (the entire range was bad, and all scores were bad). The university needs to take this seriously. They can talk about excellence blah blah blah, but it is clear that many Yale departments are much weaker than one would want at a university that claims to be one of best in the world.

  • Prof3

    I really hope the Daily *News* does some reporting on this, and does not just accept the line the administration wants to put out. Here is the basic range information for some big departments at Yale:

    Poli Sci 4-8
    History 5-15
    Econ 5-10
    Psych 4-11
    English 2-12

    These are all good, although they call into question claims that Yale has, for example, the #1 history department. More discouraging are sociology (18-46; this means SUNY-Albany has a better department!) and Physics (15-32).

    A first-rate university should have a number of departments ranked #1, and almost none below #10 or so. The NRC study says that only a small handful of Yale departments have any claim to #1 status. Perhaps the administration could focus its attention on strengthening the university they have instead of starting a new venture in Singapore.