UP CLOSE | Grad School scrutinized

Photo by David Burt.

When Graduate School of Arts and Sciences Dean Thomas Pollard asked directors of graduate studies for data about their programs last fall, department leaders saw red flags.

A two-phase review of mentoring practices at the Graduate School had just concluded months before. Some program administrators believed Pollard’s request could only be motivated by one concern: the Graduate School’s budget.

Pollard released the results of his analysis on Aug. 25, giving directors of graduate studies nine recommendations for improving mentoring and a list of corresponding best practices. He also provided median time-to-degree information for each graduate program, as well as rarely seen data detailing each program’s cost to the Graduate School.

As administrators examine the new data, many professors are perusing the statistics to see where they stand. For those in the humanities, the numbers may seem discouraging: Humanities Ph.D. candidates at Yale often take more time to complete their degrees than students in other disciplines, and they also cost the Graduate School more money on average. According to Pollard’s report, science and engineering students and professors often receive federal grants that cover more of their expenses.

Throughout the process of producing the report, Pollard said that the review was intended to increase transparency and help students complete the strongest possible dissertations on an efficient time scale rather than help the Graduate School distribute funds.

Still, many faculty members interviewed said they worry that administrators could use such data when they consider how to allocate resources in the future.

“I believe what the dean says,” said John Treat GRD ’82, director of graduate studies for the East Asian Language and Literatures Department. “But I don’t know what the next dean of the Graduate School will think.”

News of Yale’s unique take on self-evaluation has spread, and deans of other Ivy League schools have already expressed interest in adopting Pollard’s approach.

Graduate school administrators’ focus on cost and performance at their schools present a particular challenge to humanities programs, which are already struggling to justify their budgets as the academic jobs in the field grow increasingly scarce. Mark Bauerlein, former director of the Office of Research and Analysis at the National Endowment for the Arts and an English professor at Emory University, said humanities programs will need to find new ways to advocate for preserving their current consumption of university resources.

“Unless the humanities people come up with forms of justification of their own, someone else will do it for them,” he said. “The world has changed. … We need evidence. We need data.”


Pollard’s study follows a recent push by state universities to pay closer attention to costs and student outcomes, often in response to pressure from state governments to rein in spending. This led to quantitative analyses of programs, which higher education experts said they had not seen carried out with such rigor at private universities until Pollard conducted his study.

Much of the recent effort to increase productivity in universities has been focused on undergraduate education. The Lumina Foundation, a private, Indianapolis-based foundation which works to increase students’ access to higher education, recently held a conference for lawmakers and university administrators from 17 states on ways to enroll more students and increase graduation rates.

“The economic recession provided more impetus for state governments to focus on how to do more with less,” said Khadish Franklin, a spokesman for the Lumina Foundation’s College Productivity initiative.

The most high-profile statistical analysis of higher education costs came in Texas in September 2010, when the Texas A&M University System issued a “productivity spreadsheet” that listed individual professors’ net financial impact upon the school system given their salaries and the amount of revenue they generated for or drained from the system.

The data was made public to increase educators’ accountability to Texas taxpayers, said Heather Williams, a higher education policy analyst at the Texas Public Policy Foundation, which developed the model for the Texas A&M University System’s productivity spreadsheet.

“We would like to see more transparency within the universities,” she said. “We are interested in reducing the cost of college and seeing where the money is going.”

The Texas A&M University System analysis drew widespread criticism from professors, who questioned the implications and accuracy of the data. Pollard himself called the productivity spreadsheet “draconian,” and said other graduate school deans feel similarly.

Bauerlein said higher education administrators across the nation often observe the way controversial policies are received at other schools. Though less drastic than the Texas state system initiative, he said Pollard’s report is sure to draw the attention of administrators at other universities.

“The faculty will all hate it, but other administrators will be looking at it very carefully and wonder if they can do it at their campus,” Bauerlein said.


The emphasis on efficiency has not been limited to undergraduate institutions, said Frank Donoghue, director of graduate admissions in the English department at Ohio State University and author of the 2008 book “The Last Professors: The Corporate University and the Fate of the Humanities.”

Donoghue said graduate school administrators in the Big Ten conference, to which Ohio State University belongs, have also turned their attention to time-to-degree statistics and funding analyses.

On Sunday, administrators of several elite graduate schools were briefed on new uses of quantitative analysis: Pollard presented a report on his Graduate School analysis that day at a Washington, D.C., meeting of deans from “Ivy Plus” schools, which include every Ivy League institution save Dartmouth College, along with several high-ranking schools including Stanford University and the University of Chicago.

Peter Weber, dean of the Brown University Graduate School, and Cornell Graduate School Dean Barbara Knuth said they were impressed by the correlation between the use of Pollard’s recommendations and strong student outcomes at Yale — and both are interested in using some of Pollard’s tactics at their schools.

“I am intrigued by the aspects of the study relating to mentoring practices, and am considering following Yale’s lead,” Weber said.

Knuth said the Cornell Graduate School already meets many of Pollard’s recommendations, but she added that she will consider launching a similar investigation into the “size and sources” of funding for individual academic programs.

Still, Pollard’s analysis does not consist of numbers alone: The dean used statistical evidence to promote nine best practices, which are correlated with shorter time-to-degree and higher degree completion rates.

Pollard recommends that departments assign research projects early in students’ careers to prepare them for dissertation research and require students to present their work to their peers at least once per year. Other recommendations center around mentoring, suggesting that professors make an extra effort to regularly gauge students’ progress and give feedback.

All seven program heads interviewed said they already use many of the “best practices” in the report, but some said they plan to make improvements based on the information.

Meg Urry, chair of the Physics Department, said she will direct her thesis committees to meet on a more regular schedule and keep closer tabs on students. Eckart Frahm, director of graduate studies for the Near Eastern Languages and Civilizations Department, said he would consider giving qualifying exams at an earlier phase of the doctoral program.

For the majority of directors of graduate studies interviewed, though, Pollard’s recommendations are not groundbreaking.

In the Near Eastern Languages and Civilizations Department, Frahm said, students already present their work to each other even though they are not required to do so.

“We’ve done just about everything this report suggests on an ad hoc basis,” Treat said of the East Asian Languages and Literature Department. “If we [have to] formalize it to make [Pollard] happy, we’ll formalize it.”


Although slight adjustments might be all that some departments will need in order to fall in line with Pollard’s recommendations, humanities and social sciences professors said some suggestions suit science and engineering programs better than their own disciplines.

For example, Pollard recommends that advisers and students speak frequently, but professors said humanities researchers often work independently — either by necessity or by choice.

“There’s an increasing sense of trying to transition that model of lab work to other disciplines,” said Steven Smith, a political science professor. “There’s a lot of push back from those whose scholarship doesn’t fit. … What I do is pretty much impervious to that kind of work. I like to work on my own.”

Professors said they feel uncomfortable with the statistics in part because programs across the Graduate School have inherent differences.

Students in some fields may need more time to research their dissertations, they said, while other programs are so specialized that they compete with relatively few other universities for the most talented applicants in the field.

While Treat said he would prefer Yale’s East Asian Languages and Literatures Department were compared to peer programs across the country rather than other departments at Yale, a recent attempt at this kind of analysis fell flat.

The National Research Council released a much-anticipated set of rankings for roughly 5,000 doctoral programs in September 2010.

But the council used data from the 2005-’06 academic year to give programs a range of possible rankings instead of giving each program a hard-and-fast numerical ranking. The method and age of the data led professors from Yale and universities across the country to call the finished product outdated and inaccurate. Since last fall, the NRC has issued a series of corrections that address both input errors and methodological problems.

Pollard acknowledged limitations on some of the data that he collected from departments over the past year. For example, yield — which measures the percentage of students accepted who decide to matriculate — is influenced not only by the quality of Yale’s program, but by whether other programs across the country exist that may compete with Yale for applicants.

Professors still value the yield statistics: Dale Martin said he is not worried about the future of the Religious Studies Department, for which he is director of graduate studies, largely because his department’s yield is among the highest in Pollard’s report.

“Some departments feel like they got a rougher deal than others,” Martin said. “We came out looking good.”

Pollard said the median time-to-degree statistic can be difficult to interpret by itself because it does not take into account students who never complete their degrees. To help correct for this problem, Pollard used both time-to-degree statistics and completion rates to show a correlation between the use of his best practices and strong performance, and he said the correlation is strong but “noisy.”

As universities across the nation focus more on quantitative approaches, they face the challenge of using statistics that are often imprecise and hard to interpret.


Students and faculty at Yale and other universities have accused their schools of acting more like businesses than educational institutions in using statistics and financial data to evaluate academic programs.

Gaye Tuchman, a sociology professor at the University of Connecticut and the author of the book “Wannabe U: Inside the Corporate University,” said she was surprised to see an elite school like Yale produce a report like Pollard’s.

“[Pollard] has made some interesting recommendations that may improve the learning experience of doctoral students, but I wasn’t quite sure if he was interested in improving the learning experience of Yale students or improving the production of Ph.D.s at the university,” Tuchman said. “One of the problems with the model of the corporate university is that it is about producing students with degrees as much as it is about providing better education.”

On Feb. 9, Yale’s Graduate Employees and Students Organization (GESO) presented Provost Peter Salovey with a 15-page report called “Yale, Inc.: The Corporate Model in Higher Education.” The report argues that University administrators fail to adequately consult with faculty members and students when making decisions that affect them.

Graduate Student Assembly chair Andrea Stavoe GRD ’14 said Pollard met with members of the student government to discuss his findings, adding that the group read and offered suggestions on an initial draft of Pollard’s report.

Now that Pollard has publicly released the final report, Stavoe said, GSA members will discuss the recommendations amongst themselves and with the graduate students community, and will bring their findings to Pollard. She said that in recent conversations, humanities and social sciences students have registered concerns about the usefulness of the recommendations in their disciplines — much like faculty members in those fields have.

Pollard added that he is in the process of meeting with representatives from each of the programs to discuss his recommendations.

GESO co-chair Stephanie Greenlea GRD ’12 said Pollard’s collaboration is a “step in the right direction,” but she said administrators should still “include more academic workers in academic processes.”

Despite protests from some students and professors, higher education experts said they expect analyses of productivity and efficiency to grow more popular as time goes on — and at many schools, a focus on cost may leave some humanities programs looking weak against other disciplines.


Bauerlein, the Emory University English professor and former director of the National Endowment for the Arts’ Office of Research and Analysis, was candid about what he viewed as the diminishing value of a humanities doctorate: “It just doesn’t make sense for people to go to school in the humanities,” he said.

In addition to costing universities an undeniably high amount compared to the sciences, Bauerlein said humanities programs produce Ph.D.s who struggle to find work in a shrinking academic job market.

Maurice Samuels, director of graduate studies for the French Department, said even though humanities jobs are difficult to find, Yale’s programs produce the best-equipped graduates in the market.

“There are still jobs, and someone has to get them, so why shouldn’t it be Yale?” Samuels said.

In some respects, Yale’s humanities departments may be winning the numbers game: The disputed 2010 National Research Council rankings placed many of Yale’s humanities programs at the top of the nation’s offerings, and Yale’s humanities departments as a whole outranked the University’s science departments.

Frances Rosenbluth, deputy provost for faculty development, said the University will not jeopardize its top-ranked humanities offerings by underfunding them.

“Protecting ranking is one goal of the University in allocating resources,” she said. “We’re not going to give up something we’re best at to be second best at something else.”

Still, even at Yale, administrators must make sure they are granting degrees in fields where they have a strong chance of finding positions after they graduate: Salovey said the University takes into account the number of job openings available for new Ph.D.s when determining how many applicants to admit to graduate programs.

Some humanities professors said that doctoral degrees in humanities programs prepare students to think critically and excel in fields other than academia. Others argued that humanities scholarship contributes to society in a way that cannot be quantified — the humanities engage “the life of the mind for its own sake,” said Millicent Marcus GRD ’73, director of graduate studies for the Italian Department.

But some higher education experts said that such arguments carry less weight as universities across the country rely on quantitative analyses and shift their focus towards improving science and engineering offerings. Pollard acknowledged that a “new balance will have to be achieved across the whole country” to meet the shifting demands of the academic job market.

“[Humanities programs] have cause for concern,” Bauerlein said. “If they can’t show their value, then they’ll find their resources will be trimmed.”


  • The Anti-Yale

    And so the idolatry of Mercantilia spreads, as the Great God DATA, enlists new worshipers. Soon, the trumpet will sound daily from Harkness Tower and all will turn and bow their heads, as the latest productivity statistics are broadcast amidst chants of “Benchmarks!, Rubrics, Outcomes !. Hosanna to the Highest !”

    And the digitized sheep will follow, like Balanchine dancers, in perfect unison.

    God Bless the Bill and Melinda Gradgrind Foundation.

  • phosphate

    this can only end badly. Why all this faith in quantification? It worked out really well for Wall street hasnt it.

  • attila

    I guess poser’s gotta pose.

    But it’s infantile to call this “DATA” or “quantification” as if that is really what the exercise is about. The point is to look at grad programs in general, and identify what they are doing and how they can be improved.

    Who can object to that (besides the posers?)

    By the way, the paper should dig into individual department stats. Those broad categories “humanities” etc conceal most of the interesting differences. We have top grad programs in most areas. And a few departments should probably not have grad programs at all, given the weak students that seem to be all they can attract. (Yes, sociology, I am talking about you)

  • Waldemar

    It is encouraging to see graduate education subjected to this kind of scrutiny, and it is a credit to Yale to be taking the lead. This kind of information is almost never revealed. That graduate programs have become career dead-ends (not to mention miserable experiences) for far too many people is becoming apparent from sorrowful blogs like “100 reasons NOT to go to graduate school”: http://100rsns.blogspot.com/

    Now that it is clear that there is a high cost to the institution for enrolling these grad students (at least in the humanities and social sciences) perhaps there will be less incentive for universities to allow departments to lure so many people into the grad school trap.

  • The Anti-Yale

    Not “posing” at all. I sit in horror watching the worshippers of DATA as it oozes like magma into every orifice of our modern day Pompeii. Soon three Wise Men bearing gifts of frankincence and myrrh will be frozen in place in front of an eternally embedded manger honoring in perpetuity the new born deity.
    Praise Father, Son, and Bill and Melinda Gradgrind Foundation.


    Ours is a mindless idolatry of digits and dollars.


    Paul D. Keane

    M. Div. ’80

    M.A., M.Ed.

  • phosphate

    There are many ways to examine “grad programs in general, and identify what they are doing and how they can be improve”, but to rely on statistics (particularly where the Ns are small) doesn’t contribute to understanding.

    For the most part these are administrative exercises looking to justify an end or policy that has already been established but not yet implemented.

    Its fine to cut the humanities, Yale has never been as important as Harvard or Princeton in the 19th century, or Stanford, Chicago, Cornell or Johns Hopkins in the 20th so no biggie.

    Anyway its sort of fun to watch science technocrats who believe too strongly in empiricism grapple with the complexity of institutional citizenship in a university.

  • The Anti-Yale

    Right on!

  • attila

    PK confuses evaluation with the use of numbers. Surely it is appropriate for the Dean to figure out what each grad program is doing well and what it is doing badly. Some of these questions are reasonably answered with a number. Some are not. But to claim that we should not figure out what is good and bad is just posing. People with real responsibilities cannot just throw up their hands and pretend to be too subtle to think about it.

    References to “small N” can be just obscurantist. If everyone in program X completes the PhD in 5 years, and nobody in program Y does so, it does not matter what N is in each case. And it does not naturally lead to the conclusion that either X or Y is doing something wrong. Grown-ups would then have a conversation. Y might be in a field where 6 years makes more sense.

    BTW, Mr Poser Extraordinaire, the Nazi salute was “Sieg Heil,” not “Zeig Heil.”

  • The Anti-Yale

    Thanks. Hadn’t noticed the error. Have 60 blogs after all. Always happy to be corrected.

    I am not particularly irritated by THIS dean’s instance of numerical fetishism; I AM interested in the culture’s wholesale abdication of thoughtful commentary to the Imperium of DATA.

    I will be long dead in a few decades. It will be of no concern to me.

    You will be gagging in the stifling tight-jacket of statistics.

    I simply wish to project on to the screen for your consideration the possible soul-less future we are creating by our mindless embrace,and idolatrous worship, of DATA.

    Return to your keyboards and screens.

  • b

    Paul Keane, I would humbly recommend you spent a bit less time on your sixty blogs and pick up a book on numerical reasoning – perhaps “Innumeracy”, as one example of many. Numbers and data contribute to understanding, and understanding isn’t something to be feared.

    The only reason anyone will ever ‘gag in the stifling tight-jacket of statistics’ is if they never learn to process it. It would be like me warning that someone who can’t swim might ‘gag in the stifling, swirling waters of the oceans’. Guess what? Learn to swim, and being in the ocean becomes something exciting and rewarding, not something to be fearful of.



  • The Anti-Yale

    “There are three kinds of lies: lies, damned lies, and statistics.” :popularised in the United States by Mark Twain (among others), who attributed it to the 19th-century British Prime Minister Benjamin Disraeli (1804–1881):

    **Here’s data for ya:** for the price of the Viet Nam War we could have built every citizen of that divided country a seven-room, air conditioned, split level house with a two car garage. ( a popular statistic promulgated by those who do not understand that war is part of human nature, like Quakers)

    **Here’s more data for ya**: The United States used to rank first and now ranks 17th among nations in reading and writing ( a popular fear mongering statistic promulgated by the Bill and Melinda Gradgrind Foundation)

    ( To find out what this alarmist statistic really means READ Adam Gopnik’s brilliant September 12 *New Yorker* piece:

    *Decline, Fall, Rinse, Repeat
    Is America going down?*
    http://www.newyorker.com/reporting/2011/09/12/110912fa_fact_gopnik )

  • b

    I’m well aware of the quote – it’s a favorite of mine. But a witty quote doesn’t always speak the whole truth. And yes, those two bits of information are indeed data – and thus contribute to our understanding of issues. Data contributes to understanding, period. It doesn’t mean people always make the RIGHT decision, but it probably means they’re more likely to do so than if they had no data.

    Becoming comfortable with data and numbers doesn’t mean you simply take whatever statistic is thrown at you as gospel, it means you learn to discern what is and isn’t truthful, and use it to guide decisions. That’s still a far cry better than simply throwing one’s hands up, claiming there are simply far too many *numbers*, and ignoring the truths those numbers may contain once you tease it out of them.

    I will read that New Yorker article. How about you promise to read a book like ‘Innumeracy’? Since we both may learn something, we both may win.