Researchers tackle allegations of bias in college football ranking

College football fans disappointed by Sunday’s Bowl Championship Series (BCS) selection show now have a Yale professor’s study supporting allegations of bias and conflicts of interest in the coaches’ poll, which helps determines BCS standings.

In a working paper released by the National Bureau of Economic Research this November, Yale School of Forestry and Environmental Studies professor Matthew Kotchen and University of California Santa Barbara professor of corporate environmental management Matthew Potoski studied whether college football coaches’ “reputation and financial incentives” bias the USA Today coaches’ poll. The coaches’ poll of approximately 60 voting coaches, published weekly during the football season by USA Today, is one of three ranking systems used to determine teams’ BCS standing, which in turn decide which two teams receive invitations to the National Championship Game and influence the teams invited to the four other BCS bowl games. The researchers analyzed coaches’ ballots from 2005 to 2010 and concluded that coaches distort the ranking system in favor of their team.

“A lot of people raise these same issues and concerns in the media by referencing the decision of one coach,” Kotchen said. “What we did is show that it is systematically occurring.”

The researchers studied bias in the poll by calculating the difference between the coaches’ poll and the BCS computer based ranking system, in addition to the deviation between individual coaches’ ballots and the average rankings of approximately 60 coaches’ ballots. Both statistical approaches demonstrated that coaches’ votes were biased by as much as two spots in favor of their team, as much as one spot in favor of teams in their athletic conference, and as much as half a spot in favor of teams they had defeated.

The researchers also analyzed the impact on bias of disclosing the final regular season ballots, a practice which began in 2005 following a 2004 controversy in which Texas coach Mack Brown lobbied for his team to be ranked above California in order to receive the last BCS bowl invitation. They compared the deviation between the computer polls and the coaches’ poll before and after the disclosure began in 2005, concluding that bias in the coaches’ poll could be reduced if the ballots were made public throughout the season. All other weeks of the coaches’ vote remained private in 2011.

“It is hard to come up with a reason for why these ballots should continue to be confidential,” Kotchen said.

But Grant Teaff, executive director of the American Football Coaches’ Association (AFCA), the organization which conducts the coaches’ poll, said he does not expect the in-season ballots to be released. Gallup auditors recommended that the polls remain confidential in order to produce the “best results,” he said.

Bob Tortora, chief methodologist for Gallup, stated that the recommendation to keep ballots private because of concerns that “peer pressure” among coaches could skew the results of the poll. He said there was also a worry that public ballots might decrease coaches’ participation rates. Tortora declined to comment on whether the results of the study contradict the audit because he had not read the paper.

“Is it perfect — of course not,” Teaff said. “Is there bias? Of course the coaches think highly of their teams. That’s not bias. If they weren’t that way, they wouldn’t keep their jobs.”

He added that “even with its frailties” the poll has been “amazingly accurate” throughout its 62-year history.

Yale football head coach Tom Williams explained in an email to the News that while bias in the BCS does not impact the Yale football team, which plays in a different subdivision, he said he does think that there is a “natural bias” in the coaches’ poll because coaches often do not watch teams that play outside of their conference and thus “tend to vote for what they know.”

Because the coaches may not have enough time to watch every game each week, they relying on their biases, instincts, public opinion and media coverage, said Allen Sanderson, University of Chicago economics professor and member of the editorial board for the Journal of Sports Economics. Sanderson said he thought the methodologies of the study were sound and added that the problems of “imperfect information” in voting are further compounded by financial motivations.

The researchers concluded that financial incentives from the allocation of the revenue the bowl games generate skew the voting. The current system gives non-BCS teams a piece of the payoff awarded from BCS bowl games when another non-BCS team participates, while BCS conference teams receive a portion of the payoffs if a team in their own conference participates. The working paper states that net revenue from BCS bowl games were approximately $182 million in 2010.

The financial incentives led the coaches of teams in the six BCS conferences to rank the teams in their conference higher if those teams were on the cusp of receiving a bowl game invitation, and non-BCS coaches to rank non-BCS conference teams higher. The study found that a payoff of between 3.3 and five million dollars resulted in an increased ranking of one position, and larger payoffs had an even greater effect.

“I became interested in this study not just because of football but because studying conflicts of interest are a much bigger phenomenom in all aspects of economics and political activity,” Kotchen said. He added that the study’s results made him skeptical that voters can control their biases.

Football rankings, he said, were a uniquely good data set for studying conflicts of interest, because the financial incentives are clear and a biased evaluation is apparent.

In the final 2011 coaches’ poll released on Dec. 4, coaches from Auburn, LSU, Georgia and South Carolina all voted fellow Southeastern Conference (SEC) team Alabama second while coaches from Baylor, Oklahoma, Iowa State and Texas Tech all voted fellow Big 12 team Oklahoma State second. LSU will play Alabama in the National Championship Game Jan. 9 in New Orleans.

Comments