Power failure causes campus Web outage

The University’s Internet service went dark Friday for more than an hour after a power failure in an Information Technology Services equipment room.

Students were jolted into an unwanted Facebook reprieve at 1:38 p.m., and not all of campus was back online until at least 3 p.m., ITS officials said. It was the most significant unplanned outage in recent memory.

Surprisingly, perhaps, the downtime was traced to a hiccup in the basement of Commons, the University’s largest dining hall, where several key components of Yale’s Internet and telephone systems are located.

The source of the outage was the failure of a power supply in one of ITS’s equipment rooms in the Commons basement, which caused several network backbone routers to lose power, said Joseph Paolillo, senior director of infrastructure services for ITS.

ITS staff quickly identified the failure and rerouted power around the failing power supply, but network connectivity and Internet access were lost around much of campus in the meantime. Most service was restored by about 2:30 p.m., ITS officials said, although some parts of campus incurred continuing connectivity problems for about another half hour.

Once the frightening episode was over, ITS sent an e-mail to the University community later in the afternoon apologizing for the outage.

“ITS thanks you for your patience,” it said, “and regrets any inconvenience this outage may have caused.”

The power failure did not disrupt the University’s utility plant or the power distribution around campus, said Louis Annino Jr., the director of facilities infrastructure.

Ironically, the outage was caused by the failure of a system that is part of Yale’s strategy for ensuring that the University’s IT systems are not crippled by a power failure. The equipment room — which also includes a major telephone switch for the University’s phone system — is fed by two separate external power feeds, Paolillo said, with separate interruptible power supplies and battery backups for both the network equipment and the telephone system.

ITS will be replacing the failed power supply, he said, “and performing a thorough evaluation of the emergency power system.”

Friday’s outage came just three weeks after another power outage left most of the campus in the dark for an entire weekday afternoon. That outage occurred after a construction vehicle at Kroon Hall, the new home for the School of Forestry and Environmental Studies, struck an underground, high-voltage power line on Mar. 6, sending much of the campus into the dark and also knocking out Yale’s telephone system for much of the afternoon.

That outage — and, to a lesser extent, Friday’s internet loss — brought administrative offices around campus to a halt. The extent to which University affiliates rely on e-mail, for instance, is staggering: On an average day during the school year, members of the Yale community receive some 400,000 to 600,000 e-mail messages, according to ITS statistics.

Comments