Doctors: Not as smart as you thought

Recent findings suggest that a majority of medical residents make diagnoses based on statistics they actually do not have the background to understand.

As the body of medical literature continues to grow and to reference increasingly complex analytical methods, the study — conducted by Yale School of Medicine researchers — shows that 75 percent of resident physicians do not understand the biostatistics they regularly encounter in clinical research papers. But 58 percent of the same group also said they use statistical information when making medical decisions.

“Most of the residents in the study lacked the knowledge in biostatistics needed to interpret many of the results in published clinical research,” said Donna Windish, an assistant professor of medicine at the Yale School of Medicine and co-author of the study. “Physicians depend on medical literature to adapt to new trends in clinical practice. A concern that comes out of these results is that incorrect interpretation may lead to erroneous applications of clinical research.”

The study’s 277 respondents — medical residents hailing from 11 resident programs around the state — correctly answered an average of just 41.4 percent of multiple-choice questions testing their interpretation of the statistics used in clinical research studies, despite the fact that more than 68 percent of participants said they had some previous training in biostatistics.

“The poor knowledge in biostatistics likely reflects in sufficient training,” Windish said. ”With so many changes and advances in the medical field, there’s just so much information that medical schools have to get across to students, and statistics has not been considered a primary topic.”

When training does occur, the majority takes place during medical school and is not reinforced during residency, Windish said. A comprehensive study of medical school biostatistics instruction conducted in the 1990s found that statistics is typically taught in the first years of medical programs and is not revisited later on. The study also found a significant variation in the depth and breadth of instruction in the subject across schools.

The Yale School of Medicine — like many of its peer institutions — introduces biostatistics to its students early in the game, requiring students to take two courses in the subject over their first and second years.

Qi Zheng MED ’09 said she felt the course gave her a good grounding in statistical analysis, giving her confidence in her ability to read and evaluate statistics from published research. But Dayo Fadelu MED ’09 said that while such skills may be well-honed immediately after a class is completed, they risk growing dusty from disuse over time.

“With a lot of things you learn, you sort of lose them as you go on,” he said. “So if you’re a physician not involved with academic medicine, it’s not surprising you’d forget basic statistics.”

Indeed, the study showed that residents further along in their programs perform worse than their junior counterparts, possibly reflecting the deterioration of skills over time.

Fedalu added that the biostatistics course gives students a good background in the basics, but eschews more complex topics that may crop up in medical journals.

This lack of comprehensive coverage is worrying, Windish said, especially considering that medical literature is increasingly making use of more complicated statistical methods. She added that it is crucial for residency programs to include more effective biostatistics education, and to re-evaluate how this information is taught and reinforced.

“My impression working with medical students is that this study is accurate and not at all surprising,” said Robert Dubrow, director of medical studies and associate dean of academic affairs at the Yale School of Public Health. “We need to work towards better education for residents and medical students in the area of biostatistics, and study design and the intelligent reading of medical literature. This is an area definitely lacking in the field of medical education.”

But the trend may already be towards an increased focus on instruction in quantitative methods. Ting Qian MED ’11 said that she believed the study may reflect older approaches in medical education, since the participants in the study attended medical school up to a decade ago. At that time, biostatistics was relatively new and less important to the practice of medicine.

“Medicine is definitely becoming more and more quantitative,” she said. “Now, biostatistics is much more important to know. We even have a section [on biostatistics] on our Board exams.”

Of the respondents to the study, 95 percent agreed or strongly agreed that it is necessary to know something about statistics in order to be an intelligent reader of medical literature, and 77 percent expressed a desire to learn more about statistics.