Researchers explore the senses

If humans are trying to get a good perception of the world around them, Yale cognitive psychologists say, they shouldn’t trust their eyes.

Researchers working in cognitive psychology at the University are currently focusing on how the brain interprets visual information. Psychology professor Brian Scholl, who runs the Perception and Cognition Laboratory, said that vision-related studies now account for about 90 percent of the work done at the lab. Another psychology professor, Marvin Chun, is currently studying how the brain embellishes and extrapolates upon incoming visual information. Concurrently, psychology student Jonathan Flombaum GRD ’08, who works with Scholl, has been investigating how the brain is able to continuously track objects that temporarily disappear.

“Our intuitions about how we see and how our minds work in general are usually wrong,” Flombaum said. “We need to study these processes explicitly and experimentally to understand what’s really going on in our brains. The way we experience the world visually is very different from how the brain actually processes visual information.”

Scholl said the paradox of visual cognition is a simultaneous lack and overabundance of information.

“There are two themes in visual cognition and cognitive science in general — first of all that there is not enough information, and second, that there is too much information,” he said. “There is too little information in that vision is about trying to recover the structure of the world from the light that hits our eyes. There is too much information in that we would be completely overwhelmed if we were to process all incoming information.”

Scholl said the second half of the visual cognition paradox — that there is too much information to be processed by the brain — is the result of a phenomenon known as inattentional blindness.

“Inattentional blindness is an incredibly shocking phenomena whereby you are completely unaware of salient information right in front of you,” he said. “When you’re talking on a cell phone, for example, the level of inattentional blindness skyrockets.”

Flombaum said the brain is forced to make inferences to connect the tiny amount of light collected by two-dimensional retinas with the large three-dimensional world full of rich details. The brain uses various tricks to make this connection, Flombaum said, such as using the distance between a person’s two eyes, the brain is able to trigonometrically triangulate how far away something is based on the slightly different images seen by each retina.

One of Chun’s many ongoing experiments involves brain imaging to track how individuals extrapolate beyond the edges of their vision.

“What we’ve found is that when you’re looking out at a given scene you’re only seeing a limited portion of what’s really there, but when you look at the limited portion your brain extrapolates beyond the boundaries of what’s actually seen,” Chun said. “Basically, the brain represents more information than is actually coming through the eyes.”

Chun said that while it has already been demonstrated that the brain will extrapolate to see behind objects obstructing one’s view, this study is novel because it shows that the brain extrapolates to fill in information beyond the scope of vision. He added that his lab is more broadly concerned with trying to understand how people perceive the world and how visual brain machinery operates.

“How can we recognize, see and pay attention to certain things while simultaneously ignoring others?” he asked. “We are also interested in visual memory — the brain imaging involved when you see a new face or location and create a visual memory of it.”

Flombaum is involved with studying how people are able to track an object and continuously represent them while they disappear behind other objects.

“We’re basically looking at how we don’t get tripped up by the fact that objects move and disappear,” he said. “In the experiment we’re doing right now, we’re asking, ‘How can we visually follow something even when we sometimes can’t see it?’”

Flombaum said the brain is able to track something that temporarily disappears — behind a wall, for example — by predicting its trajectory and relying on heuristics. He said that if an object reappears where the brain expects it to based on when and where it disappeared, an individual will usually recognize it as the same object, whether it is or not. In one experiment involving monkeys, Flombaum said, when the monkeys saw a lemon go behind a wall and a kiwi roll out from behind the other end, brain imaging showed that they thought it was the same object.