Our eyes are continuously bombarded by an enormous amount of visual information – millions of shapes, colours and ever-changing motion all around us. For the brain, this is no easy feat. On the one hand, the visual world alters continuously because of changes in light, viewpoint and other factors. On the other, our visual input constantly changes due to blinking and the fact that our eyes, head and body are frequently in motion.
To get an idea of the “noisiness” of this visual input, place a phone in front of your eyes and record a live video while you are walking around and looking at different things. The jittery, messy result is exactly what your brain deals with in every moment of your visual experience. This can be seen also in the video below. The white circle on the right shows potential eye movements, and the blurry blob on the left reveals the jumpy visual input in every moment.
www.cogsci.nl/Sebastiaan Mathôt.Yet, seeing never feels like work for us. Rather than perceiving the fluctuations and visual noise that a video might record, we perceive a consistently stable environment. So how does our brain create this illusion of stability? This process has fascinated scientists for centuries and it is one of the fundamental questions in vision science.
The time machine brain
In our latest research, we discovered a new mechanism that, among others, can explain this illusory stability. The brain automatically smoothes our visual input over time. Instead of analysing every single visual snapshot, we perceive in a given moment an average of what we saw in the past 15 seconds. So, by pulling together objects to appear more similar to each other, our brain tricks us into perceiving a stable environment. Living “in the past” can explain why we do not notice subtle changes that occur over time.
In other words, the brain is like a time machine which keeps sending us back in time. It’s like an app that consolidates our visual input every 15 seconds into one impression so that we can handle everyday life. If our brains were always updating in real time, the world would feel like a chaotic place with constant fluctuations in light, shadow and movement. We would feel like we were hallucinating all the time.
We created an illusion to illustrate how this stabilisation mechanism works. Looking at the video below, the face on the left side slowly ages for 30 seconds, and yet, it is very difficult to notice the full extent of the change in age. In fact, observers perceive the face as ageing more slowly than it actually is.
To test this illusion we recruited hundreds of participants and asked them to view close-ups of faces morphing chronologically in age in 30-second timelapse videos. When asked to tell the age of the face at the very end of the video, the participants almost consistently reported the age of the face that was presented 15 seconds before.
As we watch the video, we are continuously biased towards the past and so the brain constantly sends us back to the previous ten to 15 seconds (where the face was younger). Instead of seeing the latest image in real time, humans actually see earlier versions because our brain’s refresh time is about 15 seconds. So this illusion demonstrates that visual smoothing over time can help stabilise perception.
What the brain is essentially doing is procrastinating. It’s too much work to constantly deal with every single snapshot it receives, so the brain sticks to the past because the past is a good predictor of the present. Basically we recycle information from the past because it’s more efficient, faster and less work.
This idea – which is also supported by other results – of mechanisms within the brain that continuously bias our visual perception towards our past visual experience is known as continuity fields. Our visual system sometimes sacrifices accuracy for the sake of a smooth visual experience of the world around us. This can explain why, for example, when watching a film we don’t notice subtle changes that occur over time, such as the difference between actors and their stunt doubles.
Repercussions
There are positive and negative implications to our brain operating with this slight lag when processing our visual world. The delay is great for preventing us from feeling bombarded by visual input every day, but it can also risk life-or-death consequences when absolute precision is needed.
For example, radiologists examine hundreds of images in batches, seeing several related images one after the other. When looking at an X-ray, clinicians are typically asked to identify any abnormalities and then classify them. During this visual search and recognition task, researchers have found that radiologists’ decisions were based not only on the present image, but also on images they had previously seen, which could have grave consequences for patients.
Our visual system’s sluggishness to update can make us blind to immediate changes because it grabs on to our first impression and pulls us toward the past. Ultimately, though, continuity fields promote our experience of a stable world. At the same time, it’s important to remember that the judgements we make every day are not totally based on the present, but strongly depend on what we have seen in the past.
License and Republishing
World Economic Forum articles may be republished in accordance with the Creative Commons Attribution-NonCommercial-NoDerivatives 4.0 International Public License, and in accordance with our Terms of Use.
The views expressed in this article are those of the author alone and not the World Economic Forum.
Imagine seeing a dozen pictures flash by in a fraction of a second. You might think it would be impossible to identify any images you see for such a short time. However, a team of neuroscientists from MIT has found that the human brain can process entire images that the eye sees for as little as 13 milliseconds — the first evidence of such rapid processing speed.
That speed is far faster than the 100 milliseconds suggested by previous studies. In the new study, which appears in the journal Attention, Perception, and Psychophysics, researchers asked subjects to look for a particular type of image, such as “picnic” or “smiling couple,” as they viewed a series of six or 12 images, each presented for between 13 and 80 milliseconds.
“The fact that you can do that at these high speeds indicates to us that what vision does is find concepts. That’s what the brain is doing all day long — trying to understand what we’re looking at,” says Mary Potter, an MIT professor of brain and cognitive sciences and senior author of the study.image: Christine Daniloff/MIT This rapid-fire processing may help direct the eyes, which shift their gaze three times per second, to their next target, Potter says. “The job of the eyes is not only to get the information into the brain, but to allow the brain to think about it rapidly enough to know what you should look at next. So in general we’re calibrating our eyes so they move around just as often as possible consistent with understanding what we’re seeing,” she says.Other authors of the paper are former MIT postdoc Brad Wyble, now at Pennsylvania State University, postdoc Carl Hagmann, and research assistant Emily McCourt.
Rapid identification
After visual input hits the retina, the information flows into the brain, where information such as shape, color, and orientation is processed. In previous studies, Potter has shown that the human brain can correctly identify images seen for as little as 100 milliseconds. In the new study, she and her colleagues decided to gradually increase the speeds until they reached a point where subjects’ answers were no better than if they were guessing. All images were new to the viewers.The researchers expected they might see a dramatic decline in performance around 50 milliseconds, because other studies have suggested that it takes at least 50 milliseconds for visual information to flow from the retina to the “top” of the visual processing chain in the brain and then back down again for further processing by so-called “re-entrant loops.” These processing loops were believed necessary to confirm identification of a particular scene or object.However, the MIT team found that although overall performance declined, subjects continued to perform better than chance as the researchers dropped the image exposure time from 80 milliseconds to 53 milliseconds, then 40 milliseconds, then 27, and finally 13 — the fastest possible rate with the computer monitor being used.“This didn’t really fit with the scientific literature we were familiar with, or with some common assumptions my colleagues and I have had for what you can see,” Potter says.Potter believes one reason for the subjects’ better performance in this study may be that they were able to practice fast detection as the images were presented progressively faster, even though each image was unfamiliar. The subjects also received feedback on their performance after each trial, allowing them to adapt to this incredibly fast presentation. At the highest rate, subjects were seeing new images more than 20 times as fast as vision typically absorbs information.“We think that under these conditions we begin to show more evidence of knowledge than in previous experiments where people hadn’t really been expecting to find success, and didn’t look very hard for it,” Potter says.Simon Thorpe, director of the Centre de Recherche Cerveau & Cognition at the University of Toulouse, says the results “illustrate beautifully” that the brain can make sense of briefly presented images.“This new paper shows that the meaning of an image can be extracted even when an image is mixed up in a sequence of six or even 12 images presented at 13 milliseconds per image — a rate of about 75 frames a second. Another striking finding was that the effect is also seen when the question concerning the target is only presented after the sequence has been run, meaning that the brain can extract meaning even when there is no way to predict what will be shown,” says Thorpe, who was not part of the research team.The findings are consistent with a 2001 study from researchers at the University of Parma and the University of St. Andrews, who found that neurons in the brains of macaque monkeys that respond to specific types of image, such as faces, could be activated even when the target images were each presented for only 14 milliseconds in a rapid sequence.“That was the only background that suggested maybe 14 milliseconds was sufficient to get something meaningful into the brain,” Potter says.One-way flow
The study offers evidence that “feedforward processing” — the flow of information in only one direction, from retina through visual processing centers in the brain — is enough for the brain to identify concepts without having to do any further feedback processing.It also suggests that while the images are seen for only 13 milliseconds before the next image appears, part of the brain continues to process those images for longer than that, Potter says, because in some cases subjects weren’t asked whether a specified image was present until after they had seen the sequence.“If images were wiped out after 13 milliseconds, people would never be able to respond positively after the sequence. There has to be something in the brain that has maintained that information at least that long,” she says.This ability to identify images seen so briefly may help the brain as it decides where to focus the eyes, which dart from point to point in brief movements called fixations about three times per second, Potter says. Deciding where to move the eyes can take 100 to 140 milliseconds, so very high-speed understanding must occur before that.The researchers are now investigating how long visual information presented so briefly can be held in the brain. They are also scanning subjects’ brains with a magnetoencephalography (MEG) scanner during the task to see what brain regions are active when a person successfully completes the identification task.The research was funded by the National Institutes of Health.