Is this the real life? Is this just fantasy?
Those aren’t just lyrics from the Queen song “Bohemian Rhapsody.” They’re also the questions that the brain must constantly answer while processing streams of visual signals from the eyes and purely mental pictures bubbling out of the imagination. Brain scan studies have repeatedly found that seeing something and imagining it evoke highly similar patterns of neural activity. Yet for most of us, the subjective experiences they produce are very different.
“I can look outside my window right now, and if I want to, I can imagine a unicorn walking down the street,” said Thomas Naselaris, an associate professor at the University of Minnesota. The street would seem real and the unicorn would not. “It’s very clear to me,” he said. The knowledge that unicorns are mythical barely plays into that: A simple imaginary white horse would seem just as unreal.
So “why are we not constantly hallucinating?” asked Nadine Dijkstra, a postdoctoral fellow at University College London. A study she led, recently published in Nature Communications, provides an intriguing answer: The brain evaluates the images it is processing against a “reality threshold.” If the signal passes the threshold, the brain thinks it’s real; if it doesn’t, the brain thinks it’s imagined.
Such a system works well most of the time because imagined signals are typically weak. But if an imagined signal is strong enough to cross the threshold, the brain takes it for reality.
Although the brain is very competent at assessing the images in our minds, it appears that “this kind of reality checking is a serious struggle,” said Lars Muckli, a professor of visual and cognitive neurosciences at the University of Glasgow. The new findings raise questions about whether variations or alterations in this system could lead to hallucinations, invasive thoughts or even dreaming.
“They’ve done a great job, in my opinion, of taking an issue that philosophers have been debating about for centuries and defining models with predictable outcomes and testing them,” Naselaris said.
When Perceptions and Imagination Mix
Dijkstra’s study of imagined images was born in the early days of the Covid-19 pandemic, when quarantines and lockdowns interrupted her scheduled work. Bored, she started going through the scientific literature on imagination — and then spent hours combing papers for historical accounts of how scientists tested such an abstract concept. That’s how she came upon a 1910 study conducted by the psychologist Mary Cheves West Perky.
Perky asked participants to picture fruits while staring at a blank wall. As they did so, she secretly projected extremely faint images of those fruits — so faint as to be barely visible — on the wall and asked the participants if they saw anything. None of them thought they saw anything real, although they commented on how vivid their imagined image seemed. “If I hadn’t known I was imagining, I would have thought it real,” one participant said.
Perky’s conclusion was that when our perception of something matches what we know we are imagining, we will assume it is imaginary. It eventually came to be known in psychology as the Perky effect. “It’s a huge classic,” said Bence Nanay, a professor of philosophical psychology at the University of Antwerp. It became kind of a “compulsory thing when you write about imagery to say your two cents about the Perky experiment.”
In the 1970s, the psychology researcher Sydney Joelson Segal revived interest in Perky’s work by updating and modifying the experiment. In one follow-up study, Segal asked participants to imagine something, such as the New York City skyline, while she projected something else faintly onto the wall — such as a tomato. What the participants saw was a mix of the imagined image and the real one, such as the New York City skyline at sunset. Segal’s findings suggested that perception and imagination can sometimes “quite literally mix,” Nanay said.
Not all studies that aimed to replicate Perky’s findings succeeded. Some of them involved repeated trials for the participants, which muddied the results: Once people know what you’re trying to test, they tend to change their answers to what they think is correct, Naselaris said.
So Dijkstra, under the direction of Steve Fleming, a metacognition expert at University College London, set up a modern version of the experiment that avoided the problem. In their study, participants never had a chance to edit their answers because they were tested only once. The work modeled and examined the Perky effect and two other competing hypotheses for how the brain tells reality and imagination apart.
Evaluation Networks
One of those alternative hypotheses says that the brain uses the same networks for reality and imagination, but that functional magnetic resonance imaging (fMRI) brain scans don’t have high enough resolution for neuroscientists to discern the differences in how the networks are used. One of Muckli’s studies, for example, suggests that in the brain’s visual cortex, which processes images, imaginary experiences are coded in a more superficial layer than real experiences are.
With functional brain imaging, “we’re squinting our eyes,” Muckli said. Within each equivalent of a pixel in a brain scan, there are about 1,000 neurons, and we can’t see what each one is doing.
The other hypothesis, suggested by studies led by Joel Pearson at the University of New South Wales, is that the same pathways in the brain code for both imagination and perception, but imagination is just a weaker form of perception.
During the pandemic lockdown, Dijkstra and Fleming recruited for an online study. Four hundred participants were told to look at a series of static-filled images and imagine diagonal lines tilting through them to the right or left. Between each trial, they were asked to rate how vivid the imagery was on a scale of 1 to 5. What the participants did not know was that in the last trial, the researchers slowly raised the intensity of a faint projected image of diagonal lines — tilted either in the direction the participants were told to imagine or in the opposite direction. The researchers then asked the participants if what they saw was real or imagined.
Dijkstra expected that she would find the Perky effect — that when the imagined image matched the projected one, the participants would see the projection as the product of their imagination. Instead, the participants were much more likely to think the image was really there.
Yet there was at least an echo of the Perky effect in those results: Participants who thought the image was there saw it more vividly than the participants who thought it was all their imagination.
In a second experiment, Dijkstra and her team didn’t present an image during the last trial. But the result was the same: The people who rated what they were seeing as more vivid were also more likely to rate it as real.
The observations suggest that imagery in our mind’s eye and real perceived images in the world do get mixed together, Dijkstra said. “When this mixed signal is strong or vivid enough, we think it reflects reality.” It’s likely that there’s some threshold above which visual signals feel real to the brain and below which they feel imagined, she thinks. But there could also be a more gradual continuum.
To learn what’s happening within a brain trying to distinguish reality from imagination, the researchers reanalyzed brain scans from a previous study in which 35 participants vividly imagined and perceived various images, from watering cans to roosters.
In keeping with other studies, they found that the activity patterns in the visual cortex in the two scenarios were very similar. “Vivid imagery is more like perception, but whether faint perception is more like imagery is less clear,” Dijkstra said. There were hints that looking at a faint image could produce a pattern similar to that of imagination, but the differences weren’t significant and need to be examined further.
