The Brain Trick That Makes Videos Work
Thank you to Wren for supporting PBS.
Hey smart people, Joe here. In the 1820s, there was a dinner party.
Mathematician Charles Babbage was there with astronomer William Herschel. You might know
Babbage as the father of computing and Herschel as the guy who found Uranus. It
sounds like a fun dinner party. Well, after dinner, Herschel asked Babbage a question,
"How can you see two sides of a coin at the same time?" Well, Babbage's answer,
"Just look at its reflection in the mirror," pretty creative solution,
but Herschel had an even better answer. He spun the coin on the table, like magic both sides of
the coin seemed to blend together as if they were seeing them at the same time. Now, this
story got around until an Irish doctor heard it and it gave him an idea. He put two pictures on
either side of a disc. And when he spun them between the string, the two images became one.
This became known as the thaumatrope.
This was one of the most popular toys of the 19th century, but it was eventually mostly forgotten,
except that it wasn't. This and the toys that it inspired are the reason you can watch this video,
it's all why movies exist and cartoons and video games. Every moving image we view on screens,
big or small, can trace its origin to this simple illusion, one that can take still
images flashed in sequence and fool our brain into perceiving motion. So what makes this all work?
Spinning discs are one of the simplest ways to create illusions that blend images into
one. And in the 19th century, they inspired a whole range of so-called philosophical toys,
including this one, a spinning disc with several slits cut around it. Now, when viewed in a mirror,
a series of still images on one side appear as one moving image. The illusion
of apparent motion. These inspired a series of Victorian cinematic toys that were commercialized
throughout the night 19th century, selling by the thousands and captivating people around the world.
These cinematic toys became more and more complex, eventually giving birth to the era
of the moving picture. Despite the fact that no one quite knew how they worked.
Ultimately, those toys directly led to this invention: the film projector.
To create the illusion of movement, a sequence of still images is fed between a light source
and a set of lenses, projecting a series of pictures on a screen, but simply projecting
a scrolling stream of still images alone is not enough to create the illusion of motion,
we merely see an indecipherable blur. This illusion is about what you don't see.
What actually happens in a projector is one still frame is projected on screen, then the
screen goes black while the film is in advanced one full step and the next still image is shown.
The film doesn't move continuously, but instead is advanced frame by frame many times every second.
Inside the projector is a carefully timed shutter
that blocks the light so that whenever the film is moving, you only see black. When these images are
flashed on the screen at a fast enough rate, we perceive a moving picture. Before around
2010, it's likely that any movie that you saw in a theater was projected like this.
Most of the time, the movie wasn't actually moving and you spent half your time in the theater
completely in the dark. Now today, of course, films and other moving pictures are projected
digitally or displayed directly on digital screens. As you can see in these incredible clips
from our friends The Slow Mo Guys, modern screens no longer flicker to black between still images,
instead millions of individual pixels are refreshed dozens of times per second,
but effect is still the same. What's getting beamed into your eyes is a series of still
images and that's true on screens big or small. Thanks to this technology,
in the last century and a half or so, billions of minds have been tricked into seeing moving images
that aren't really there. Many of us spend hours every day staring at
these illusions and never even think twice about it and it all traces back to these.
These were the first forms of moving picture entertainment. Though, there are some hints from
caves and artifacts that prehistoric cultures may have made versions of their own long before
as well, which is pretty cool. It's been said that our understanding of vision was changed
as much by these toys as the field of biology was changed by the invention of the microscope.
Literal toys inspired fundamental questions about how our brains work,
how we perceive the world and how we construct
reality itself and scientists today are still using these illusions to tackle those questions.
I made something that I have to show you right off the bat. I hope this works.
It's my Eagleman trope.
Yes, that works surprisingly well even over video. I like it.
Why does this work? I mean, this is like a toy that people played
with in the 1830s or something? What is happening.
So why does that work? It's because when the brain sees something,
even if it's very rapid, your brain's unable to turn it on and off that quickly, so your brain
sees things for longer than your eye does. This is what's called persistence of vision.
The idea of some kind of visual persistence goes back long before Babbage's dinner party,
even to ancient Greece and Egypt. Early philosophers noticed that streaks from
lightning seem to stick around for a split second after the flash or that
the sun stayed in their vision after they looked away. Don't stare at the sun ever,
okay. The guy who invented this did an experiment where he stared at the sun and he went blind.
Just don't do it, people. It's bad for you. In the 11th century, an Arab mathematician and
philosopher noted that a flickering flame seemed to appear where it was a moment before. DaVinci
and Newton even devoted time to these mysteries. All of those early explanations centered on the
idea that the light that enters our peepers temporarily burns a scene into our eyes,
then it's basically wiped clean, our eyes refresh and then the next scene is burned in and so on.
In 1765, an Irish mathematician did this experiment with glowing embers.
He calculated the velocity you'd have to spin to see the streak of light make a complete circle
and estimated the supposed refresh rate the eye to be 130 milliseconds. You can try this
experiment for yourself. If you rapidly move a point of light, it appears to leave trails
behind. These illusions are not simply artifacts of my camera or the shutter speed, you can see
them if you try this in person. This theory that the eye acts like a camera with some refresh rate,
capturing slices of the world in single frames stuck around for a long time, but it's wrong.
That's right. It has to do with what's going on in the brain. There are many things that happen
in the retina and that's just the first stage and then you go back into the brain to an area
of the visual thalamus and then the visual cortex and then all these areas of the visual cortex.
All of these things are interplaying.
The effects in these toys and movies and screens today create an illusion neuroscientists call
apparent motion. When you see a series of still images appear to move, it's your brain that's been
fooled, not your eye, and how it really works is pretty freaky. Look at this arrangement of
dots and here's a slightly different arrangement and another. These dots are just dots, right?
But played rapidly in succession, at some point these dots become something else. And depending on
just how those dots are arranged relative to each other, as they move our brains can write
very different stories about what we're seeing, even invent different characters.
Now, look at these two dots. Does it look like one is chasing the other? Or perhaps they've
switched? Well, none of those are true. One dot is simply moving erratically around the other,
but when you add a moving background, suddenly you perceive something else.
That may feel like a trick, but there's no reason to think dots chase each other. They're just
dots. The position of one of the dots never even changes,
you made it do that. It's a story invented by your brain. How about this? Did it move or do they just
blink on and off? So why are our brains inventing stories that don't exist? Well, these little white
lies our brains tell us are an important part of understanding the visual information in our world.
If I see a bird flying and it goes behind the tree,
then a moment later I see the bird emerge on the other side of the trunk and for my motion
detectors, that's perfectly fine. I say, "Oh great, there was motion there, smooth motion."
The bird didn't disappear. Birds just don't do that in the real world.
Exactly, exactly. And this is this notion of object permanence.
Your brain is wanting to say, "I want to still hang on to that bird wherever it went."
I played a lot of Mario Brothers growing up on the old Nintendo.
And actually, when you look at what's happening frame by frame, like you got a little plumber man
standing there and then all of a sudden you have like a plumber guy
with his fist up a few pixels away and there's nothing in between that. But when you play the
game, you have this sensation that this character smoothly jumped up in the air.
That's exactly right and that's what apparent motion is about. Your brain
does all kinds of computations and says, "Oh wait,
here's Mario and there was a Mario over here. It must be the same Mario and he has moved."
Your brain makes an unconscious decision that images seen at different places
and at different times represent the same object. This is called correspondence. The
snake in the classic game isn't moving, it's simply a series of blocky shapes,
but our brains object permanence interprets the shape as the same snake between frames.
And based on our understanding of how snakes usually move in the real world,
we assemble those shapes into motion. Even in modern video games while the picture's quality
has definitely improved and the technology has increased the number of images played each second,
what we see as continuous motion is just a series of still images.
What's really interesting is that the brain can only do this in retrospect. So in other words,
it collects up the data from frame one, then from frame two and then it retrospectively says, "Ah,
it must have moved smoothly between those things." And part of the way we know this
is from a visual illusion called the colorify effect, which is if I show, let's say, a red dot
and then I show a blue dot, you will think that it moved from one position to the other, but you
will also have the impression that it changed color halfway between, and that's only possible
after the second dot has appeared, even though it feels like, "Oh, I'm seeing this thing and
it changes color to blue," you can only know the position and the color after it's over.
So your brain is going back and writing a story that never happened in between those two things.
That's exactly right. And actually, I coined the term for this some years ago called postdiction.
It's the opposite of prediction, which is to say your brain collects up all the information
about a scene before it retrospectively says what it thinks it saw. You're living about...
probably about half a second in the past. So when you think the moment now occurs,
it's already happened a long time ago, your brain's collecting up all this information,
including what's coming through your eyes and your ears and your fingertips and
your toes and everything, which comes in at different speeds. Your brain has to collect
all that up, stitch it together and then say, "Okay, here's what I think happened."
I'm really starting to question reality more and more as we talk here. Here's another example of
this. There are cells in your eyes, even in the visual processing centers of your brain that
respond fast enough to detect rapidly flickering light, but what's really interesting is there's
a threshold where your brain flips, it decides to ignore that flickering and you suddenly see a
constant light source. When the rate of flicker is higher than 35 flashes per second or so,
your brain says, "I'll just smooth this out to look like real life," even though your eyes
and parts of your brain are sensing flickering light. Movie cameras record 24 images per second,
but if you played that back at 24 flickers per second, it would look like a strobe light.
Film projector shutters flicker two or three times for each still image. Digital displays refresh two
or three times before the image changes. It's over that threshold, so your brain ignores the flicker.
When you think about this, that your brain is taking in all of this information from
your senses, it's comparing it with your past experience about how things in the universe
should behave, then your brain goes back and writes the story of what it thinks you saw,
it brings up a lot of interesting questions, like how did our brains end up this way? I mean,
our species has only been staring at illusions like these for a couple of centuries, but
our brains are way older than that. We didn't evolve with cartoons and TV and video games around
or even spinning picture discs, but being able to sense motion, even the illusion of motion,
has been a big part of our specie's survival. Maybe 99 times out of a hundred that wasn't
a tiger in the grass, it was just an illusion of motion, but our only ancestors who survived
are the ones whose motion detectors got it right the hundredth time too.
To me, what I think it illustrates is that your whole brain, your whole
perceptual world is built out of cells that are just trying to do the best job they can.
Essentially, we found a little loophole, a trick that we can play on these cells,
but we certainly didn't need to evolve to see movies.
I'm glad we did though. Otherwise,
we wouldn't be able to talk here and no one would watch YouTube. I like this reality.
I've always been attracted to visual illusions because we open our eyes and there's the world
and we take it to be reality and so it's very interesting to see how the brain
constructs this reality. It's like if you were a fish in water and you were asked to describe
water, you wouldn't be able to do it. But if you see a bubble that comes up past you,
you might think, "Whoa, what is that thing?" Well, that's exactly what visual illusions are to us.
You almost have to break the brain or find its shortcomings to figure out how it actually
works. It's doing the best that it can, I guess, with limited information.
Well, we wouldn't know what we're missing,
so we think it's doing a good job, but maybe in a hundred years when we all
wear glasses for detecting infrared and ultraviolet and the rest of the electromagnetic
spectrum, maybe we'll say, "Wow, it's actually been doing a terrible job this whole time."
I look forward to living like a bee, that sounds amazing.
So why do videos work or movies or games? Well, the old idea that our eyes work like a camera,
that motion is burned into our eyes the way light hits film, that turns out to be wrong or at least
incomplete. Your eyes are really good at sensing the universe and they do sense the
individual still images you are watching and send that information to your brain, but our
brains know that isn't how the universe really works so it fills in the gaps and blends this
with something else, something that isn't really there. Imagine if you lived in a black,
silent room, cut off from the outside world. The only descriptions of the outside world that you
get are periodic notes passed under the door. Your image of reality will be a story you write
based on that limited information. That's how it is for your brain. In something as normal
as watching TV or a movie, it isn't the actors that are creating the story, it's your brain.
So what does this all mean? Well, it means that this video is an illusion,
but it's one that you can trust. Stay curious.
Thank you to Wren for supporting PBS. Wren is a website where you calculate your carbon footprint
and then offset it by funding projects that plant trees, protect rainforests and suck carbon from
the sky. By answering a few questions about your lifestyle, you can find out your carbon footprint
and how you can reduce it. Now, no one can reduce their carbon footprint to zero,
so you can offset what you have left after reducing. Subscribers receive monthly updates
from the tree planting, rainforest protection and carbon removal projects that they support.
You get to see the trees you planted and what your money is spent on.
You can learn more about Wren's climate projects at wren.co. And as always, a big thank you to
everyone who supports the show on Patreon. We could not make these videos without you.
If you'd like to learn more about how you can support the show,
just click the link down in the description or watch this rapidly spinning disc.
You're getting very sleepy. You want to click the button
and support the show on Patreon. of computing and Herschel as the guy who found Uranus.
