• Hello everybody! We have tons of new awards for the new year that can be requested through our Awards System thanks to Antifa Lockhart! Some are limited-time awards so go claim them before they are gone forever...

    CLICK HERE FOR AWARDS

Scientists Reconstruct Brains’ Visions Into Digital Video



REGISTER TO REMOVE ADS
Status
Not open for further replies.

CaptainMarvelQ8

Bronze Member
Joined
Mar 14, 2009
Messages
1,334
Awards
4

UC Berkeley scientists have developed a system to capture visual activity in human brains and reconstruct it as digital video clips. Eventually, this process will allow you to record and reconstruct your own dreams on a computer screen.

I just can't believe this is happening for real, but according to Professor Jack Gallant—UC Berkeley neuroscientist and coauthor of the research published today in the journal Current Biology—"this is a major leap toward reconstructing internal imagery. We are opening a window into the movies in our minds."

Indeed, it's mindblowing. I'm simultaneously excited and terrified. This is how it works:
They used three different subjects for the experiments—incidentally, they were part of the research team because it requires being inside a functional Magnetic Resonance Imaging system for hours at a time. The subjects were exposed to two different groups of Hollywood movie trailers as the fMRI system recorded the brain's blood flow through their brains' visual cortex.

The readings were fed into a computer program in which they were divided into three-dimensional pixels units called voxels (volumetric pixels). This process effectively decodes the brain signals generated by moving pictures, connecting the shape and motion information from the movies to specific brain actions. As the sessions progressed, the computer learned more and more about how the visual activity presented on the screen corresponded to the brain activity.

An 18-million-second picture palette

After recording this information, another group of clips was used to reconstruct the videos shown to the subjects. The computer analyzed 18 million seconds of random YouTube video, building a database of potential brain activity for each clip. From all these videos, the software picked the one hundred clips that caused a brain activity more similar to the ones the subject watched, combining them into one final movie. Although the resulting video is low resolution and blurry, it clearly matched the actual clips watched by the subjects.

Think about those 18 million seconds of random videos as a painter's color palette. A painter sees a red rose in real life and tries to reproduce the color using the different kinds of reds available in his palette, combining them to match what he's seeing. The software is the painter and the 18 million seconds of random video is its color palette. It analyzes how the brain reacts to certain stimuli, compares it to the brain reactions to the 18-million-second palette, and picks what more closely matches those brain reactions. Then it combines the clips into a new one that duplicates what the subject was seeing. Notice that the 18 million seconds of motion video are not what the subject is seeing. They are random bits used just to compose the brain image.

Given a big enough database of video material and enough computing power, the system would be able to re-create any images in your brain.

In this other video you can see how this process worked in the three experimental targets. On the top left square you can see the movie the subjects were watching while they were in the fMRI machine. Right below you can see the movie "extracted" from their brain activity. It shows that this technique gives consistent results independent of what's being watched—or who's watching. The three lines of clips next to the left column show the random movies that the computer program used to reconstruct the visual information.

Right now, the resulting quality is not good, but the potential is enormous. Lead research author—and one of the lab test bunnies—Shinji Nishimoto thinks this is the first step to tap directly into what our brain sees and imagines:

Our natural visual experience is like watching a movie. In order for this technology to have wide applicability, we must understand how the brain processes these dynamic visual experiences.

gizmodo

This is all very interesting, i can't wait to see how it advances in the future
 

inasuma

i'm gonna be
Joined
Dec 28, 2005
Messages
1,289
Awards
23
Location
Indigo Plateau
So, it uses youtube videos to reconstruct what the person saw? How is that reconstructing from the brain?

Maybe I'm just not getting this. lol
 

Dogenzaka

PLATINUM USERNAME WINS
Joined
Aug 28, 2006
Messages
17,730
Awards
4
Location
Killing is easy once you forget the taste of sugar
People watched the imagery that was on the left side of the YouTube video.
They then took the information that the BRAIN was interpreting/receiving from that, decoded it, and spit out THAT transcription in the form of a video.
Then they compared that video to the original content they watched.

Is that right?
 

CaptainMarvelQ8

Bronze Member
Joined
Mar 14, 2009
Messages
1,334
Awards
4
So, it uses youtube videos to reconstruct what the person saw? How is that reconstructing from the brain?

Maybe I'm just not getting this. lol


People watched the imagery that was on the left side of the YouTube video.
They then took the information that the BRAIN was interpreting/receiving from that, decoded it, and spit out THAT transcription in the form of a video.
Then they compared that video to the original content they watched.

Is that right?

Apparently the system takes from the the color pallete,examines how the brain reacts to them and then creates new duplicate clips that resemble what the person was seeing.That's why some of the clips do NOT look like the original clip(one time you see a woman instead of the black man in the original clip). Meaning we're not talking about direct comparison between the original clips and the one's generated from the brains activity.
 

stephaknee

Hakuna Matata
Joined
Apr 17, 2004
Messages
1,808
Location
Boston
What I'm gathering is that subjects watched the original video, and the researchers recorded the brain activity. They then compared that activity to "potential" brain activity of 18 million seconds of clips from random youtube videos and produced the second video. So say that Point A of the original video generates a specific pattern of activation-- the computer analyzes this activation and chooses clips from random youtube videos that it assumes will produce the same activation. It seems kind of shady.

I think it's more interesting that scientists can accurately predict what you're going to do before you do it.
 

inasuma

i'm gonna be
Joined
Dec 28, 2005
Messages
1,289
Awards
23
Location
Indigo Plateau
Hmm alright this is making more sense then. I can see where the "brain activity" plays into this with all the text overlaying the images from the video in the right. Naturally when you see or hear something, the brain relates it to certain words or shapes.

Or vice versa.
 

Marly

All right, don't have a crap attack
Joined
Nov 11, 2007
Messages
4,354
Awards
2
It would be fascinating to use this technology in optical illusions, I wonder how an illusion would be interpreted, considering it's literal visual deception of the brain that even with knowledge of the trickery the brain cannot even process and implement that knowledge to fix itself.
 

Chuman

Dad of Boy
Joined
Jun 30, 2008
Messages
12,694
Awards
44
Age
23
Oh shit. Technology advances faster and faster every year.
 

Luap

sans 911
Joined
Oct 8, 2008
Messages
5,233
Awards
6
Age
27
Website
www.facebook.com
Why do the images they make look different from what the brain was seeing? It's weird how when you see the black kid, there's a woman instead, or when it showed the words "All bets are off" a bunch of random text all bunched together is made.
 

Chuman

Dad of Boy
Joined
Jun 30, 2008
Messages
12,694
Awards
44
Age
23
Why do the images they make look different from what the brain was seeing? It's weird how when you see the black kid, there's a woman instead, or when it showed the words "All bets are off" a bunch of random text all bunched together is made.

Because our brains are fucked up. Have you ever noticed how weird you're dreams are? Everything comes out different then when it was put in.
 

Orion

Prepared To Die
Joined
Sep 17, 2007
Messages
20,385
Awards
10
Haha, damn you're negative.
I just mean that saying technology is always advancing faster than ever is something that goes without saying. Technology works by exponential growth, just like population - it will always get bigger and better based on how much big and how good it is now.
speak for yourself, mayne. I'm perfectly fine with the way my brain works
haha yeah you showed him something guud
 

Chuman

Dad of Boy
Joined
Jun 30, 2008
Messages
12,694
Awards
44
Age
23
I just mean that saying technology is always advancing faster than ever is something that goes without saying. Technology works by exponential growth, just like population - it will always get bigger and better based on how much big and how good it is now.haha yeah you showed him something guud

I know. Anyway, back to subject.
 

Jadentheman

Bronze Member
Joined
Oct 21, 2009
Messages
1,355
Age
28
Location
Texas
This could be good. Not for thoughts, but to extract memories especially the ones you don't remember
 

Orion

Prepared To Die
Joined
Sep 17, 2007
Messages
20,385
Awards
10
This could be good. Not for thoughts, but to extract memories especially the ones you don't remember
That's not how memories work. At best, this process might somehow be reversible to allow the sharper recall of forgotten or vague memories.
 
Status
Not open for further replies.
Top