“Glass Brain” Offers Tours of the Space between Your Ears

 

3-D visualizations combine EEG and MRI data to illustrate how brain signals propagate and could be used to study neural disorders

Sep 29, 2014 By Larry Greenemeier

MINDFUL: Former Grateful Dead percussionist Mickey Hart wore a 64-electrode EEG skullcap and an Oculus Rift virtual reality headset to demonstrate Neurodrummer and the Glass Brain at NVIDIA’s GPU Technology Conference in March. This enabled the audience to see a visual representation of his brain signals as he played. Hart was joined on stage by Tim Mullen, a chief scientist on the Glass Brain project. Courtesy of the Neuroscape Lab, University of California, San Francisco

Former Grateful Dead percussionist Mickey Hart takes pride in his brain. Large, anatomically realistic 3-D animations representing the inner workings of his gray and white matter have graced video screens at several science and technology conferences. These “Glass Brain” visualizations use imaging and advanced computing systems to depict in colorful detail the fiber pathways that make Hart’s brain tick. The researchers behind the project hope it will also form the basis of a new type of tool for the diagnosis and treatment of neurological disorders.
Each Glass Brain animation overlays electroencephalography (EEG) data collected in real time atop a magnetic resonance imaging (MRI) scan—in this case Hart’s—to illustrate how different brain areas communicate with each other. Special algorithms coded into software digitally reconstruct this activity within the brain. The result is a tour of the brain that captures both the timing and location of brain signals.
Hart demonstrated the Glass Brain at a computer conference in San Jose, Calif., this past March by playing a video game called NeuroDrummer on stage. The drummer is working with the Studio Bee digital animation house in San Francisco as well as the Glass Brain’s creators to develop NeuroDrummer into a tool that can determine whether teaching someone to keep a drumbeat might help improve the neural signals responsible for cognition, memory and other functions. The Glass Brain’s brain trust includes the University of California, San Francisco’s Neuroscape Lab as well as the University of California, San Diego’s Swartz Center for Computational Neuroscience, EEG maker Cognionics, Inc. and NVIDIA, a maker of extremely fast graphics processing unit (GPU) computer chips and host of the conference where Hart performed.
Hart donned a black EEG skullcap and took the stage during the closing keynote to demonstrate a prototype of NeuroDrummer. The game came across as a therapeutic, cognition-boosting version of the once-popular Guitar Hero game, creating a very different kind of jam session for Hart. He wore an Oculus Rift virtual reality headset while tapping an electronic drum pad as he appeared to travel through a trippy digital realm resembling outer space. His attempts to keep the beat were frequently interrupted by the need to multitask—in this case, zapping virtual asteroids with a pointed finger.The conference audience watched the action on two wall-sized video screens at the rear of the stage. The first showed what Hart was seeing in his headset while the other displayed the Glass Brain in action. The MRI scan of Hart’s brain—taken in advance of the demo—rippled with streams of color representing different frequencies of neural activity collected by the EEG headset during gameplay.
Researchers have been fitting EEG electrodes to people’s heads for 90 years to measure the electrical signals that pulse through the brain at different frequencies as brain cells communicate with one another. A fundamental feature of how our brains work, such communication is related to higher-order abilities like attention, memory and perception, says Adam Gazzaley, founding director of the Neuroscience Imaging Center at U.C.S.F. and head of both the Glass Brain and NeuroDrummer projects.
The ability to integrate EEG and MRI data in this way is relatively new, made possible in recent years by the development of more sophisticated algorithms for decoding neural activity and the use of GPUs—originally designed to render complex video game graphics—to quickly process streaming data. EEG scanners can detect the timing of brain signals in milliseconds, although they do not provide much information about where those signals originate. That plays to MRI’s strengths: these scanners are much slower to pick up neural pulses but can pinpoint the location of neural activity by registering the relatively slow changes in brain blood flow.
Hart’s interest in neuroscience is oddly fitting. As a young musician, he played a literally instrumental role ushering in the 1960s counterculture movement that challenged conventional thinking in many areas and introduced psychedelia to the mainstream. Now Hart is involved in a new movement to expand the mind, or at least our understanding of it—this time decoding the brain’s mysteries with the help of sensors, scanners and software.
This is a personal quest for Hart, whose grandmother suffered from Alzheimer’s disease. The significance of brain rhythms first became clear to Hart about 40 years ago when he drummed for his grandmother, he said following his NeuroDrummer demo. “She hadn’t spoken in years,” he said. “I played the drum, she spoke my name. That’s power. That’s medicine. When I saw that, I knew that vibrations, in this case music, reconnected those broken pathways and brought her speech, brought her cognition back, at least for the time I was drumming.”
In the lab, Gazzaley and his team are working to ensure these visualizations depict distinct brain events that are interpretable in real time. “We will determine how to best pull out these subtle signals and how consistent they are from trial to trial and across individuals,” he says.
Gazzaley has demonstrated the Glass Brain several times since the NVIDIA conference and is currently working with his team to evaluate the project’s performance thus far. Perhaps the Glass Brain’s greatest success, he says, is the way it educates audiences about the study of how the brain works.

Broadcast live streaming video on Ustream