Several years ago, Keith Johnson, a Professor of Linguistics at UC Berkeley, was teaching his department’s introductory course in phonetics. Phonetics is the study of the sounds in human speech, everything from the physical placement of your tongue when you say “ch” or “sh” to the social meanings of different sound enunciations. After Johnson’s first lecture, a young man came up and introduced himself as Edward F. Chang. “He was literally a brain surgeon,” Johnson says, “and he told me, ‘I need to know everything you know.’”
Chang, now a faculty member at the University of California, San Francisco (UCSF) School of Medicine, is in fact a brain surgeon. And he needed to know about phonetics because his lab is trying to map how language functions in the human brain. At the heart of this challenge is a well-known but little understood phenomenon: when we listen to a language we understand, the sounds are meaningful, not just a series of vocalizations.
Since that phonetics course, Johnson has become a team member affiliated with the Chang Lab, which brings neurosurgeons, bioengineers, and linguists together to help address questions related to language and the brain. For the surgical part of the research, Chang placed a dense mesh of 264 electrodes on the brains of volunteers, all of whom were having brain surgery for severe epileptic seizures. “You open the skull and look in at part of the brain, and this is done awake,” says Chang.
The network of electrodes allows the research team to see is how the signals triggered by hearing words travels through the brain. “We’re starting at the bottom by looking at what parts of the brain light up with speaking and hearing,” says Johnson. “We are able to detect difference in sound at the millisecond range, so we can see how brain works in real time.”
It is in the nature of these sounds that Johnson adds his contribution. All languages are made up of a limited number of phonemes, the building-block sounds, like “b” or “a,” that are combined into words and meanings. Phonetic linguistics has a sophisticated vocabulary for the sounds that make up human speech, as well as an in-depth understanding of where sounds originate in our throats and mouths.
By using the Timit Database, a vast trove of recorded English language speech, patients “listened to sounds over and over. Then we could go back and classify what they were hearing based on acoustic properties, like ‘these were vowels made in the front of the mouth,’” says Johnson. By combining this with the real-time visualizations of auditory signals moving through the brain, “we could see the sequence of events, from a signal coming into the brain to the brain’s response of recognizing something. From outside the brain, this is mysterious, but getting into it, you can see the signal morph and change.”
The result of taking these two forms of expertise—linguistics and neuroscience—and combining them, Johnson says, is that Chang’s lab is learning how the signals generated in the motor cortex make the lips, tongue, larynx and jaw move, Johnson explains. The paper he co-authored with Chang last year in Nature is the first to describe these neurological mechanisms for speech. “A lot of this sort of linguistics is not so high-powered on the neuroscience side, with behavioral linguistics people sort of dabbling, or there are neuroscientists who don’t know what the physical side is doing,” Chang says.
The ultimate goal of Chang’s lab is to learn enough about neural speech mechanisms that the lab can develop a computerized prosthetic that would give speech back to people with paralyses or victims of stroke. “The therapeutic application,” Johnson says, “is to be able to take the intention to speak, in your head, and turn it into actual sound,” even for people no longer physically able to use their mouths.
Johnson has found the interdisciplinary nature of the research fulfilling, and working with Chang’s lab has offered exposure to skills and expertise that no single researcher can access. “I wasn’t going to go out and learn brain surgery,” Johnson says. “Some questions are really addressed best or can only be address by working together across disciplines.”
