%0 Journal Article %J J Neural Eng %D 2011 %T Decoding vowels and consonants in spoken and imagined words using electrocorticographic signals in humans. %A Pei, Xiao-Mei %A Barbour, Dennis L %A Leuthardt, E C %A Gerwin Schalk %K Adolescent %K Adult %K Brain %K Brain Mapping %K Cerebral Cortex %K Communication Aids for Disabled %K Data Interpretation, Statistical %K Discrimination (Psychology) %K Electrodes, Implanted %K Electroencephalography %K Epilepsy %K Female %K Functional Laterality %K Humans %K Male %K Middle Aged %K Movement %K Speech Perception %K User-Computer Interface %X

Several stories in the popular media have speculated that it may be possible to infer from the brain which word a person is speaking or even thinking. While recent studies have demonstrated that brain signals can give detailed information about actual and imagined actions, such as different types of limb movements or spoken words, concrete experimental evidence for the possibility to 'read the mind', i.e. to interpret internally-generated speech, has been scarce. In this study, we found that it is possible to use signals recorded from the surface of the brain (electrocorticography) to discriminate the vowels and consonants embedded in spoken and in imagined words, and we defined the cortical areas that held the most information about discrimination of vowels and consonants. The results shed light on the distinct mechanisms associated with production of vowels and consonants, and could provide the basis for brain-based communication using imagined speech.

%B J Neural Eng %V 8 %P 046028 %8 08/2011 %G eng %U http://www.ncbi.nlm.nih.gov/pubmed/21750369 %N 4 %R 10.1088/1741-2560/8/4/046028 %0 Journal Article %J J Neurosci %D 2011 %T Nonuniform high-gamma (60-500 Hz) power changes dissociate cognitive task and anatomy in human cortex. %A Charles M Gaona %A Sharma, Mohit %A Zachary V. Freudenberg %A Breshears, Jonathan %A Bundy, David T %A Roland, Jarod %A Barbour, Dennis L %A Gerwin Schalk %A Leuthardt, E C %K Acoustic Stimulation %K Adolescent %K Adult %K Analysis of Variance %K Brain Mapping %K Brain Waves %K Cerebral Cortex %K Cognition Disorders %K Electroencephalography %K Epilepsy %K Evoked Potentials %K Female %K Humans %K Male %K Middle Aged %K Neuropsychological Tests %K Nonlinear Dynamics %K Photic Stimulation %K Reaction Time %K Spectrum Analysis %K Time Factors %K Vocabulary %X

High-gamma-band (>60 Hz) power changes in cortical electrophysiology are a reliable indicator of focal, event-related cortical activity. Despite discoveries of oscillatory subthreshold and synchronous suprathreshold activity at the cellular level, there is an increasingly popular view that high-gamma-band amplitude changes recorded from cellular ensembles are the result of asynchronous firing activity that yields wideband and uniform power increases. Others have demonstrated independence of power changes in the low- and high-gamma bands, but to date, no studies have shown evidence of any such independence above 60 Hz. Based on nonuniformities in time-frequency analyses of electrocorticographic (ECoG) signals, we hypothesized that induced high-gamma-band (60-500 Hz) power changes are more heterogeneous than currently understood. Using single-word repetition tasks in six human subjects, we showed that functional responsiveness of different ECoG high-gamma sub-bands can discriminate cognitive task (e.g., hearing, reading, speaking) and cortical locations. Power changes in these sub-bands of the high-gamma range are consistently present within single trials and have statistically different time courses within the trial structure. Moreover, when consolidated across all subjects within three task-relevant anatomic regions (sensorimotor, Broca's area, and superior temporal gyrus), these behavior- and location-dependent power changes evidenced nonuniform trends across the population. Together, the independence and nonuniformity of power changes across a broad range of frequencies suggest that a new approach to evaluating high-gamma-band cortical activity is necessary. These findings show that in addition to time and location, frequency is another fundamental dimension of high-gamma dynamics.

%B J Neurosci %V 31 %P 2091-100 %8 02/2011 %G eng %U http://www.ncbi.nlm.nih.gov/pubmed/21307246 %N 6 %R 10.1523/JNEUROSCI.4722-10.2011 %0 Journal Article %J Neuroimage %D 2011 %T Spatiotemporal dynamics of electrocorticographic high gamma activity during overt and covert word repetition. %A Pei, Xiao-Mei %A Leuthardt, E C %A Charles M Gaona %A Peter Brunner %A Jonathan Wolpaw %A Gerwin Schalk %K Adolescent %K Adult %K Brain %K Brain Mapping %K Electroencephalography %K Female %K Humans %K Male %K Middle Aged %K Signal Processing, Computer-Assisted %K Verbal Behavior %X

Language is one of the defining abilities of humans. Many studies have characterized the neural correlates of different aspects of language processing. However, the imaging techniques typically used in these studies were limited in either their temporal or spatial resolution. Electrocorticographic (ECoG) recordings from the surface of the brain combine high spatial with high temporal resolution and thus could be a valuable tool for the study of neural correlates of language function. In this study, we defined the spatiotemporal dynamics of ECoG activity during a word repetition task in nine human subjects. ECoG was recorded while each subject overtly or covertly repeated words that were presented either visually or auditorily. ECoG amplitudes in the high gamma (HG) band confidently tracked neural changes associated with stimulus presentation and with the subject's verbal response. Overt word production was primarily associated with HG changes in the superior and middle parts of temporal lobe, Wernicke's area, the supramarginal gyrus, Broca's area, premotor cortex (PMC), primary motor cortex. Covert word production was primarily associated with HG changes in superior temporal lobe and the supramarginal gyrus. Acoustic processing from both auditory stimuli as well as the subject's own voice resulted in HG power changes in superior temporal lobe and Wernicke's area. In summary, this study represents a comprehensive characterization of overt and covert speech using electrophysiological imaging with high spatial and temporal resolution. It thereby complements the findings of previous neuroimaging studies of language and thus further adds to current understanding of word processing in humans.

%B Neuroimage %V 54 %P 2960-72 %8 02/2011 %G eng %U http://www.ncbi.nlm.nih.gov/pubmed/21029784 %N 4 %R 10.1016/j.neuroimage.2010.10.029 %0 Journal Article %J J Neural Eng %D 2011 %T Using the electrocorticographic speech network to control a brain-computer interface in humans. %A Leuthardt, E C %A Charles M Gaona %A Sharma, Mohit %A Szrama, Nicholas %A Roland, Jarod %A Zachary V. Freudenberg %A Solisb, Jamie %A Breshears, Jonathan %A Gerwin Schalk %K Adult %K Brain %K Brain Mapping %K Computer Peripherals %K Electroencephalography %K Evoked Potentials %K Feedback, Physiological %K Female %K Humans %K Imagination %K Male %K Middle Aged %K Nerve Net %K Speech Production Measurement %K User-Computer Interface %X

Electrocorticography (ECoG) has emerged as a new signal platform for brain-computer interface (BCI) systems. Classically, the cortical physiology that has been commonly investigated and utilized for device control in humans has been brain signals from the sensorimotor cortex. Hence, it was unknown whether other neurophysiological substrates, such as the speech network, could be used to further improve on or complement existing motor-based control paradigms. We demonstrate here for the first time that ECoG signals associated with different overt and imagined phoneme articulation can enable invasively monitored human patients to control a one-dimensional computer cursor rapidly and accurately. This phonetic content was distinguishable within higher gamma frequency oscillations and enabled users to achieve final target accuracies between 68% and 91% within 15 min. Additionally, one of the patients achieved robust control using recordings from a microarray consisting of 1 mm spaced microwires. These findings suggest that the cortical network associated with speech could provide an additional cognitive and physiologic substrate for BCI operation and that these signals can be acquired from a cortical array that is small and minimally invasive.

%B J Neural Eng %V 8 %P 036004 %8 06/2011 %G eng %U http://www.ncbi.nlm.nih.gov/pubmed/21471638 %N 3 %R 10.1088/1741-2560/8/3/036004 %0 Journal Article %J Neurosurgery %D 2010 %T Electrocorticographic frequency alteration mapping for extraoperative localization of speech cortex. %A Wu, Melinda %A Wisneski, Kimberly %A Gerwin Schalk %A Sharma, Mohit %A Roland, Jarod %A Breshears, Jonathan %A Charles M Gaona %A Leuthardt, E C %K Acoustic Stimulation %K Adolescent %K Adult %K Brain Mapping %K Cerebral Cortex %K Chi-Square Distribution %K Electroencephalography %K Epilepsy %K Female %K Humans %K Male %K Mass Spectrometry %K Middle Aged %K Photic Stimulation %K Speech %K Verbal Behavior %K Young Adult %X

OBJECTIVE: 

Electrocortical stimulation (ECS) has long been established for delineating eloquent cortex in extraoperative mapping. However, ECS is still coarse and inefficient in delineating regions of functional cortex and can be hampered by afterdischarges. Given these constraints, an adjunct approach to defining motor cortex is the use of electrocorticographic (ECoG) signal changes associated with active regions of cortex. The broad range of frequency oscillations are categorized into 2 main groups with respect to sensorimotor cortex: low-frequency bands (LFBs) and high-frequency bands (HFBs). The LFBs tend to show a power reduction, whereas the HFBs show power increases with cortical activation. These power changes associated with activated cortex could potentially provide a powerful tool in delineating areas of speech cortex. We explore ECoG signal alterations as they occur with activated region of speech cortex and its potential in clinical brain mapping applications.

METHODS: 

We evaluated 7 patients who underwent invasive monitoring for seizure localization. Each had extraoperative ECS mapping to identify speech cortex. Additionally, all subjects performed overt speech tasks with an auditory or a visual cue to identify associated frequency power changes in regard to location and degree of concordance with ECS results.

RESULTS: 

Electrocorticographic frequency alteration mapping (EFAM) had an 83.9% sensitivity and a 40.4% specificity in identifying any language site when considering both frequency bands and both stimulus cues. Electrocorticographic frequency alteration mapping was more sensitive in identifying the Wernicke area (100%) than the Broca area (72.2%). The HFB is uniquely suited to identifying the Wernicke area, whereas a combination of the HFB and LFB is important for Broca localization.

CONCLUSION: 

The concordance between stimulation and spectral power changes demonstrates the possible utility of EFAM as an adjunct method to improve the efficiency and resolution of identifying speech cortex.

%B Neurosurgery %V 66 %P E407-9 %8 02/2010 %G eng %U http://www.ncbi.nlm.nih.gov/pubmed/20087111 %N 2 %R 10.1227/01.NEU.0000345352.13696.6F %0 Journal Article %J Epilepsy Behav %D 2009 %T A practical procedure for real-time functional mapping of eloquent cortex using electrocorticographic signals in humans. %A Peter Brunner %A A L Ritaccio %A Lynch, Timothy M %A Emrich, Joseph F %A Adam J Wilson %A Williams, Justin C %A Aarnoutse, Erik J %A Ramsey, Nick F %A Leuthardt, E C %A H Bischof %A Gerwin Schalk %K Adult %K Brain Mapping %K Cerebral Cortex %K Electric Stimulation %K Electrodes, Implanted %K Electroencephalography %K Epilepsy %K Female %K Humans %K Male %K Middle Aged %K Practice Guidelines as Topic %K Signal Processing, Computer-Assisted %K Young Adult %X

Functional mapping of eloquent cortex is often necessary prior to invasive brain surgery, but current techniques that derive this mapping have important limitations. In this article, we demonstrate the first comprehensive evaluation of a rapid, robust, and practical mapping system that uses passive recordings of electrocorticographic signals. This mapping procedure is based on the BCI2000 and SIGFRIED technologies that we have been developing over the past several years. In our study, we evaluated 10 patients with epilepsy from four different institutions and compared the results of our procedure with the results derived using electrical cortical stimulation (ECS) mapping. The results show that our procedure derives a functional motor cortical map in only a few minutes. They also show a substantial concurrence with the results derived using ECS mapping. Specifically, compared with ECS maps, a next-neighbor evaluation showed no false negatives, and only 0.46 and 1.10% false positives for hand and tongue maps, respectively. In summary, we demonstrate the first comprehensive evaluation of a practical and robust mapping procedure that could become a new tool for planning of invasive brain surgeries.

%B Epilepsy Behav %V 15 %P 278-86 %8 07/2009 %G eng %U http://www.ncbi.nlm.nih.gov/pubmed/19366638 %N 3 %R 10.1016/j.yebeh.2009.04.001 %0 Journal Article %J Neuroimage %D 2008 %T Real-time detection of event-related brain activity. %A Gerwin Schalk %A Leuthardt, E C %A Peter Brunner %A Ojemann, J G %A Lester A Gerhardt %A Jonathan Wolpaw %K Adult %K Algorithms %K Brain Mapping %K Computer Systems %K Diagnosis, Computer-Assisted %K Electroencephalography %K Epilepsy %K Evoked Potentials %K Female %K Humans %K Male %K Pattern Recognition, Automated %K Reproducibility of Results %K Sensitivity and Specificity %X

The complexity and inter-individual variation of brain signals impedes real-time detection of events in raw signals. To convert these complex signals into results that can be readily understood, current approaches usually apply statistical methods to data from known conditions after all data have been collected. The capability to provide meaningful visualization of complex brain signals without the requirement to initially collect data from all conditions would provide a new tool, essentially a new imaging technique, that would open up new avenues for the study of brain function. Here we show that a new analysis approach, called SIGFRIED, can overcome this serious limitation of current methods. SIGFRIED can visualize brain signal changes without requiring prior data collection from all conditions. This capacity is particularly well suited to applications in which comprehensive prior data collection is impossible or impractical, such as intraoperative localization of cortical function or detection of epileptic seizures.

%B Neuroimage %V 43 %P 245-9 %8 11/2008 %G eng %U http://www.ncbi.nlm.nih.gov/pubmed/18718544 %N 2 %R 10.1016/j.neuroimage.2008.07.037 %0 Journal Article %J J Neural Eng %D 2008 %T Two-dimensional movement control using electrocorticographic signals in humans. %A Gerwin Schalk %A Miller, K.J. %A Nicholas R Anderson %A Adam J Wilson %A Smyth, Matt %A Ojemann, J G %A Moran, D %A Jonathan Wolpaw %A Leuthardt, E C %K Adolescent %K Adult %K Brain Mapping %K Data Interpretation, Statistical %K Drug Resistance %K Electrocardiography %K Electrodes, Implanted %K Electroencephalography %K Epilepsy %K Female %K Humans %K Male %K Movement %K User-Computer Interface %X

We show here that a brain-computer interface (BCI) using electrocorticographic activity (ECoG) and imagined or overt motor tasks enables humans to control a computer cursor in two dimensions. Over a brief training period of 12-36 min, each of five human subjects acquired substantial control of particular ECoG features recorded from several locations over the same hemisphere, and achieved average success rates of 53-73% in a two-dimensional four-target center-out task in which chance accuracy was 25%. Our results support the expectation that ECoG-based BCIs can combine high performance with technical and clinical practicality, and also indicate promising directions for further research.

%B J Neural Eng %V 5 %P 75-84 %8 03/2008 %G eng %U http://www.ncbi.nlm.nih.gov/pubmed/18310813 %N 1 %R 10.1088/1741-2560/5/1/008 %0 Journal Article %J Stroke %D 2008 %T Unique cortical physiology associated with ipsilateral hand movements and neuroprosthetic implications. %A Wisneski, Kimberly %A Nicholas R Anderson %A Gerwin Schalk %A Smyth, Matt %A Moran, D %A Leuthardt, E C %K Adolescent %K Adult %K Artificial Limbs %K Bionics %K Brain Mapping %K Child %K Dominance, Cerebral %K Electroencephalography %K Female %K Hand %K Humans %K Male %K Middle Aged %K Motor Cortex %K Movement %K Paresis %K Prosthesis Design %K Psychomotor Performance %K Stroke %K User-Computer Interface %K Volition %X

BACKGROUND AND PURPOSE: 

Brain computer interfaces (BCIs) offer little direct benefit to patients with hemispheric stroke because current platforms rely on signals derived from the contralateral motor cortex (the same region injured by the stroke). For BCIs to assist hemiparetic patients, the implant must use unaffected cortex ipsilateral to the affected limb. This requires the identification of distinct electrophysiological features from the motor cortex associated with ipsilateral hand movements.

METHODS: 

In this study we studied 6 patients undergoing temporary placement of intracranial electrode arrays. Electrocorticographic (ECoG) signals were recorded while the subjects engaged in specific ipsilateral or contralateral hand motor tasks. Spectral changes were identified with regards to frequency, location, and timing.

RESULTS: 

Ipsilateral hand movements were associated with electrophysiological changes that occur in lower frequency spectra, at distinct anatomic locations, and earlier than changes associated with contralateral hand movements. In a subset of 3 patients, features specific to ipsilateral and contralateral hand movements were used to control a cursor on a screen in real time. In ipsilateral derived control this was optimal with lower frequency spectra.

CONCLUSIONS: 

There are distinctive cortical electrophysiological features associated with ipsilateral movements which can be used for device control. These findings have implications for patients with hemispheric stroke because they offer a potential methodology for which a single hemisphere can be used to enhance the function of a stroke induced hemiparesis.

%B Stroke %V 39 %P 3351-9 %8 12/2008 %G eng %U http://www.ncbi.nlm.nih.gov/pubmed/18927456 %N 12 %R 10.1161/STROKEAHA.108.518175 %0 Journal Article %J J Neural Eng %D 2007 %T Decoding two-dimensional movement trajectories using electrocorticographic signals in humans. %A Gerwin Schalk %A Kubánek, J %A Miller, John W %A Nicholas R Anderson %A Leuthardt, E C %A Ojemann, J G %A Limbrick, D %A Moran, D %A Lester A Gerhardt %A Jonathan Wolpaw %K Adult %K Algorithms %K Arm %K Brain Mapping %K Cerebral Cortex %K Electroencephalography %K Evoked Potentials, Motor %K Female %K Humans %K Male %K Movement %X

Signals from the brain could provide a non-muscular communication and control system, a brain-computer interface (BCI), for people who are severely paralyzed. A common BCI research strategy begins by decoding kinematic parameters from brain signals recorded during actual arm movement. It has been assumed that these parameters can be derived accurately only from signals recorded by intracortical microelectrodes, but the long-term stability of such electrodes is uncertain. The present study disproves this widespread assumption by showing in humans that kinematic parameters can also be decoded from signals recorded by subdural electrodes on the cortical surface (ECoG) with an accuracy comparable to that achieved in monkey studies using intracortical microelectrodes. A new ECoG feature labeled the local motor potential (LMP) provided the most information about movement. Furthermore, features displayed cosine tuning that has previously been described only for signals recorded within the brain. These results suggest that ECoG could be a more stable and less invasive alternative to intracortical electrodes for BCI systems, and could also prove useful in studies of motor function.

%B J Neural Eng %V 4 %P 264-75 %8 09/2007 %G eng %U http://www.ncbi.nlm.nih.gov/pubmed/17873429 %N 3 %R 10.1088/1741-2560/4/3/012 %0 Journal Article %J Neurosurgery %D 2007 %T Electrocorticographic Frequency Alteration Mapping: A Clinical Technique for Mapping the Motor Cortex. %A Leuthardt, E C %A Miller, John W %A Nicholas R Anderson %A Gerwin Schalk %A Dowling, Joshua %A Miller, John W %A Moran, D %A Ojemann, J G %K Adult %K Biological Clocks %K Brain Mapping %K Electric Stimulation %K Electrodes, Implanted %K Electroencephalography %K Female %K Hand %K Humans %K Male %K Middle Aged %K Motor Cortex %K Oscillometry %K Signal Processing, Computer-Assisted %K Tongue %X

OBJECTIVE: 

Electrocortical stimulation (ECS) has been well established for delineating the eloquent cortex. However, ECS is still coarse and inefficient in delineating regions of the functional cortex and can be hampered by after-discharges. Given these constraints, an adjunct approach to defining the motor cortex is the use of electrocorticographic signal changes associated with active regions of the cortex. The broad range of frequency oscillations are categorized into two main groups with respect to the sensorimotor cortex: low and high frequency bands. The low frequency bands tend to show a power reduction with cortical activation, whereas the high frequency bands show power increases. These power changes associated with the activated cortex could potentially provide a powerful tool in delineating areas of the motor cortex. We explore electrocorticographic signal alterations as they occur with activated regions of the motor cortex, as well as its potential in clinical brain mapping applications.

METHODS: 

We evaluated seven patients who underwent invasive monitoring for seizure localization. Each patient had extraoperative ECS mapping to identify the motor cortex. All patients also performed overt hand and tongue motor tasks to identify associated frequency power changes in regard to location and degree of concordance with ECS results that localized either hand or tongue motor function.

RESULTS: 

The low frequency bands had a high sensitivity (88.9-100%) and a lower specificity (79.0-82.6%) for identifying electrodes with either hand or tongue ECS motor responses. The high frequency bands had a lower sensitivity (72.7-88.9%) and a higher specificity (92.4-94.9%) in correlation with the same respective ECS positive electrodes.

CONCLUSION: 

The concordance between stimulation and spectral power changes demonstrate the possible utility of electrocorticographic frequency alteration mapping as an adjunct method to improve the efficiency and resolution of identifying the motor cortex.

%B Neurosurgery %V 60 %P 260-70; discussion 270-1 %8 04/2007 %G eng %U http://www.ncbi.nlm.nih.gov/pubmed/17415162 %N 4 Suppl 2 %R 10.1227/01.NEU.0000255413.70807.6E %0 Journal Article %J J Neurosci %D 2007 %T Spectral Changes in Cortical Surface Potentials During Motor Movement. %A Miller, John W %A Leuthardt, E C %A Gerwin Schalk %A Rao, Rajesh P N %A Nicholas R Anderson %A Moran, D %A Miller, John W %A Ojemann, J G %K Adult %K Brain Mapping %K Female %K Humans %K Male %K Middle Aged %K Motor Cortex %K Movement %X

In the first large study of its kind, we quantified changes in electrocorticographic signals associated with motor movement across 22 subjects with subdural electrode arrays placed for identification of seizure foci. Patients underwent a 5-7 d monitoring period with array placement, before seizure focus resection, and during this time they participated in the study. An interval-based motor-repetition task produced consistent and quantifiable spectral shifts that were mapped on a Talairach-standardized template cortex. Maps were created independently for a high-frequency band (HFB) (76-100 Hz) and a low-frequency band (LFB) (8-32 Hz) for several different movement modalities in each subject. The power in relevant electrodes consistently decreased in the LFB with movement, whereas the power in the HFB consistently increased. In addition, the HFB changes were more focal than the LFB changes. Sites of power changes corresponded to stereotactic locations in sensorimotor cortex and to the results of individual clinical electrical cortical mapping. Sensorimotor representation was found to be somatotopic, localized in stereotactic space to rolandic cortex, and typically followed the classic homunculus with limited extrarolandic representation.

%B J Neurosci %V 27 %P 2424-32 %8 02/2007 %G eng %U http://www.ncbi.nlm.nih.gov/pubmed/17329441 %N 9 %R 10.1523/JNEUROSCI.3886-06.2007 %0 Journal Article %J J Neural Eng %D 2004 %T A brain-computer interface using electrocorticographic signals in humans. %A Leuthardt, E C %A Gerwin Schalk %A Jonathan Wolpaw %A Ojemann, J G %A Moran, D %K Adult %K Brain %K Communication Aids for Disabled %K Computer Peripherals %K Diagnosis, Computer-Assisted %K Electrodes, Implanted %K Electroencephalography %K Evoked Potentials %K Female %K Humans %K Imagination %K Male %K Movement Disorders %K User-Computer Interface %X

Brain-computer interfaces (BCIs) enable users to control devices with electroencephalographic (EEG) activity from the scalp or with single-neuron activity from within the brain. Both methods have disadvantages: EEG has limited resolution and requires extensive training, while single-neuron recording entails significant clinical risks and has limited stability. We demonstrate here for the first time that electrocorticographic (ECoG) activity recorded from the surface of the brain can enable users to control a one-dimensional computer cursor rapidly and accurately. We first identified ECoG signals that were associated with different types of motor and speech imagery. Over brief training periods of 3-24 min, four patients then used these signals to master closed-loop control and to achieve success rates of 74-100% in a one-dimensional binary task. In additional open-loop experiments, we found that ECoG signals at frequencies up to 180 Hz encoded substantial information about the direction of two-dimensional joystick movements. Our results suggest that an ECoG-based BCI could provide for people with severe motor disabilities a non-muscular communication and control option that is more powerful than EEG-based BCIs and is potentially more stable and less traumatic than BCIs that use electrodes penetrating the brain.

%B J Neural Eng %V 1 %P 63-71 %8 06/2004 %G eng %U http://www.ncbi.nlm.nih.gov/pubmed/15876624 %N 2 %R 10.1088/1741-2560/1/2/001