%0 Journal Article %J Current Biology %D 2022 %T A neural population selective for song in human auditory cortex %A Sam V. Norman-Haignere %A Jenelle Feather %A Dana Boebinger %A Peter Brunner %A Anthony Ritaccio %A Josh H. McDermott %A Gerwin Schalk %A Nancy Kanwisher %K Auditory Cortex %K component %K ECoG %K Electrocorticography %K fMRI %K music %K natural sounds %K song %K Speech %K voice %X Summary How is music represented in the brain? While neuroimaging has revealed some spatial segregation between responses to music versus other sounds, little is known about the neural code for music itself. To address this question, we developed a method to infer canonical response components of human auditory cortex using intracranial responses to natural sounds, and further used the superior coverage of fMRI to map their spatial distribution. The inferred components replicated many prior findings, including distinct neural selectivity for speech and music, but also revealed a novel component that responded nearly exclusively to music with singing. Song selectivity was not explainable by standard acoustic features, was located near speech- and music-selective responses, and was also evident in individual electrodes. These results suggest that representations of music are fractionated into subpopulations selective for different types of music, one of which is specialized for the analysis of song. %B Current Biology %V 32 %P 1470-1484.e12 %G eng %U https://www.sciencedirect.com/science/article/pii/S0960982222001312 %R https://doi.org/10.1016/j.cub.2022.01.069 %0 Journal Article %J Journal of Neuroscience Methods %D 2019 %T A quantitative method for evaluating cortical responses to electrical stimulation %A Lawrence J. Crowther %A Peter Brunner %A Christoph Kapeller %A Christoph Guger %A Kyousuke Kamada %A Marjorie E. Bunch %A Bridget K. Frawley %A Timothy M. Lynch %A Anthony L. Ritaccio %A Gerwin Schalk %K Connectivity %K Cortico-cortical evoked potentials %K Electrical stimulation %K Electrocorticography %X Background Electrical stimulation of the cortex using subdurally implanted electrodes can causally reveal structural connectivity by eliciting cortico-cortical evoked potentials (CCEPs). While many studies have demonstrated the potential value of CCEPs, the methods to evaluate them were often relatively subjective, did not consider potential artifacts, and did not lend themselves to systematic scientific investigations. New method We developed an automated and quantitative method called SIGNI (Stimulation-Induced Gamma-based Network Identification) to evaluate cortical population-level responses to electrical stimulation that minimizes the impact of electrical artifacts. We applied SIGNI to electrocorticographic (ECoG) data from eight human subjects who were implanted with a total of 978 subdural electrodes. Across the eight subjects, we delivered 92 trains of approximately 200 discrete electrical stimuli each (amplitude 4–15 mA) to a total of 64 electrode pairs. Results We verified SIGNI's efficacy by demonstrating a relationship between the magnitude of evoked cortical activity and stimulation amplitude, as well as between the latency of evoked cortical activity and the distance from the stimulated locations. Conclusions SIGNI reveals the timing and amplitude of cortical responses to electrical stimulation as well as the structural connectivity supporting these responses. With these properties, it enables exploration of new and important questions about the neurophysiology of cortical communication and may also be useful for pre-surgical planning. %B Journal of Neuroscience Methods %V 311 %P 67 - 75 %G eng %U http://www.sciencedirect.com/science/article/pii/S0165027018302796 %R https://doi.org/10.1016/j.jneumeth.2018.09.034 %0 Journal Article %J Current Biology %D 2018 %T Encoding of Multiple Reward-Related Computations in Transient and Sustained High-Frequency Activity in Human OFC %A Ignacio Saez %A Jack Lin %A Arjen Stolk %A Edward Chang %A Josef Parvizi %A Gerwin Schalk %A Robert T. Knight %A Ming Hsu %K ECoC %K Electrocorticography %K ERP %K event-related potential %K field potential %K FP %K HFA %K high-frequency activity %K OFC %K orbitofrontal cortex %K reward-prediction error %K RPE %X Summary Human orbitofrontal cortex (OFC) has long been implicated in value-based decision making. In recent years, convergent evidence from human and model organisms has further elucidated its role in representing reward-related computations underlying decision making. However, a detailed description of these processes remains elusive due in part to (1) limitations in our ability to observe human OFC neural dynamics at the timescale of decision processes and (2) methodological and interspecies differences that make it challenging to connect human and animal findings or to resolve discrepancies when they arise. Here, we sought to address these challenges by conducting multi-electrode electrocorticography (ECoG) recordings in neurosurgical patients during economic decision making to elucidate the electrophysiological signature, sub-second temporal profile, and anatomical distribution of reward-related computations within human OFC. We found that high-frequency activity (HFA) (70–200 Hz) reflected multiple valuation components grouped in two classes of valuation signals that were dissociable in temporal profile and information content: (1) fast, transient responses reflecting signals associated with choice and outcome processing, including anticipated risk and outcome regret, and (2) sustained responses explicitly encoding what happened in the immediately preceding trial. Anatomically, these responses were widely distributed in partially overlapping networks, including regions in the central OFC (Brodmann areas 11 and 13), which have been consistently implicated in reward processing in animal single-unit studies. Together, these results integrate insights drawn from human and animal studies and provide evidence for a role of human OFC in representing multiple reward computations. %B Current Biology %V 28 %P 2889 - 2899.e3 %G eng %U http://www.sciencedirect.com/science/article/pii/S0960982218309758 %R https://doi.org/10.1016/j.cub.2018.07.045 %0 Conference Proceedings %B American Epilepsy Society 72nd Annual Meeting %D 2018 %T Rapid Identification of Cortical Connectivity During Functional Mapping %A Lawrence J. Crowther %A Peter Brunner %A Anthony L. Ritaccio %A Gerwin Schalk %B American Epilepsy Society 72nd Annual Meeting %C New Orleans, LA %8 12/2018 %G eng %0 Journal Article %J Stroke %D 2017 %T Contralesional Brain-Computer Interface Control of a Powered Exoskeleton for Motor Recovery in Chronic Stroke Survivors. %A Bundy, David T. %A Souders, Lauren %A Baranyai, Kelly %A Leonard, Laura %A Gerwin Schalk %A Coker, Robert %A Moran, Daniel W. %A Huskey, Thy %A Leuthardt, Eric C. %X There are few effective therapies to achieve functional recovery from motor-related disabilities affecting the upper limb after stroke. This feasibility study tested whether a powered exoskeleton driven by a brain-computer interface (BCI), using neural activity from the unaffected cortical hemisphere, could affect motor recovery in chronic hemiparetic stroke survivors. This novel system was designed and configured for a home-based setting to test the feasibility of BCI-driven neurorehabilitation in outpatient environments. %B Stroke %8 May %G eng %U http://www.ncbi.nlm.nih.gov/pubmed/28550098 %R 10.1161/STROKEAHA.116.016304 %0 Journal Article %J Proc. Natl. Acad. Sci. U.S.A. %D 2017 %T Facephenes and rainbows: Causal evidence for functional and anatomical specificity of face and color processing in the human brain %A Gerwin Schalk %A Kapeller, C. %A Guger, C. %A Ogawa, H. %A Hiroshima, S. %A Lafer-Sousa, R. %A Saygin, Z. M. %A Kamada, K. %A Kanwisher, N. %K cortical specificity; electrical stimulation; fusiform face area %X Neuroscientists have long debated whether some regions of the human brain are exclusively engaged in a single specific mental process. Consistent with this view, fMRI has revealed cortical regions that respond selectively to certain stimulus classes such as faces. However, results from multivoxel pattern analyses (MVPA) challenge this view by demonstrating that category-selective regions often contain information about "nonpreferred" stimulus dimensions. But is this nonpreferred information causally relevant to behavior? Here we report a rare opportunity to test this question in a neurosurgical patient implanted for clinical reasons with strips of electrodes along his fusiform gyri. Broadband gamma electrocorticographic responses in multiple adjacent electrodes showed strong selectivity for faces in a region corresponding to the fusiform face area (FFA), and preferential responses to color in a nearby site, replicating earlier reports. To test the causal role of these regions in the perception of nonpreferred dimensions, we then electrically stimulated individual sites while the patient viewed various objects. When stimulated in the FFA, the patient reported seeing an illusory face (or "facephene"), independent of the object viewed. Similarly, stimulation of color-preferring sites produced illusory "rainbows." Crucially, the patient reported no change in the object viewed, apart from the facephenes and rainbows apparently superimposed on them. The functional and anatomical specificity of these effects indicate that some cortical regions are exclusively causally engaged in a single specific mental process, and prompt caution about the widespread assumption that any information scientists can decode from the brain is causally relevant to behavior. %B Proc. Natl. Acad. Sci. U.S.A. %V 114 %P 12285–12290 %8 Nov %G eng %U http://www.pnas.org/content/114/46/12285 %R doi.org/10.1073/pnas.1713447114 %0 Journal Article %J Proceedings of the National Academy of Sciences of the United States of America %D 2017 %T Spatiotemporal dynamics of word retrieval in speech production revealed by cortical high-frequency band activity. %A Riès, Stephanie K. %A Dhillon, Rummit K. %A Clarke, Alex %A King-Stephens, David %A Laxer, Kenneth D. %A Weber, Peter B. %A Kuperman, Rachel A. %A Auguste, Kurtis I. %A Peter Brunner %A Gerwin Schalk %A Lin, Jack J. %A Parvizi, Josef %A Crone, Nathan E. %A Dronkers, Nina F. %A Robert T. Knight %X Word retrieval is core to language production and relies on complementary processes: the rapid activation of lexical and conceptual representations and word selection, which chooses the correct word among semantically related competitors. Lexical and conceptual activation is measured by semantic priming. In contrast, word selection is indexed by semantic interference and is hampered in semantically homogeneous (HOM) contexts. We examined the spatiotemporal dynamics of these complementary processes in a picture naming task with blocks of semantically heterogeneous (HET) or HOM stimuli. We used electrocorticography data obtained from frontal and temporal cortices, permitting detailed spatiotemporal analysis of word retrieval processes. A semantic interference effect was observed with naming latencies longer in HOM versus HET blocks. Cortical response strength as indexed by high-frequency band (HFB) activity (70-150 Hz) amplitude revealed effects linked to lexical-semantic activation and word selection observed in widespread regions of the cortical mantle. Depending on the subsecond timing and cortical region, HFB indexed semantic interference (i.e., more activity in HOM than HET blocks) or semantic priming effects (i.e., more activity in HET than HOM blocks). These effects overlapped in time and space in the left posterior inferior temporal gyrus and the left prefrontal cortex. The data do not support a modular view of word retrieval in speech production but rather support substantial overlap of lexical-semantic activation and word selection mechanisms in the brain. %B Proceedings of the National Academy of Sciences of the United States of America %8 May %G eng %U http://www.ncbi.nlm.nih.gov/pubmed/28533406 %R 10.1073/pnas.1620669114 %0 Journal Article %J NeuroImage %D 2016 %T Alpha power indexes task-related networks on large and small scales: A multimodal ECoG study in humans and a non-human primate. %A de Pesters, A. %A Coon, W. G. %A Peter Brunner %A Gunduz, A. %A A L Ritaccio %A Brunet, N. M. %A de Weerd, P. %A Roberts, M. J. %A Oostenveld, R. %A Fries, P. %A Gerwin Schalk %X Performing different tasks, such as generating motor movements or processing sensory input, requires the recruitment of specific networks of neuronal populations. Previous studies suggested that power variations in the alpha band (8-12Hz) may implement such recruitment of task-specific populations by increasing cortical excitability in task-related areas while inhibiting population-level cortical activity in task-unrelated areas (Klimesch et al., 2007; Jensen and Mazaheri, 2010). However, the precise temporal and spatial relationships between the modulatory function implemented by alpha oscillations and population-level cortical activity remained undefined. Furthermore, while several studies suggested that alpha power indexes task-related populations across large and spatially separated cortical areas, it was largely unclear whether alpha power also differentially indexes smaller networks of task-related neuronal populations. Here we addressed these questions by investigating the temporal and spatial relationships of electrocorticographic (ECoG) power modulations in the alpha band and in the broadband gamma range (70-170Hz, indexing population-level activity) during auditory and motor tasks in five human subjects and one macaque monkey. In line with previous research, our results confirm that broadband gamma power accurately tracks task-related behavior and that alpha power decreases in task-related areas. More importantly, they demonstrate that alpha power suppression lags population-level activity in auditory areas during the auditory task, but precedes it in motor areas during the motor task. This suppression of alpha power in task-related areas was accompanied by an increase in areas not related to the task. In addition, we show for the first time that these differential modulations of alpha power could be observed not only across widely distributed systems (e.g., motor vs. auditory system), but also within the auditory system. Specifically, alpha power was suppressed in the locations within the auditory system that most robustly responded to particular sound stimuli. Altogether, our results provide experimental evidence for a mechanism that preferentially recruits task-related neuronal populations by increasing cortical excitability in task-related cortical areas and decreasing cortical excitability in task-unrelated areas. This mechanism is implemented by variations in alpha power and is common to humans and the non-human primate under study. These results contribute to an increasingly refined understanding of the mechanisms underlying the selection of the specific neuronal populations required for task execution. %B NeuroImage %V 134 %P 122–131 %8 Jul %G eng %U http://www.ncbi.nlm.nih.gov/pubmed/27057960 %R 10.1016/j.neuroimage.2016.03.074 %0 Journal Article %J Epilepsy & behavior case reports %D 2016 %T Electrocorticographic mapping of expressive language function without requiring the patient to speak: A report of three cases. %A de Pesters, Adriana %A Taplin, AmiLyn M. %A Adamo, Matthew A. %A A L Ritaccio %A Gerwin Schalk %X Patients requiring resective brain surgery often undergo functional brain mapping during perioperative planning to localize expressive language areas. Currently, all established protocols to perform such mapping require substantial time and patient participation during verb generation or similar tasks. These issues can make language mapping impractical in certain clinical circumstances (e.g., during awake craniotomies) or with certain populations (e.g., pediatric patients). Thus, it is important to develop new techniques that reduce mapping time and the requirement for active patient participation. Several neuroscientific studies reported that the mere auditory presentation of speech stimuli can engage not only receptive but also expressive language areas. Here, we tested the hypothesis that submission of electrocorticographic (ECoG) recordings during a short speech listening task to an appropriate analysis procedure can identify eloquent expressive language cortex without requiring the patient to speak. %B Epilepsy & behavior case reports %V 6 %P 13–18 %8 Mar %G eng %U http://www.ncbi.nlm.nih.gov/pubmed/27408803 %R 10.1016/j.ebcr.2016.02.002 %0 Journal Article %J Epilepsy & behavior case reports %D 2016 %T Intraoperative mapping of expressive language cortex using passive real-time electrocorticography. %A Taplin, AmiLyn M. %A de Pesters, Adriana %A Peter Brunner %A Hermes, Dora %A Dalfino, John C. %A Adamo, Matthew A. %A A L Ritaccio %A Gerwin Schalk %X In this case report, we investigated the utility and practicality of passive intraoperative functional mapping of expressive language cortex using high-resolution electrocorticography (ECoG). The patient presented here experienced new-onset seizures caused by a medium-grade tumor in very close proximity to expressive language regions. In preparation of tumor resection, the patient underwent multiple functional language mapping procedures. We examined the relationship of results obtained with intraoperative high-resolution ECoG, extraoperative ECoG utilizing a conventional subdural grid, extraoperative electrical cortical stimulation (ECS) mapping, and functional magnetic resonance imaging (fMRI). Our results demonstrate that intraoperative mapping using high-resolution ECoG is feasible and, within minutes, produces results that are qualitatively concordant to those achieved by extraoperative mapping modalities. They also suggest that functional language mapping of expressive language areas with ECoG may prove useful in many intraoperative conditions given its time efficiency and safety. Finally, they demonstrate that integration of results from multiple functional mapping techniques, both intraoperative and extraoperative, may serve to improve the confidence in or precision of functional localization when pathology encroaches upon eloquent language cortex. %B Epilepsy & behavior case reports %V 5 %P 46–51 %8 Mar %G eng %U http://www.ncbi.nlm.nih.gov/pubmed/27408802 %R 10.1016/j.ebcr.2016.03.003 %0 Journal Article %J Journal of neuroscience methods %D 2016 %T A method to establish the spatiotemporal evolution of task-related cortical activity from electrocorticographic signals in single trials. %A Coon, W. G. %A Gerwin Schalk %X Progress in neuroscience depends substantially on the ability to establish the detailed spatial and temporal sequence of neuronal population-level activity across large areas of the brain. Because there is substantial inter-trial variability in neuronal activity, traditional techniques that rely on signal averaging obscure where and when neuronal activity occurs. Thus, up to the present, it has been difficult to examine the detailed progression of neuronal activity across large areas of the brain. %B Journal of neuroscience methods %V 271 %P 76–85 %8 Sep %G eng %U http://www.ncbi.nlm.nih.gov/pubmed/27427301 %R 10.1016/j.jneumeth.2016.06.024 %0 Journal Article %J Proceedings of the National Academy of Sciences of the United States of America %D 2016 %T Neural correlate of the construction of sentence meaning. %A Fedorenko, Evelina %A Scott, Terri L. %A Peter Brunner %A Coon, William G. %A Pritchett, Brianna %A Gerwin Schalk %A Kanwisher, Nancy %X The neural processes that underlie your ability to read and understand this sentence are unknown. Sentence comprehension occurs very rapidly, and can only be understood at a mechanistic level by discovering the precise sequence of underlying computational and neural events. However, we have no continuous and online neural measure of sentence processing with high spatial and temporal resolution. Here we report just such a measure: intracranial recordings from the surface of the human brain show that neural activity, indexed by $\gamma$-power, increases monotonically over the course of a sentence as people read it. This steady increase in activity is absent when people read and remember nonword-lists, despite the higher cognitive demand entailed, ruling out accounts in terms of generic attention, working memory, and cognitive load. Response increases are lower for sentence structure without meaning (``Jabberwocky'' sentences) and word meaning without sentence structure (word-lists), showing that this effect is not explained by responses to syntax or word meaning alone. Instead, the full effect is found only for sentences, implicating compositional processes of sentence understanding, a striking and unique feature of human language not shared with animal communication systems. This work opens up new avenues for investigating the sequence of neural events that underlie the construction of linguistic meaning. %B Proceedings of the National Academy of Sciences of the United States of America %V 113 %P E6256–E6262 %8 Oct %G eng %U http://www.ncbi.nlm.nih.gov/pubmed/27671642 %R 10.1073/pnas.1612132113 %0 Journal Article %J NeuroImage %D 2016 %T Oscillatory phase modulates the timing of neuronal activations and resulting behavior. %A Coon, W. G. %A Gunduz, A. %A Peter Brunner %A A L Ritaccio %A Pesaran, B. %A Gerwin Schalk %X Human behavioral response timing is highly variable from trial to trial. While it is generally understood that behavioral variability must be due to trial-by-trial variations in brain function, it is still largely unknown which physiological mechanisms govern the timing of neural activity as it travels through networks of neuronal populations, and how variations in the timing of neural activity relate to variations in the timing of behavior. In our study, we submitted recordings from the cortical surface to novel analytic techniques to chart the trajectory of neuronal population activity across the human cortex in single trials, and found joint modulation of the timing of this activity and of consequent behavior by neuronal oscillations in the alpha band (8-12Hz). Specifically, we established that the onset of population activity tends to occur during the trough of oscillatory activity, and that deviations from this preferred relationship are related to changes in the timing of population activity and the speed of the resulting behavioral response. These results indicate that neuronal activity incurs variable delays as it propagates across neuronal populations, and that the duration of each delay is a function of the instantaneous phase of oscillatory activity. We conclude that the results presented in this paper are supportive of a general model for variability in the effective speed of information transmission in the human brain and for variability in the timing of human behavior. %B NeuroImage %V 133 %P 294–301 %8 Jun %G eng %U http://www.ncbi.nlm.nih.gov/pubmed/26975551 %R 10.1016/j.neuroimage.2016.02.080 %0 Journal Article %J Epilepsy & behavior : E&B %D 2016 %T Proceedings of the Eighth International Workshop on Advances in Electrocorticography. %A A L Ritaccio %A Williams, Justin %A Denison, Tim %A Foster, Brett L. %A Starr, Philip A. %A Gunduz, Aysegul %A Zijlmans, Maeike %A Gerwin Schalk %X Excerpted proceedings of the Eighth International Workshop on Advances in Electrocorticography (ECoG), which convened October 15-16, 2015 in Chicago, IL, are presented. The workshop series has become the foremost gathering to present current basic and clinical research in subdural brain signal recording and analysis. %B Epilepsy & behavior : E&B %V 64 %P 248–252 %8 Nov %G eng %U http://www.ncbi.nlm.nih.gov/pubmed/27780085 %R 10.1016/j.yebeh.2016.08.020 %0 Journal Article %J PloS one %D 2016 %T Spatio-Temporal Progression of Cortical Activity Related to Continuous Overt and Covert Speech Production in a Reading Task. %A Brumberg, Jonathan S. %A Krusienski, Dean J. %A Chakrabarti, Shreya %A Gunduz, Aysegul %A Peter Brunner %A A L Ritaccio %A Gerwin Schalk %X How the human brain plans, executes, and monitors continuous and fluent speech has remained largely elusive. For example, previous research has defined the cortical locations most important for different aspects of speech function, but has not yet yielded a definition of the temporal progression of involvement of those locations as speech progresses either overtly or covertly. In this paper, we uncovered the spatio-temporal evolution of neuronal population-level activity related to continuous overt speech, and identified those locations that shared activity characteristics across overt and covert speech. Specifically, we asked subjects to repeat continuous sentences aloud or silently while we recorded electrical signals directly from the surface of the brain (electrocorticography (ECoG)). We then determined the relationship between cortical activity and speech output across different areas of cortex and at sub-second timescales. The results highlight a spatio-temporal progression of cortical involvement in the continuous speech process that initiates utterances in frontal-motor areas and ends with the monitoring of auditory feedback in superior temporal gyrus. Direct comparison of cortical activity related to overt versus covert conditions revealed a common network of brain regions involved in speech that may implement orthographic and phonological processing. Our results provide one of the first characterizations of the spatiotemporal electrophysiological representations of the continuous speech process, and also highlight the common neural substrate of overt and covert speech. These results thereby contribute to a refined understanding of speech functions in the human brain. %B PloS one %V 11 %P e0166872 %8 Nov %G eng %U http://www.ncbi.nlm.nih.gov/pubmed/27875590 %R 10.1371/journal.pone.0166872 %0 Journal Article %J PLoS computational biology %D 2016 %T Spontaneous Decoding of the Timing and Content of Human Object Perception from Cortical Surface Recordings Reveals Complementary Information in the Event-Related Potential and Broadband Spectral Change. %A Miller, Kai J. %A Gerwin Schalk %A Hermes, Dora %A Ojemann, Jeffrey G. %A Rao, Rajesh P. N. %K Visual Perception %X The link between object perception and neural activity in visual cortical areas is a problem of fundamental importance in neuroscience. Here we show that electrical potentials from the ventral temporal cortical surface in humans contain sufficient information for spontaneous and near-instantaneous identification of a subject's perceptual state. Electrocorticographic (ECoG) arrays were placed on the subtemporal cortical surface of seven epilepsy patients. Grayscale images of faces and houses were displayed rapidly in random sequence. We developed a template projection approach to decode the continuous ECoG data stream spontaneously, predicting the occurrence, timing and type of visual stimulus. In this setting, we evaluated the independent and joint use of two well-studied features of brain signals, broadband changes in the frequency power spectrum of the potential and deflections in the raw potential trace (event-related potential; ERP). Our ability to predict both the timing of stimulus onset and the type of image was best when we used a combination of both the broadband response and ERP, suggesting that they capture different and complementary aspects of the subject's perceptual state. Specifically, we were able to predict the timing and type of 96% of all stimuli, with less than 5% false positive rate and a {\textasciitilde}20ms error in timing. %B PLoS computational biology %V 12 %P e1004660 %8 Jan %G eng %U http://www.ncbi.nlm.nih.gov/pubmed/26820899 %R 10.1371/journal.pcbi.1004660 %0 Journal Article %J Scientific reports %D 2016 %T Word pair classification during imagined speech using direct brain recordings. %A Martin, Stéphanie %A Peter Brunner %A Iturrate, Iñaki %A Millán, José Del R. %A Gerwin Schalk %A Robert T. Knight %A Pasley, Brian N. %X People that cannot communicate due to neurological disorders would benefit from an internal speech decoder. Here, we showed the ability to classify individual words during imagined speech from electrocorticographic signals. In a word imagery task, we used high gamma (70-150þinspaceHz) time features with a support vector machine model to classify individual words from a pair of words. To account for temporal irregularities during speech production, we introduced a non-linear time alignment into the SVM kernel. Classification accuracy reached 88% in a two-class classification framework (50% chance level), and average classification accuracy across fifteen word-pairs was significant across five subjects (meanþinspace=þinspace58%; pþinspace<þinspace0.05). We also compared classification accuracy between imagined speech, overt speech and listening. As predicted, higher classification accuracy was obtained in the listening and overt speech conditions (meanþinspace=þinspace89% and 86%, respectively; pþinspace<þinspace0.0001), where speech stimuli were directly presented. The results provide evidence for a neural representation for imagined words in the temporal lobe, frontal lobe and sensorimotor cortex, consistent with previous findings in speech perception and production. These data represent a proof of concept study for basic decoding of speech imagery, and delineate a number of key challenges to usage of speech imagery neural representations for clinical applications. %B Scientific reports %V 6 %P 25803 %8 May %G eng %U http://www.ncbi.nlm.nih.gov/pubmed/27165452 %R 10.1038/srep25803 %0 Journal Article %J Journal of Neural Engineering %D 2015 %T Brain-to-text: Decoding spoken sentences from phone representations in the brain. %A Herff, C. %A Heger, D. %A Pesters, Adriana de %A Telaar, D. %A Peter Brunner %A Gerwin Schalk %A Schultz, T. %K automatic speech recognition %K brain-computer interface %K broadband gamma %K ECoG %K Electrocorticography %K pattern recognition %K speech decoding %K speech production %X It has long been speculated whether communication between humans and machines based on natural speech related cortical activity is possible. Over the past decade, studies have suggested that it is feasible to recognize isolated aspects of speech from neural signals, such as auditory features, phones or one of a few isolated words. However, until now it remained an unsolved challenge to decode continuously spoken speech from the neural substrate associated with speech and language processing. Here, we show for the first time that continuously spoken speech can be decoded into the expressed words from intracranial electrocorticographic (ECoG) recordings.Specifically, we implemented a system, which we call Brain-To-Text that models single phones, employs techniques from automatic speech recognition (ASR), and thereby transforms brain activity while speaking into the corresponding textual representation. Our results demonstrate that our system can achieve word error rates as low as 25% and phone error rates below 50%. Additionally, our approach contributes to the current understanding of the neural basis of continuous speech production by identifying those cortical regions that hold substantial information about individual phones. In conclusion, the Brain-To- Text system described in this paper represents an important step toward human-machine communication based on imagined speech. %B Journal of Neural Engineering %8 06/2015 %G eng %U http://journal.frontiersin.org/article/10.3389/fnins.2015.00217/abstract %R 10.3389/fnins.2015.00217 %0 Journal Article %J Front. Neurosci %D 2015 %T Cortical alpha activity predicts the confidence in an impending action. %A Kubánek, J %A Jeremy Jeremy Hill %A Snyder, Lawrence H. %A Gerwin Schalk %K certainty %K EEG %K human %K neural correlates %K perceptual decision-making %X When we make a decision, we experience a degree of confidence that our choice may lead to a desirable outcome. Recent studies in animals have probed the subjective aspects of the choice confidence using confidence-reporting tasks. These studies showed that estimates of the choice confidence substantially modulate neural activity in multiple regions of the brain. Building on these findings, we investigated the neural representation of the confidence in a choice in humans who explicitly reported the confidence in their choice. Subjects performed a perceptual decision task in which they decided between choosing a button press or a saccade while we recorded EEG activity. Following each choice, subjects indicated whether they were sure or unsure about the choice. We found that alpha activity strongly encodes a subject's confidence level in a forthcoming button press choice. The neural effect of the subjects' confidence was independent of the reaction time and independent of the sensory input modeled as a decision variable. Furthermore, the effect is not due to a general cognitive state, such as reward expectation, because the effect was specifically observed during button press choices and not during saccade choices. The neural effect of the confidence in the ensuing button press choice was strong enough that we could predict, from independent single trial neural signals, whether a subject was going to be sure or unsure of an ensuing button press choice. In sum, alpha activity in human cortex provides a window into the commitment to make a hand movement. %B Front. Neurosci %8 07/2015 %G eng %U http://journal.frontiersin.org/article/10.3389/fnins.2015.00243/abstract %R 10.3389/fnins.2015.00243 %0 Journal Article %J Journal of neural engineering %D 2015 %T The effects of spatial filtering and artifacts on electrocorticographic signals. %A Liu, Y. %A Coon, W. G. %A de Pesters, A. %A Peter Brunner %A Gerwin Schalk %K Young Adult %X Electrocorticographic (ECoG) signals contain noise that is common to all channels and noise that is specific to individual channels. Most published ECoG studies use common average reference (CAR) spatial filters to remove common noise, but CAR filters may introduce channel-specific noise into other channels. To address this concern, scientists often remove artifactual channels prior to data analysis. However, removing these channels depends on expert-based labeling and may also discard useful data. Thus, the effects of spatial filtering and artifacts on ECoG signals have been largely unknown. This study aims to quantify these effects and thereby address this gap in knowledge. %B Journal of neural engineering %V 12 %P 056008 %8 Oct %G eng %U http://www.ncbi.nlm.nih.gov/pubmed/26268446 %R 10.1088/1741-2560/12/5/056008 %0 Journal Article %J Front Hum Neurosci %D 2015 %T Electrocorticographic representations of segmental features in continuous speech. %A Lotte, Fabien %A Jonathan S Brumberg %A Peter Brunner %A Gunduz, Aysegul %A A L Ritaccio %A Guan, Cuntai %A Gerwin Schalk %K electrocorticography (ECoG) %K manner of articulation %K place of articulation %K speech processing %K voicing %X Acoustic speech output results from coordinated articulation of dozens of muscles, bones and cartilages of the vocal mechanism. While we commonly take the fluency and speed of our speech productions for granted, the neural mechanisms facilitating the requisite muscular control are not completely understood. Previous neuroimaging and electrophysiology studies of speech sensorimotor control has typically concentrated on speech sounds (i.e., phonemes, syllables and words) in isolation; sentence-length investigations have largely been used to inform coincident linguistic processing. In this study, we examined the neural representations of segmental features (place and manner of articulation, and voicing status) in the context of fluent, continuous speech production. We used recordings from the cortical surface [electrocorticography (ECoG)] to simultaneously evaluate the spatial topography and temporal dynamics of the neural correlates of speech articulation that may mediate the generation of hypothesized gestural or articulatory scores. We found that the representation of place of articulation involved broad networks of brain regions during all phases of speech production: preparation, execution and monitoring. In contrast, manner of articulation and voicing status were dominated by auditory cortical responses after speech had been initiated. These results provide a new insight into the articulatory and auditory processes underlying speech production in terms of their motor requirements and acoustic correlates. %B Front Hum Neurosci %V 9 %P 97 %8 02/2015 %G eng %U http://www.ncbi.nlm.nih.gov/pubmed/25759647 %R 10.3389/fnhum.2015.00097 %0 Journal Article %J Front. Hum. Neurosci. %D 2015 %T A general framework for dynamic cortical function: the function-through-biased-oscillations (FBO) hypothesis. %A Gerwin Schalk %K communication-through-coherence %K gating-by-inhibition %K information routing %K oscillations %K oscillatory modulation %X A central goal of neuroscience is to determine how the brain’s relatively static anatomy can support dynamic cortical function, i.e., cortical function that varies according to task demands. In pursuit of this goal, scientists have produced a large number of experimental results and established influential conceptual frameworks, in particular communication-through-coherence (CTC) and gating-by-inhibition (GBI), but these data and frameworks have not provided a parsimonious view of the principles that underlie cortical function. Here I synthesize these existing experimental results and the CTC and GBI frameworks, and propose the function-through-biased-oscillations (FBO) hypothesis as a model to understand dynamic cortical function. The FBO hypothesis suggests that oscillatory voltage amplitude is the principal measurement that directly reflects cortical excitability, that asymmetries in voltage amplitude explain a range of brain signal phenomena, and that predictive variations in such asymmetric oscillations provide a simple and general model for information routing that can help to explain dynamic cortical function. %B Front. Hum. Neurosci. %V 9 %8 06/2015 %G eng %U http://journal.frontiersin.org/article/10.3389/fnhum.2015.00352/abstract %N 352 %R 10.3389/fnhum.2015.00352 %0 Journal Article %J Journal of Neural Engineering %D 2015 %T Identifying the Attended Speaker Using Electrocorticographic (ECoG) Signals. %A Dijkstra, K. %A Peter Brunner %A Gunduz, Aysegul %A Coon, W.G. %A A L Ritaccio %A Farquhar, Jason %A Gerwin Schalk %K auditory attention %K Brain-computer interface (BCI) %K Cocktail Party %K electrocorticography (ECoG) %X People affected by severe neuro-degenerative diseases (e.g., late-stage amyotrophic lateral sclerosis (ALS) or locked-in syndrome) eventually lose all muscular control. Thus, they cannot use traditional assistive communication devices that depend on muscle control, or brain-computer interfaces (BCIs) that depend on the ability to control gaze. While auditory and tactile BCIs can provide communication to such individuals, their use typically entails an artificial mapping between the stimulus and the communication intent. This makes these BCIs difficult to learn and use. In this study, we investigated the use of selective auditory attention to natural speech as an avenue for BCI communication. In this approach, the user communicates by directing his/her attention to one of two simultaneously presented speakers. We used electrocorticographic (ECoG) signals in the gamma band (70–170 Hz) to infer the identity of attended speaker, thereby removing the need to learn such an artificial mapping. Our results from twelve human subjects show that a single cortical location over superior temporal gyrus or pre-motor cortex is typically sufficient to identify the attended speaker within 10 s and with 77% accuracy (50% accuracy due to chance). These results lay the groundwork for future studies that may determine the real-time performance of BCIs based on selective auditory attention to speech. %B Journal of Neural Engineering %G eng %U https://www.ncbi.nlm.nih.gov/pmc/articles/PMC4776341/ %R 10.1080/2326263X.2015.1063363 %0 Book Section %B Brain-Computer Interface Research: A State-of-the-Art Summary %D 2015 %T Near-Instantaneous Classification of Perceptual States from Cortical Surface Recordings %A Miller, Kai J %A Gerwin Schalk %A Hermes, Dora %A Ojemann, Jeffrey G %A Rao, Rajesh P N %K broadband power %K Electrocorticography %K event-related potential %K fusiform cortex %K human vision %X Human visual processing is of such complexity that, despite decades of focused research, many basic questions remain unanswered. Although we know that the inferotemporal cortex is a key region in object recognition, we don’t fully understand its physiologic role in brain function, nor do we have the full set of tools to explore this question. Here we show that electrical potentials from the surface of the human brain contain enough information to decode a subject’s perceptual state accurately, and with fine temporal precision. Electrocorticographic (ECoG) arrays were placed over the inferotemporal cortical areas of seven subjects. Pictures of faces and houses were quickly presented while each subject performed a simple visual task. Results showed that two well-known types of brain signals—event-averaged broadband power and event-averaged raw potential—can independently or together be used to classify the presented image. When applied to continuously recorded brain activity, our decoding technique could accurately predict whether each stimulus was a face, house, or neither, with  20 ms timing error. These results provide a roadmap for improved brain-computer interfacing tools to help neurosurgeons, research scientists, engineers, and, ultimately, patients. %B Brain-Computer Interface Research: A State-of-the-Art Summary %I Springer International Publishing %C New York City, NY %P 105-114 %@ 978-3-319-25188-2 %G eng %U http://link.springer.com/chapter/10.1007/978-3-319-25190-5_10 %R 10.1007/978-3-319-25190-5_10 %0 Journal Article %J Neuroinformatics %D 2015 %T NeuralAct: A Tool to Visualize Electrocortical (ECoG) Activity on a Three-Dimensional Model of the Cortex. %A Kubanek, Jan %A Gerwin Schalk %K Brain %K DOT %K ECoG %K EEG %K imaging %K Matlab %K MEG %X

Electrocorticography (ECoG) records neural signals directly from the surface of the cortex. Due to its high temporal and favorable spatial resolution, ECoG has emerged as a valuable new tool in acquiring cortical activity in cognitive and systems neuroscience. Many studies using ECoG visualized topographies of cortical activity or statistical tests on a three-dimensional model of the cortex, but a dedicated tool for this function has not yet been described. In this paper, we describe the NeuralAct package that serves this purpose. This package takes as input the 3D coordinates of the recording sensors, a cortical model in the same coordinate system (e.g., Talairach), and the activation data to be visualized at each sensor. It then aligns the sensor coordinates with the cortical model, convolves the activation data with a spatial kernel, and renders the resulting activations in color on the cortical model. The NeuralAct package can plot cortical activations of an individual subject as well as activations averaged over subjects. It is capable to render single images as well as sequences of images. The software runs under Matlab and is stable and robust. We here provide the tool and describe its visualization capabilities and procedures. The provided package contains thoroughly documented code and includes a simple demo that guides the researcher through the functionality of the tool.

%B Neuroinformatics %V 13 %P 167-74 %8 04/2015 %G eng %U http://www.ncbi.nlm.nih.gov/pubmed/25381641 %N 2 %R 10.1007/s12021-014-9252-3 %0 Journal Article %J Proceedings of the IEEE %D 2015 %T The Plurality of Human Brain-Computer Interfacing. %A Mueller-Putz, G. %A Millán, José del R %A Gerwin Schalk %A Mueller, K.R. %K Brain-computer interface (BCI) %X The articles in this special issue focus on brain-computer interfacing. The papers are dedicated to this growing and diversifying research enterprise, and features important review articles as well as some important current examples of research in this area. The field of brain-computer interface (BCI) research began to develop about 25 years ago and transformed from initially isolated demonstrations by a few groups into a large scientific enterprise that is currently producing hundreds of peer-reviewed articles and several dedicated conferences and workshops each year. This level of productivity is reflective of the large and continually growing enthusiasm by the scientific community, funding agencies, and the public. %B Proceedings of the IEEE %P 868-870 %8 06/2015 %G eng %U http://ieeexplore.ieee.org/xpl/articleDetails.jsp?arnumber=7115302 %R 10.1109/JPROC.2015.2425835 %0 Journal Article %J Epilepsy & behavior : E&B %D 2015 %T Proceedings of the Seventh International Workshop on Advances in Electrocorticography. %A A L Ritaccio %A Matsumoto, Riki %A Morrell, Martha %A Kamada, Kyousuke %A Koubeissi, Mohamad %A Poeppel, David %A Lachaux, Jean-Philippe %A Yanagisawa, Yakufumi %A Hirata, Masayuki %A Guger, Christoph %A Gerwin Schalk %K Humans %X The Seventh International Workshop on Advances in Electrocorticography (ECoG) convened in Washington, DC, on November 13-14, 2014. Electrocorticography-based research continues to proliferate widely across basic science and clinical disciplines. The 2014 workshop highlighted advances in neurolinguistics, brain-computer interface, functional mapping, and seizure termination facilitated by advances in the recording and analysis of the ECoG signal. The following proceedings document summarizes the content of this successful multidisciplinary gathering. %B Epilepsy & behavior : E&B %V 51 %P 312–320 %8 Oct %G eng %U http://www.ncbi.nlm.nih.gov/pubmed/26322594 %R 10.1016/j.yebeh.2015.08.002 %0 Book Section %B Brain-Computer Interface Research: A State-of-the-Art Summary %D 2015 %T Towards an Auditory Attention BCI %A Peter Brunner %A Dijkstra, K. %A Coon, W.G. %A Mellinger, Jürgen %A A L Ritaccio %A Gerwin Schalk %X People affected by severe neuro-degenerative diseases (e.g., late-stage amyotrophic lateral sclerosis (ALS) or locked-in syndrome) eventually lose all muscular control and are no longer able to gesture or speak. For this population, an auditory BCI is one of only a few remaining means of communication. All currently used auditory BCIs require a relatively artificial mapping between a stimulus and a communication output. This mapping is cumbersome to learn and use. Recent studies suggest that electrocorticographic (ECoG) signals in the gamma band (i.e., 70–170 Hz) can be used to infer the identity of auditory speech stimuli, effectively removing the need to learn such an artificial mapping. However, BCI systems that use this physiological mechanism for communication purposes have not yet been described. In this study, we explore this possibility by implementing a BCI2000-based real-time system that uses ECoG signals to identify the attended speaker. %B Brain-Computer Interface Research: A State-of-the-Art Summary %I Springer International Publishing %C New York City, NY %P 29-42 %@ 978-3-319-25188-2 %G eng %U http://link.springer.com/chapter/10.1007%2F978-3-319-25190-5_4 %R 10.1007/978-3-319-25190-5_4 %0 Journal Article %J Frontiers in Computational Neuroscience %D 2014 %T Assessing dynamics, spatial scale, and uncertainty in task-related brain network analyses. %A Stephen, Emily P %A Lepage, Kyle Q %A Eden, Uri T %A Peter Brunner %A Gerwin Schalk %A Jonathan S Brumberg %A Guenther, Frank H %A Kramer, Mark A %K canonical correlation %K coherence %K ECoG %K EEG %K functional connectivity %K MEG %X The brain is a complex network of interconnected elements, whose interactions evolve dynamically in time to cooperatively perform specific functions. A common technique to probe these interactions involves multi-sensor recordings of brain activity during a repeated task. Many techniques exist to characterize the resulting task-related activity, including establishing functional networks, which represent the statistical associations between brain areas. Although functional network inference is commonly employed to analyze neural time series data, techniques to assess the uncertainty—both in the functional network edges and the corresponding aggregate measures of network topology—are lacking. To address this, we describe a statistically principled approach for computing uncertainty in functional networks and aggregate network measures in task-related data. The approach is based on a resampling procedure that utilizes the trial structure common in experimental recordings. We show in simulations that this approach successfully identifies functional networks and associated measures of confidence emergent during a task in a variety of scenarios, including dynamically evolving networks. In addition, we describe a principled technique for establishing functional networks based on predetermined regions of interest using canonical correlation. Doing so provides additional robustness to the functional network inference. Finally, we illustrate the use of these methods on example invasive brain voltage recordings collected during an overt speech task. The general strategy described here—appropriate for static and dynamic network inference and different statistical measures of coupling—permits the evaluation of confidence in network measures in a variety of settings common to neuroscience. %B Frontiers in Computational Neuroscience %V 8 %8 03/2014 %G eng %U http://www.ncbi.nlm.nih.gov/pubmed/24678295 %N 31 %R 10.3389/fncom.2014.00031 %0 Journal Article %J Frontiers in Neuroengineering %D 2014 %T Decoding spectrotemporal features of overt and covert speech from the human cortex. %A Martin, Stéphanie %A Peter Brunner %A Holdgraf, Chris %A Heinze, Hans-Jochen %A Nathan E. Crone %A Rieger, Jochem %A Gerwin Schalk %A Robert T. Knight %A Pasley, Brian N. %K covert speech %K decoding model %K Electrocorticography %K pattern recognition %K speech production %X Auditory perception and auditory imagery have been shown to activate overlapping brain regions. We hypothesized that these phenomena also share a common underlying neural representation. To assess this, we used electrocorticography intracranial recordings from epileptic patients performing an out loud or a silent reading task. In these tasks, short stories scrolled across a video screen in two conditions: subjects read the same stories both aloud (overt) and silently (covert). In a control condition the subject remained in a resting state. We first built a high gamma (70–150 Hz) neural decoding model to reconstruct spectrotemporal auditory features of self-generated overt speech. We then evaluated whether this same model could reconstruct auditory speech features in the covert speech condition. Two speech models were tested: a spectrogram and a modulation-based feature space. For the overt condition, reconstruction accuracy was evaluated as the correlation between original and predicted speech features, and was significant in each subject (p < 0.00001; paired two-sample t-test). For the covert speech condition, dynamic time warping was first used to realign the covert speech reconstruction with the corresponding original speech from the overt condition. Reconstruction accuracy was then evaluated as the correlation between original and reconstructed speech features. Covert reconstruction accuracy was compared to the accuracy obtained from reconstructions in the baseline control condition. Reconstruction accuracy for the covert condition was significantly better than for the control condition (p < 0.005; paired two-sample t-test). The superior temporal gyrus, pre- and post-central gyrus provided the highest reconstruction information. The relationship between overt and covert speech reconstruction depended on anatomy. These results provide evidence that auditory representations of covert speech can be reconstructed from models that are built from an overt speech data set, supporting a partially shared neural substrate. %B Frontiers in Neuroengineering %V 7 %8 03/2014 %G eng %U http://www.ncbi.nlm.nih.gov/pubmed/24904404 %N 14 %R 10.3389/fneng.2014.00014 %0 Journal Article %J Front Hum Neurosci %D 2014 %T ECoG high gamma activity reveals distinct cortical representations of lyrics passages, harmonic and timbre-related changes in a rock song. %A Sturm, Irene %A Benjamin Blankertz %A Potes, Cristhian %A Gerwin Schalk %A Curio, Gabriel %K acoustic features %K electrocorticography (ECoG) %K high gamma %K music processing %K natural music %X

Listening to music moves our minds and moods, stirring interest in its neural underpinnings. A multitude of compositional features drives the appeal of natural music. How such original music, where a composer's opus is not manipulated for experimental purposes, engages a listener's brain has not been studied until recently. Here, we report an in-depth analysis of two electrocorticographic (ECoG) data sets obtained over the left hemisphere in ten patients during presentation of either a rock song or a read-out narrative. First, the time courses of five acoustic features (intensity, presence/absence of vocals with lyrics, spectral centroid, harmonic change, and pulse clarity) were extracted from the audio tracks and found to be correlated with each other to varying degrees. In a second step, we uncovered the specific impact of each musical feature on ECoG high-gamma power (70-170 Hz) by calculating partial correlations to remove the influence of the other four features. In the music condition, the onset and offset of vocal lyrics in ongoing instrumental music was consistently identified within the group as the dominant driver for ECoG high-gamma power changes over temporal auditory areas, while concurrently subject-individual activation spots were identified for sound intensity, timbral, and harmonic features. The distinct cortical activations to vocal speech-related content embedded in instrumental music directly demonstrate that song integrated in instrumental music represents a distinct dimension in complex music. In contrast, in the speech condition, the full sound envelope was reflected in the high gamma response rather than the onset or offset of the vocal lyrics. This demonstrates how the contributions of stimulus features that modulate the brain response differ across the two examples of a full-length natural stimulus, which suggests a context-dependent feature selection in the processing of complex auditory stimuli.

%B Front Hum Neurosci %V 8 %P 798 %8 10/2014 %G eng %U http://www.ncbi.nlm.nih.gov/pubmed/25352799 %R 10.3389/fnhum.2014.00798 %0 Journal Article %J Journal of Neural Engineering %D 2014 %T A general method for assessing brain–computer interface performance and its limitations. %A Jeremy Jeremy Hill %A Häuser, Ann-Katrin %A Gerwin Schalk %K brain-computer interface %K information gain %K information transfer rate %K Neuroprosthetics %K performance evaluation %X Objective. When researchers evaluate brain–computer interface (BCI) systems, we want quantitative answers to questions such as: How good is the system's performance? How good does it need to be? and: Is it capable of reaching the desired level in future? In response to the current lack of objective, quantitative, study-independent approaches, we introduce methods that help to address such questions. We identified three challenges: (I) the need for efficient measurement techniques that adapt rapidly and reliably to capture a wide range of performance levels; (II) the need to express results in a way that allows comparison between similar but non-identical tasks; (III) the need to measure the extent to which certain components of a BCI system (e.g. the signal processing pipeline) not only support BCI performance, but also potentially restrict the maximum level it can reach. Approach. For challenge (I), we developed an automatic staircase method that adjusted task difficulty adaptively along a single abstract axis. For challenge (II), we used the rate of information gain between two Bernoulli distributions: one reflecting the observed success rate, the other reflecting chance performance estimated by a matched random-walk method. This measure includes Wolpaw's information transfer rate as a special case, but addresses the latter's limitations including its restriction to item-selection tasks. To validate our approach and address challenge (III), we compared four healthy subjects' performance using an EEG-based BCI, a 'Direct Controller' (a high-performance hardware input device), and a 'Pseudo-BCI Controller' (the same input device, but with control signals processed by the BCI signal processing pipeline). Main results. Our results confirm the repeatability and validity of our measures, and indicate that our BCI signal processing pipeline reduced attainable performance by about 33% (21 bits/min). Significance. Our approach provides a flexible basis for evaluating BCI performance and its limitations, across a wide range of tasks and task difficulties. %B Journal of Neural Engineering %V 11 %8 03/2014 %G eng %U http://www.ncbi.nlm.nih.gov/pubmed/24658406 %N 026018 %R 10.1088/1741-2560/11/2/026018 %0 Journal Article %J Neuroimage Clin %D 2014 %T Localizing ECoG electrodes on the cortical anatomy without post-implantation imaging. %A Disha Gupta %A Jeremy Jeremy Hill %A Adamo, Matthew A %A A L Ritaccio %A Gerwin Schalk %K auditory processing %K electrocorticography (ECoG) %K electrode localization %K fiducials %K interaoperative localization %X

INTRODUCTION: Electrocorticographic (ECoG) grids are placed subdurally on the cortex in people undergoing cortical resection to delineate eloquent cortex. ECoG signals have high spatial and temporal resolution and thus can be valuable for neuroscientific research. The value of these data is highest when they can be related to the cortical anatomy. Existing methods that establish this relationship rely either on post-implantation imaging using computed tomography (CT), magnetic resonance imaging (MRI) or X-Rays, or on intra-operative photographs. For research purposes, it is desirable to localize ECoG electrodes on the brain anatomy even when post-operative imaging is not available or when intra-operative photographs do not readily identify anatomical landmarks.

METHODS: We developed a method to co-register ECoG electrodes to the underlying cortical anatomy using only a pre-operative MRI, a clinical neuronavigation device (such as BrainLab VectorVision), and fiducial markers. To validate our technique, we compared our results to data collected from six subjects who also had post-grid implantation imaging available. We compared the electrode coordinates obtained by our fiducial-based method to those obtained using existing methods, which are based on co-registering pre- and post-grid implantation images.

RESULTS: Our fiducial-based method agreed with the MRI-CT method to within an average of 8.24 mm (mean, median = 7.10 mm) across 6 subjects in 3 dimensions. It showed an average discrepancy of 2.7 mm when compared to the results of the intra-operative photograph method in a 2D coordinate system. As this method does not require post-operative imaging such as CTs, our technique should prove useful for research in intra-operative single-stage surgery scenarios. To demonstrate the use of our method, we applied our method during real-time mapping of eloquent cortex during a single-stage surgery. The results demonstrated that our method can be applied intra-operatively in the absence of post-operative imaging to acquire ECoG signals that can be valuable for neuroscientific investigations.

%B Neuroimage Clin %V 6 %P 64-76 %8 08/2014 %G eng %U http://www.ncbi.nlm.nih.gov/pubmed/25379417 %R 10.1016/j.nicl.2014.07.015 %0 Journal Article %J Epilepsy Behav %D 2014 %T Proceedings of the Fifth International Workshop on Advances in Electrocorticography. %A A L Ritaccio %A Peter Brunner %A Gunduz, Aysegul %A Hermes, Dora %A Hirsch, Lawrence J %A Jacobs, Joshua %A Kamada, Kyousuke %A Kastner, Sabine %A Robert T. Knight %A Lesser, Ronald P %A Miller, Kai %A Sejnowski, Terrence %A Worrell, Gregory %A Gerwin Schalk %K Brain Mapping %K brain-computer interface %K electrical stimulation mapping %K Electrocorticography %K functional mapping %K Gamma-frequency electroencephalography %K High-frequency oscillations %K Neuroprosthetics %K Seizure detection %K Subdural grid %X

The Fifth International Workshop on Advances in Electrocorticography convened in San Diego, CA, on November 7-8, 2013. Advancements in methodology, implementation, and commercialization across both research and in the interval year since the last workshop were the focus of the gathering. Electrocorticography (ECoG) is now firmly established as a preferred signal source for advanced research in functional, cognitive, and neuroprosthetic domains. Published output in ECoG fields has increased tenfold in the past decade. These proceedings attempt to summarize the state of the art.

%B Epilepsy Behav %V 41 %P 183-92 %8 12/2014 %G eng %U http://www.ncbi.nlm.nih.gov/pubmed/25461213 %R 10.1016/j.yebeh.2014.09.015 %0 Journal Article %J J Neurosurg Pediatr %D 2014 %T Real-time functional mapping: potential tool for improving language outcome in pediatric epilepsy surgery. %A Korostenskaja, Milena %A Chen, Po-Ching %A Salinas, Christine M %A Westerveld, Michael %A Peter Brunner %A Gerwin Schalk %A Cook, Jane C %A Baumgartner, James %A Lee, Ki H %K Adolescent %K Anticonvulsants %K Brain Mapping %K Cerebral Cortex %K Electric Stimulation %K Electroencephalography %K Epilepsies, Partial %K Female %K Humans %K Language %K Neuropsychological Tests %K Sensitivity and Specificity %K Speech %X

Accurate language localization expands surgical treatment options for epilepsy patients and reduces the risk of postsurgery language deficits. Electrical cortical stimulation mapping (ESM) is considered to be the clinical gold standard for language localization. While ESM affords clinically valuable results, it can be poorly tolerated by children, requires active participation and compliance, carries a risk of inducing seizures, is highly time consuming, and is labor intensive. Given these limitations, alternative and/or complementary functional localization methods such as analysis of electrocorticographic (ECoG) activity in high gamma frequency band in real time are needed to precisely identify eloquent cortex in children. In this case report, the authors examined 1) the use of real-time functional mapping (RTFM) for language localization in a high gamma frequency band derived from ECoG to guide surgery in an epileptic pediatric patient and 2) the relationship of RTFM mapping results to postsurgical language outcomes. The authors found that RTFM demonstrated relatively high sensitivity (75%) and high specificity (90%) when compared with ESM in a "next-neighbor" analysis. While overlapping with ESM in the superior temporal region, RTFM showed a few other areas of activation related to expressive language function, areas that were eventually resected during the surgery. The authors speculate that this resection may be associated with observed postsurgical expressive language deficits. With additional validation in more subjects, this finding would suggest that surgical planning and associated assessment of the risk/benefit ratio would benefit from information provided by RTFM mapping.

%B J Neurosurg Pediatr %V 14 %P 287-95 %8 09/2014 %G eng %U http://www.ncbi.nlm.nih.gov/pubmed/24995815 %N 3 %R 10.3171/2014.6.PEDS13477 %0 Journal Article %J Clinical EEG and neuroscience %D 2014 %T Real-Time Functional Mapping with Electrocorticography in Pediatric Epilepsy: Comparison with fMRI and ESM Findings. %A Korostenskaja, Milena %A Adam J Wilson %A Rose, Douglas F %A Peter Brunner %A Gerwin Schalk %A Leach, James %A Mangano, Francesco T %A Fujiwara, Hisako %A Rozhkov, Leonid %A Harris, Elana %A Chen, Po-Ching %A Seo, Joo-Hee %A Lee, Ki H %K Brain-computer interface (BCI) %K cortical stimulation %K electrocorticography (ECoG) %K epilepsy surgery %K functional magnetic resonance imaging (fMRI) %K functional mapping %K pediatrics %K SIGFRIED %X SIGFRIED (SIGnal modeling For Real-time Identification and Event Detection) software provides real-time functional mapping (RTFM) of eloquent cortex for epilepsy patients preparing to undergo resective surgery. This study presents the first application of paradigms used in functional magnetic resonance (fMRI) and electrical cortical stimulation mapping (ESM) studies for shared functional cortical mapping in the context of RTFM. Results from the 3 modalities are compared. A left-handed 13-year-old male with intractable epilepsy participated in functional mapping for localization of eloquent language cortex with fMRI, ESM, and RTFM. For RTFM, data were acquired over the frontal and temporal cortex. Several paradigms were sequentially presented: passive (listening to stories) and active (picture naming and verb generation). For verb generation and story processing, fMRI showed atypical right lateralizing language activation within temporal lobe regions of interest and bilateral frontal activation with slight right lateralization. Left hemisphere ESM demonstrated no eloquent language areas. RTFM procedures using story processing and picture naming elicited activity in the right lateral and basal temporal regions. Verb generation elicited strong right lateral temporal lobe activation, as well as left frontal lobe activation. RTFM results confirmed atypical language lateralization evident from fMRI and ESM. We demonstrated the feasibility and usefulness of a new RTFM stimulation paradigm during presurgical evaluation. Block design paradigms used in fMRI may be optimal for this purpose. Further development is needed to create age-appropriate RTFM test batteries. %B Clinical EEG and neuroscience %8 07/2014 %G eng %U http://www.ncbi.nlm.nih.gov/pubmed/24293161 %R 10.1177/1550059413492960 %0 Journal Article %J Journal of Neural Engineering %D 2014 %T Simultaneous Real-Time Monitoring of Multiple Cortical Systems. %A Disha Gupta %A Jeremy Jeremy Hill %A Peter Brunner %A Gunduz, Aysegul %A A L Ritaccio %A Gerwin Schalk %K auditory processing %K Electrocorticography %K movement intention %K realtime decoding %K simultaneous decoding %X OBJECTIVE: Real-time monitoring of the brain is potentially valuable for performance monitoring, communication, training or rehabilitation. In natural situations, the brain performs a complex mix of various sensory, motor or cognitive functions. Thus, real-time brain monitoring would be most valuable if (a) it could decode information from multiple brain systems simultaneously, and (b) this decoding of each brain system were robust to variations in the activity of other (unrelated) brain systems. Previous studies showed that it is possible to decode some information from different brain systems in retrospect and/or in isolation. In our study, we set out to determine whether it is possible to simultaneously decode important information about a user from different brain systems in real time, and to evaluate the impact of concurrent activity in different brain systems on decoding performance. APPROACH: We study these questions using electrocorticographic signals recorded in humans. We first document procedures for generating stable decoding models given little training data, and then report their use for offline and for real-time decoding from 12 subjects (six for offline parameter optimization, six for online experimentation). The subjects engage in tasks that involve movement intention, movement execution and auditory functions, separately, and then simultaneously. Main Results: Our real-time results demonstrate that our system can identify intention and movement periods in single trials with an accuracy of 80.4% and 86.8%, respectively (where 50% would be expected by chance). Simultaneously, the decoding of the power envelope of an auditory stimulus resulted in an average correlation coefficient of 0.37 between the actual and decoded power envelopes. These decoders were trained separately and executed simultaneously in real time. SIGNIFICANCE: This study yielded the first demonstration that it is possible to decode simultaneously the functional activity of multiple independent brain systems. Our comparison of univariate and multivariate decoding strategies, and our analysis of the influence of their decoding parameters, provides benchmarks and guidelines for future research on this topic. %B Journal of Neural Engineering %8 10/2014 %G eng %U http://www.ncbi.nlm.nih.gov/pubmed/25080161 %R 10.1088/1741-2560/11/5/056001 %0 Journal Article %J NeuroImage %D 2014 %T Spatial and temporal relationships of electrocorticographic alpha and gamma activity during auditory processing. %A Potes, Cristhian %A Peter Brunner %A Gunduz, Aysegul %A Robert T. Knight %A Gerwin Schalk %K alpha and high gamma activity %K auditory processing %K electrocorticography (ECoG) %K functional connectivity %K granger causality %K thalamo-cortical interactions %X Neuroimaging approaches have implicated multiple brain sites in musical perception, including the posterior part of the superior temporal gyrus and adjacent perisylvian areas. However, the detailed spatial and temporal relationship of neural signals that support auditory processing is largely unknown. In this study, we applied a novel inter-subject analysis approach to electrophysiological signals recorded from the surface of the brain (electrocorticography (ECoG)) in ten human subjects. This approach allowed us to reliably identify those ECoG features that were related to the processing of a complex auditory stimulus (i.e., continuous piece of music) and to investigate their spatial, temporal, and causal relationships. Our results identified stimulus-related modulations in the alpha (8-12 Hz) and high gamma (70-110 Hz) bands at neuroanatomical locations implicated in auditory processing. Specifically, we identified stimulus-related ECoG modulations in the alpha band in areas adjacent to primary auditory cortex, which are known to receive afferent auditory projections from the thalamus (80 of a total of 15,107 tested sites). In contrast, we identified stimulus-related ECoG modulations in the high gamma band not only in areas close to primary auditory cortex but also in other perisylvian areas known to be involved in higher-order auditory processing, and in superior premotor cortex (412/15,107 sites). Across all implicated areas, modulations in the high gamma band preceded those in the alpha band by 280 ms, and activity in the high gamma band causally predicted alpha activity, but not vice versa (Granger causality, p<1e(-8)). Additionally, detailed analyses using Granger causality identified causal relationships of high gamma activity between distinct locations in early auditory pathways within superior temporal gyrus (STG) and posterior STG, between posterior STG and inferior frontal cortex, and between STG and premotor cortex. Evidence suggests that these relationships reflect direct cortico-cortical connections rather than common driving input from subcortical structures such as the thalamus. In summary, our inter-subject analyses defined the spatial and temporal relationships between music-related brain activity in the alpha and high gamma bands. They provide experimental evidence supporting current theories about the putative mechanisms of alpha and gamma activity, i.e., reflections of thalamo-cortical interactions and local cortical neural activity, respectively, and the results are also in agreement with existing functional models of auditory processing. %B NeuroImage %V 97 %P 188-95 %8 08/2014 %G eng %U http://www.ncbi.nlm.nih.gov/pubmed/24768933 %R 10.1016/j.neuroimage.2014.04.045 %0 Book Section %B Towards Practical Brain-Computer Interfaces %D 2013 %T BCI Software Platforms. %A Brunner, Clemens %A Andreoni, G %A Bianchi, L %A Benjamin Blankertz %A Breitwieser, C. %A Kanoh, S. %A Kothe, C. A. %A Lecuyer, A. %A Makeig, S %A Mellinger, J. %A Perego, P. %A Renard, Y. %A Gerwin Schalk %A Susila, I.P. %A Venthur, B %A Mueller-Putz, G.R. %A Brendan Z. Allison %A Dunne, S. %A Leeb, R. %A Del R. Millán, J. %A A. Nijholt %X In this chapter, we provide an overview of publicly available software platforms for brain–computer interfaces. We have identified seven major BCI platforms and one platform specifically targeted towards feedback and stimulus presentation. We describe the intended target user group (which includes researchers, programmers, and end users), the most important features of each platform such as availability on different operating systems, licences, programming languages involved, supported devices, and so on. These seven platforms are: (1) BCI2000, (2) OpenViBE, (3) TOBI Common Implementation Platform (CIP), (4) BCILAB, (5) BCI++, (6) xBCI, and (7) BF++. The feedback framework is called Pyff. Our conclusion discusses possible synergies and future developments, such as combining different components of different platforms. With this overview, we hope to identify the strengths and weaknesses of each available platform, which should help anyone in the BCI research field in their decision which platform to use for their specific purposes. %B Towards Practical Brain-Computer Interfaces %I Biological and Medical Physics %@ 978-3-642-29745-8 %G eng %U http://link.springer.com/chapter/10.1007/978-3-642-29746-5_16 %R DOI: 10.1007/978-3-642-29746-5 %0 Generic %D 2013 %T Brain-Computer Interfaces Yesterday, Today, and Tomorrow: A Status Report of Bioengineering Research Partnership EB0085 %A Gerwin Schalk %X National Institute of Biomedical Imaging and Bioengineering, National Institutes of Health, Bethesda, MD %8 04/2013 %G eng %0 Generic %D 2013 %T Brain-Computer Interfacing Using P300 Evoked Potentials %A Gerwin Schalk %X Guest lecture in course Brain-Computer Interfaces, Electrical and Computer Engineering Department, NYU Poly %8 04/2013 %G eng %0 Generic %D 2013 %T Brain-Computer Interfacing Using P300 Evoked Potentials %A Gerwin Schalk %X Guest lecture in course Brain-Computer Interfaces, Departments of Neurosurgery/Bioengineering, University of Pennsylvania %8 04/2013 %G eng %0 Generic %D 2013 %T Communicating Directly With the Brain %A Gerwin Schalk %X Annual Gala of Fondazione Neurone, Rome, Italy %8 03/2013 %G eng %0 Conference Proceedings %B 35th Annual International IEEE EMBS Conference (EMBC) %D 2013 %T cortiQ – Clinical Software for Electrocorticographic Real-Time Functional Mapping of the Eloquent Cortex %A Prueckl, R. %A Kapeller, C %A Potes, Cristhian %A Korostenskaja, M %A Gerwin Schalk %A Lee, K.H. %A Guger, C %B 35th Annual International IEEE EMBS Conference (EMBC) %G eng %0 Journal Article %J Conf Proc IEEE Eng Med Biol Soc %D 2013 %T CortiQ - clinical software for electrocorticographic real-time functional mapping of the eloquent cortex. %A Prueckl, Robert %A Kapeller, Christoph %A Potes, Cristhian %A Korostenskaja, Milena %A Gerwin Schalk %A Lee, Ki H %A Guger, Christoph %X Planning for epilepsy surgery depends substantially on the localization of brain cortical areas responsible for sensory, motor, or cognitive functions, clinically also known as eloquent cortex. In this paper, we present the novel software package 'cortiQ' that allows clinicians to localize eloquent cortex, thus providing a safe margin for surgical resection with a low incidence of neurological deficits. This software can be easily used in addition to traditional mapping procedures such as the electrical cortical stimulation (ECS) mapping. The software analyses task-related changes in gamma activity recorded from implanted subdural electrocorticography electrodes using extensions to previously published methods. In this manuscript, we describe the system's architecture and workflow required to obtain a map of the eloquent cortex. We validate the system by comparing our mapping results with those acquired using ECS mapping in two subjects. Our results indicate that cortiQ reliably identifies eloquent cortex much faster (several minutes compared to an hour or more) than ECS mapping. Next-neighbour analyses show that there are no false positives and an average of 1.24% false negatives. %B Conf Proc IEEE Eng Med Biol Soc %V 2013 %P 6365-8 %8 07/2013 %G eng %U http://www.ncbi.nlm.nih.gov/pubmed/24111197 %R 10.1109/EMBC.2013.6611010 %0 Generic %D 2013 %T ECoG-Based Neuroscience and Neuroengineering %A Gerwin Schalk %X Center for Neuropharmacology and Neuroscience Seminar Series, Albany Medical College, Albany, NY %8 05/2013 %G eng %0 Generic %D 2013 %T Exciting Opportunities for Neuroengineering %A Gerwin Schalk %X Hershey Medical Center, Penn State University %8 01/2013 %G eng %0 Generic %D 2013 %T The Exciting World of Brain-Computer Interfaces %A Gerwin Schalk %X Society of Physics Students, State University of New York at Albany, Albany, NY %8 05/2013 %G eng %0 Generic %D 2013 %T Long-term Cortical Neuroprostheses: Prospects and Challenges %A Gerwin Schalk %X 1st Bernstein Sparks Workshop on Cortical Neurointerfaces, Delmenhorst, Germany %8 03/2013 %G eng %0 Journal Article %J NeuroImage %D 2013 %T A low-frequency oscillatory neural signal in humans encodes a developing decision variable. %A Kubanek, Jan %A Snyder, Lawrence H. %A Brunton, Bingni W. %A Brody, Carlos D. %A Gerwin Schalk %X We often make decisions based on sensory evidence that is accumulated over a period of time. How the evidence for such decisions is represented in the brain and how such a neural representation is used to guide a subsequent action are questions of considerable interest to decision sciences. The neural correlates of developing perceptual decisions have been thoroughly investigated in the oculomotor system of macaques who communicated their decisions using an eye movement. It has been found that the evidence informing a decision to make an eye movement is in part accumulated within the same oculomotor circuits that signal the upcoming eye movement. Recent evidence suggests that the somatomotor system may exhibit an analogous property for choices made using a hand movement. To investigate this possibility, we engaged humans in a decision task in which they integrated discrete quanta of sensory information over a period of time and signaled their decision using a hand movement or an eye movement. The discrete form of the sensory evidence allowed us to infer the decision variable on which subjects base their decision on each trial and to assess the neural processes related to each quantum of the incoming decision evidence. We found that a low-frequency electrophysiological signal recorded over centroparietal regions strongly encodes the decision variable inferred in this task, and that it does so specifically for hand movement choices. The signal ramps up with a rate that is proportional to the decision variable, remains graded by the decision variable throughout the delay period, reaches a common peak shortly before a hand movement, and falls off shortly after the hand movement. Furthermore, the signal encodes the polarity of each evidence quantum, with a short latency, and retains the response level over time. Thus, this neural signal shows properties of evidence accumulation. These findings suggest that the decision-related effects observed in the oculomotor system of the monkey during eye movement choices may share the same basic properties with the decision-related effects in the somatomotor system of humans during hand movement choices. %B NeuroImage %V 83 %P 795–808 %8 12/2013 %G eng %U http://www.ncbi.nlm.nih.gov/pubmed/23872495 %R 10.1016/j.neuroimage.2013.06.085 %0 Generic %D 2013 %T Opportunities in Computation Electrophysiological Imaging %A Gerwin Schalk %X Department of Psychology, NYU, New York, NY %8 04/2013 %G eng %0 Generic %D 2013 %T Opportunities in Computation Electrophysiological Imaging %A Gerwin Schalk %X McGovern Institute for Brain Research, MIT %8 02/2013 %G eng %0 Journal Article %J Epilepsy & Behavior %D 2013 %T Proceedings of the Fourth International Workshop on Advances in Electrocorticography. %A A L Ritaccio %A Peter Brunner %A Nathan E. Crone %A Gunduz, Aysegul %A Hirsch, Lawrence J. %A Kanwisher, Nancy %A Litt, Brian %A Kai J. Miller %A Morani, Daniel %A Parvizi, Josef %A Ramsey, Nick F %A Richner, Thomas J. %A Tandon, Niton %A Williams, Justin %A Gerwin Schalk %K Brain Mapping %K Brain–computer interface %K Electrocorticography %K Gamma-frequency electroencephalography %K High-frequency oscillations %K Neuroprosthetics %K Seizure detection %K Subdural grid %X The Fourth International Workshop on Advances in Electrocorticography (ECoG) convened in New Orleans, LA, on October 11–12, 2012. The proceedings of the workshop serves as an accurate record of the most contemporary clinical and experimental work on brain surface recording and represents the insights of a unique multidisciplinary ensemble of expert clinicians and scientists. Presentations covered a broad range of topics, including innovations in passive functional mapping, increased understanding of pathologic high-frequency oscillations, evolving sensor technologies, a human trial of ECoG-driven brain–machine interface, as well as fresh insights into brain electrical stimulation. %B Epilepsy & Behavior %V 29 %P 259–68 %8 11/2013 %G eng %U http://www.ncbi.nlm.nih.gov/pubmed/24034899 %N 2 %R 10.1016/j.yebeh.2013.08.012 %0 Journal Article %J Clinical Neurophysiology %D 2013 %T Toward Gaze-Independent Brain-Computer Interfaces. %A Peter Brunner %A Gerwin Schalk %B Clinical Neurophysiology %V 125 %P 831-3 %8 05/2013 %G eng %U http://www.ncbi.nlm.nih.gov/pubmed/23465431 %N 5 %R 10.1016/j.clinph.2013.01.017 %0 Journal Article %J PLoS ONE %D 2013 %T The Tracking of Speech Envelope in the Human Cortex. %A Kubanek, Jan %A Peter Brunner %A Gunduz, Aysegul %A Poeppel, David %A Gerwin Schalk %X Humans are highly adept at processing speech. Recently, it has been shown that slow temporal information in speech (i.e., the envelope of speech) is critical for speech comprehension. Furthermore, it has been found that evoked electric potentials in human cortex are correlated with the speech envelope. However, it has been unclear whether this essential linguistic feature is encoded differentially in specific regions, or whether it is represented throughout the auditory system. To answer this question, we recorded neural data with high temporal resolution directly from the cortex while human subjects listened to a spoken story. We found that the gamma activity in human auditory cortex robustly tracks the speech envelope. The effect is so marked that it is observed during a single presentation of the spoken story to each subject. The effect is stronger in regions situated relatively early in the auditory pathway (belt areas) compared to other regions involved in speech processing, including the superior temporal gyrus (STG) and the posterior inferior frontal gyrus (Broca's region). To further distinguish whether speech envelope is encoded in the auditory system as a phonological (speech-related), or instead as a more general acoustic feature, we also probed the auditory system with a melodic stimulus. We found that belt areas track melody envelope weakly, and as the only region considered. Together, our data provide the first direct electrophysiological evidence that the envelope of speech is robustly tracked in non-primary auditory cortex (belt areas in particular), and suggest that the considered higher-order regions (STG and Broca's region) partake in a more abstract linguistic analysis. %B PLoS ONE %V 8 %P e53398 - %8 01/2013 %G eng %U http://dx.doi.org/10.1371%2Fjournal.pone.0053398 %N 1 %R 10.1371/journal.pone.0053398 %0 Generic %D 2012 %T BCI2000: A General-Purpose BCI System and its Application to ECoG Signals %A Gerwin Schalk %X g.tec Brain-Computer Interface Workshop, New Orleans, LA %8 10/2012 %G eng %0 Generic %D 2012 %T BCI2000: A General-Purpose BCI System and Its Application to ECoG Signals %A Gerwin Schalk %X g.tec Brain-Computer Interface Workshop, New Orleans, LA %8 10/13/2012 %G eng %0 Book Section %B Brain-Computer Interfaces: Principles and Practice %D 2012 %T BCIs That Use Electrocorticographic Activity. %A Jonathan Wolpaw %A E. Winter-Wolpaw %A Gerwin Schalk %K brain signals %K brain-computer interfaces %K ECoG %K intracortically recorded signals %X This chapter discusses the potential of electrocorticography (ECoG) as a clinically useful brain-computer interface signal modality. ECoG has greater amplitude, higher topographical resolution, and a much broader frequency range than scalp-recorded electroencephalography and is less susceptible to artifacts. With current and foreseeable recording methodologies, ECoG is likely to have greater long-term stability than intracortically recorded signals. Furthermore, it can more readily be recorded from larger cortical areas, and it requires much lower digitization rates, thus greatly reducing the power requirements of wholly implanted systems. %B Brain-Computer Interfaces: Principles and Practice %I Oxford University Press %G eng %U http://www.oxfordscholarship.com/view/10.1093/acprof:oso/9780195388855.001.0001/acprof-9780195388855-chapter-015 %R 10.1093/acprof:oso/9780195388855.003.0015 %0 Generic %D 2012 %T Brain-Computer Interfacing Using P300 Evoked Potentials %A Gerwin Schalk %X Guest lecture in course Brain-Computer Interfaces, Departments of Neurosurgery/Bioengineering, University of Pennsylvania, Philadelphia, Pennsylvania %8 03/20/2012 %G eng %0 Generic %D 2012 %T Brain-Computer Interfacing Using P300 Evoked Potentials %A Gerwin Schalk %X Guest lecture in course Brain-Computer Interfaces, Electrical and Computer Engineering Department, NYU poly, New York, NY %8 03/21/2012 %G eng %0 Generic %D 2012 %T Communicating Directly With the Brain %A Gerwin Schalk %X Introductory lecture at the initial public presentation of the €20m Italian project "cyber brain." Chamber of Commerce, Avellino, Italy. %8 06/15/2012 %G eng %0 Journal Article %J Frontiers in Neuroscience %D 2012 %T Communication and control by listening: towards optimal design of a two-class auditory streaming brain-computer interface. %A Jeremy Jeremy Hill %A Moinuddin, Aisha %A Häuser, Ann-Katrin %A Kienzle, Stephan %A Gerwin Schalk %K auditory attention %K auditory event-related potentials %K brain-computer interface %K dichotic listening %K N1 potential %K P3 potential %X Most brain-computer interface (BCI) systems require users to modulate brain signals in response to visual stimuli. Thus, they may not be useful to people with limited vision, such as those with severe paralysis. One important approach for overcoming this issue is auditory streaming, an approach whereby a BCI system is driven by shifts of attention between two simultaneously presented auditory stimulus streams. Motivated by the long-term goal of translating such a system into a reliable, simple yes-no interface for clinical usage, we aim to answer two main questions. First, we asked which of two previously published variants provides superior performance: a fixed-phase (FP) design in which the streams have equal period and opposite phase, or a drifting-phase (DP) design where the periods are unequal. We found FP to be superior to DP (p = 0.002): average performance levels were 80 and 72% correct, respectively. We were also able to show, in a pilot with one subject, that auditory streaming can support continuous control and neurofeedback applications: by shifting attention between ongoing left and right auditory streams, the subject was able to control the position of a paddle in a computer game. Second, we examined whether the system is dependent on eye movements, since it is known that eye movements and auditory attention may influence each other, and any dependence on the ability to move one’s eyes would be a barrier to translation to paralyzed users. We discovered that, despite instructions, some subjects did make eye movements that were indicative of the direction of attention. However, there was no correlation, across subjects, between the reliability of the eye movement signal and the reliability of the BCI system, indicating that our system was configured to work independently of eye movement. Together, these findings are an encouraging step forward toward BCIs that provide practical communication and control options for the most severely paralyzed users. %B Frontiers in Neuroscience %V 6 %8 12/2012 %G eng %U http://www.ncbi.nlm.nih.gov/pubmed/23267312 %R 10.3389/fnins.2012.00181 %0 Journal Article %J Neuroimage %D 2012 %T Decoding covert spatial attention using electrocorticographic (ECoG) signals in humans. %A Gunduz, Aysegul %A Peter Brunner %A Amy Daitch %A Leuthardt, E C %A A L Ritaccio %A Pesaran, Bijan %A Gerwin Schalk %K covert attention %K electrocorticography (ECoG) %K visual spatial attention %X

This study shows that electrocorticographic (ECoG) signals recorded from the surface of the brain provide detailed information about shifting of visual attention and its directional orientation in humans. ECoG allows for the identification of the cortical areas and time periods that hold the most information about covert attentional shifts. Our results suggest a transient distributed fronto-parietal mechanism for orienting of attention that is represented by different physiological processes. This neural mechanism encodes not only whether or not a subject shifts their attention to a location, but also the locus of attention. This work contributes to our understanding of the electrophysiological representation of attention in humans. It may also eventually lead to brain-computer interfaces (BCIs) that optimize user interaction with their surroundings or that allow people to communicate choices simply by shifting attention to them.

%B Neuroimage %V 60 %P 2285-93 %8 05/2012 %G eng %U http://www.ncbi.nlm.nih.gov/pubmed/22366333 %N 4 %R 10.1016/j.neuroimage.2012.02.017 %0 Journal Article %J Frontiers in Neuroengineering %D 2012 %T Decoding Onset and Direction of Movements using Electrocorticographic (ECoG) Signals in Humans. %A Wang, Z. %A Gunduz, Aysegul %A Peter Brunner %A A L Ritaccio %A Ji, Q %A Gerwin Schalk %K brain computer interface %K ECoG %K movement direction prediction %K movement onset prediction %K neurorehabilitation %K performance augmentation %X Communication of intent usually requires motor function. This requirement can be limiting when a person is engaged in a task, or prohibitive for some people suffering from neuromuscular disorders. Determining a person's intent, e.g., where and when to move, from brain signals rather than from muscles would have important applications in clinical or other domains. For example, detection of the onset and direction of intended movements may provide the basis for restoration of simple grasping function in people with chronic stroke, or could be used to optimize a user's interaction with the surrounding environment. Detecting the onset and direction of actual movements are a first step in this direction. In this study, we demonstrate that we can detect the onset of intended movements and their direction using electrocorticographic (ECoG) signals recorded from the surface of the cortex in humans. We also demonstrate in a simulation that the information encoded in ECoG about these movements may improve performance in a targeting task. In summary, the results in this paper suggest that detection of intended movement is possible, and may serve useful functions. %B Frontiers in Neuroengineering %V 5 %8 08/2012 %G eng %U http://www.ncbi.nlm.nih.gov/pubmed/22891058 %N 15 %R 10.3389/fneng.2012.00015 %0 Journal Article %J Neuroimage %D 2012 %T Dynamics of electrocorticographic (ECoG) activity in human temporal and frontal cortical areas during music listening. %A Potes, Cristhian %A Gunduz, Aysegul %A Peter Brunner %A Gerwin Schalk %K auditory processing %K electrocorticography (ECoG) %K high gamma activity %K sound intensity %X

Previous studies demonstrated that brain signals encode information about specific features of simple auditory stimuli or of general aspects of natural auditory stimuli. How brain signals represent the time course of specific features in natural auditory stimuli is not well understood. In this study, we show in eight human subjects that signals recorded from the surface of the brain (electrocorticography (ECoG)) encode information about the sound intensity of music. ECoG activity in the high gamma band recorded from the posterior part of the superior temporal gyrus as well as from an isolated area in the precentral gyrus was observed to be highly correlated with the sound intensity of music. These results not only confirm the role of auditory cortices in auditory processing but also point to an important role of premotor and motor cortices. They also encourage the use of ECoG activity to study more complex acoustic features of simple or natural auditory stimuli.

%B Neuroimage %V 61 %P 841-8 %8 07/2012 %G eng %U http://www.ncbi.nlm.nih.gov/pubmed/22537600 %N 4 %R 10.1016/j.neuroimage.2012.04.022 %0 Generic %D 2012 %T ECoG-Based Neuroscience and Neuroengineering %A Gerwin Schalk %X BBCI Workshop 2012, Advances in Neurotechnology, Berlin, Germany %8 09/19/2012 %G eng %0 Journal Article %J Exp Brain Res %D 2012 %T Electrocorticographic (ECoG) Correlates of Human Arm Movements. %A Nicholas R Anderson %A Blakely, Timothy %A Gerwin Schalk %A Leuthardt, E C %A Moran, Daniel W %K arm tuning %K brain-computer interfaces %K cosine tuning %K Electrocorticography %K Motor Cortex %K subdural electroencephalography %X Invasive and non-invasive brain-computer interface (BCI) studies have long focused on the motor cortex for kinematic control of artificial devices. Most of these studies have used single-neuron recordings or electroencephalography (EEG). Electrocorticography (ECoG) is a relatively new recording modality in BCI research that has primarily been built on successes in EEG recordings. We built on prior experiments related to single-neuron recording and quantitatively compare the extent to which different brain regions reflect kinematic tuning parameters of hand speed, direction, and velocity in both a reaching and tracing task in humans. Hand and arm movement experiments using ECoG have shown positive results before, but the tasks were not designed to tease out which kinematics are encoded. In non-human primates, the relationships among these kinematics have been more carefully documented, and we sought to begin elucidating that relationship in humans using ECoG. The largest modulation in ECoG activity for direction, speed, and velocity representation was found in the primary motor cortex. We also found consistent cosine tuning across both tasks, to hand direction and velocity in the high gamma band (70-160 Hz). Thus, the results of this study clarify the neural substrates involved in encoding aspects of motor preparation and execution and confirm the important role of the motor cortex in BCI applications. %B Exp Brain Res %V 223 %P 1-10 %8 11/2012 %G eng %U http://www.ncbi.nlm.nih.gov/pubmed/23001369 %N 1 %R 10.1007/s00221-012-3226-1 %0 Generic %D 2012 %T Exciting Adventures in Neuroscience and Neuroengineering %A Gerwin Schalk %X Colloquium at Electrical Engineering and Computer Science Department, Case Western Reserve University, Cleveland OH %8 04/19/2012 %G eng %0 Generic %D 2012 %T Exciting Directions in Neuroscience and Neuroengineering %A Gerwin Schalk %X Kent State University, Kent, OH %8 04/18/2012 %G eng %0 Generic %D 2012 %T Exciting Opportunities in Neuroscience and Neuroengineering %A Gerwin Schalk %X University of Washington, Seattle, WA %8 01/27/2012 %G eng %0 Generic %D 2012 %T The Exciting World of Brain-Computer Interfaces %A Gerwin Schalk %X Wadsworth Center Research Experience for Undergraduates (REU) program %8 08/01/2012 %G eng %0 Generic %D 2012 %T The Exciting World of Brain-Computer Interfaces %A Gerwin Schalk %X Lecture in course Science in the News, Sage College, Albany, NY %8 10/19/2012 %G eng %0 Generic %D 2012 %T The Exciting World of Brain-Computer Interfacing %A Gerwin Schalk %X Keynote Address, Workshop in "Solving the Mystery of how the Brian Works." Walt Disney Pavilion, Florida Hospital for Children, FL %8 05/10/2012 %G eng %0 Generic %D 2012 %T Future Aspects of Functional Mapping %A Gerwin Schalk %8 10/15/2012 %G eng %0 Generic %D 2012 %T Future Aspects of Functional Mapping %A Gerwin Schalk %X 1st International Workshop on Functional Mapping with ECoG, New Orleans, LA %8 10/2012 %G eng %0 Book Section %B Brain-Computer Interfaces: Principles and Practice %D 2012 %T Hardware and Software Technologies. %A Gerwin Schalk %A Guger, C %A Adam J Wilson %E Jonathan Wolpaw %E E. Winter-Wolpaw %B Brain-Computer Interfaces: Principles and Practice %I Oxford University Press %G eng %0 Conference Proceedings %B Neural Information Processing Systems (NIPS) Conference %D 2012 %T Learning with Target Prior %A Wang, Z. %A Lyu, S. %A Gerwin Schalk %A Ji, Q %B Neural Information Processing Systems (NIPS) Conference %8 11/2012 %G eng %0 Generic %D 2012 %T Past and Present Aspects of Functional Mapping %A Gerwin Schalk %X 1st International Workshop on Functional Mapping with ECoG, New Orleans, LA %8 10/15/2012 %G eng %0 Generic %D 2012 %T Perspectives on ECoG Research and Application %A Gerwin Schalk %X 4th International Workshop on Advances in Electrocorticography, New Orleans, LA %8 10/12/2012 %G eng %0 Generic %D 2012 %T Principles of Real-Time Passive Functional Mapping %A Gerwin Schalk %X Department of Neurology, Yale University, New Haven, CT %8 11/27/2012 %G eng %0 Journal Article %J Epilepsy Behav %D 2012 %T Proceedings of the Third International Workshop on Advances in Electrocorticography. %A A L Ritaccio %A Beauchamp, Michael %A Bosman, Conrado %A Peter Brunner %A Chang, Edward %A Nathan E. Crone %A Gunduz, Aysegul %A Disha Gupta %A Robert T. Knight %A Leuthardt, Eric %A Litt, Brian %A Moran, Daniel %A Ojemann, Jeffrey %A Parvizi, Josef %A Ramsey, Nick %A Rieger, Jochem %A Viventi, Jonathan %A Voytek, Bradley %A Williams, Justin %A Gerwin Schalk %K Brain Mapping %K brain-computer interface %K Electrocorticography %K Gamma-frequency electroencephalography %K high-frequency oscillation %K Neuroprosthetics %K Seizure detection %K Subdural grid %X The Third International Workshop on Advances in Electrocorticography (ECoG) was convened in Washington, DC, on November 10-11, 2011. As in prior meetings, a true multidisciplinary fusion of clinicians, scientists, and engineers from many disciplines gathered to summarize contemporary experiences in brain surface recordings. The proceedings of this meeting serve as evidence of a very robust and transformative field but will yet again require revision to incorporate the advances that the following year will surely bring. %B Epilepsy Behav %V 25 %P 605-13 %8 12/2012 %G eng %U http://www.ncbi.nlm.nih.gov/pubmed/23160096 %N 4 %R 10.1016/j.yebeh.2012.09.016 %0 Generic %D 2012 %T Real-Time Functional Mapping Using ECoG %A Gerwin Schalk %X g.tec ECoG/Spike Workshop, New Orleans, LA %8 10/15/2012 %G eng %0 Journal Article %J J Vis Exp %D 2012 %T Recording Human Electrocorticographic (ECoG) Signals for Neuroscientific Research and Real-time Functional Cortical Mapping. %A Jeremy Jeremy Hill %A Disha Gupta %A Peter Brunner %A Gunduz, Aysegul %A Adamo, Matthew A %A A L Ritaccio %A Gerwin Schalk %K BCI2000 %K brain-computer interfacing %K Electrocorticography %K epilepsy monitoring %K functional brain mapping %K issue 64 %K Magnetic Resonance Imaging %K MRI %K neuroscience %K SIGFRIED %X

Neuroimaging studies of human cognitive, sensory, and motor processes are usually based on noninvasive techniques such as electroencephalography (EEG), magnetoencephalography or functional magnetic-resonance imaging. These techniques have either inherently low temporal or low spatial resolution, and suffer from low signal-to-noise ratio and/or poor high-frequency sensitivity. Thus, they are suboptimal for exploring the short-lived spatio-temporal dynamics of many of the underlying brain processes. In contrast, the invasive technique of electrocorticography (ECoG) provides brain signals that have an exceptionally high signal-to-noise ratio, less susceptibility to artifacts than EEG, and a high spatial and temporal resolution (i.e., <1 cm/<1 millisecond, respectively). ECoG involves measurement of electrical brain signals using electrodes that are implanted subdurally on the surface of the brain. Recent studies have shown that ECoG amplitudes in certain frequency bands carry substantial information about task-related activity, such as motor execution and planning, auditory processing and visual-spatial attention. Most of this information is captured in the high gamma range (around 70-110 Hz). Thus, gamma activity has been proposed as a robust and general indicator of local cortical function. ECoG can also reveal functional connectivity and resolve finer task-related spatial-temporal dynamics, thereby advancing our understanding of large-scale cortical processes. It has especially proven useful for advancing brain-computer interfacing (BCI) technology for decoding a user's intentions to enhance or improve communication and control. Nevertheless, human ECoG data are often hard to obtain because of the risks and limitations of the invasive procedures involved, and the need to record within the constraints of clinical settings. Still, clinical monitoring to localize epileptic foci offers a unique and valuable opportunity to collect human ECoG data. We describe our methods for collecting recording ECoG, and demonstrate how to use these signals for important real-time applications such as clinical mapping and brain-computer interfacing. Our example uses the BCI2000 software platform and the SIGFRIED method, an application for real-time mapping of brain functions. This procedure yields information that clinicians can subsequently use to guide the complex and laborious process of functional mapping by electrical stimulation. PREREQUISITES AND PLANNING: Patients with drug-resistant partial epilepsy may be candidates for resective surgery of an epileptic focus to minimize the frequency of seizures. Prior to resection, the patients undergo monitoring using subdural electrodes for two purposes: first, to localize the epileptic focus, and second, to identify nearby critical brain areas (i.e., eloquent cortex) where resection could result in long-term functional deficits. To implant electrodes, a craniotomy is performed to open the skull. Then, electrode grids and/or strips are placed on the cortex, usually beneath the dura. A typical grid has a set of 8 x 8 platinum-iridium electrodes of 4 mm diameter (2.3 mm exposed surface) embedded in silicon with an inter-electrode distance of 1cm. A strip typically contains 4 or 6 such electrodes in a single line. The locations for these grids/strips are planned by a team of neurologists and neurosurgeons, and are based on previous EEG monitoring, on a structural MRI of the patient's brain, and on relevant factors of the patient's history. Continuous recording over a period of 5-12 days serves to localize epileptic foci, and electrical stimulation via the implanted electrodes allows clinicians to map eloquent cortex. At the end of the monitoring period, explantation of the electrodes and therapeutic resection are performed together in one procedure. In addition to its primary clinical purpose, invasive monitoring also provides a unique opportunity to acquire human ECoG data for neuroscientific research. The decision to include a prospective patient in the research is based on the planned location of their electrodes, on the patient's performance scores on neuropsychological assessments, and on their informed consent, which is predicated on their understanding that participation in research is optional and is not related to their treatment. As with all research involving human subjects, the research protocol must be approved by the hospital's institutional review board. The decision to perform individual experimental tasks is made day-by-day, and is contingent on the patient's endurance and willingness to participate. Some or all of the experiments may be prevented by problems with the clinical state of the patient, such as post-operative facial swelling, temporary aphasia, frequent seizures, post-ictal fatigue and confusion, and more general pain or discomfort. At the Epilepsy Monitoring Unit at Albany Medical Center in Albany, New York, clinical monitoring is implemented around the clock using a 192-channel Nihon-Kohden Neurofax monitoring system. Research recordings are made in collaboration with the Wadsworth Center of the New York State Department of Health in Albany. Signals from the ECoG electrodes are fed simultaneously to the research and the clinical systems via splitter connectors. To ensure that the clinical and research systems do not interfere with each other, the two systems typically use separate grounds. In fact, an epidural strip of electrodes is sometimes implanted to provide a ground for the clinical system. Whether research or clinical recording system, the grounding electrode is chosen to be distant from the predicted epileptic focus and from cortical areas of interest for the research. Our research system consists of eight synchronized 16-channel g.USBamp amplifier/digitizer units (g.tec, Graz, Austria). These were chosen because they are safety-rated and FDA-approved for invasive recordings, they have a very low noise-floor in the high-frequency range in which the signals of interest are found, and they come with an SDK that allows them to be integrated with custom-written research software. In order to capture the high-gamma signal accurately, we acquire signals at 1200Hz sampling rate-considerably higher than that of the typical EEG experiment or that of many clinical monitoring systems. A built-in low-pass filter automatically prevents aliasing of signals higher than the digitizer can capture. The patient's eye gaze is tracked using a monitor with a built-in Tobii T-60 eye-tracking system (Tobii Tech., Stockholm, Sweden). Additional accessories such as joystick, bluetooth Wiimote (Nintendo Co.), data-glove (5(th) Dimension Technologies), keyboard, microphone, headphones, or video camera are connected depending on the requirements of the particular experiment. Data collection, stimulus presentation, synchronization with the different input/output accessories, and real-time analysis and visualization are accomplished using our BCI2000 software. BCI2000 is a freely available general-purpose software system for real-time biosignal data acquisition, processing and feedback. It includes an array of pre-built modules that can be flexibly configured for many different purposes, and that can be extended by researchers' own code in C++, MATLAB or Python. BCI2000 consists of four modules that communicate with each other via a network-capable protocol: a Source module that handles the acquisition of brain signals from one of 19 different hardware systems from different manufacturers; a Signal Processing module that extracts relevant ECoG features and translates them into output signals; an Application module that delivers stimuli and feedback to the subject; and the Operator module that provides a graphical interface to the investigator. A number of different experiments may be conducted with any given patient. The priority of experiments will be determined by the location of the particular patient's electrodes. However, we usually begin our experimentation using the SIGFRIED (SIGnal modeling For Realtime Identification and Event Detection) mapping method, which detects and displays significant task-related activity in real time. The resulting functional map allows us to further tailor subsequent experimental protocols and may also prove as a useful starting point for traditional mapping by electrocortical stimulation (ECS). Although ECS mapping remains the gold standard for predicting the clinical outcome of resection, the process of ECS mapping is time consuming and also has other problems, such as after-discharges or seizures. Thus, a passive functional mapping technique may prove valuable in providing an initial estimate of the locus of eloquent cortex, which may then be confirmed and refined by ECS. The results from our passive SIGFRIED mapping technique have been shown to exhibit substantial concurrence with the results derived using ECS mapping. The protocol described in this paper establishes a general methodology for gathering human ECoG data, before proceeding to illustrate how experiments can be initiated using the BCI2000 software platform. Finally, as a specific example, we describe how to perform passive functional mapping using the BCI2000-based SIGFRIED system.

%B J Vis Exp %8 05/2012 %G eng %U http://www.ncbi.nlm.nih.gov/pubmed/22782131 %N 64 %R 10.3791/3993 %0 Journal Article %J Frontiers in Neuroprosthetics %D 2012 %T Review of the BCI Competition IV. %A Tangermann, M. %A Muller, K.R. %A Aertsen, A. %A Niels Birbaumer %A Christoph Braun %A Brunner, Clemens %A Leeb, R. %A Mehring, C. %A Miller, K.J. %A Mueller-Putz, G. %A Nolte, G. %A Pfurtscheller, G. %A Preissl, H. %A Gerwin Schalk %A Schlögl, A. %A Vidaurre, C. %A Waldert, S. %A Benjamin Blankertz %K BCI %K brain-computer interface %K competition %X The BCI competition IV stands in the tradition of prior BCI competitions that aim to provide high quality neuroscientific data for open access to the scientific community. As experienced already in prior competitions not only scientists from the narrow field of BCI compete, but scholars with a broad variety of backgrounds and nationalities. They include high specialists as well as students. The goals of all BCI competitions have always been to challenge with respect to novel paradigms and complex data. We report on the following challenges: (1) asynchronous data, (2) synthetic, (3) multi-class continuous data, (4) session-to-session transfer, (5) directionally modulated MEG, (6) finger movements recorded by ECoG. As after past competitions, our hope is that winning entries may enhance the analysis methods of future BCIs. %B Frontiers in Neuroprosthetics %V 6 %P 1-31 %8 07/2012 %G eng %U http://www.ncbi.nlm.nih.gov/pubmed/22811657 %N 55 %R 10.3389/fnins.2012.00055 %0 Journal Article %J IEEE Pulse %D 2012 %T Silent Communication: toward using brain signals. %A Pei, Xiao-Mei %A Jeremy Jeremy Hill %A Gerwin Schalk %K Animals %K Brain %K Brain Waves %K Humans %K Movement %K User-Computer Interface %X

From the 1980s movie Firefox to the more recent Avatar, popular science fiction has speculated about the possibility of a persons thoughts being read directly from his or her brain. Such braincomputer interfaces (BCIs) might allow people who are paralyzed to communicate with and control their environment, and there might also be applications in military situations wherever silent user-to-user communication is desirable. Previous studies have shown that BCI systems can use brain signals related to movements and movement imagery or attention-based character selection. Although these systems have successfully demonstrated the possibility to control devices using brain function, directly inferring which word a person intends to communicate has been elusive. A BCI using imagined speech might provide such a practical, intuitive device. Toward this goal, our studies to date addressed two scientific questions: (1) Can brain signals accurately characterize different aspects of speech? (2) Is it possible to predict spoken or imagined words or their components using brain signals?

%B IEEE Pulse %V 3 %P 43-6 %8 01/2012 %G eng %U http://www.ncbi.nlm.nih.gov/pubmed/22344951 %N 1 %R 10.1109/MPUL.2011.2175637 %0 Journal Article %J Front Hum Neurosci %D 2012 %T Temporal evolution of gamma activity in human cortex during an overt and covert word repetition task. %A Leuthardt, E C %A Pei, Xiao-Mei %A Breshears, Jonathan %A Charles M Gaona %A Sharma, Mohit %A Zachary V. Freudenberg %A Barbour, Dennis L %A Gerwin Schalk %K cortex %K Electrocorticography %K gamma rhythms %K human %K Speech %X

Several scientists have proposed different models for cortical processing of speech. Classically, the regions participating in language were thought to be modular with a linear sequence of activations. More recently, modern theoretical models have posited a more hierarchical and distributed interaction of anatomic areas for the various stages of speech processing. Traditional imaging techniques can only define the location or time of cortical activation, which impedes the further evaluation and refinement of these models. In this study, we take advantage of recordings from the surface of the brain [electrocorticography (ECoG)], which can accurately detect the location and timing of cortical activations, to study the time course of ECoG high gamma (HG) modulations during an overt and covert word repetition task for different cortical areas. For overt word production, our results show substantial perisylvian cortical activations early in the perceptual phase of the task that were maintained through word articulation. However, this broad activation is attenuated during the expressive phase of covert word repetition. Across the different repetition tasks, the utilization of the different cortical sites within the perisylvian region varied in the degree of activation dependent on which stimulus was provided (auditoryor visual cue) and whether the word was to be spoken or imagined. Taken together, the data support current models of speech that have been previously described with functional imaging. Moreover, this study demonstrates that the broad perisylvian speech network activates early and maintains suprathreshold activation throughout the word repetition task that appears to be modulated by the demands of different conditions.

%B Front Hum Neurosci %V 6 %P 99 %8 05/2012 %G eng %U http://www.ncbi.nlm.nih.gov/pubmed/22563311 %R 10.3389/fnhum.2012.00099 %0 Generic %D 2012 %T Using Machines to Read the Mind %A Gerwin Schalk %X Department of Neurology and Neurological Services, Stanford University School of Medicine, Palo Alto, CA %8 02/09/2012 %G eng %0 Generic %D 2011 %T Advanced BCI2000 Concepts %A Gerwin Schalk %X 8th BCI2000 Workshop, University Medical Center, Utrecht, The Netherlands %8 05/18/2011 %G eng %0 Conference Proceedings %B NIPS %D 2011 %T Anatomically Constrained Decoding of Finger Flexion from Electrocorticographic Signals %A Zuoguan Wang %A Gerwin Schalk %A Ji, Q %B NIPS %G eng %0 Generic %D 2011 %T BCI2000 %A Gerwin Schalk %X FBNCI Cluster Workshop for Roadmap Development, Graz University of Technology %8 09/22/2011 %G eng %0 Generic %D 2011 %T BCI2000: A General-Purpose BCI System and its Application to ECoG Signals %A Gerwin Schalk %X g.tec Brain-Computer Interface Workshop, Washington, DC %8 11/12/2011 %G eng %0 Generic %D 2011 %T BCI2000: A General-Purpose BCI System and its Application to ECoG Signals %A Gerwin Schalk %X g.tec Brain-Computer Interface Workshop, IEEE EMBC Conference, Boston, MA %8 08/30/2011 %G eng %0 Generic %D 2011 %T Brain-Computer Interfaces: Integrating Bioengineering and Neuroscience Research %A Gerwin Schalk %X Keynote, 37th Annual Northeast Bioengineering Conference, Rensselaer Polytechnic Institute, Troy, New York %8 04/03/2011 %G eng %0 Generic %D 2011 %T Brain-Computer Interfaces: The Hope, The Hype, The Power, and The Pain %A Gerwin Schalk %X Brain-Computer Interfacing in 2011, Rudolf Magnus Institute for Neuroscience, Utrecht, The Netherlands %8 05/21/2011 %G eng %0 Journal Article %J IEEE Rev Biomed Eng %D 2011 %T Brain-computer interfaces using electrocorticographic signals. %A Gerwin Schalk %A Leuthardt, E C %K Brain-computer interface (BCI) %K brain-machine interface (BMI) %K electrocorticography (ECoG) %X

Many studies over the past two decades have shown that people and animals can use brain signals to convey their intent to a computer using brain-computer interfaces (BCIs). BCI systems measure specific features of brain activity and translate them into control signals that drive an output. The sensor modalities that have most commonly been used in BCI studies have been electroencephalographic (EEG) recordings from the scalp and single-neuron recordings from within the cortex. Over the past decade, an increasing number of studies has explored the use of electrocorticographic (ECoG) activity recorded directly from the surface of the brain. ECoG has attracted substantial and increasing interest, because it has been shown to reflect specific details of actual and imagined actions, and because its technical characteristics should readily support robust and chronic implementations of BCI systems in humans. This review provides general perspectives on the ECoG platform; describes the different electrophysiological features that can be detected in ECoG; elaborates on the signal acquisition issues, protocols, and online performance of ECoG-based BCI studies to date; presents important limitations of current ECoG studies; discusses opportunities for further research; and finally presents a vision for eventual clinical implementation. In summary, the studies presented to date strongly encourage further research using the ECoG platform for basic neuroscientific research, as well as for translational neuroprosthetic applications.

%B IEEE Rev Biomed Eng %V 4 %P 140-54 %8 10/2011 %G eng %U http://www.ncbi.nlm.nih.gov/pubmed/22273796 %R 10.1109/RBME.2011.2172408 %0 Generic %D 2011 %T Communicating Directly from the Brain %A Gerwin Schalk %X EmTech Conference, MIT Campus %8 10/19/2011 %G eng %0 Journal Article %J J Neural Eng %D 2011 %T Current Trends in Hardware and Software for Brain-Computer Interfaces (BCIs). %A Peter Brunner %A Bianchi, L %A Guger, C %A Cincotti, F %A Gerwin Schalk %K Biofeedback, Psychology %K Brain %K Brain Mapping %K Electroencephalography %K Equipment Design %K Equipment Failure Analysis %K Humans %K Man-Machine Systems %K Software %K User-Computer Interface %X

brain-computer interface (BCI) provides a non-muscular communication channel to people with and without disabilities. BCI devices consist of hardware and software. BCI hardware records signals from the brain, either invasively or non-invasively, using a series of device components. BCI software then translates these signals into device output commands and provides feedback. One may categorize different types of BCI applications into the following four categories: basic research, clinical/translational research, consumer products, and emerging applications. These four categories use BCI hardware and software, but have different sets of requirements. For example, while basic research needs to explore a wide range of system configurations, and thus requires a wide range of hardware and software capabilities, applications in the other three categories may be designed for relatively narrow purposes and thus may only need a very limited subset of capabilities. This paper summarizes technical aspects for each of these four categories of BCI applications. The results indicate that BCI technology is in transition from isolated demonstrations to systematic research and commercial development. This process requires several multidisciplinary efforts, including the development of better integrated and more robust BCI hardware and software, the definition of standardized interfaces, and the developmentof certification, dissemination and reimbursement procedures.

%B J Neural Eng %V 8 %P 025001 %8 04/2011 %G eng %U http://www.ncbi.nlm.nih.gov/pubmed/21436536 %N 2 %R 10.1088/1741-2560/8/2/025001 %0 Journal Article %J J Neural Eng %D 2011 %T Decoding vowels and consonants in spoken and imagined words using electrocorticographic signals in humans. %A Pei, Xiao-Mei %A Barbour, Dennis L %A Leuthardt, E C %A Gerwin Schalk %K Adolescent %K Adult %K Brain %K Brain Mapping %K Cerebral Cortex %K Communication Aids for Disabled %K Data Interpretation, Statistical %K Discrimination (Psychology) %K Electrodes, Implanted %K Electroencephalography %K Epilepsy %K Female %K Functional Laterality %K Humans %K Male %K Middle Aged %K Movement %K Speech Perception %K User-Computer Interface %X

Several stories in the popular media have speculated that it may be possible to infer from the brain which word a person is speaking or even thinking. While recent studies have demonstrated that brain signals can give detailed information about actual and imagined actions, such as different types of limb movements or spoken words, concrete experimental evidence for the possibility to 'read the mind', i.e. to interpret internally-generated speech, has been scarce. In this study, we found that it is possible to use signals recorded from the surface of the brain (electrocorticography) to discriminate the vowels and consonants embedded in spoken and in imagined words, and we defined the cortical areas that held the most information about discrimination of vowels and consonants. The results shed light on the distinct mechanisms associated with production of vowels and consonants, and could provide the basis for brain-based communication using imagined speech.

%B J Neural Eng %V 8 %P 046028 %8 08/2011 %G eng %U http://www.ncbi.nlm.nih.gov/pubmed/21750369 %N 4 %R 10.1088/1741-2560/8/4/046028 %0 Conference Paper %D 2011 %T Defense-related insights and solutions from neuroscience and neuroengineering. %A Gunduz, Aysegul %A Gerwin Schalk %X Communication of intent usually requires motor function, which can be limiting during military missions. Determining a soldier's intent from brain signals rather than using muscles would have numerous applications for tactical combat. Brain-computer interfaces (BCIs) translate brain signals into machine readable form and could optimize a soldier's interaction with the surrounding environment. However, current BCI devices have largely remained laboratory curiosities, because current techniques either require extended training or do not have the requisite signal fidelity, because they are highly invasive and thus not safe or practical for use in humans, or because they rely on equipment (such as magnetic resonance imaging scanners) that do not allow for real-time applications and/or field deployment. The objective of our research program is to create a prototype of a system for communication and monitoring of orientation that uses brain signals to provide, in real time, an accurate assessment of the users intentional focus and imagined speech. We expect that our efforts will provide a prototype of the first intuitive brain-based communication and orientation system for human use. %8 06/2011 %G eng %U http://spie.org/Publications/Proceedings/Paper/10.1117/12.888189 %R 10.1117/12.888189 %0 Conference Proceedings %D 2011 %T Defense-related insights and solutions from neuroscience and neuroengineering %A Gunduz, Aysegul %A Gerwin Schalk %X Communication of intent usually requires motor function, which can be limiting during military missions. Determining a soldier's intent from brain signals rather than using muscles would have numerous applications for tactical combat. Brain-computer interfaces (BCIs) translate brain signals into machine readable form and could optimize a soldier's interaction with the surrounding environment. However, current BCI devices have largely remained laboratory curiosities, because current techniques either require extended training or do not have the requisite signal fidelity, because they are highly invasive and thus not safe or practical for use in humans, or because they rely on equipment (such as magnetic resonance imaging scanners) that do not allow for real-time applications and/or field deployment. The objective of our research program is to create a prototype of a system for communication and monitoring of orientation that uses brain signals to provide, in real time, an accurate assessment of the users intentional focus and imagined speech. We expect that our efforts will provide a prototype of the first intuitive brain-based communication and orientation system for human use. %8 06/2011 %G eng %R DOI: 10.1117/12.888189 %0 Journal Article %J International Journal of Human-Computer Interaction %D 2011 %T Editorial: Current Trends in Brain-Computer Interface (BCI) Research and Development. %A Nam, C.S. %A Gerwin Schalk %A Moore-Jackson, M. %B International Journal of Human-Computer Interaction %8 01/2011 %G eng %U http://www.tandfonline.com/doi/abs/10.1080/10447318.2011.535748?journalCode=hihc20#.VYwf82AxI4g %R 10.1080/10447318.2011.535748 %0 Generic %D 2011 %T Electrocorticography: A New Window into Brain Function %A Gerwin Schalk %X Department of Brain and Cognitive Sciences, Massachusetts Institute of Technology, Boston, MA %8 02/22/2011 %G eng %0 Generic %D 2011 %T Exciting Adventures in Neuroscience and Neuroengineering %A Gerwin Schalk %X Electrical and Computer Engineering Department, University of Houston, Houston, Texas %8 06/20/2011 %G eng %0 Generic %D 2011 %T Exciting Adventures in Neuroscience and Neuroengineering %A Gerwin Schalk %X Department of Biomedical Engineering, Rensselaer Polytechnic Institute, Troy, New York %8 04/06/2011 %G eng %0 Generic %D 2011 %T Exciting Adventures in Neuroscience and Neuroengineering %A Gerwin Schalk %X Institute for Knowledge Discovery, Graz Technical University, Graz, Austria %8 05/23/2011 %G eng %0 Generic %D 2011 %T Introduction to BCI2000 %A Gerwin Schalk %X 8th BCI2000 Workshop, University Medical Center, Utrecht, The Netherlands %8 05/18/2011 %G eng %0 Journal Article %J Front Hum Neurosci %D 2011 %T Neural Correlates of Covert Attention in Electrocorticographic (ECoG) Signals in Humans. %A Gunduz, Aysegul %A Peter Brunner %A Amy Daitch %A Leuthardt, E C %A A L Ritaccio %A Pesaran, Bijan %A Gerwin Schalk %K covert attention %K Electrocorticography %K intention %K motor response %K visual-spatial attention %X

Attention is a cognitive selection mechanism that allocates the limited processing resources of the brain to the sensory streams most relevant to our immediate goals, thereby enhancing responsiveness and behavioral performance. The underlying neural mechanisms of orienting attention are distributed across a widespread cortical network. While aspects of this network have been extensively studied, details about the electrophysiological dynamics of this network are scarce. In this study, we investigated attentional networks using electrocorticographic (ECoG) recordings from the surface of the brain, which combine broad spatial coverage with high temporal resolution, in five human subjects. ECoG was recorded when subjects covertly attended to a spatial location and responded to contrast changes in the presence of distractors in a modified Posner cueing task. ECoG amplitudes in the alpha, beta, and gamma bands identified neural changes associated with covert attention and motor preparation/execution in the different stages of the task. The results show that attentional engagement was primarily associated with ECoG activity in the visual, prefrontal, premotor, and parietal cortices. Motor preparation/execution was associated with ECoG activity in premotor/sensorimotor cortices. In summary, our results illustrate rich and distributed cortical dynamics that are associated with orienting attention and the subsequent motor preparation and execution. These findings are largely consistent with and expand on primate studies using intracortical recordings and human functional neuroimaging studies.

%B Front Hum Neurosci %V 5 %P 89 %8 09/2011 %G eng %U http://www.ncbi.nlm.nih.gov/pubmed/22046153 %R 10.3389/fnhum.2011.00089 %0 Journal Article %J J Neurosci %D 2011 %T Nonuniform high-gamma (60-500 Hz) power changes dissociate cognitive task and anatomy in human cortex. %A Charles M Gaona %A Sharma, Mohit %A Zachary V. Freudenberg %A Breshears, Jonathan %A Bundy, David T %A Roland, Jarod %A Barbour, Dennis L %A Gerwin Schalk %A Leuthardt, E C %K Acoustic Stimulation %K Adolescent %K Adult %K Analysis of Variance %K Brain Mapping %K Brain Waves %K Cerebral Cortex %K Cognition Disorders %K Electroencephalography %K Epilepsy %K Evoked Potentials %K Female %K Humans %K Male %K Middle Aged %K Neuropsychological Tests %K Nonlinear Dynamics %K Photic Stimulation %K Reaction Time %K Spectrum Analysis %K Time Factors %K Vocabulary %X

High-gamma-band (>60 Hz) power changes in cortical electrophysiology are a reliable indicator of focal, event-related cortical activity. Despite discoveries of oscillatory subthreshold and synchronous suprathreshold activity at the cellular level, there is an increasingly popular view that high-gamma-band amplitude changes recorded from cellular ensembles are the result of asynchronous firing activity that yields wideband and uniform power increases. Others have demonstrated independence of power changes in the low- and high-gamma bands, but to date, no studies have shown evidence of any such independence above 60 Hz. Based on nonuniformities in time-frequency analyses of electrocorticographic (ECoG) signals, we hypothesized that induced high-gamma-band (60-500 Hz) power changes are more heterogeneous than currently understood. Using single-word repetition tasks in six human subjects, we showed that functional responsiveness of different ECoG high-gamma sub-bands can discriminate cognitive task (e.g., hearing, reading, speaking) and cortical locations. Power changes in these sub-bands of the high-gamma range are consistently present within single trials and have statistically different time courses within the trial structure. Moreover, when consolidated across all subjects within three task-relevant anatomic regions (sensorimotor, Broca's area, and superior temporal gyrus), these behavior- and location-dependent power changes evidenced nonuniform trends across the population. Together, the independence and nonuniformity of power changes across a broad range of frequencies suggest that a new approach to evaluating high-gamma-band cortical activity is necessary. These findings show that in addition to time and location, frequency is another fundamental dimension of high-gamma dynamics.

%B J Neurosci %V 31 %P 2091-100 %8 02/2011 %G eng %U http://www.ncbi.nlm.nih.gov/pubmed/21307246 %N 6 %R 10.1523/JNEUROSCI.4722-10.2011 %0 Generic %D 2011 %T Opportunities for Clinical Application of Emerging Neuroscientific and Neuroengineering Understanding %A Gerwin Schalk %X Neruophysiology Seminar Series, Baylor Hospital, Houston, TX %8 08/23/2011 %G eng %0 Generic %D 2011 %T Perspectives on ECoG Research and Application %A Gerwin Schalk %X 3rd International Workshop on Advances in Electrocorticography, Washington, DC %8 11/11/2011 %G eng %0 Journal Article %J Front Neurosci %D 2011 %T Prior knowledge improves decoding of finger flexion from electrocorticographic signals. %A Zuoguan Wang %A Ji, Q %A Miller, John W %A Gerwin Schalk %K brain-computer interface %K decoding algorithm %K electrocorticographic %K finger flexion %K machine learning %K prior knowledge %X

Brain-computer interfaces (BCIs) use brain signals to convey a user's intent. Some BCI approaches begin by decoding kinematic parameters of movements from brain signals, and then proceed to using these signals, in absence of movements, to allow a user to control an output. Recent results have shown that electrocorticographic (ECoG) recordings from the surface of the brain in humans can give information about kinematic parameters (e.g., hand velocity or finger flexion). The decoding approaches in these studies usually employed classical classification/regression algorithms that derive a linear mapping between brain signals and outputs. However, they typically only incorporate little prior information about the target movement parameter. In this paper, we incorporate prior knowledge using a Bayesian decoding method, and use it to decode finger flexion from ECoG signals. Specifically, we exploit the constraints that govern finger flexion and incorporate these constraints in the construction, structure, and the probabilistic functions of the prior model of a switched non-parametric dynamic system (SNDS). Given a measurement model resulting from a traditional linear regression method, we decoded finger flexion using posterior estimation that combined the prior and measurement models. Our results show that the application of the Bayesian decoding model, which incorporates prior knowledge, improves decoding performance compared to the application of a linear regression model, which does not incorporate prior knowledge. Thus, the results presented in this paper may ultimately lead to neurally controlled hand prostheses with full fine-grained finger articulation.

%B Front Neurosci %V 5 %P 127 %8 11/2011 %G eng %U http://www.ncbi.nlm.nih.gov/pubmed/22144944 %R 10.3389/fnins.2011.00127 %0 Journal Article %J Epilepsy Behav %D 2011 %T Proceedings of the Second International Workshop on Advances in Electrocorticography. %A A L Ritaccio %A Boatman-Reich, Dana %A Peter Brunner %A Cervenka, Mackenzie C %A Cole, Andrew J %A Nathan E. Crone %A Duckrow, Robert %A Korzeniewska, Anna %A Litt, Brian %A Miller, John W %A Moran, D %A Parvizi, Josef %A Viventi, Jonathan %A Williams, Justin C %A Gerwin Schalk %K Brain %K Brain Mapping %K Brain Waves %K Diagnosis, Computer-Assisted %K Electroencephalography %K Epilepsy %K Humans %K United States %K User-Computer Interface %X

The Second International Workshop on Advances in Electrocorticography (ECoG) was convened in San Diego, CA, USA, on November 11-12, 2010. Between this meeting and the inaugural 2009 event, a much clearer picture has been emerging of cortical ECoG physiology and its relationship to local field potentials and single-cell recordings. Innovations in material engineering are advancing the goal of a stable long-term recording interface. Continued evolution of ECoG-driven brain-computer interface technology is determining innovation in neuroprosthetics. Improvements in instrumentation and statistical methodologies continue to elucidate ECoG correlates of normal human function as well as the ictal state. This proceedings document summarizes the current status of this rapidly evolving field.

%B Epilepsy Behav %V 22 %P 641-50 %8 12/2011 %G eng %U http://www.ncbi.nlm.nih.gov/pubmed/22036287 %N 4 %R 10.1016/j.yebeh.2011.09.028 %0 Journal Article %J Front Neurosci %D 2011 %T Rapid Communication with a "P300" Matrix Speller Using Electrocorticographic Signals (ECoG). %A Peter Brunner %A A L Ritaccio %A Emrich, Joseph F %A H Bischof %A Gerwin Schalk %K brain-computer interface %K Electrocorticography %K event-related potential %K P300 %K speller %X

brain-computer interface (BCI) can provide a non-muscular communication channel to severely disabled people. One particular realization of a BCI is the P300 matrix speller that was originally described by Farwell and Donchin (1988). This speller uses event-related potentials (ERPs) that include the P300 ERP. All previous online studies of the P300 matrix speller used scalp-recorded electroencephalography (EEG) and were limited in their communication performance to only a few characters per minute. In our study, we investigated the feasibility of using electrocorticographic (ECoG) signals for online operation of the matrix speller, and determined associated spelling rates. We used the matrix speller that is implemented in the BCI2000 system. This speller used ECoG signals that were recorded from frontal, parietal, and occipital areas in one subject. This subject spelled a total of 444 characters in online experiments. The results showed that the subject sustained a rate of 17 characters/min (i.e., 69 bits/min), and achieved a peak rate of 22 characters/min (i.e., 113 bits/min). Detailed analysis of the results suggests that ERPs over visual areas (i.e., visual evoked potentials) contribute significantly to the performance of the matrix speller BCI system. Our results also point to potential reasons for the apparent advantages in spelling performance of ECoG compared to EEG. Thus, with additional verification in more subjects, these results may further extend the communication options for people with serious neuromuscular disabilities.

%B Front Neurosci %V 5 %P 5 %8 02/2011 %G eng %U http://www.ncbi.nlm.nih.gov/pubmed/21369351 %R 10.3389/fnins.2011.00005 %0 Generic %D 2011 %T Real-Time Functional Mapping Using ECoG %A Gerwin Schalk %X g.tec ECoG/Spike Workshop, Washington, DC %8 11/14/2011 %G eng %0 Journal Article %J Neuroimage %D 2011 %T Spatiotemporal dynamics of electrocorticographic high gamma activity during overt and covert word repetition. %A Pei, Xiao-Mei %A Leuthardt, E C %A Charles M Gaona %A Peter Brunner %A Jonathan Wolpaw %A Gerwin Schalk %K Adolescent %K Adult %K Brain %K Brain Mapping %K Electroencephalography %K Female %K Humans %K Male %K Middle Aged %K Signal Processing, Computer-Assisted %K Verbal Behavior %X

Language is one of the defining abilities of humans. Many studies have characterized the neural correlates of different aspects of language processing. However, the imaging techniques typically used in these studies were limited in either their temporal or spatial resolution. Electrocorticographic (ECoG) recordings from the surface of the brain combine high spatial with high temporal resolution and thus could be a valuable tool for the study of neural correlates of language function. In this study, we defined the spatiotemporal dynamics of ECoG activity during a word repetition task in nine human subjects. ECoG was recorded while each subject overtly or covertly repeated words that were presented either visually or auditorily. ECoG amplitudes in the high gamma (HG) band confidently tracked neural changes associated with stimulus presentation and with the subject's verbal response. Overt word production was primarily associated with HG changes in the superior and middle parts of temporal lobe, Wernicke's area, the supramarginal gyrus, Broca's area, premotor cortex (PMC), primary motor cortex. Covert word production was primarily associated with HG changes in superior temporal lobe and the supramarginal gyrus. Acoustic processing from both auditory stimuli as well as the subject's own voice resulted in HG power changes in superior temporal lobe and Wernicke's area. In summary, this study represents a comprehensive characterization of overt and covert speech using electrophysiological imaging with high spatial and temporal resolution. It thereby complements the findings of previous neuroimaging studies of language and thus further adds to current understanding of word processing in humans.

%B Neuroimage %V 54 %P 2960-72 %8 02/2011 %G eng %U http://www.ncbi.nlm.nih.gov/pubmed/21029784 %N 4 %R 10.1016/j.neuroimage.2010.10.029 %0 Book Section %B Recent Advances in Brain-Computer Interface Systems %D 2011 %T State-of-the-Art in BCI Research: BCI Award 2010. %A Guger, C %A Gerwin Schalk %E Reza Fazel %B Recent Advances in Brain-Computer Interface Systems %I InTech Press %P 193-222 %G eng %U http://www.intechopen.com/books/recent-advances-in-brain-computer-interface-systems/state-of-the-art-in-bci-research-bci-award-2010 %R 10.5772/15017 %0 Journal Article %J Clin Neurophysiol %D 2011 %T Toward a gaze-independent matrix speller brain-computer interface. %A Peter Brunner %A Gerwin Schalk %K Attention %K Brain %K Fixation, Ocular %K Humans %K User-Computer Interface %B Clin Neurophysiol %V 122 %P 1063-4 %8 06/2011 %G eng %U http://www.ncbi.nlm.nih.gov/pubmed/21183404 %N 6 %R 10.1016/j.clinph.2010.11.014 %0 Journal Article %J J Neural Eng %D 2011 %T Using the electrocorticographic speech network to control a brain-computer interface in humans. %A Leuthardt, E C %A Charles M Gaona %A Sharma, Mohit %A Szrama, Nicholas %A Roland, Jarod %A Zachary V. Freudenberg %A Solisb, Jamie %A Breshears, Jonathan %A Gerwin Schalk %K Adult %K Brain %K Brain Mapping %K Computer Peripherals %K Electroencephalography %K Evoked Potentials %K Feedback, Physiological %K Female %K Humans %K Imagination %K Male %K Middle Aged %K Nerve Net %K Speech Production Measurement %K User-Computer Interface %X

Electrocorticography (ECoG) has emerged as a new signal platform for brain-computer interface (BCI) systems. Classically, the cortical physiology that has been commonly investigated and utilized for device control in humans has been brain signals from the sensorimotor cortex. Hence, it was unknown whether other neurophysiological substrates, such as the speech network, could be used to further improve on or complement existing motor-based control paradigms. We demonstrate here for the first time that ECoG signals associated with different overt and imagined phoneme articulation can enable invasively monitored human patients to control a one-dimensional computer cursor rapidly and accurately. This phonetic content was distinguishable within higher gamma frequency oscillations and enabled users to achieve final target accuracies between 68% and 91% within 15 min. Additionally, one of the patients achieved robust control using recordings from a microarray consisting of 1 mm spaced microwires. These findings suggest that the cortical network associated with speech could provide an additional cognitive and physiologic substrate for BCI operation and that these signals can be acquired from a cortical array that is small and minimally invasive.

%B J Neural Eng %V 8 %P 036004 %8 06/2011 %G eng %U http://www.ncbi.nlm.nih.gov/pubmed/21471638 %N 3 %R 10.1088/1741-2560/8/3/036004 %0 Generic %D 2010 %T Advanced BCI2000 Concepts %A Gerwin Schalk %X 7th BCI2000 Workshop, Asilomar Conference Center, Monterey, California %8 05/30/2010 %G eng %0 Generic %D 2010 %T A brain-based communication and orientation system %A Gerwin Schalk %X 2010 US Army DDRE MURI Conference in Arlington, VA %8 07/22/2010 %G eng %0 Generic %D 2010 %T Brain-Computer Interfaces: Prospects and Problems %A Gerwin Schalk %X Cog Sci Issues Colloquium, Department of Cognitive Sciences, Rensselaer Polytechnic Institute, Troy, New York %8 01/27/2010 %G eng %0 Journal Article %J Ann Neurol %D 2010 %T Brain-computer interfacing based on cognitive control. %A Vansteensel, Mariska J %A Hermes, Dora %A Aarnoutse, Erik J %A Bleichner, Martin G %A Gerwin Schalk %A van Rijen, Peter C %A Leijten, Frans S S %A Ramsey, Nick F %K Cognition %K Computers %K Electrodes %K Electroencephalography %K Epilepsy %K Humans %K Image Processing, Computer-Assisted %K Magnetic Resonance Imaging %K Neuropsychological Tests %K Oxygen %K Prefrontal Cortex %K Psychomotor Performance %K Spectrum Analysis %K Time Factors %K User-Computer Interface %X

OBJECTIVE: 

Brain-computer interfaces (BCIs) translate deliberate intentions and associated changes in brain activity into action, thereby offering patients with severe paralysis an alternative means of communication with and control over their environment. Such systems are not available yet, partly due to the high performance standard that is required. A major challenge in the development of implantable BCIs is to identify cortical regions and related functions that an individual can reliably and consciously manipulate. Research predominantly focuses on the sensorimotor cortex, which can be activated by imagining motor actions. However, because this region may not provide an optimal solution to all patients, other neuronal networks need to be examined. Therefore, we investigated whether the cognitive control network can be used for BCI purposes. We also determined the feasibility of using functional magnetic resonance imaging (fMRI) for noninvasive localization of the cognitive control network.

METHODS: 

Three patients with intractable epilepsy, who were temporarily implanted with subdural grid electrodes for diagnostic purposes, attempted to gain BCI control using the electrocorticographic (ECoG) signal of the left dorsolateral prefrontal cortex (DLPFC).

RESULTS: 

All subjects quickly gained accurate BCI control by modulation of gamma-power of the left DLPFC. Prelocalization of the relevant region was performed with fMRI and was confirmed using the ECoG signals obtained during mental calculation localizer tasks.

INTERPRETATION: 

The results indicate that the cognitive control network is a suitable source of signals for BCI applications. They also demonstrate the feasibility of translating understanding about cognitive networks derived from functional neuroimaging into clinical applications.

%B Ann Neurol %V 67 %P 809-16 %8 06/2010 %G eng %U http://www.ncbi.nlm.nih.gov/pubmed/20517943 %N 6 %R 10.1002/ana.21985 %0 Journal Article %J Front Neuroeng %D 2010 %T Can Electrocorticography (ECoG) Support Robust and Powerful Brain-Computer Interfaces?. %A Gerwin Schalk %B Front Neuroeng %V 3 %P 9 %8 06/2010 %G eng %U http://www.ncbi.nlm.nih.gov/pubmed/20631853 %R 10.3389/fneng.2010.00009 %0 Journal Article %J Proc Natl Acad Sci U S A %D 2010 %T Cortical activity during motor execution, motor imagery, and imagery-based online feedback. %A Miller, K.J. %A Gerwin Schalk %A Fetz, Eberhard E %A den Nijs, Marcel %A Ojemann, J G %A Rao, Rajesh P N %K Adolescent %K Adult %K Biofeedback, Psychology %K Cerebral Cortex %K Child %K Electric Stimulation %K Electrocardiography %K Female %K Humans %K Male %K Middle Aged %K Motor Activity %K Young Adult %X

Imagery of motor movement plays an important role in learning of complex motor skills, from learning to serve in tennis to perfecting a pirouette in ballet. What and where are the neural substrates that underlie motor imagery-based learning? We measured electrocorticographic cortical surface potentials in eight human subjects during overt action and kinesthetic imagery of the same movement, focusing on power in "high frequency" (76-100 Hz) and "low frequency" (8-32 Hz) ranges. We quantitatively establish that the spatial distribution of local neuronal population activity during motor imagery mimics the spatial distribution of activity during actual motor movement. By comparing responses to electrocortical stimulation with imagery-induced cortical surface activity, we demonstrate the role of primary motor areas in movement imagery. The magnitude of imagery-induced cortical activity change was approximately 25% of that associated with actual movement. However, when subjects learned to use this imagery to control a computer cursor in a simple feedback task, the imagery-induced activity change was significantly augmented, even exceeding that of overt movement.

%B Proc Natl Acad Sci U S A %V 107 %P 4430-5 %8 03/2010 %G eng %U http://www.ncbi.nlm.nih.gov/pubmed/20160084 %N 9 %R 10.1073/pnas.0913697107 %0 Conference Proceedings %B International Conference on Pattern Recognition - ICPR %D 2010 %T Decoding finger flexion from electrocorticographic signals using sparse Gaussian process. %A Zuoguan Wang %A Ji, Q %A Kai J. Miller %A Gerwin Schalk %X A brain-computer interface (BCI) creates a direct communication pathway between the brain and an external device, and can thereby restore function in people with severe motor disabilities. A core component in a BCI system is the decoding algorithm that translates brain signals into action commands of an output device. Most of current decoding algorithms are based on linear models (e.g., derived using linear regression) that may have important shortcomings. The use of nonlinear models (e.g., neural networks) could overcome some of these shortcomings, but has difficulties with high dimensional feature spaces. Here we propose another decoding algorithm that is based on the sparse gaussian process with pseudo-inputs (SPGP). As a nonparametric method, it can model more complex relationships compared to linear methods. As a kernel method, it can readily deal with high dimensional feature space. The evaluations shown in this paper demonstrate that SPGP can decode the flexion of finger movements from electrocorticographic (ECoG) signals more accurately than a previously described algorithm that used a linear model. In addition, by formulating problems in the bayesian probabilistic framework, SPGP can provide estimation of the prediction uncertainty. Furthermore, the trained SPGP offers a very effective way for identifying important features. %B International Conference on Pattern Recognition - ICPR %G eng %R 10.1109/ICPR.2010.915 %0 Journal Article %J J Neural Eng %D 2010 %T Does the 'P300' speller depend on eye gaze?. %A Peter Brunner %A Joshi, S %A S Briskin %A Jonathan Wolpaw %A H Bischof %A Gerwin Schalk %K Adult %K Event-Related Potentials, P300 %K Eye Movements %K Female %K Humans %K Male %K Middle Aged %K Models, Neurological %K Photic Stimulation %K User-Computer Interface %K Young Adult %X

Many people affected by debilitating neuromuscular disorders such as amyotrophic lateral sclerosis, brainstem stroke or spinal cord injury are impaired in their ability to, or are even unable to, communicate. A brain-computer interface (BCI) uses brain signals, rather than muscles, to re-establish communication with the outside world. One particular BCI approach is the so-called 'P300 matrix speller' that was first described by Farwell and Donchin (1988 Electroencephalogr. Clin. Neurophysiol. 70 510-23). It has been widely assumed that this method does not depend on the ability to focus on the desired character, because it was thought that it relies primarily on the P300-evoked potential and minimally, if at all, on other EEG features such as the visual-evoked potential (VEP). This issue is highly relevant for the clinical application of this BCI method, because eye movements may be impaired or lost in the relevant user population. This study investigated the extent to which the performance in a 'P300' speller BCI depends on eye gaze. We evaluated the performance of 17 healthy subjects using a 'P300' matrix speller under two conditions. Under one condition ('letter'), the subjects focused their eye gaze on the intended letter, while under the second condition ('center'), the subjects focused their eye gaze on a fixation cross that was located in the center of the matrix. The results show that the performance of the 'P300' matrix speller in normal subjects depends in considerable measure on gaze direction. They thereby disprove a widespread assumption in BCI research, and suggest that this BCI might function more effectively for people who retain some eye-movement control. The applicability of these findings to people with severe neuromuscular disabilities (particularly in eye-movements) remains to be determined.

%B J Neural Eng %V 7 %P 056013 %8 10/2010 %G eng %U http://www.ncbi.nlm.nih.gov/pubmed/20858924 %N 5 %R 10.1088/1741-2560/7/5/056013 %0 Journal Article %J Neurosurgery %D 2010 %T Electrocorticographic frequency alteration mapping for extraoperative localization of speech cortex. %A Wu, Melinda %A Wisneski, Kimberly %A Gerwin Schalk %A Sharma, Mohit %A Roland, Jarod %A Breshears, Jonathan %A Charles M Gaona %A Leuthardt, E C %K Acoustic Stimulation %K Adolescent %K Adult %K Brain Mapping %K Cerebral Cortex %K Chi-Square Distribution %K Electroencephalography %K Epilepsy %K Female %K Humans %K Male %K Mass Spectrometry %K Middle Aged %K Photic Stimulation %K Speech %K Verbal Behavior %K Young Adult %X

OBJECTIVE: 

Electrocortical stimulation (ECS) has long been established for delineating eloquent cortex in extraoperative mapping. However, ECS is still coarse and inefficient in delineating regions of functional cortex and can be hampered by afterdischarges. Given these constraints, an adjunct approach to defining motor cortex is the use of electrocorticographic (ECoG) signal changes associated with active regions of cortex. The broad range of frequency oscillations are categorized into 2 main groups with respect to sensorimotor cortex: low-frequency bands (LFBs) and high-frequency bands (HFBs). The LFBs tend to show a power reduction, whereas the HFBs show power increases with cortical activation. These power changes associated with activated cortex could potentially provide a powerful tool in delineating areas of speech cortex. We explore ECoG signal alterations as they occur with activated region of speech cortex and its potential in clinical brain mapping applications.

METHODS: 

We evaluated 7 patients who underwent invasive monitoring for seizure localization. Each had extraoperative ECS mapping to identify speech cortex. Additionally, all subjects performed overt speech tasks with an auditory or a visual cue to identify associated frequency power changes in regard to location and degree of concordance with ECS results.

RESULTS: 

Electrocorticographic frequency alteration mapping (EFAM) had an 83.9% sensitivity and a 40.4% specificity in identifying any language site when considering both frequency bands and both stimulus cues. Electrocorticographic frequency alteration mapping was more sensitive in identifying the Wernicke area (100%) than the Broca area (72.2%). The HFB is uniquely suited to identifying the Wernicke area, whereas a combination of the HFB and LFB is important for Broca localization.

CONCLUSION: 

The concordance between stimulation and spectral power changes demonstrates the possible utility of EFAM as an adjunct method to improve the efficiency and resolution of identifying speech cortex.

%B Neurosurgery %V 66 %P E407-9 %8 02/2010 %G eng %U http://www.ncbi.nlm.nih.gov/pubmed/20087111 %N 2 %R 10.1227/01.NEU.0000345352.13696.6F %0 Generic %D 2010 %T Emerging Opportunities in Neuroengineering %A Gerwin Schalk %X Department of Electrical Engineering and Computer Science, Technical University of Berlin, Berlin, Germany %8 10/01/2010 %G eng %0 Generic %D 2010 %T Emerging Opportunities in Neuroengineering %A Gerwin Schalk %X Graduate School of Biomedical Engineering, The University of New South Wales, Sydney, Australia %8 08/27/2010 %G eng %0 Generic %D 2010 %T Encoding of Perception and Cognition in Human Electrocorticographic Signals %A Gerwin Schalk %X Keynote Address, Bernstein Conference on Computational Neuroscience, Berlin, Germany %8 09/30/2010 %G eng %0 Generic %D 2010 %T Exciting Directions in Human Electrocorticography %A Gerwin Schalk %X University of California San Francisco Medical School, San Francisco, California %8 05/28/2010 %G eng %0 Generic %D 2010 %T Exciting Directions in Human Electrocorticography %A Gerwin Schalk %X Small Scale Systems and Integration and Packaging Center's Seminar Series, Binghamton University, Binghamton, New York %8 01/20/2010 %G eng %0 Generic %D 2010 %T Exciting Directions in Neuroscience and Neuroengineering %A Gerwin Schalk %X Department of Physical Therapy and Human Movement Sciences, Northwestern University, Feinberg School of Medicine, Chicago, IL %8 12/13/2010 %G eng %0 Generic %D 2010 %T Inferring Detailed Aspects of COgnition Using Electrocorticographic (ECoG) Signals in Humans %A Gerwin Schalk %X Seattle Children's Research Institute, Seattle, WA %8 09/17/2010 %G eng %0 Generic %D 2010 %T Introduction to BCI2000 %A Gerwin Schalk %X 7th BCI2000 Workshop, Asilomar Conference Center, Monterey, California %8 05/30/2010 %G eng %0 Generic %D 2010 %T Neuroscience and Brain-Computer Interface Research Using Signals Recorded from the Surface of the Brain %A Gerwin Schalk %X Hudson Valley-Berkshire Chapter of the Society for Neuroscience, Albany, New York %8 02/22/2010 %G eng %0 Generic %D 2010 %T Novel Methods and Applications in Brain-Computer Interface Research %A Gerwin Schalk %X U.S. Army Research Laboratory, Aberdeen Proving Ground, Aberdeen MD %8 07/21/2010 %G eng %0 Journal Article %J Epilepsy Behav %D 2010 %T Passive real-time identification of speech and motor cortex during an awake craniotomy. %A Roland, Jarod %A Peter Brunner %A Johnston, James %A Gerwin Schalk %A Leuthardt, E C %K Brain Mapping %K Brain Neoplasms %K Cerebral Cortex %K Craniotomy %K Electric Stimulation %K Electroencephalography %K Humans %K Neurologic Examination %X

Precise localization of eloquent cortex is a clinical necessity prior to surgical resections adjacent to speech or motor cortex. In the intraoperative setting, this traditionally requires inducing temporary lesions by direct electrocortical stimulation (DECS). In an attempt to increase efficiency and potentially reduce the amount of necessary stimulation, we used a passive mapping procedure in the setting of an awake craniotomy for tumor in two patients resection. We recorded electrocorticographic (ECoG) signals from exposed cortex while patients performed simple cue-directed motor and speech tasks. SIGFRIED, a procedure for real-time event detection, was used to identify areas of cortical activation by detecting task-related modulations in the ECoG high gamma band. SIGFRIED's real-time output quickly localized motor and speech areas of cortex similar to those identified by DECS. In conclusion, real-time passive identification of cortical function using SIGFRIED may serve as a useful adjunct to cortical stimulation mapping in the intraoperative setting.

%B Epilepsy Behav %V 18 %P 123-8 %8 05/2010 %G eng %U http://www.ncbi.nlm.nih.gov/pubmed/20478745 %N 1-2 %R 10.1016/j.yebeh.2010.02.017 %0 Book %D 2010 %T A Practical Guide to Brain-Computer Interfacing with BCI2000. %A Gerwin Schalk %A Mellinger, Jürgen %I Springer London Dordrecht Heidelberg New York %P 288 %G eng %U http://link.springer.com/book/10.1007%2F978-1-84996-092-2 %R 10.1007/978-1-84996-092-2 %0 Journal Article %J IEEE Trans Biomed Eng %D 2010 %T A procedure for measuring latencies in brain-computer interfaces. %A Adam J Wilson %A Mellinger, Jürgen %A Gerwin Schalk %A Williams, Justin C %K Brain %K Computer Systems %K Electroencephalography %K Evoked Potentials %K Humans %K Models, Neurological %K Reproducibility of Results %K Signal Processing, Computer-Assisted %K Time Factors %K User-Computer Interface %X

Brain-computer interface (BCI) systems must process neural signals with consistent timing in order to support adequate system performance. Thus, it is important to have the capability to determine whether a particular BCI configuration (i.e., hardware and software) provides adequate timing performance for a particular experiment. This report presents a method of measuring and quantifying different aspects of system timing in several typical BCI experiments across a range of settings, and presents comprehensive measures of expected overall system latency for each experimental configuration.

%B IEEE Trans Biomed Eng %V 57 %P 1785-97 %8 06/2010 %G eng %U http://www.ncbi.nlm.nih.gov/pubmed/20403781 %N 7 %R 10.1109/TBME.2010.2047259 %0 Journal Article %J Epilepsy Behav %D 2010 %T Proceedings of the first international workshop on advances in electrocorticography. %A A L Ritaccio %A Peter Brunner %A Cervenka, Mackenzie C %A Nathan E. Crone %A Guger, C %A Leuthardt, E C %A Oostenveld, Robert %A Stacey, William %A Gerwin Schalk %K Brain %K Brain Mapping %K Diagnosis, Computer-Assisted %K Electroencephalography %K Humans %K International Cooperation %K Seizures %K Signal Detection, Psychological %X

In October 2009, a group of neurologists, neurosurgeons, computational neuroscientists, and engineers congregated to present novel developments transforming human electrocorticography (ECoG) beyond its established relevance in clinical epileptology. The contents of the proceedings advanced the role of ECoG in seizure detection and prediction, neurobehavioral research, functional mapping, and brain-computer interface technology. The meeting established the foundation for future work on the methodology and application of surface brain recordings.

%B Epilepsy Behav %V 19 %P 204-15 %8 10/2010 %G eng %U http://www.ncbi.nlm.nih.gov/pubmed/20889384 %N 3 %R 10.1016/j.yebeh.2010.08.028 %0 Generic %D 2010 %T Real-Time Functional Mapping Using Electrocorticographic Signals %A Gerwin Schalk %X Department of Neurology, Seattle Children's Hospital, Seattle, WA %8 09/17/2010 %G eng %0 Generic %D 2010 %T Toward Brain-Computer Symbiosis %A Gerwin Schalk %X Keynote Address, X-Prize Workshop on Brain-Computer Interfaces, MIT Campus, Cambridge, MA %8 01/07/2010 %G eng %0 Book Section %B Brain-Computer Interfaces: Applying our Minds to Human-Computer Interaction %D 2010 %T Using BCI2000 for HCI-Centered BCI Research. %A Adam J Wilson %A Gerwin Schalk %E A. Nijholt %E D. Tan %X BCI2000 is a general-purpose software suite designed for brain-computer interface (BCI) and related research. BCI2000 has been in development since 2000 and is currently used in close to 500 laboratories around the world. BCI2000 can provide stimulus presentation while simultaneously recording brain signals and subject responses from a number of data acquisition and input devices, respectively. Furthermore, BCI2000 provides a number of services (such as a generic data format that can accommodate any hardware or experimental setup) that can greatly facilitate research. In summary, BCI2000 is ideally suited to support investigations in the area of human-computer interfaces (HCI), in particular those that include recording and processing of brain signals. This chapter provides an overview of the BCI2000 system, and gives examples of its utility for HCI research. %B Brain-Computer Interfaces: Applying our Minds to Human-Computer Interaction %I Springer London %P 261-274 %G eng %U http://link.springer.com/chapter/10.1007%2F978-1-84996-272-8_15 %R 10.1007/978-1-84996-272-8_15 %0 Book Section %B Brain-Computer Interfaces: Revolutionizing Human-Computer Interaction %D 2010 %T Using BCI2000 in BCI Research. %A Mellinger, Jürgen %A Gerwin Schalk %E Graimann, Bernhard %E Pfurtscheller, Gert %E Brendan Z. Allison %X

BCI2000 is a general-purpose system for brain–computer interface (BCI) research. It can also be used for data acquisition, stimulus presentation, and brain monitoring applications [18,27]. The mission of the BCI2000 project is to facilitate research and applications in these areas. BCI2000 has been in development since 2000 in a collaboration between the Wadsworth Center of the New York State Department of Health in Albany, New York, and the Institute of Medical Psychology and Behavioral Neurobiology at the University of Tübingen, Germany. Many other individuals at different institutions world-wide have contributed to this project.

%B Brain-Computer Interfaces: Revolutionizing Human-Computer Interaction %S The Frontiers Collection %I Springer Berlin Heidelberg %P 259-280 %G eng %U http://dx.doi.org/10.1007/978-3-642-02091-9_15 %0 Generic %D 2010 %T Using Neuroscience and Neuroengineering to Augment Human Performance %A Gerwin Schalk %X Topical Panel on Neuroscience, 27th Army Science Conference Orlando, FL %8 12/01/2010 %G eng %0 Generic %D 2009 %T BCI2000: A General-Purpose BCI System and its Application to ECoG Signals %A Gerwin Schalk %X Tutorial T02: Brain-Computer Interface, HCI International, San Diego, California %8 07/19/2009 %G eng %0 Generic %D 2009 %T BCI2000: A General-Purpose BCI System and its Application to ECoG Signals %A Gerwin Schalk %X g.tec Brain-Computer Interface Workshop, Society for Neuroscience Annual Meeting, Chicago, IL %8 10/17/2009 %G eng %0 Generic %D 2009 %T The BCI2000 Framework %A Gerwin Schalk %X 5th BCI2000 Workshop, The Sagamore Conference Center, Bolton Landing, new York %8 10/01/2009 %G eng %0 Generic %D 2009 %T Brain-Computer Interaction %A Gerwin Schalk %X Session Applications and Challenges in Neurally-Driven System Interfaces, Intl. Conference on Augmented Cognition, San Diego, California %8 07/22/2009 %G eng %0 Conference Paper %B 5th Intl. Conference on Augmented Cognition %D 2009 %T Brain-Computer Interaction. %A Peter Brunner %A Gerwin Schalk %K BCI %K brain-computer interface %K neural engineering %K neural prosthesis %X

Detection and automated interpretation of attention-related or intention-related brain activity carries significant promise for many military and civilian applications. This interpretation of brain activity could provide information about a person’s intended movements, imagined movements, or attentional focus, and thus could be valuable for optimizing or replacing traditional motor-based communication between a person and a computer or other output devices. We describe here the objective and preliminary results of our studies in this area.

%B 5th Intl. Conference on Augmented Cognition %I Springer %8 2009 %@ 978-3-642-02811-3 %G eng %U http://link.springer.com/chapter/10.1007%2F978-3-642-02812-0_81 %R 10.1007/978-3-642-02812-0_81 %0 Generic %D 2009 %T Brain-Computer Interfacing Using Electrocorticography (ECoG) %A Gerwin Schalk %X Swartz Center for Computational Neuroscience, Institute for Neural Computation, University of California San Diego, La Jolla, California %8 07/20/2009 %G eng %0 Generic %D 2009 %T Brain-Computer Interfacing Using Electrocorticography (ECoG) %A Gerwin Schalk %X Beijing BCI20009 Symposium, Tsinghua University, Beijing, China %8 12/05/2009 %G eng %0 Generic %D 2009 %T Brain-Computer Interfacing Using P300 Evoked Potentials %A Gerwin Schalk %X Guest lecture in course Brain-Computer Interfaces, Departments of Neurosurgery/Bioengineering, University of Pennsylvania, Philadelphia, Pennsylvania %8 03/25 %G eng %0 Generic %D 2009 %T Brain-COmputer Interfacing Using P300 Evoked Potentials %A Gerwin Schalk %X Guest lecture in course Brain-Computer Interface Systems, Department of Cognitive Sciences, University of California San Diego, La Jolla, California %8 07/21/2009 %G eng %0 Journal Article %J J Neural Eng %D 2009 %T Decoding flexion of individual fingers using electrocorticographic signals in humans. %A Kubánek, J %A Miller, John W %A Ojemann, J G %A Jonathan Wolpaw %A Gerwin Schalk %K Adolescent %K Adult %K Biomechanics %K Brain %K Electrodiagnosis %K Epilepsy %K Female %K Fingers %K Humans %K Male %K Microelectrodes %K Middle Aged %K Motor Activity %K Rest %K Thumb %K Time Factors %K Young Adult %X

Brain signals can provide the basis for a non-muscular communication and control system, a brain-computer interface (BCI), for people with motor disabilities. A common approach to creating BCI devices is to decode kinematic parameters of movements using signals recorded by intracortical microelectrodes. Recent studies have shown that kinematic parameters of hand movements can also be accurately decoded from signals recorded by electrodes placed on the surface of the brain (electrocorticography (ECoG)). In the present study, we extend these results by demonstrating that it is also possible to decode the time course of the flexion of individual fingers using ECoG signals in humans, and by showing that these flexion time courses are highly specific to the moving finger. These results provide additional support for the hypothesis that ECoG could be the basis for powerful clinically practical BCI systems, and also indicate that ECoG is useful for studying cortical dynamics related to motor function.

%B J Neural Eng %V 6 %P 066001 %8 12/2009 %G eng %U http://www.ncbi.nlm.nih.gov/pubmed/19794237 %N 6 %R 10.1088/1741-2560/6/6/066001 %0 Generic %D 2009 %T Detecting Detailed Aspects of Behavior in ECoG Signals %A Gerwin Schalk %X International Workshop on Advances in Electrocorticography, Bolton Landing, New York %8 10/02/2009 %G eng %0 Conference Proceedings %B Conf Proc IEEE Eng Med Biol Soc %D 2009 %T Detection of spontaneous class-specific visual stimuli with high temporal accuracy in human electrocorticography. %A Miller, John W %A Hermes, Dora %A Gerwin Schalk %A Ramsey, Nick F %A Jagadeesh, Bharathi %A den Nijs, Marcel %A Ojemann, J G %A Rao, Rajesh P N %K Algorithms %K Electrocardiography %K Evoked Potentials, Visual %K Humans %K Male %K Pattern Recognition, Automated %K Pattern Recognition, Visual %K Photic Stimulation %K Reproducibility of Results %K Sensitivity and Specificity %K User-Computer Interface %K Visual Cortex %X Most brain-computer interface classification experiments from electrical potential recordings have been focused on the identification of classes of stimuli or behavior where the timing of experimental parameters is known or pre-designated. Real world experience, however, is spontaneous, and to this end we describe an experiment predicting the occurrence, timing, and types of visual stimuli perceived by a human subject from electrocorticographic recordings. All 300 of 300 presented stimuli were correctly detected, with a temporal precision of order 20 ms. The type of stimulus (face/house) was correctly identified in 95% of these cases. There were approximately 20 false alarm events, corresponding to a late 2nd neuronal response to a previously identified event. %B Conf Proc IEEE Eng Med Biol Soc %V 2009 %P 6465-8 %8 2009 %G eng %R 10.1109/IEMBS.2009.5333546 %0 Generic %D 2009 %T EEG/ECoG-based BCIs for People with Little or No Motor Function %A Gerwin Schalk %X Seminar Brain-Computer Interfaces: Frontiers in Neurology and Neuroscience, American Academy of Neurology Meeting, Seattle, Washington %8 04/27/2009 %G eng %0 Conference Proceedings %B Conf Proc IEEE Eng Med Biol Soc %D 2009 %T Effective brain-computer interfacing using BCI2000. %A Gerwin Schalk %K Algorithms %K Brain %K Electrocardiography %K Equipment Design %K Equipment Failure Analysis %K Rehabilitation %K Reproducibility of Results %K Sensitivity and Specificity %K Signal Processing, Computer-Assisted %K User-Computer Interface %X To facilitate research and development in Brain-Computer Interface (BCI) research, we have been developing a general-purpose BCI system, called BCI2000, over the past nine years. This system has enjoyed a growing adoption in BCI and related areas and has been the basis for some of the most impressive studies reported to date. This paper gives an update on the status of this project by describing the principles of the BCI2000 system, its benefits, and impact on the field to date. %B Conf Proc IEEE Eng Med Biol Soc %V 2009 %P 5498-501 %8 2009 %G eng %R 10.1109/IEMBS.2009.5334558 %0 Generic %D 2009 %T Effective Brain-Computer Interfacing Using BCI2000 %A Gerwin Schalk %X Session Brain-Machine Interface I, 31st Annual International IEEE EMBS Conference, Minneapolis, Minnesota %8 09/05/2009 %G eng %0 Generic %D 2009 %T Emerging Opportunities in Neuroengineering %A Gerwin Schalk %X University of Pennsylvania, Philadelphia, Pennsylvania %8 06/18/2009 %G eng %0 Generic %D 2009 %T Engineering the Future in Biomedicine: Using Brain Signals for Communication and Diagnosis %A Gerwin Schalk %X IEEE Schenectady Section, Niskayuna, New York %8 04/03/2009 %G eng %0 Journal Article %J Neurosurg Focus %D 2009 %T Evolution of brain-computer interfaces: going beyond classic motor physiology. %A Leuthardt, E C %A Gerwin Schalk %A Roland, Jarod %A Rouse, Adam %A Moran, D %K Brain %K Cerebral Cortex %K Humans %K Man-Machine Systems %K Motor Cortex %K Movement %K Movement Disorders %K Neuronal Plasticity %K Prostheses and Implants %K Research %K Signal Processing, Computer-Assisted %K User-Computer Interface %X

The notion that a computer can decode brain signals to infer the intentions of a human and then enact those intentions directly through a machine is becoming a realistic technical possibility. These types of devices are known as brain-computer interfaces (BCIs). The evolution of these neuroprosthetic technologies could have significant implications for patients with motor disabilities by enhancing their ability to interact and communicate with their environment. The cortical physiology most investigated and used for device control has been brain signals from the primary motor cortex. To date, this classic motor physiology has been an effective substrate for demonstrating the potential efficacy of BCI-based control. However, emerging research now stands to further enhance our understanding of the cortical physiology underpinning human intent and provide further signals for more complex brain-derived control. In this review, the authors report the current status of BCIs and detail the emerging research trends that stand to augment clinical applications in the future.

%B Neurosurg Focus %V 27 %P E4 %8 07/2009 %G eng %U http://www.ncbi.nlm.nih.gov/pubmed/19569892 %N 1 %R 10.3171/2009.4.FOCUS0979 %0 Generic %D 2009 %T Introduction to BCI2000 %A Gerwin Schalk %X 6th BCI2000 Workshop, Tsinghua University, Beijing, China %8 12/06/2009 %G eng %0 Generic %D 2009 %T Overview of Available BCI2000 Components %A Gerwin Schalk %X 5th BCI2000 Workshop, The Sagamore Conference Center, Bolton Landing, New York %8 10/01/2009 %G eng %0 Journal Article %J Epilepsy Behav %D 2009 %T A practical procedure for real-time functional mapping of eloquent cortex using electrocorticographic signals in humans. %A Peter Brunner %A A L Ritaccio %A Lynch, Timothy M %A Emrich, Joseph F %A Adam J Wilson %A Williams, Justin C %A Aarnoutse, Erik J %A Ramsey, Nick F %A Leuthardt, E C %A H Bischof %A Gerwin Schalk %K Adult %K Brain Mapping %K Cerebral Cortex %K Electric Stimulation %K Electrodes, Implanted %K Electroencephalography %K Epilepsy %K Female %K Humans %K Male %K Middle Aged %K Practice Guidelines as Topic %K Signal Processing, Computer-Assisted %K Young Adult %X

Functional mapping of eloquent cortex is often necessary prior to invasive brain surgery, but current techniques that derive this mapping have important limitations. In this article, we demonstrate the first comprehensive evaluation of a rapid, robust, and practical mapping system that uses passive recordings of electrocorticographic signals. This mapping procedure is based on the BCI2000 and SIGFRIED technologies that we have been developing over the past several years. In our study, we evaluated 10 patients with epilepsy from four different institutions and compared the results of our procedure with the results derived using electrical cortical stimulation (ECS) mapping. The results show that our procedure derives a functional motor cortical map in only a few minutes. They also show a substantial concurrence with the results derived using ECS mapping. Specifically, compared with ECS maps, a next-neighbor evaluation showed no false negatives, and only 0.46 and 1.10% false positives for hand and tongue maps, respectively. In summary, we demonstrate the first comprehensive evaluation of a practical and robust mapping procedure that could become a new tool for planning of invasive brain surgeries.

%B Epilepsy Behav %V 15 %P 278-86 %8 07/2009 %G eng %U http://www.ncbi.nlm.nih.gov/pubmed/19366638 %N 3 %R 10.1016/j.yebeh.2009.04.001 %0 Generic %D 2009 %T Real-Time Data Acquisition, Signal Processing, and Stimulus Presentation Using BCI2000 %A Gerwin Schalk %X Workshop Neural Engineering in Real Time, Pittsburgh, Pennsylvania %8 10/07/2009 %G eng %0 Generic %D 2009 %T Real-Time Functional Mapping Using Electrocorticographic Signals %A Gerwin Schalk %X Clinical Neurophysiology Research Seminar, Langone Medical Center, New York University, New York, New York %8 11/18/2009 %G eng %0 Generic %D 2009 %T Research and Clinical Application of Electrocorticographic Signals in Humans %A Gerwin Schalk %X Helen Wills Neuroscience Institute, University of California Berkeley, Berkeley, CA %8 12/01/2009 %G eng %0 Conference Proceedings %B 13th Intl. Conference on Human-Computer Interaction %D 2009 %T Sensor Modalities for Brain-Computer Interfacing %A Gerwin Schalk %X Many people have neuromuscular conditions or disorders that impair the neural pathways that control muscles. Those most severely affected lose all voluntary muscle control and hence lose the ability to communicate. Brain-computer interfaces (BCIs) might be able to restore some communication or control functions for these people by creating a new communication channel – directly from the brain to an output device. Many studies over the past two decades have shown that such BCI communication is possible and that it can serve useful functions. This paper reviews the different sensor methodologies that have been explored in these studies. %B 13th Intl. Conference on Human-Computer Interaction %I Springer %V 5611 %G eng %R 10.1007/978-3-642-02577-8_67 %0 Generic %D 2009 %T Sensor Modalities for Brain-Computer Interfacing %A Gerwin Schalk %X Session Brain-Computer Interface (BCI); Towards Understanding Neural Bases of Human-Computer Interaction, HCI International, San Diego, California %8 07/22/2009 %G eng %0 Generic %D 2009 %T Technical Basis for Real-Time Functional Mapping of Eloquent Cortex %A Gerwin Schalk %X Department of Neurology, Weill Cornell Medical Center, New York City, New York %8 03/17/2009 %G eng %0 Generic %D 2009 %T Theory and Application of Electrocorticographic (ECoG) Signals in Humans %A Gerwin Schalk %X Invited 1.5 hour tutorial, BBCI Workshop 2009, Advances in Neurotechnology, Berlin, Germany %8 07/08/2009 %G eng %0 Journal Article %J J Vis Exp %D 2009 %T Using an EEG-based brain-computer interface for virtual cursor movement with BCI2000. %A Adam J Wilson %A Gerwin Schalk %A Walton, Léo M %A Williams, Justin C %K Brain %K Calibration %K Electrodes %K Electroencephalography %K Humans %K User-Computer Interface %X

A brain-computer interface (BCI) functions by translating a neural signal, such as the electroencephalogram (EEG), into a signal that can be used to control a computer or other device. The amplitude of the EEG signals in selected frequency bins are measured and translated into a device command, in this case the horizontal and vertical velocity of a computer cursor. First, the EEG electrodes are applied to the user s scalp using a cap to record brain activity. Next, a calibration procedure is used to find the EEG electrodes and features that the user will learn to voluntarily modulate to use the BCI. In humans, the power in the mu (8-12 Hz) and beta (18-28 Hz) frequency bands decrease in amplitude during a real or imagined movement. These changes can be detected in the EEG in real-time, and used to control a BCI ([1],[2]). Therefore, during a screening test, the user is asked to make several different imagined movements with their hands and feet to determine the unique EEG features that change with the imagined movements. The results from this calibration will show the best channels to use, which are configured so that amplitude changes in the mu and beta frequency bands move the cursor either horizontally or vertically. In this experiment, the general purpose BCI system BCI2000 is used to control signal acquisition, signal processing, and feedback to the user [3].

%B J Vis Exp %8 07/2009 %G eng %U http://www.ncbi.nlm.nih.gov/pubmed/19641479 %N 29 %R 10.3791/1319 %0 Generic %D 2009 %T Using Electrocorticographic (ECoG) Signals in Humans for Communication and Diagnosis %A Gerwin Schalk %X University of Pennsylvania, Philadelphia, Pennsylvania %8 03/25/2009 %G eng %0 Generic %D 2009 %T Using Subdural Signals in Humans for Communication and Diagnosis %A Gerwin Schalk %X Epilepsy Research Program, Georgetown University, Washington, DC. %8 11/10/2009 %G eng %0 Journal Article %J J Neurosci %D 2008 %T Advanced neurotechnologies for chronic neural interfaces: new horizons and clinical opportunities. %A Kipke, Daryl R %A Shain, William %A Buzsáki, György %A Fetz, Eberhard E %A Henderson, Jaimie M %A Hetke, Jamille F %A Gerwin Schalk %K Cerebral Cortex %K Electrodes, Implanted %K Electroencephalography %K Electronics, Medical %K Electrophysiology %K Evoked Potentials %K Movement Disorders %K Neurons %K Prostheses and Implants %K User-Computer Interface %B J Neurosci %V 28 %P 11830-8 %8 11/2008 %G eng %U http://www.ncbi.nlm.nih.gov/pubmed/19005048?report=abstract %N 46 %R 10.1523/JNEUROSCI.3879-08.2008 %0 Generic %D 2008 %T BCI2000: A General-Purpose BCI System and its Application to ECoG Signals %A Gerwin Schalk %X g.tec Brain-Computer Interface Workshop, Society for Neuroscience Annual Meeting, Washington, DC %8 11/15/2008 %G eng %0 Generic %D 2008 %T BCI2000: A General-Purpose Brain-Computer Interface System %A Gerwin Schalk %X 4th BCI2000 Workshop, Utrecht, The Netherlands %8 07/05/2008 %G eng %0 Generic %D 2008 %T The BCI2000 Framework %A Gerwin Schalk %X 4th BCI2000 Workshop, Utrecht, The Netherlands %8 07/06/2008 %G eng %0 Generic %D 2008 %T Brain-Based Communication and Orientation %A Gerwin Schalk %X Multi-disciplinary University Research Initiative sponsored by the US Army Research Office. University of Maryland College Park, College Park, Maryland. %8 09/17/2008 %G eng %0 Journal Article %J J Neurosci Methods %D 2008 %T Brain-computer interfaces (BCIs): Detection Instead of Classification. %A Gerwin Schalk %A Peter Brunner %A Lester A Gerhardt %A H Bischof %A Jonathan Wolpaw %K Adult %K Algorithms %K Brain %K Brain Mapping %K Electrocardiography %K Electroencephalography %K Humans %K Male %K Man-Machine Systems %K Normal Distribution %K Online Systems %K Signal Detection, Psychological %K Signal Processing, Computer-Assisted %K Software Validation %K User-Computer Interface %X

Many studies over the past two decades have shown that people can use brain signals to convey their intent to a computer through brain-computer interfaces (BCIs). These devices operate by recording signals from the brain and translating these signals into device commands. They can be used by people who are severely paralyzed to communicate without any use of muscle activity. One of the major impediments in translating this novel technology into clinical applications is the current requirement for preliminary analyses to identify the brain signal features best suited for communication. This paper introduces and validates signal detection, which does not require such analysis procedures, as a new concept in BCI signal processing. This detection concept is realized with Gaussian mixture models (GMMs) that are used to model resting brain activity so that any change in relevant brain signals can be detected. It is implemented in a package called SIGFRIED (SIGnal modeling For Real-time Identification and Event Detection). The results indicate that SIGFRIED produces results that are within the range of those achieved using a common analysis strategy that requires preliminary identification of signal features. They indicate that such laborious analysis procedures could be replaced by merely recording brain signals during rest. In summary, this paper demonstrates how SIGFRIED could be used to overcome one of the present impediments to translation of laboratory BCI demonstrations into clinically practical applications.

%B J Neurosci Methods %V 167 %P 51-62 %8 01/2008 %G eng %U http://www.ncbi.nlm.nih.gov/pubmed/17920134 %N 1 %R 10.1016/j.jneumeth.2007.08.010 %0 Generic %D 2008 %T Brain-Computer Interfacing Using Electrocorticography and Electrocorticography %A Gerwin Schalk %X EuroNeuro (European Congress on Neurology, Neurosurgery, Intensive Care and Anesthesiology), Maastricht, The Netherlands %8 01/18/2008 %G eng %0 Generic %D 2008 %T Brain-Computer Interfacing Using Electrocorticography (ECoG) %A Gerwin Schalk %X Bernstein Seminar, Bernstein Center for Computation Neuroscience, Albert-Ludwigs-Universitaet Freiburg, Freiburg, Germany %8 01/22/2008 %G eng %0 Generic %D 2008 %T Brain-Computer Interfacing Using Electrocorticography (ECoG) %A Gerwin Schalk %X Institute for Automation, University of Bremen, Bremen, Germany %8 01/14/2008 %G eng %0 Generic %D 2008 %T Brain-Computer Interfacing Using Electrocorticography (ECoG) %A Gerwin Schalk %X University of Pittsburgh, Pittsburgh, Pennsylvania %8 10/30/2008 %G eng %0 Generic %D 2008 %T Brain-Computer Interfacing Using Electrocorticography (ECoG) %A Gerwin Schalk %X MEG Center, Eberhard Karls University of Tübingen, Tübingen, Germany %8 01/24/2008 %G eng %0 Generic %D 2008 %T Brain-Computer Interfacing Using Non-Invasive and Invasive Methods %A Gerwin Schalk %X Workshop Brain-Computer Interfacing in 2008, Utrecht, The Netherlands %8 07/03/2008 %G eng %0 Generic %D 2008 %T Brain-Computer Interfacing Using Non-Invasive and Invasive Methods %A Gerwin Schalk %X Institute for Computer Graphics and Vision, Graz University of Technology, Graz, Austria %8 07/14/2008 %G eng %0 Journal Article %J J Neural Eng %D 2008 %T Brain-computer symbiosis. %A Gerwin Schalk %K Brain %K Computers %K Humans %K User-Computer Interface %X

The theoretical groundwork of the 1930s and 1940s and the technical advance of computers in the following decades provided the basis for dramatic increases in human efficiency. While computers continue to evolve, and we can still expect increasing benefits from their use, the interface between humans and computers has begun to present a serious impediment to full realization of the potential payoff. This paper is about the theoretical and practical possibility that direct communication between the brain and the computer can be used to overcome this impediment by improving or augmenting conventional forms of human communication. It is about the opportunity that the limitations of our body's input and output capacities can be overcome using direct interaction with the brain, and it discusses the assumptions, possible limitations and implications of a technology that I anticipate will be a major source of pervasive changes in the coming decades.

%B J Neural Eng %V 5 %P P1-P15 %8 03/2008 %G eng %U http://www.ncbi.nlm.nih.gov/pubmed/18310804 %N 1 %R 10.1088/1741-2560/5/1/P01 %0 Generic %D 2008 %T Combining Fidelity With Practicality: Interrogation of the Brain Using Electrocorticography (ECoG) %A Gerwin Schalk %X Symposium Advanced Neurotechnologies for Chronic Neural Interfaces: New Horizons and Clinical Opportunities, Society for Neuroscience Annual Meeting, Washington, DC %8 11/17/2008 %G eng %0 Generic %D 2008 %T Decoding Detailed Information fromt he Brain Using Electrocorticographic (ECoG) Signals in Humans %A Gerwin Schalk %X Lecture in Workshop on Research Efforts and Future Directions in Neuroergonamics and Neuromorphics sponsored by the US Army Research Office. University of Maryland Colelge Park, College Park, Maryland %8 10/24/2008 %G eng %0 Generic %D 2008 %T A Device For Real-Time Functional Mapping Using ECoG %A Gerwin Schalk %X CIMIT Epilepsy Innovation Summit, Boston, Massachusetts %8 05/07/2008 %G eng %0 Generic %D 2008 %T Direct Interaction With The Brain %A Gerwin Schalk %X Guest lecture in Information Technology Capstone Course, Department of Information Technology, Rensselaer Polytechnic Institute, Troy, New York, 11/24/2008 %8 11/24/2008 %G eng %0 Generic %D 2008 %T Direct Interaction With The Brain %A Gerwin Schalk %X Guest lecture in Business Issues for Engineers and Scientists, Department of Information Technology, Rensselaer Polytechnic Institute, Troy, New York %8 10/08/2008 %G eng %0 Book Section %B Neuroengineering %D 2008 %T General Clinical Issues Relevant to Brain-Computer Interfaces. %A Leuthardt, E C %A Ojemann, J G %A Gerwin Schalk %A Moran, D %E Daniel DiLorenzo %B Neuroengineering %I Taylor and Francis Group %C Boca Raton %G eng %0 Generic %D 2008 %T Inferring Details of Motor/Language Function Using ECoG Signals in Humans %A Gerwin Schalk %X Workshop Advances in Theory and Clinical Application of Subdural Recordings, American Epilepsy Society Annual Meeting, Seattle, Washington %8 12/09/2008 %G eng %0 Generic %D 2008 %T Movement Control Using Field Potentials Recorded on the Surface of the Brain %A Gerwin Schalk %X Workshop on Real-Time Brain Interfacing Applications, Mathematical Biosciences Institute, The Ohio State University, Columbus, Ohio %8 05/12/2008 %G eng %0 Journal Article %J Brain Res Bull %D 2008 %T Non-invasive brain-computer interface system: towards its application as assistive technology. %A Cincotti, F %A Mattia, Donatella %A Aloise, Fabio %A Bufalari, Simona %A Gerwin Schalk %A Oriolo, Giuseppe %A Cherubini, Andrea %A Marciani, Maria Grazia %A Babiloni, Fabio %K Activities of Daily Living %K Adolescent %K Adult %K Brain %K Child %K Electroencephalography %K Evoked Potentials, Motor %K Female %K Humans %K Learning %K Male %K Middle Aged %K Motor Skills %K Muscular Dystrophy, Duchenne %K Pilot Projects %K Prostheses and Implants %K Robotics %K Self-Help Devices %K Software %K Spinal Muscular Atrophies of Childhood %K User-Computer Interface %K Volition %X

The quality of life of people suffering from severe motor disabilities can benefit from the use of current assistive technology capable of ameliorating communication, house-environment management and mobility, according to the user's residual motor abilities. Brain-computer interfaces (BCIs) are systems that can translate brain activity into signals that control external devices. Thus they can represent the only technology for severely paralyzed patients to increase or maintain their communication and control options. Here we report on a pilot study in which a system was implemented and validated to allow disabled persons to improve or recover their mobility (directly or by emulation) and communication within the surrounding environment. The system is based on a software controller that offers to the user a communication interface that is matched with the individual's residual motor abilities. Patients (n=14) with severe motor disabilities due to progressive neurodegenerative disorders were trained to use the system prototype under a rehabilitation program carried out in a house-like furnished space. All users utilized regular assistive control options (e.g., microswitches or head trackers). In addition, four subjects learned to operate the system by means of a non-invasive EEG-based BCI. This system was controlled by the subjects' voluntary modulations of EEG sensorimotor rhythms recorded on the scalp; this skill was learnt even though the subjects have not had control over their limbs for a long time. We conclude that such a prototype system, which integrates several different assistive technologies including a BCI system, can potentially facilitate the translation from pre-clinical demonstrations to a clinical useful BCI.

%B Brain Res Bull %V 75 %P 796-803 %8 04/2008 %G eng %U http://www.ncbi.nlm.nih.gov/pubmed/18394526 %N 6 %R 10.1016/j.brainresbull.2008.01.007 %0 Generic %D 2008 %T Overview of Available BCI2000 Components %A Gerwin Schalk %X 4th BCI2000 Workshop, Utrecht, The Netherlands %8 07/06/2008 %G eng %0 Journal Article %J Neuroimage %D 2008 %T Real-time detection of event-related brain activity. %A Gerwin Schalk %A Leuthardt, E C %A Peter Brunner %A Ojemann, J G %A Lester A Gerhardt %A Jonathan Wolpaw %K Adult %K Algorithms %K Brain Mapping %K Computer Systems %K Diagnosis, Computer-Assisted %K Electroencephalography %K Epilepsy %K Evoked Potentials %K Female %K Humans %K Male %K Pattern Recognition, Automated %K Reproducibility of Results %K Sensitivity and Specificity %X

The complexity and inter-individual variation of brain signals impedes real-time detection of events in raw signals. To convert these complex signals into results that can be readily understood, current approaches usually apply statistical methods to data from known conditions after all data have been collected. The capability to provide meaningful visualization of complex brain signals without the requirement to initially collect data from all conditions would provide a new tool, essentially a new imaging technique, that would open up new avenues for the study of brain function. Here we show that a new analysis approach, called SIGFRIED, can overcome this serious limitation of current methods. SIGFRIED can visualize brain signal changes without requiring prior data collection from all conditions. This capacity is particularly well suited to applications in which comprehensive prior data collection is impossible or impractical, such as intraoperative localization of cortical function or detection of epileptic seizures.

%B Neuroimage %V 43 %P 245-9 %8 11/2008 %G eng %U http://www.ncbi.nlm.nih.gov/pubmed/18718544 %N 2 %R 10.1016/j.neuroimage.2008.07.037 %0 Generic %D 2008 %T Theory and Application of Subdural Recordings in Humans %A Gerwin Schalk %X Neurosciences Grand Rounds, The Neurosciences Institute, Albany Medical Center, Albany, New York %8 12/18/2008 %G eng %0 Conference Proceedings %B Conf Proc IEEE Eng Med Biol Soc %D 2008 %T Three cases of feature correlation in an electrocorticographic BCI. %A Miller, John W %A Blakely, Timothy %A Gerwin Schalk %A den Nijs, Marcel %A Rao, Rajesh P N %A Ojemann, J G %K Adolescent %K Adult %K Algorithms %K Electrocardiography %K Evoked Potentials, Motor %K Female %K Humans %K Male %K Middle Aged %K Motor Cortex %K Pattern Recognition, Automated %K Statistics as Topic %K Task Performance and Analysis %K User-Computer Interface %X Three human subjects participated in a closed-loop brain computer interface cursor control experiment mediated by implanted subdural electrocorticographic arrays. The paradigm consisted of several stages: baseline recording, hand and tongue motor tasks as the basis for feature selection, two closed-loop one-dimensional feedback experiments with each of these features, and a two-dimensional feedback experiment using both of the features simultaneously. The two selected features were simple channel and frequency band combinations associated with change during hand and tongue movement. Inter-feature correlation and cross-correlation between features during different epochs of each task were quantified for each stage of the experiment. Our anecdotal, three subject, result suggests that while high correlation between horizontal and vertical control signal can initially preclude successful two-dimensional cursor control, a feedback-based learning strategy can be successfully employed by the subject to overcome this limitation and progressively decorrelate these control signals. %B Conf Proc IEEE Eng Med Biol Soc %P 5318-21 %8 2008 %G eng %R 10.1109/IEMBS.2008.4650415 %0 Conference Paper %B Engineering in Medicine and Biology Society, 2008. %D 2008 %T Three cases of feature correlation in an electrocorticographic BCI. %A Miller, Kai J %A Blakely, Timothy %A Gerwin Schalk %A den Nijs, Marcel %A Rao, Rajesh PN %A Ojemann, Jeffrey G %K Adolescent %K Adult %K Algorithms %K automated pattern recognition %K control systems %K decorrelation %K Electrocardiography %K Electrodes %K Electroencephalography %K evoked motor potentials %K Feedback %K Female %K frequency %K hospitals %K Humans %K Male %K Middle Aged %K Motor Cortex %K Signal Processing %K Statistics as Topic %K Task Performance and Analysis %K Tongue %K User-Computer Interface %X Three human subjects participated in a closed-loop brain computer interface cursor control experiment mediated by implanted subdural electrocorticographic arrays. The paradigm consisted of several stages: baseline recording, hand and tongue motor tasks as the basis for feature selection, two closed-loop one-dimensional feedback experiments with each of these features, and a two-dimensional feedback experiment using both of the features simultaneously. The two selected features were simple channel and frequency band combinations associated with change during hand and tongue movement. Inter-feature correlation and cross-correlation between features during different epochs of each task were quantified for each stage of the experiment. Our anecdotal, three subject, result suggests that while high correlation between horizontal and vertical control signal can initially preclude successful two-dimensional cursor control, a feedback-based learning strategy can be successfully employed by the subject to overcome this limitation and progressively decorrelate these control signals. %B Engineering in Medicine and Biology Society, 2008. %I IEEE %C Vancouver, BC %8 08/2008 %G eng %U http://www.ncbi.nlm.nih.gov/pubmed/19163918 %R 10.1109/IEMBS.2008.4650415 %0 Journal Article %J Clin Neurophysiol %D 2008 %T Towards an independent brain-computer interface using steady state visual evoked potentials. %A Brendan Z. Allison %A Dennis J. McFarland %A Gerwin Schalk %A Zheng, Shi Dong %A Moore-Jackson, Melody %A Jonathan Wolpaw %K Adolescent %K Adult %K Attention %K Brain %K Brain Mapping %K Dose-Response Relationship, Radiation %K Electroencephalography %K Evoked Potentials, Visual %K Female %K Humans %K Male %K Pattern Recognition, Visual %K Photic Stimulation %K Spectrum Analysis %K User-Computer Interface %X

OBJECTIVE: 

Brain-computer interface (BCI) systems using steady state visual evoked potentials (SSVEPs) have allowed healthy subjects to communicate. However, these systems may not work in severely disabled users because they may depend on gaze shifting. This study evaluates the hypothesis that overlapping stimuli can evoke changes in SSVEP activity sufficient to control a BCI. This would provide evidence that SSVEP BCIs could be used without shifting gaze.

METHODS: 

Subjects viewed a display containing two images that each oscillated at a different frequency. Different conditions used overlapping or non-overlapping images to explore dependence on gaze function. Subjects were asked to direct attention to one or the other of these images during each of 12 one-minute runs.

RESULTS: 

Half of the subjects produced differences in SSVEP activity elicited by overlapping stimuli that could support BCI control. In all remaining users, differences did exist at corresponding frequencies but were not strong enough to allow effective control.

CONCLUSIONS: 

The data demonstrate that SSVEP differences sufficient for BCI control may be elicited by selective attention to one of two overlapping stimuli. Thus, some SSVEP-based BCI approaches may not depend on gaze control. The nature and extent of any BCI's dependence on muscle activity is a function of many factors, including the display, task, environment, and user.

SIGNIFICANCE: 

SSVEP BCIs might function in severely disabled users unable to reliably control gaze. Further research with these users is necessary to explore the optimal parameters of such a system and validate online performance in a home environment.

%B Clin Neurophysiol %V 119 %P 399-408 %8 02/2008 %G eng %U http://www.ncbi.nlm.nih.gov/pubmed/18077208 %N 2 %R 10.1016/j.clinph.2007.09.121 %0 Journal Article %J J Neural Eng %D 2008 %T Two-dimensional movement control using electrocorticographic signals in humans. %A Gerwin Schalk %A Miller, K.J. %A Nicholas R Anderson %A Adam J Wilson %A Smyth, Matt %A Ojemann, J G %A Moran, D %A Jonathan Wolpaw %A Leuthardt, E C %K Adolescent %K Adult %K Brain Mapping %K Data Interpretation, Statistical %K Drug Resistance %K Electrocardiography %K Electrodes, Implanted %K Electroencephalography %K Epilepsy %K Female %K Humans %K Male %K Movement %K User-Computer Interface %X

We show here that a brain-computer interface (BCI) using electrocorticographic activity (ECoG) and imagined or overt motor tasks enables humans to control a computer cursor in two dimensions. Over a brief training period of 12-36 min, each of five human subjects acquired substantial control of particular ECoG features recorded from several locations over the same hemisphere, and achieved average success rates of 53-73% in a two-dimensional four-target center-out task in which chance accuracy was 25%. Our results support the expectation that ECoG-based BCIs can combine high performance with technical and clinical practicality, and also indicate promising directions for further research.

%B J Neural Eng %V 5 %P 75-84 %8 03/2008 %G eng %U http://www.ncbi.nlm.nih.gov/pubmed/18310813 %N 1 %R 10.1088/1741-2560/5/1/008 %0 Journal Article %J Stroke %D 2008 %T Unique cortical physiology associated with ipsilateral hand movements and neuroprosthetic implications. %A Wisneski, Kimberly %A Nicholas R Anderson %A Gerwin Schalk %A Smyth, Matt %A Moran, D %A Leuthardt, E C %K Adolescent %K Adult %K Artificial Limbs %K Bionics %K Brain Mapping %K Child %K Dominance, Cerebral %K Electroencephalography %K Female %K Hand %K Humans %K Male %K Middle Aged %K Motor Cortex %K Movement %K Paresis %K Prosthesis Design %K Psychomotor Performance %K Stroke %K User-Computer Interface %K Volition %X

BACKGROUND AND PURPOSE: 

Brain computer interfaces (BCIs) offer little direct benefit to patients with hemispheric stroke because current platforms rely on signals derived from the contralateral motor cortex (the same region injured by the stroke). For BCIs to assist hemiparetic patients, the implant must use unaffected cortex ipsilateral to the affected limb. This requires the identification of distinct electrophysiological features from the motor cortex associated with ipsilateral hand movements.

METHODS: 

In this study we studied 6 patients undergoing temporary placement of intracranial electrode arrays. Electrocorticographic (ECoG) signals were recorded while the subjects engaged in specific ipsilateral or contralateral hand motor tasks. Spectral changes were identified with regards to frequency, location, and timing.

RESULTS: 

Ipsilateral hand movements were associated with electrophysiological changes that occur in lower frequency spectra, at distinct anatomic locations, and earlier than changes associated with contralateral hand movements. In a subset of 3 patients, features specific to ipsilateral and contralateral hand movements were used to control a cursor on a screen in real time. In ipsilateral derived control this was optimal with lower frequency spectra.

CONCLUSIONS: 

There are distinctive cortical electrophysiological features associated with ipsilateral movements which can be used for device control. These findings have implications for patients with hemispheric stroke because they offer a potential methodology for which a single hemisphere can be used to enhance the function of a stroke induced hemiparesis.

%B Stroke %V 39 %P 3351-9 %8 12/2008 %G eng %U http://www.ncbi.nlm.nih.gov/pubmed/18927456 %N 12 %R 10.1161/STROKEAHA.108.518175 %0 Generic %D 2008 %T Using Brain Signals for Clinical Diagnosis and Communication %A Gerwin Schalk %X Center for Neuropharmacology & Neuroscience, Albany Medical College, Albany, New York %8 09/24/2008 %G eng %0 Generic %D 2008 %T Using Electrocorticographic (ECoG) Signals in Humans for Communication and Diagnosis %A Gerwin Schalk %X Department of Physiology and Biophysics, University of Washington, Seattle, Washington %8 12/10/2008 %G eng %0 Generic %D 2008 %T Using Electrocorticographic (ECoG) Signals in Humans for Communication and Diagnosis %A Gerwin Schalk %X Technical University of Berlin, Berlin, Germany %8 12/01/2008 %G eng %0 Generic %D 2008 %T Using Electrocorticography for Brain-Computer Interfacing and Detailed Single-Trial Decoding of Human Behavior %A Gerwin Schalk %X Brain Gain Lecture, F.C. Donders Center for Cognitive Neuroimaging, Radboud University Nijmegen, Nijmegen, The Netherlands %8 01/15/2008 %G eng %0 Generic %D 2007 %T BCI2000: A General-Purpose Brain-Computer Interface System %A Gerwin Schalk %X 2nd BCI2000 Workshop, Beijing, China %8 07/24/2007 %G eng %0 Book Section %B Brain-Computer Interfaces %D 2007 %T BCI2000: A General-Purpose Software Platform for BCI Research. %A Mellinger, Jürgen %A Gerwin Schalk %B Brain-Computer Interfaces %I MIT Press %G eng %0 Generic %D 2007 %T Brain-Computer Interfaces: Controlling A Computer With Your Thought %A Gerwin Schalk %X Science Today Seminar Series. Bethlehem High School, Delmar, New York. %8 10/25/2007 %G eng %0 Generic %D 2007 %T Brain-Computer Interfacing Using Electrocorticography (ECoG) %A Gerwin Schalk %X International Workshop on Brain-Computer Interface Technology, Beijing, China %8 07/23/2007 %G eng %0 Generic %D 2007 %T Brain-Computer Interfacing Using Non-Invasive and Invasive Methods %A Gerwin Schalk %X Computer Science and Electrical Engineering Department, Orgegon Health & Science University, Beaverton, Oregon %8 12/04/2007 %G eng %0 Generic %D 2007 %T Brain-Computer Interfacing Using Non-Invasive, Intra-Cortical, and Subdural Methods %A Gerwin Schalk %X Max-Planck-Institute for Brain Research, Frankfurt, Germany %8 09/25/2007 %G eng %0 Generic %D 2007 %T Communicating Directly from the Brain: Brain-Computer Interfaces (BCIs) %A Gerwin Schalk %X Wadsworth Center Research Experience for Undergraduates Program, Albany, New York %8 06/19/2007 %G eng %0 Journal Article %J J Neural Eng %D 2007 %T Decoding two-dimensional movement trajectories using electrocorticographic signals in humans. %A Gerwin Schalk %A Kubánek, J %A Miller, John W %A Nicholas R Anderson %A Leuthardt, E C %A Ojemann, J G %A Limbrick, D %A Moran, D %A Lester A Gerhardt %A Jonathan Wolpaw %K Adult %K Algorithms %K Arm %K Brain Mapping %K Cerebral Cortex %K Electroencephalography %K Evoked Potentials, Motor %K Female %K Humans %K Male %K Movement %X

Signals from the brain could provide a non-muscular communication and control system, a brain-computer interface (BCI), for people who are severely paralyzed. A common BCI research strategy begins by decoding kinematic parameters from brain signals recorded during actual arm movement. It has been assumed that these parameters can be derived accurately only from signals recorded by intracortical microelectrodes, but the long-term stability of such electrodes is uncertain. The present study disproves this widespread assumption by showing in humans that kinematic parameters can also be decoded from signals recorded by subdural electrodes on the cortical surface (ECoG) with an accuracy comparable to that achieved in monkey studies using intracortical microelectrodes. A new ECoG feature labeled the local motor potential (LMP) provided the most information about movement. Furthermore, features displayed cosine tuning that has previously been described only for signals recorded within the brain. These results suggest that ECoG could be a more stable and less invasive alternative to intracortical electrodes for BCI systems, and could also prove useful in studies of motor function.

%B J Neural Eng %V 4 %P 264-75 %8 09/2007 %G eng %U http://www.ncbi.nlm.nih.gov/pubmed/17873429 %N 3 %R 10.1088/1741-2560/4/3/012 %0 Generic %D 2007 %T Definition of Motor Responses and Brain-Computer Interfacing Using Electrocorticography %A Gerwin Schalk %X Neuroscience Group, Laborator of Nervous System Disorders, Wadsworth Center, New York State Department of Health, Albany New York %8 07/06/2007 %G eng %0 Generic %D 2007 %T Direct Communication From the Brain %A Gerwin Schalk %X Guest lecture for BS in IT Capstone Course. Department of Information Technology, Rensselaer Polytechnic Institute, Troy, New York %8 11/19/2007 %G eng %0 Journal Article %J Neurosurgery %D 2007 %T Electrocorticographic Frequency Alteration Mapping: A Clinical Technique for Mapping the Motor Cortex. %A Leuthardt, E C %A Miller, John W %A Nicholas R Anderson %A Gerwin Schalk %A Dowling, Joshua %A Miller, John W %A Moran, D %A Ojemann, J G %K Adult %K Biological Clocks %K Brain Mapping %K Electric Stimulation %K Electrodes, Implanted %K Electroencephalography %K Female %K Hand %K Humans %K Male %K Middle Aged %K Motor Cortex %K Oscillometry %K Signal Processing, Computer-Assisted %K Tongue %X

OBJECTIVE: 

Electrocortical stimulation (ECS) has been well established for delineating the eloquent cortex. However, ECS is still coarse and inefficient in delineating regions of the functional cortex and can be hampered by after-discharges. Given these constraints, an adjunct approach to defining the motor cortex is the use of electrocorticographic signal changes associated with active regions of the cortex. The broad range of frequency oscillations are categorized into two main groups with respect to the sensorimotor cortex: low and high frequency bands. The low frequency bands tend to show a power reduction with cortical activation, whereas the high frequency bands show power increases. These power changes associated with the activated cortex could potentially provide a powerful tool in delineating areas of the motor cortex. We explore electrocorticographic signal alterations as they occur with activated regions of the motor cortex, as well as its potential in clinical brain mapping applications.

METHODS: 

We evaluated seven patients who underwent invasive monitoring for seizure localization. Each patient had extraoperative ECS mapping to identify the motor cortex. All patients also performed overt hand and tongue motor tasks to identify associated frequency power changes in regard to location and degree of concordance with ECS results that localized either hand or tongue motor function.

RESULTS: 

The low frequency bands had a high sensitivity (88.9-100%) and a lower specificity (79.0-82.6%) for identifying electrodes with either hand or tongue ECS motor responses. The high frequency bands had a lower sensitivity (72.7-88.9%) and a higher specificity (92.4-94.9%) in correlation with the same respective ECS positive electrodes.

CONCLUSION: 

The concordance between stimulation and spectral power changes demonstrate the possible utility of electrocorticographic frequency alteration mapping as an adjunct method to improve the efficiency and resolution of identifying the motor cortex.

%B Neurosurgery %V 60 %P 260-70; discussion 270-1 %8 04/2007 %G eng %U http://www.ncbi.nlm.nih.gov/pubmed/17415162 %N 4 Suppl 2 %R 10.1227/01.NEU.0000255413.70807.6E %0 Generic %D 2007 %T Electrocorticography (ECoG) for Feedback and Decoding of Function %A Gerwin Schalk %X Workshop on Large Scale Brain Dynamics, Neural Information Processing Systems (NIPS), Whistler, British Columbia, Canada %8 12/08/2007 %G eng %0 Generic %D 2007 %T Electrocorticography for Brain-Computer Interfacing and Motor/Language Mapping %A Gerwin Schalk %X Neurosciences Grand Rounds, The Neurosciences Institute, Albany Medical Center, Albany, New York. %8 02/22/2007 %G eng %0 Journal Article %J Neuroimage %D 2007 %T An MEG-based brain-computer interface (BCI). %A Mellinger, Jürgen %A Gerwin Schalk %A Christoph Braun %A Preissl, Hubert %A Rosenstiel, W. %A Niels Birbaumer %A Kübler, A. %K Adult %K Algorithms %K Artifacts %K Brain %K Electroencephalography %K Electromagnetic Fields %K Electromyography %K Feedback %K Female %K Foot %K Hand %K Head Movements %K Humans %K Magnetic Resonance Imaging %K Magnetoencephalography %K Male %K Movement %K Principal Component Analysis %K Signal Processing, Computer-Assisted %K User-Computer Interface %X

Brain-computer interfaces (BCIs) allow for communicating intentions by mere brain activity, not involving muscles. Thus, BCIs may offer patients who have lost all voluntary muscle control the only possible way to communicate. Many recent studies have demonstrated that BCIs based on electroencephalography(EEG) can allow healthy and severely paralyzed individuals to communicate. While this approach is safe and inexpensive, communication is slow. Magnetoencephalography (MEG) provides signals with higher spatiotemporal resolution than EEG and could thus be used to explore whether these improved signal properties translate into increased BCI communication speed. In this study, we investigated the utility of an MEG-based BCI that uses voluntary amplitude modulation of sensorimotor mu and beta rhythms. To increase the signal-to-noise ratio, we present a simple spatial filtering method that takes the geometric properties of signal propagation in MEG into account, and we present methods that can process artifacts specifically encountered in an MEG-based BCI. Exemplarily, six participants were successfully trained to communicate binary decisions by imagery of limb movements using a feedback paradigm. Participants achieved significant mu rhythm self control within 32 min of feedback training. For a subgroup of three participants, we localized the origin of the amplitude modulated signal to the motor cortex. Our results suggest that an MEG-based BCI is feasible and efficient in terms of user training.

%B Neuroimage %V 36 %P 581-93 %8 07/2007 %G eng %U http://www.ncbi.nlm.nih.gov/pubmed/17475511 %N 3 %R 10.1016/j.neuroimage.2007.03.019 %0 Conference Proceedings %B Conf Proc IEEE Eng Med Biol Soc %D 2007 %T Non-invasive brain-computer interface system to operate assistive devices. %A Cincotti, F %A Aloise, Fabio %A Bufalari, Simona %A Gerwin Schalk %A Oriolo, Giuseppe %A Cherubini, Andrea %A Davide, Fabrizio %A Babiloni, Fabio %A Marciani, Maria Grazia %A Mattia, Donatella %K Brain %K Communication Aids for Disabled %K Computer Systems %K Humans %K Neurodegenerative Diseases %K Quality of Life %K Self-Help Devices %K Software %K User-Computer Interface %X In this pilot study, a system that allows disabled persons to improve or recover their mobility and communication within the surrounding environment was implemented and validated. The system is based on a software controller that offers to the user a communication interface that is matched with the individual's residual motor abilities. Fourteen patients with severe motor disabilities due to progressive neurodegenerative disorders were trained to use the system prototype under a rehabilitation program. All users utilized regular assistive control options (e.g., microswitches or head trackers) while four patients learned to operate the system by means of a non-invasive EEG-based Brain-Computer Interface, based on the subjects' voluntary modulations of EEG sensorimotor rhythms recorded on the scalp. %B Conf Proc IEEE Eng Med Biol Soc %P 2532-5 %8 04/2007 %G eng %R 10.1109/IEMBS.2007.4352844 %0 Journal Article %J J Neurosci %D 2007 %T Spectral Changes in Cortical Surface Potentials During Motor Movement. %A Miller, John W %A Leuthardt, E C %A Gerwin Schalk %A Rao, Rajesh P N %A Nicholas R Anderson %A Moran, D %A Miller, John W %A Ojemann, J G %K Adult %K Brain Mapping %K Female %K Humans %K Male %K Middle Aged %K Motor Cortex %K Movement %X

In the first large study of its kind, we quantified changes in electrocorticographic signals associated with motor movement across 22 subjects with subdural electrode arrays placed for identification of seizure foci. Patients underwent a 5-7 d monitoring period with array placement, before seizure focus resection, and during this time they participated in the study. An interval-based motor-repetition task produced consistent and quantifiable spectral shifts that were mapped on a Talairach-standardized template cortex. Maps were created independently for a high-frequency band (HFB) (76-100 Hz) and a low-frequency band (LFB) (8-32 Hz) for several different movement modalities in each subject. The power in relevant electrodes consistently decreased in the LFB with movement, whereas the power in the HFB consistently increased. In addition, the HFB changes were more focal than the LFB changes. Sites of power changes corresponded to stereotactic locations in sensorimotor cortex and to the results of individual clinical electrical cortical mapping. Sensorimotor representation was found to be somatotopic, localized in stereotactic space to rolandic cortex, and typically followed the classic homunculus with limited extrarolandic representation.

%B J Neurosci %V 27 %P 2424-32 %8 02/2007 %G eng %U http://www.ncbi.nlm.nih.gov/pubmed/17329441 %N 9 %R 10.1523/JNEUROSCI.3886-06.2007 %0 Journal Article %J IEEE Trans Biomed Eng %D 2007 %T A µ-rhythm Matched Filter for Continuous Control of a Brain-Computer Interface. %A Krusienski, Dean J %A Gerwin Schalk %A Dennis J. McFarland %A Jonathan Wolpaw %K Algorithms %K Cerebral Cortex %K Cortical Synchronization %K Electroencephalography %K Evoked Potentials %K Humans %K Imagination %K Pattern Recognition, Automated %K User-Computer Interface %X

A brain-computer interface (BCI) is a system that provides an alternate nonmuscular communication/control channel for individuals with severe neuromuscular disabilities. With proper training, individuals can learn to modulate the amplitude of specific electroencephalographic (EEG) components (e.g., the 8-12 Hz mu rhythm and 18-26 Hz beta rhythm) over the sensorimotor cortex and use them to control a cursor on a computer screen. Conventional spectral techniques for monitoring the continuousamplitude fluctuations fail to capture essential amplitude/phase relationships of the mu and beta rhythms in a compact fashion and, therefore, are suboptimal. By extracting the characteristic mu rhythm for a user, the exact morphology can be characterized and exploited as a matched filter. A simple, parameterized model for the characteristic mu rhythm is proposed and its effectiveness as a matched filter is examined online for a one-dimensional cursor control task. The results suggest that amplitude/phase coupling exists between the mu and beta bands during event-related desynchronization, and that an appropriate matched filter can provide improved performance.

%B IEEE Trans Biomed Eng %V 54 %P 273-80 %8 02/2007 %G eng %U http://www.ncbi.nlm.nih.gov/pubmed/17278584 %N 2 %R 10.1109/TBME.2006.886661 %0 Journal Article %J IEEE Trans Neural Syst Rehabil Eng %D 2006 %T The BCI competition III: Validating alternative approaches to actual BCI problems. %A Benjamin Blankertz %A Müller, Klaus-Robert %A Krusienski, Dean J %A Gerwin Schalk %A Jonathan Wolpaw %A Schlögl, Alois %A Pfurtscheller, Gert %A Millán, José del R %A Schröder, Michael %A Niels Birbaumer %K Algorithms %K Brain %K Communication Aids for Disabled %K Databases, Factual %K Electroencephalography %K Evoked Potentials %K Humans %K Neuromuscular Diseases %K Software Validation %K Technology Assessment, Biomedical %K User-Computer Interface %X

brain-computer interface (BCI) is a system that allows its users to control external devices with brainactivity. Although the proof-of-concept was given decades ago, the reliable translation of user intent into device control commands is still a major challenge. Success requires the effective interaction of two adaptive controllers: the user's brain, which produces brain activity that encodes intent, and the BCI system, which translates that activity into device control commands. In order to facilitate this interaction, many laboratories are exploring a variety of signal analysis techniques to improve the adaptation of the BCI system to the user. In the literature, many machine learning and pattern classification algorithms have been reported to give impressive results when applied to BCI data in offline analyses. However, it is more difficult to evaluate their relative value for actual online use. BCI data competitions have been organized to provide objective formal evaluations of alternative methods. Prompted by the great interest in the first two BCI Competitions, we organized the third BCI Competition to address several of the most difficult and important analysis problems in BCI research. The paper describes the data sets that were provided to the competitors and gives an overview of the results.

%B IEEE Trans Neural Syst Rehabil Eng %V 14 %P 153-9 %8 06/2006 %G eng %U http://www.ncbi.nlm.nih.gov/pubmed/16792282 %N 2 %R 10.1109/TNSRE.2006.875642 %0 Journal Article %J IEEE Trans Neural Syst Rehabil Eng %D 2006 %T BCI meeting 2005 - Workshop on Technology: Hardware and Software. %A Cincotti, F %A Bianchi, L %A Birch, Gary %A Guger, C %A Mellinger, Jürgen %A Scherer, Reinhold %A Schmidt, Robert N %A Yáñez Suárez, Oscar %A Gerwin Schalk %K Algorithms %K Biotechnology %K Brain %K Communication Aids for Disabled %K Computers %K Electroencephalography %K Equipment Design %K Humans %K Internationality %K Man-Machine Systems %K Neuromuscular Diseases %K Software %K User-Computer Interface %X

This paper describes the outcome of discussions held during the Third International BCI Meeting at a workshop to review and evaluate the current state of BCI-related hardware and software. Technical requirements and current technologies, standardization procedures and future trends are covered. The main conclusion was recognition of the need to focus technical requirements on the users' needs and the need for consistent standards in BCI research.

%B IEEE Trans Neural Syst Rehabil Eng %V 14 %P 128-31 %8 06/2006 %G eng %U http://www.ncbi.nlm.nih.gov/pubmed/16792276 %N 2 %R 10.1109/TNSRE.2006.875584 %0 Generic %D 2006 %T BCI2000: Software for Brain-Computer Interface Research %A Gerwin Schalk %X Neuroscience Group, Laboratory of Nervous System Disorders, Wadsworth Center, New York State Department of Health, Albany, New York. %8 11/30/2006 %G eng %0 Generic %D 2006 %T Brain Interfacing with Materials %A Gerwin Schalk %X Task group presentation, National Academy of Sciences Keck Futures Initiative "Smart Prosthetics." Beckman Center, Irvine, California. %8 11/10/2006 %G eng %0 Generic %D 2006 %T Brain-Computer Interfaces (BCIs): Towards Clinical Applications %A Gerwin Schalk %X Biomedical Engineering Colloquium, Washington University in St. Louis, St. Louis, Missouri. %8 11/03/2006 %G eng %0 Generic %D 2006 %T Brain-Computer Interfaces: Challenges and Perspectives %A Gerwin Schalk %X Rudolf Magnus Institute of Neuroscience, University Medical Center, Utrecht, The Netherlands %8 09/11/2006 %G eng %0 Generic %D 2006 %T Brain-Computer Interfaces: Ready for Clinical Use? %A Gerwin Schalk %X Center for Disability Services, Albany, New York %8 03/02/2006 %G eng %0 Generic %D 2006 %T Brain-Computer Interfacing Using BCI2000 %A Gerwin Schalk %X Keynote address, g.tec Brain-Computer Interface Workshop, Graz, Austria %8 09/20/2006 %G eng %0 Generic %D 2006 %T Direct Communication From the Brain %A Gerwin Schalk %X Guest lecture for course Services Science, Management, and Engineering. Department of Information Technology, Rensselaer Polytechnic Institute, Troy, New York. %8 10/05/2006 %G eng %0 Journal Article %J IEEE Trans Neural Syst Rehabil Eng %D 2006 %T ECoG factors underlying multimodal control of a brain-computer interface. %A Adam J Wilson %A Felton, Elizabeth A %A Garell, P Charles %A Gerwin Schalk %A Williams, Justin C %K Adult %K Brain Mapping %K Cerebral Cortex %K Communication Aids for Disabled %K Computer Peripherals %K Evoked Potentials %K Female %K Humans %K Imagination %K Male %K Man-Machine Systems %K Neuromuscular Diseases %K Systems Integration %K User-Computer Interface %K Volition %X

Most current brain-computer interface (BCI) systems for humans use electroencephalographic activity recorded from the scalp, and may be limited in many ways. Electrocorticography (ECoG) is believed to be a minimally-invasive alternative to electroencephalogram (EEG) for BCI systems, yielding superior signal characteristics that could allow rapid user training and faster communication rates. In addition, our preliminary results suggest that brain regions other than the sensorimotor cortex, such as auditory cortex, may be trained to control a BCI system using similar methods as those used to train motor regions of the brain. This could prove to be vital for users who have neurological disease, head trauma, or other conditions precluding the use of sensorimotor cortex for BCI control.

%B IEEE Trans Neural Syst Rehabil Eng %V 14 %P 246-50 %8 06/2006 %G eng %U http://www.ncbi.nlm.nih.gov/pubmed/16792305 %N 2 %R 10.1109/TNSRE.2006.875570 %0 Journal Article %J IEEE Trans Neural Syst Rehabil Eng %D 2006 %T Electrocorticography-based brain computer interface--the Seattle experience. %A Leuthardt, E C %A Miller, John W %A Gerwin Schalk %A Rao, Rajesh P N %A Ojemann, J G %K Cerebral Cortex %K Electroencephalography %K Epilepsy %K Evoked Potentials %K Humans %K Therapy, Computer-Assisted %K User-Computer Interface %K Washington %X

Electrocorticography (ECoG) has been demonstrated to be an effective modality as a platform for brain-computer interfaces (BCIs). Through our experience with ten subjects, we further demonstrate evidence to support the power and flexibility of this signal for BCI usage. In a subset of four patients, closed-loop BCI experiments were attempted with the patient receiving online feedback that consisted of one-dimensional cursor movement controlled by ECoG features that had shown correlation with various real and imagined motor and speech tasks. All four achieved control, with final target accuracies between 73%-100%. We assess the methods for achieving control and the manner in which enhancing online control can be accomplished by rescreening during online tasks. Additionally, we assess the relevant issues of the current experimental paradigm in light of their clinical constraints.

%B IEEE Trans Neural Syst Rehabil Eng %V 14 %P 194-8 %8 06/2006 %G eng %U http://www.ncbi.nlm.nih.gov/pubmed/16792292 %N 2 %R 10.1109/TNSRE.2006.875536 %0 Journal Article %J Neurosurgery %D 2006 %T The emerging world of motor neuroprosthetics: a neurosurgical perspective. %A Leuthardt, E C %A Gerwin Schalk %A Moran, D %A Ojemann, J G %K Brain %K Humans %K Man-Machine Systems %K Movement %K Neurosurgery %K Prostheses and Implants %K User-Computer Interface %X

A MOTOR NEUROPROSTHETIC device, or brain computer interface, is a machine that can take some type of signal from the brain and convert that information into overt device control such that it reflects the intentions of the user's brain. In essence, these constructs can decode the electrophysiological signals representing motor intent. With the parallel evolution of neuroscience, engineering, and rapid computing, the era of clinical neuroprosthetics is approaching as a practical reality for people with severe motor impairment. Patients with such diseases as spinal cord injury, stroke, limb loss, and neuromuscular disorders may benefit through the implantation of these brain computer interfaces that serve to augment their ability to communicate and interact with their environment. In the upcoming years, it will be important for the neurosurgeon to understand what a brain computer interface is, its fundamental principle of operation, and what the salient surgical issues are when considering implantation. We review the current state of the field of motor neuroprosthetics research, the early clinical applications, and the essential considerations from a neurosurgical perspective for the future.

%B Neurosurgery %V 59 %P 1-14; discussion 1-14 %8 07/2006 %G eng %U http://www.ncbi.nlm.nih.gov/pubmed/16823294 %N 1 %R 10.1227/01.NEU.0000221506.06947.AC %0 Journal Article %J IEEE Trans Neural Syst Rehabil Eng %D 2006 %T The Wadsworth BCI Research and Development Program: At Home with BCI. %A Theresa M Vaughan %A Dennis J. McFarland %A Gerwin Schalk %A Sarnacki, William A %A Krusienski, Dean J %A Sellers, Eric W %A Jonathan Wolpaw %K Animals %K Brain %K Electroencephalography %K Evoked Potentials %K Humans %K Neuromuscular Diseases %K New York %K Research %K Switzerland %K Therapy, Computer-Assisted %K Universities %K User-Computer Interface %X

The ultimate goal of brain-computer interface (BCI) technology is to provide communication and control capacities to people with severe motor disabilities. BCI research at the Wadsworth Center focuses primarily on noninvasive, electroencephalography (EEG)-based BCI methods. We have shown that people, including those with severe motor disabilities, can learn to use sensorimotor rhythms (SMRs) to move a cursor rapidly and accurately in one or two dimensions. We have also improved P300-based BCI operation. We are now translating this laboratory-proven BCI technology into a system that can be used by severely disabled people in their homes with minimal ongoing technical oversight. To accomplish this, we have: improved our general-purpose BCI software (BCI2000); improved online adaptation and feature translation for SMR-based BCI operation; improved the accuracy and bandwidth of P300-based BCI operation; reduced the complexity of system hardware and software and begun to evaluate home system use in appropriate users. These developments have resulted in prototype systems for every day use in people's homes.

%B IEEE Trans Neural Syst Rehabil Eng %V 14 %P 229-33 %8 06/2006 %G eng %U http://www.ncbi.nlm.nih.gov/pubmed/16792301 %N 2 %R 10.1109/TNSRE.2006.875577 %0 Generic %D 2005 %T Communicating Directly from the Brain: Brain-Computer Interfaces %A Gerwin Schalk %X Condensed Matter Physics Seminar Series. Rensselaer Polytechnic Institute, Troy, New York %8 11/2005 %G eng %0 Journal Article %J The Journal of neuroscience : the official journal of the Society for Neuroscience %D 2005 %T The interaction of a new motor skill and an old one: H-reflex conditioning and locomotion in rats. %A Yi Chen %A Xiang Yang Chen %A Jakeman, Lyn B. %A Gerwin Schalk %A Stokes, Bradford T. %A Jonathan Wolpaw %K H-reflex conditioning %K Learning %K Locomotion %K memory consolidation %K Motor control %K Rehabilitation %K spinal cord plasticity %X New and old motor skills can interfere with each other or interact in other ways. Because each skill entails a distributed pattern of activity-dependent plasticity, investigation of their interactions is facilitated by simple models. In a well characterized model of simple learning, rats and monkeys gradually change the size of the H-reflex, the electrical analog of the spinal stretch reflex. This study evaluates in normal rats the interactions of this new skill of H-reflex conditioning with the old well established skill of overground locomotion. In rats in which the soleus H-reflex elicited in the conditioning protocol (i.e., the conditioning H-reflex) had been decreased by down-conditioning, the H-reflexes elicited during the stance and swing phases of locomotion (i.e., the locomotor H-reflexes) were also smaller. Similarly, in rats in which the conditioning H-reflex had been increased by up-conditioning, the locomotor H-reflexes were also larger. Soleus H-reflex conditioning did not affect the duration, length, or right/left symmetry of the step cycle. However, the conditioned change in the stance H-reflex was positively correlated with change in the amplitude of the soleus locomotor burst, and the correlation was consistent with current estimates of the contribution of primary afferent input to the burst. Although H-reflex conditioning and locomotion did not interfere with each other, H-reflex conditioning did affect how locomotion was produced: it changed soleus burst amplitude and may have induced compensatory changes in the activity of other muscles. These results illustrate and clarify the subtlety and complexity of skill interactions. They also suggest that H-reflex conditioning might be used to improve the abnormal locomotion produced by spinal cord injury or other disorders of supraspinal control. %B The Journal of neuroscience : the official journal of the Society for Neuroscience %V 25 %P 6898–6906 %8 07/2005 %G eng %U http://www.ncbi.nlm.nih.gov/pubmed/16033899 %R 10.1523/JNEUROSCI.1684-05.2005 %0 Journal Article %J Journal of neuroscience methods %D 2005 %T Long-term spinal reflex studies in awake behaving mice. %A Jonathan S. Carp %A Tennissen, Ann M. %A Xiang Yang Chen %A Gerwin Schalk %A Jonathan Wolpaw %K Electromyography %K implanted electrodes %K Monosynaptic %K Spinal Cord %X The increasing availability of genetic variants of mice has facilitated studies of the roles of specific molecules in specific behaviors. The contributions of such studies could be strengthened and extended by correlation with detailed information on the patterns of motor commands throughout the course of specific behaviors in freely moving animals. Previously reported methodologies for long-term recording of electromyographic activity (EMG) in mice using implanted electrodes were designed for intermittent, but not continuous operation. This report describes the fabrication, implantation, and utilization of fine wire electrodes for continuous long-term recordings of spontaneous and nerve-evoked EMG in mice. Six mice were implanted with a tibial nerve cuff electrode and EMG electrodes in soleus and gastrocnemius muscles. Wires exited through a skin button and traveled through an armored cable to an electrical commutator. In mice implanted for 59-144 days, ongoing EMG was monitored continuously (i.e., 24 h/day, 7 days/week) by computer for 18-92 days (total intermittent recording for 25-130 days). When the ongoing EMG criteria were met, the computer applied the nerve stimulus, recorded the evoked EMG response, and determined the size of the M-response (MR) and the H-reflex (HR). It continually adjusted stimulation intensity to maintain a stable MR size. Stable recordings of ongoing EMG, MR, and HR were obtained typically 3 weeks after implantation. This study demonstrates the feasibility of long-term continuous EMG recordings in mice for addressing a variety of neurophysiological and behavioral issues. %B Journal of neuroscience methods %V 149 %P 134–143 %8 12/2005 %G eng %U http://www.ncbi.nlm.nih.gov/pubmed/16026848 %R 10.1016/j.jneumeth.2005.05.012 %0 Generic %D 2005 %T News from the Wadsworth BCI R&D Program: Pushing the Envelope %A Gerwin Schalk %X Eberhard-Karls University, Tübingen, Germany %8 04/2005 %G eng %0 Journal Article %J Neurology %D 2005 %T Patients with ALS can use sensorimotor rhythms to operate a brain-computer interface. %A Kübler, A. %A Nijboer, F %A Mellinger, Jürgen %A Theresa M Vaughan %A Pawelzik, H %A Gerwin Schalk %A Dennis J. McFarland %A Niels Birbaumer %A Jonathan Wolpaw %K Aged %K Amyotrophic Lateral Sclerosis %K Electroencephalography %K Evoked Potentials, Motor %K Evoked Potentials, Somatosensory %K Female %K Humans %K Imagination %K Male %K Middle Aged %K Motor Cortex %K Movement %K Paralysis %K Photic Stimulation %K Prostheses and Implants %K Somatosensory Cortex %K Treatment Outcome %K User-Computer Interface %X

People with severe motor disabilities can maintain an acceptable quality of life if they can communicate. Brain-computer interfaces (BCIs), which do not depend on muscle control, can provide communication. Four people severely disabled by ALS learned to operate a BCI with EEG rhythms recorded over sensorimotor cortex. These results suggest that a sensorimotor rhythm-based BCI could help maintain quality of life for people with ALS.

%B Neurology %V 64 %P 1775-7 %8 05/2005 %G eng %U http://www.ncbi.nlm.nih.gov/pubmed/15911809 %N 10 %R 10.1212/01.WNL.0000158616.43002.6D %0 Journal Article %J Neurology %D 2005 %T Patients with ALS can use sensorimotor rhythms to operate a brain-computer interface. %A Kübler, A. %A Nijboer, F. %A Mellinger, J. %A Theresa M Vaughan %A Pawelzik, H. %A Gerwin Schalk %A Dennis J. McFarland %A Niels Birbaumer %A Jonathan Wolpaw %K User-Computer Interface %X People with severe motor disabilities can maintain an acceptable quality of life if they can communicate. Brain-computer interfaces (BCIs), which do not depend on muscle control, can provide communication. Four people severely disabled by ALS learned to operate a BCI with EEG rhythms recorded over sensorimotor cortex. These results suggest that a sensorimotor rhythm-based BCI could help maintain quality of life for people with ALS. %B Neurology %V 64 %P 1775–1777 %8 05/2005 %G eng %U http://www.ncbi.nlm.nih.gov/pubmed/15911809 %R 10.1212/01.WNL.0000158616.43002.6D %0 Generic %D 2005 %T Recording Options for Brain-Computer Interfaces %A Gerwin Schalk %X Augmented Cognition Conference / Satellite to 11th International Conference on Human-Computer Interaction, Caesars Palace, Las Vegas, Nevada %8 07/2005 %G eng %0 Conference Proceedings %B Proceedings to the 1st Conference on Augmented Cognition %D 2005 %T Recording Options for Brain-Computer Interfaces %A Gerwin Schalk %A Jonathan Wolpaw %B Proceedings to the 1st Conference on Augmented Cognition %8 2005 %G eng %0 Conference Paper %B Proceedings to the 1st Conference on Augmented Cognition %D 2005 %T Recording Options for Brain-Computer Interfaces. %A Gerwin Schalk %A Jonathan Wolpaw %B Proceedings to the 1st Conference on Augmented Cognition %8 2005 %G eng %0 Generic %D 2005 %T Towards 2D Brain Control Using ECoG %A Gerwin Schalk %X 11th International Conference on Human-Computer Interaction, Caesars Palace, Las Vegas, Nevada %8 07/2005 %G eng %0 Conference Paper %B Proceedings to the 11th International Conference on Human-Computer Interaction %D 2005 %T Towards two-dimensional cursor control using electrocorticographic signals. %A Gerwin Schalk %A Leuthardt, E C %A Moran, D %A Miller, K.J. %A Ojemann, J G %A Jonathan Wolpaw %B Proceedings to the 11th International Conference on Human-Computer Interaction %8 2005 %G eng %0 Conference Proceedings %B Proceedings to the 11th International Conference on Human-Computer Interaction %D 2005 %T Towards two-dimensional cursor control using electrocorticographic signals %A Gerwin Schalk %A Leuthardt, E C %A Moran, D %A Miller, K.J. %A Ojemann, J G %A Jonathan Wolpaw %B Proceedings to the 11th International Conference on Human-Computer Interaction %G eng %0 Conference Paper %B Proc. IEEE International Conference of Neural Engineering %D 2005 %T Tracking of the mu rhythm using an empirically derived matched filter. %A Krusienski, Dean J %A Gerwin Schalk %A Dennis J. McFarland %A Jonathan Wolpaw %K bioelectric potentials %K Brain Computer Interfaces %K brain modeling %K brain-computer interface %K communication device %K communication system control %K cortical mu rhythm modulation %K EEG %K Electroencephalography %K empirically derived matched filter %K handicapped aids %K laboratories %K matched filters %K medical signal detection %K medical signal processing %K monitoring %K motor imagery %K mu rhythm tracking %K noninvasive treatment %K rhythm %K synchronous motors %K two-dimensional cursor control data %B Proc. IEEE International Conference of Neural Engineering %I IEEE %C Arlington, VA %8 03/2005 %@ 0-7803-8710-4 %G eng %U http://ieeexplore.ieee.org/xpl/articleDetails.jsp?arnumber=1419559 %R 10.1109/CNE.2005.1419559 %0 Conference Proceedings %B Proc. IEEE International Conference of Neural Engineering %D 2005 %T Tracking of the mu rhythm using an empirically derived matched filter %A Krusienski, Dean J %A Gerwin Schalk %A Dennis J. McFarland %A Jonathan Wolpaw %B Proc. IEEE International Conference of Neural Engineering %8 03/2005 %G eng %0 Journal Article %J IEEE transactions on bio-medical engineering %D 2004 %T The BCI Competition 2003: progress and perspectives in detection and discrimination of EEG single trials. %A Benjamin Blankertz %A Müller, Klaus-Robert %A Curio, Gabriel %A Theresa M Vaughan %A Gerwin Schalk %A Jonathan Wolpaw %A Schlögl, Alois %A Neuper, Christa %A Pfurtscheller, Gert %A Hinterberger, Thilo %A Schröder, Michael %A Niels Birbaumer %K augmentative communication %K BCI %K beta-rhythm %K brain-computer interface %K EEG %K ERP %K imagined hand movements %K lateralized readiness potential %K mu-rhythm %K P300 %K Rehabilitation %K single-trial classification %K slow cortical potentials %X Interest in developing a new method of man-to-machine communication–a brain-computer interface (BCI)–has grown steadily over the past few decades. BCIs create a new communication channel between the brain and an output device by bypassing conventional motor output pathways of nerves and muscles. These systems use signals recorded from the scalp, the surface of the cortex, or from inside the brain to enable users to control a variety of applications including simple word-processing software and orthotics. BCI technology could therefore provide a new communication and control option for individuals who cannot otherwise express their wishes to the outside world. Signal processing and classification methods are essential tools in the development of improved BCI technology. We organized the BCI Competition 2003 to evaluate the current state of the art of these tools. Four laboratories well versed in EEG-based BCI research provided six data sets in a documented format. We made these data sets (i.e., labeled training sets and unlabeled test sets) and their descriptions available on the Internet. The goal in the competition was to maximize the performance measure for the test labels. Researchers worldwide tested their algorithms and competed for the best classification results. This paper describes the six data sets and the results and function of the most successful algorithms. %B IEEE transactions on bio-medical engineering %V 51 %P 1044–1051 %8 06/2004 %G eng %U http://www.ncbi.nlm.nih.gov/pubmed/15188876 %R 10.1109/TBME.2004.826692 %0 Journal Article %J IEEE Trans Biomed Eng %D 2004 %T The BCI Competition 2003: Progress and perspectives in detection and discrimination of EEG single trials. %A Benjamin Blankertz %A Müller, Klaus-Robert %A Curio, Gabriel %A Theresa M Vaughan %A Gerwin Schalk %A Jonathan Wolpaw %A Schlögl, Alois %A Neuper, Christa %A Pfurtscheller, Gert %A Hinterberger, T. %A Schröder, Michael %A Niels Birbaumer %K Adult %K Algorithms %K Amyotrophic Lateral Sclerosis %K Artificial Intelligence %K Brain %K Cognition %K Databases, Factual %K Electroencephalography %K Evoked Potentials %K Humans %K Reproducibility of Results %K Sensitivity and Specificity %K User-Computer Interface %X Interest in developing a new method of man-to-machine communication--a brain-computer interface (BCI)--has grown steadily over the past few decades. BCIs create a new communication channel between the brain and an output device by bypassing conventional motor output pathways of nerves and muscles. These systems use signals recorded from the scalp, the surface of the cortex, or from inside the brain to enable users to control a variety of applications including simple word-processing software and orthotics. BCI technology could therefore provide a new communication and control option for individuals who cannot otherwise express their wishes to the outside world. Signal processing and classification methods are essential tools in the development of improved BCI technology. We organized the BCI Competition 2003 to evaluate the current state of the art of these tools. Four laboratories well versed in EEG-based BCI research provided six data sets in a documented format. We made these data sets (i.e., labeled training sets and unlabeled test sets) and their descriptions available on the Internet. The goal in the competition was to maximize the performance measure for the test labels. Researchers worldwide tested their algorithms and competed for the best classification results. This paper describes the six data sets and the results and function of the most successful algorithms. %B IEEE Trans Biomed Eng %V 51 %P 1044-51 %8 06/2004 %G eng %N 6 %R 10.1109/TBME.2004.826692 %0 Journal Article %J IEEE Trans Biomed Eng %D 2004 %T BCI2000: a general-purpose brain-computer interface (BCI) system. %A Gerwin Schalk %A Dennis J. McFarland %A Hinterberger, T. %A Niels Birbaumer %A Jonathan Wolpaw %K Algorithms %K Brain %K Cognition %K Communication Aids for Disabled %K Computer Peripherals %K Electroencephalography %K Equipment Design %K Equipment Failure Analysis %K Evoked Potentials %K Humans %K Systems Integration %K User-Computer Interface %X Many laboratories have begun to develop brain-computer interface (BCI) systems that provide communication and control capabilities to people with severe motor disabilities. Further progress and realization of practical applications depends on systematic evaluations and comparisons of different brain signals, recording methods, processing algorithms, output formats, and operating protocols. However, the typical BCI system is designed specifically for one particular BCI method and is, therefore, not suited to the systematic studies that are essential for continued progress. In response to this problem, we have developed a documented general-purpose BCI research and development platform called BCI2000. BCI2000 can incorporate alone or in combination any brain signals, signal processing methods, output devices, and operating protocols. This report is intended to describe to investigators, biomedical engineers, and computer scientists the concepts that the BC12000 system is based upon and gives examples of successful BCI implementations using this system. To date, we have used BCI2000 to create BCI systems for a variety of brain signals, processing methods, and applications. The data show that these systems function well in online operation and that BCI2000 satisfies the stringent real-time requirements of BCI systems. By substantially reducing labor and cost, BCI2000 facilitates the implementation of different BCI systems and other psychophysiological experiments. It is available with full documentation and free of charge for research or educational purposes and is currently being used in a variety of studies by many research groups. %B IEEE Trans Biomed Eng %V 51 %P 1034-43 %8 06/2004 %G eng %N 6 %R 10.1109/TBME.2004.827072 %0 Journal Article %J IEEE transactions on bio-medical engineering %D 2004 %T BCI2000: a general-purpose brain-computer interface (BCI) system. %A Gerwin Schalk %A Dennis J. McFarland %A Hinterberger, Thilo %A Niels Birbaumer %A Jonathan Wolpaw %K User-Computer Interface %X Many laboratories have begun to develop brain-computer interface (BCI) systems that provide communication and control capabilities to people with severe motor disabilities. Further progress and realization of practical applications depends on systematic evaluations and comparisons of different brain signals, recording methods, processing algorithms, output formats, and operating protocols. However, the typical BCI system is designed specifically for one particular BCI method and is, therefore, not suited to the systematic studies that are essential for continued progress. In response to this problem, we have developed a documented general-purpose BCI research and development platform called BCI2000. BCI2000 can incorporate alone or in combination any brain signals, signal processing methods, output devices, and operating protocols. This report is intended to describe to investigators, biomedical engineers, and computer scientists the concepts that the BC12000 system is based upon and gives examples of successful BCI implementations using this system. To date, we have used BCI2000 to create BCI systems for a variety of brain signals, processing methods, and applications. The data show that these systems function well in online operation and that BCI2000 satisfies the stringent real-time requirements of BCI systems. By substantially reducing labor and cost, BCI2000 facilitates the implementation of different BCI systems and other psychophysiological experiments. It is available with full documentation and free of charge for research or educational purposes and is currently being used in a variety of studies by many research groups. %B IEEE transactions on bio-medical engineering %V 51 %P 1034–1043 %8 06/2004 %G eng %U http://www.ncbi.nlm.nih.gov/pubmed/15188875 %R 10.1109/TBME.2004.827072 %0 Journal Article %J J Neural Eng %D 2004 %T A brain-computer interface using electrocorticographic signals in humans. %A Leuthardt, E C %A Gerwin Schalk %A Jonathan Wolpaw %A Ojemann, J G %A Moran, D %K Adult %K Brain %K Communication Aids for Disabled %K Computer Peripherals %K Diagnosis, Computer-Assisted %K Electrodes, Implanted %K Electroencephalography %K Evoked Potentials %K Female %K Humans %K Imagination %K Male %K Movement Disorders %K User-Computer Interface %X

Brain-computer interfaces (BCIs) enable users to control devices with electroencephalographic (EEG) activity from the scalp or with single-neuron activity from within the brain. Both methods have disadvantages: EEG has limited resolution and requires extensive training, while single-neuron recording entails significant clinical risks and has limited stability. We demonstrate here for the first time that electrocorticographic (ECoG) activity recorded from the surface of the brain can enable users to control a one-dimensional computer cursor rapidly and accurately. We first identified ECoG signals that were associated with different types of motor and speech imagery. Over brief training periods of 3-24 min, four patients then used these signals to master closed-loop control and to achieve success rates of 74-100% in a one-dimensional binary task. In additional open-loop experiments, we found that ECoG signals at frequencies up to 180 Hz encoded substantial information about the direction of two-dimensional joystick movements. Our results suggest that an ECoG-based BCI could provide for people with severe motor disabilities a non-muscular communication and control option that is more powerful than EEG-based BCIs and is potentially more stable and less traumatic than BCIs that use electrodes penetrating the brain.

%B J Neural Eng %V 1 %P 63-71 %8 06/2004 %G eng %U http://www.ncbi.nlm.nih.gov/pubmed/15876624 %N 2 %R 10.1088/1741-2560/1/2/001 %0 Generic %D 2004 %T Brain-Computer Interfaces; EGI Amp Server; Event-Detection %A Gerwin Schalk %X Electrical Geodesics, Eugene, Oregon %8 08/2004 %G eng %0 Generic %D 2004 %T Brain-Computer Interfaces: Present and Future %A Gerwin Schalk %X "BrainDays" Symposium, Rudolf Magnus Institute of Neuroscience, University Medical Center, Utrecht, The Netherlands %8 10/2004 %G eng %0 Generic %D 2004 %T Brain-Computer Interfaces: Present and Future %A Gerwin Schalk %X Fondazione Santa Lucia, Rome, Italy %8 06/2004 %G eng %0 Generic %D 2004 %T Brain-Computer Interfaces: Present and Future %A Gerwin Schalk %X University of Washington, Seattle, Washington %8 08/2004 %G eng %0 Generic %D 2004 %T Brain-Computer Interfaces: Signals, Methods, and Systems %A Gerwin Schalk %X Seminar SeriesNew Frontiers in Brain Machine Interfaces Research. Institute for Infocomm Research(I2R), Singapore %8 02/2004 %G eng %0 Generic %D 2004 %T Business in Austria %A Gerwin Schalk %X International Business Panel, Executive MBA Program, Lally School of Management, Rensselaer Polytechnic Institute, Troy, NY %8 05/2004 %G eng %0 Generic %D 2004 %T Introduction to Brain-Computer Interfaces %A Gerwin Schalk %X University of Rome "La Sapienza," Rome, Italy %8 06/2004 %G eng %0 Journal Article %J Biomedizinische Technik %D 2004 %T P300 for communication: Evidence from patients with amyotrophic lateral sclerosis (ALS). %A Mellinger, Jürgen %A Nijboer, F %A Pawelzik, H %A Gerwin Schalk %A Dennis J. McFarland %A Theresa M Vaughan %A Jonathan Wolpaw %A Niels Birbaumer %A Kuebler, A. %B Biomedizinische Technik %G eng %0 Generic %D 2003 %T Brain-Computer Interfaces: Signals, Methods, and Systems %A Gerwin Schalk %X Eberhard Karls University of Tübingen, Tübingen, Germany %8 12/2003 %G eng %0 Generic %D 2003 %T Brain-Computer Interfaces: Signals, Methods, and Systems %A Gerwin Schalk %X NASA Ames Research Center, Moffett Field, CA %8 06/2003 %G eng %0 Generic %D 2003 %T Brain-Computer Interfaces: Signals, Methods, and Systems %A Gerwin Schalk %X World Congress on Medical Physics and Biomedical Engineering, Sydney, Australia %8 8/2003 %G eng %0 Generic %D 2003 %T Brain-Computer Interfaces: Signals, Methods, and Systems %A Gerwin Schalk %X Society for Neuroscience Hudson-Berkshire Chapter, State University of Albany, Albany, NY %8 10/2003 %G eng %0 Generic %D 2003 %T Business in Austria %A Gerwin Schalk %X International Business Panel, Executive MBA Program, Lally School of Management, Rensselaer Polytechnic Institute, Troy, NY %8 05/2003 %G eng %0 Journal Article %J IEEE Trans Neural Syst Rehabil Eng %D 2003 %T The Wadsworth Center brain-computer interface (BCI) research and development program. %A Jonathan Wolpaw %A Dennis J. McFarland %A Theresa M Vaughan %A Gerwin Schalk %K Academic Medical Centers %K Adult %K Algorithms %K Artifacts %K Brain %K Brain Mapping %K Electroencephalography %K Evoked Potentials, Visual %K Feedback %K Humans %K Middle Aged %K Nervous System Diseases %K Research %K Research Design %K User-Computer Interface %K Visual Perception %X

Brain-computer interface (BCI) research at the Wadsworth Center has focused primarily on using electroencephalogram (EEG) rhythms recorded from the scalp over sensorimotor cortex to control cursor movement in one or two dimensions. Recent and current studies seek to improve the speed and accuracy of this control by improving the selection of signal features and their translation into device commands, by incorporating additional signal features, and by optimizing the adaptive interaction between the user and system. In addition, to facilitate the evaluation, comparison, and combination of alternative BCI methods, we have developed a general-purpose BCI system called BCI-2000 and have made it available to other research groups. Finally, in collaboration with several other groups, we are developing simple BCI applications and are testing their practicality and long-term value for people with severe motor disabilities.

%B IEEE Trans Neural Syst Rehabil Eng %V 11 %P 204-7 %8 06/2003 %G eng %U http://www.ncbi.nlm.nih.gov/pubmed/12899275 %N 2 %R 10.1109/TNSRE.2003.814442 %0 Generic %D 2002 %T Brain-Computer Interfaces and BCI2000 %A Gerwin Schalk %X Georgia State University, Atlanta, Georgia %8 03/2002 %G eng %0 Generic %D 2002 %T Brain-Computer Interfaces for Communication and Control %A Gerwin Schalk %X 8th International Conference on Functional Mapping of the Human Brain, Sendai, Japan %8 06/2002 %G eng %0 Generic %D 2002 %T General-Purpose Brain-Computer Interfaces (BCI) System %A Gerwin Schalk %X 33rd Neural Prosthesis Workshop, National Library of Medicine / NIH, Bethesda, Maryland %8 10/2002 %G eng %0 Journal Article %J J Neurosci Methods %D 2002 %T Temporal transformation of multiunit activity improves identification of single motor units. %A Gerwin Schalk %A Jonathan S. Carp %A Jonathan Wolpaw %K Action Potentials %K Animals %K Electromyography %K H-Reflex %K Motor Neurons %K Muscle, Skeletal %K Rats %K Signal Processing, Computer-Assisted %X

This report describes a temporally based method for identifying repetitive firing of motor units. This approach is ideally suited to spike trains with negative serially correlated inter-spike intervals (ISIs). It can also be applied to spike trains in which ISIs exhibit little serial correlation if their coefficient of variation (COV) is sufficiently low. Using a novel application of the Hough transform, this method (i.e. the modified Hough transform (MHT)) maps motor unit action potential (MUAP) firing times into a feature space with ISI and offset (defined as the latency from an arbitrary starting time to the first MUAP in the train) as dimensions. Each MUAP firing time corresponds to a pattern in the feature space that represents all possible MUAP trains with a firing at that time. Trains with stable ISIs produce clusters in the feature space, whereas randomly firing trains do not. The MHT provides a direct estimate of mean firing rate and its variability for the entire data segment, even if several individual MUAPs are obscured by firings from other motor units. Addition of this method to a shape-based classification approach markedly improved rejection of false positives using simulated data and identified spike trains in whole muscle electromyographic recordings from rats. The relative independence of the MHT from the need to correctly classify individual firings permits a global description of stable repetitive firing behavior that is complementary to shape-based approaches to MUAP classification.

%B J Neurosci Methods %V 114 %P 87-98 %8 02/2002 %G eng %U http://www.ncbi.nlm.nih.gov/pubmed/11850043 %N 1 %R 10.1016/S0165-0270(01)00517-9 %0 Generic %D 2001 %T Brain-Computer Interfaces for Communication and Control %A Gerwin Schalk %X NIPS*2001 Brain-Computer Interface Workshop, Whistler, British Columbia, Canda %8 12/2001 %G eng %0 Generic %D 2001 %T Improved Motor Unit Detection Using the Hough Transform %A Gerwin Schalk %A Jonathan Carp %X Neuro-Muscular Research Center, Boston University %8 01/2001 %G eng %0 Generic %D 2000 %T BCI2000: A Generic Brain-Computer Interface %A Gerwin Schalk %X Department of Medical Informatics, Technical University of Graz %8 06/2000 %G eng %0 Journal Article %J IEEE transactions on rehabilitation engineering : a publication of the IEEE Engineering in Medicine and Biology Society %D 2000 %T Brain-computer interface technology: a review of the first international meeting. %A Jonathan Wolpaw %A Niels Birbaumer %A Heetderks, W. J. %A Dennis J. McFarland %A Peckham, P. H. %A Gerwin Schalk %A Emanuel Donchin %A Quatrano, L. A. %A Robinson, C. J. %A Theresa M Vaughan %K augmentative communication %K Brain-computer interface (BCI) %K electroencephalography (EEG) %X Over the past decade, many laboratories have begun to explore brain-computer interface (BCI) technology as a radically new communication option for those with neuromuscular impairments that prevent them from using conventional augmentative communication methods. BCI's provide these users with communication channels that do not depend on peripheral nerves and muscles. This article summarizes the first international meeting devoted to BCI research and development. Current BCI's use electroencephalographic (EEG) activity recorded at the scalp or single-unit activity recorded from within cortex to control cursor movement, select letters or icons, or operate a neuroprosthesis. The central element in each BCI is a translation algorithm that converts electrophysiological input from the user into output that controls external devices. BCI operation depends on effective interaction between two adaptive controllers, the user who encodes his or her commands in the electrophysiological input provided to the BCI, and the BCI which recognizes the commands contained in the input and expresses them in device control. Current BCI's have maximum information transfer rates of 5-25 b/min. Achievement of greater speed and accuracy depends on improvements in signal processing, translation algorithms, and user training. These improvements depend on increased interdisciplinary cooperation between neuroscientists, engineers, computer programmers, psychologists, and rehabilitation specialists, and on adoption and widespread application of objective methods for evaluating alternative methods. The practical use of BCI technology depends on the development of appropriate applications, identification of appropriate user groups, and careful attention to the needs and desires of individual users. BCI research and development will also benefit from greater emphasis on peer-reviewed publications, and from adoption of standard venues for presentations and discussion. %B IEEE transactions on rehabilitation engineering : a publication of the IEEE Engineering in Medicine and Biology Society %V 8 %P 164–173 %8 06/2000 %G eng %U http://www.ncbi.nlm.nih.gov/pubmed/10896178 %R 10.1109/TRE.2000.847807 %0 Journal Article %J IEEE Trans Rehabil Eng %D 2000 %T Brain-computer interface technology: a review of the first international meeting. %A Jonathan Wolpaw %A Niels Birbaumer %A Heetderks, W J %A Dennis J. McFarland %A Peckham, P H %A Gerwin Schalk %A Emanuel Donchin %A Quatrano, L A %A Robinson, C J %A Theresa M Vaughan %K Algorithms %K Cerebral Cortex %K Communication Aids for Disabled %K Disabled Persons %K Electroencephalography %K Evoked Potentials %K Humans %K Neuromuscular Diseases %K Signal Processing, Computer-Assisted %K User-Computer Interface %X

Over the past decade, many laboratories have begun to explore brain-computer interface (BCI) technology as a radically new communication option for those with neuromuscular impairments that prevent them from using conventional augmentative communication methods. BCI's provide these users with communication channels that do not depend on peripheral nerves and muscles. This article summarizes the first international meeting devoted to BCI research and development. Current BCI's use electroencephalographic (EEG) activity recorded at the scalp or single-unit activity recorded from within cortex to control cursor movement, select letters or icons, or operate a neuroprosthesis. The central element in each BCI is a translation algorithm that converts electrophysiological input from the user into output that controls external devices. BCI operation depends on effective interaction between two adaptive controllers, the user who encodes his or her commands in the electrophysiological input provided to the BCI, and the BCI which recognizes the commands contained in the input and expresses them in device control. Current BCI's have maximum information transfer rates of 5-25 b/min. Achievement of greater speed and accuracy depends on improvements in signal processing, translation algorithms, and user training. These improvements depend on increased interdisciplinary cooperation between neuroscientists, engineers, computer programmers, psychologists, and rehabilitation specialists, and on adoption and widespread application of objective methods for evaluating alternative methods. The practical use of BCI technology depends on the development of appropriate applications, identification of appropriate user groups, and careful attention to the needs and desires of individual users. BCI research and development will also benefit from greater emphasis on peer-reviewed publications, and from adoption of standard venues for presentations and discussion.

%B IEEE Trans Rehabil Eng %V 8 %P 164-73 %8 06/2000 %G eng %U http://www.ncbi.nlm.nih.gov/pubmed/10896178 %N 2 %R 10.1109/TRE.2000.847807 %0 Journal Article %J Clinical neurophysiology : official journal of the International Federation of Clinical Neurophysiology %D 2000 %T EEG-based communication: presence of an error potential. %A Gerwin Schalk %A Jonathan Wolpaw %A Dennis J. McFarland %A Pfurtscheller, G. %K augmentative communication %K brain-computer interface %K Electroencephalography %K error potential %K error related negativity %K event related potential %K mu rhythm %K Rehabilitation %K sensorimotor cortex %X EEG-based communication could be a valuable new augmentative communication technology for those with severe motor disabilities. Like all communication methods, it faces the problem of errors in transmission. In the Wadsworth EEG-based brain-computer interface (BCI) system, subjects learn to use mu or beta rhythm amplitude to move a cursor to targets on a computer screen. While cursor movement is highly accurate in trained subjects, it is not perfect. %B Clinical neurophysiology : official journal of the International Federation of Clinical Neurophysiology %V 111 %P 2138–2144 %8 12/2000 %G eng %U http://www.ncbi.nlm.nih.gov/pubmed/11090763 %R 10.1016/S1388-2457(00)00457-0