%0 Book Section %B Brain–Computer Interfaces Handbook: Technological and Theoretical Advances %D 2018 %T ECoG-Based BCIs %A Gunduz, Aysegul %A Schalk, Gerwin %B Brain–Computer Interfaces Handbook: Technological and Theoretical Advances %P 297 %G eng %0 Journal Article %J Brain-Computer Interfaces %D 2016 %T Differential roles of high gamma and local motor potentials for movement preparation and execution %A Gunduz, Aysegul %A Peter Brunner %A Sharma, Mohit %A Leuthardt, Eric C. %A Ritaccio, Anthony L. %A Pesaran, Bijan %A Schalk, Gerwin %K BCI %K brain-computer interfaces %K ECoG %K Electrocorticography %K sensorimotor systems %X Determining a person’s intent, such as the planned direction of their movement, directly from their cortical activity could support important applications such as brain-computer interfaces (BCIs). Continuing development of improved BCI systems requires a better understanding of how the brain prepares for and executes movements. To contribute to this understanding, we recorded surface cortical potentials (electrocorticographic signals; ECoG) in 11 human subjects performing a delayed center-out task to establish the differential role of high gamma activity (HGA) and the local motor potential (LMP) as a function of time and anatomical area during movement preparation and execution. High gamma modulations mostly confirm previous findings of sensorimotor cortex involvement, whereas modulations in LMPs are observed in prefrontal cortices. These modulations include directional information during movement planning as well as execution. Our results suggest that sampling signals from these widely distributed cortical areas improves decoding accuracy. %B Brain-Computer Interfaces %V 3 %P 88-102 %8 May %G eng %N 2 %R https://doi.org/10.1080/2326263X.2016.1179087 %0 Journal Article %J Epilepsy & behavior : E&B %D 2016 %T Proceedings of the Eighth International Workshop on Advances in Electrocorticography. %A A L Ritaccio %A Williams, Justin %A Denison, Tim %A Foster, Brett L. %A Starr, Philip A. %A Gunduz, Aysegul %A Zijlmans, Maeike %A Gerwin Schalk %X Excerpted proceedings of the Eighth International Workshop on Advances in Electrocorticography (ECoG), which convened October 15-16, 2015 in Chicago, IL, are presented. The workshop series has become the foremost gathering to present current basic and clinical research in subdural brain signal recording and analysis. %B Epilepsy & behavior : E&B %V 64 %P 248–252 %8 Nov %G eng %U http://www.ncbi.nlm.nih.gov/pubmed/27780085 %R 10.1016/j.yebeh.2016.08.020 %0 Journal Article %J PloS one %D 2016 %T Spatio-Temporal Progression of Cortical Activity Related to Continuous Overt and Covert Speech Production in a Reading Task. %A Brumberg, Jonathan S. %A Krusienski, Dean J. %A Chakrabarti, Shreya %A Gunduz, Aysegul %A Peter Brunner %A A L Ritaccio %A Gerwin Schalk %X How the human brain plans, executes, and monitors continuous and fluent speech has remained largely elusive. For example, previous research has defined the cortical locations most important for different aspects of speech function, but has not yet yielded a definition of the temporal progression of involvement of those locations as speech progresses either overtly or covertly. In this paper, we uncovered the spatio-temporal evolution of neuronal population-level activity related to continuous overt speech, and identified those locations that shared activity characteristics across overt and covert speech. Specifically, we asked subjects to repeat continuous sentences aloud or silently while we recorded electrical signals directly from the surface of the brain (electrocorticography (ECoG)). We then determined the relationship between cortical activity and speech output across different areas of cortex and at sub-second timescales. The results highlight a spatio-temporal progression of cortical involvement in the continuous speech process that initiates utterances in frontal-motor areas and ends with the monitoring of auditory feedback in superior temporal gyrus. Direct comparison of cortical activity related to overt versus covert conditions revealed a common network of brain regions involved in speech that may implement orthographic and phonological processing. Our results provide one of the first characterizations of the spatiotemporal electrophysiological representations of the continuous speech process, and also highlight the common neural substrate of overt and covert speech. These results thereby contribute to a refined understanding of speech functions in the human brain. %B PloS one %V 11 %P e0166872 %8 Nov %G eng %U http://www.ncbi.nlm.nih.gov/pubmed/27875590 %R 10.1371/journal.pone.0166872 %0 Journal Article %J Front Hum Neurosci %D 2015 %T Electrocorticographic representations of segmental features in continuous speech. %A Lotte, Fabien %A Jonathan S Brumberg %A Peter Brunner %A Gunduz, Aysegul %A A L Ritaccio %A Guan, Cuntai %A Gerwin Schalk %K electrocorticography (ECoG) %K manner of articulation %K place of articulation %K speech processing %K voicing %X Acoustic speech output results from coordinated articulation of dozens of muscles, bones and cartilages of the vocal mechanism. While we commonly take the fluency and speed of our speech productions for granted, the neural mechanisms facilitating the requisite muscular control are not completely understood. Previous neuroimaging and electrophysiology studies of speech sensorimotor control has typically concentrated on speech sounds (i.e., phonemes, syllables and words) in isolation; sentence-length investigations have largely been used to inform coincident linguistic processing. In this study, we examined the neural representations of segmental features (place and manner of articulation, and voicing status) in the context of fluent, continuous speech production. We used recordings from the cortical surface [electrocorticography (ECoG)] to simultaneously evaluate the spatial topography and temporal dynamics of the neural correlates of speech articulation that may mediate the generation of hypothesized gestural or articulatory scores. We found that the representation of place of articulation involved broad networks of brain regions during all phases of speech production: preparation, execution and monitoring. In contrast, manner of articulation and voicing status were dominated by auditory cortical responses after speech had been initiated. These results provide a new insight into the articulatory and auditory processes underlying speech production in terms of their motor requirements and acoustic correlates. %B Front Hum Neurosci %V 9 %P 97 %8 02/2015 %G eng %U http://www.ncbi.nlm.nih.gov/pubmed/25759647 %R 10.3389/fnhum.2015.00097 %0 Journal Article %J Journal of Neural Engineering %D 2015 %T Identifying the Attended Speaker Using Electrocorticographic (ECoG) Signals. %A Dijkstra, K. %A Peter Brunner %A Gunduz, Aysegul %A Coon, W.G. %A A L Ritaccio %A Farquhar, Jason %A Gerwin Schalk %K auditory attention %K Brain-computer interface (BCI) %K Cocktail Party %K electrocorticography (ECoG) %X People affected by severe neuro-degenerative diseases (e.g., late-stage amyotrophic lateral sclerosis (ALS) or locked-in syndrome) eventually lose all muscular control. Thus, they cannot use traditional assistive communication devices that depend on muscle control, or brain-computer interfaces (BCIs) that depend on the ability to control gaze. While auditory and tactile BCIs can provide communication to such individuals, their use typically entails an artificial mapping between the stimulus and the communication intent. This makes these BCIs difficult to learn and use. In this study, we investigated the use of selective auditory attention to natural speech as an avenue for BCI communication. In this approach, the user communicates by directing his/her attention to one of two simultaneously presented speakers. We used electrocorticographic (ECoG) signals in the gamma band (70–170 Hz) to infer the identity of attended speaker, thereby removing the need to learn such an artificial mapping. Our results from twelve human subjects show that a single cortical location over superior temporal gyrus or pre-motor cortex is typically sufficient to identify the attended speaker within 10 s and with 77% accuracy (50% accuracy due to chance). These results lay the groundwork for future studies that may determine the real-time performance of BCIs based on selective auditory attention to speech. %B Journal of Neural Engineering %G eng %U https://www.ncbi.nlm.nih.gov/pmc/articles/PMC4776341/ %R 10.1080/2326263X.2015.1063363 %0 Journal Article %J Epilepsy Behav %D 2014 %T Proceedings of the Fifth International Workshop on Advances in Electrocorticography. %A A L Ritaccio %A Peter Brunner %A Gunduz, Aysegul %A Hermes, Dora %A Hirsch, Lawrence J %A Jacobs, Joshua %A Kamada, Kyousuke %A Kastner, Sabine %A Robert T. Knight %A Lesser, Ronald P %A Miller, Kai %A Sejnowski, Terrence %A Worrell, Gregory %A Gerwin Schalk %K Brain Mapping %K brain-computer interface %K electrical stimulation mapping %K Electrocorticography %K functional mapping %K Gamma-frequency electroencephalography %K High-frequency oscillations %K Neuroprosthetics %K Seizure detection %K Subdural grid %X

The Fifth International Workshop on Advances in Electrocorticography convened in San Diego, CA, on November 7-8, 2013. Advancements in methodology, implementation, and commercialization across both research and in the interval year since the last workshop were the focus of the gathering. Electrocorticography (ECoG) is now firmly established as a preferred signal source for advanced research in functional, cognitive, and neuroprosthetic domains. Published output in ECoG fields has increased tenfold in the past decade. These proceedings attempt to summarize the state of the art.

%B Epilepsy Behav %V 41 %P 183-92 %8 12/2014 %G eng %U http://www.ncbi.nlm.nih.gov/pubmed/25461213 %R 10.1016/j.yebeh.2014.09.015 %0 Journal Article %J Journal of Neural Engineering %D 2014 %T Simultaneous Real-Time Monitoring of Multiple Cortical Systems. %A Disha Gupta %A Jeremy Jeremy Hill %A Peter Brunner %A Gunduz, Aysegul %A A L Ritaccio %A Gerwin Schalk %K auditory processing %K Electrocorticography %K movement intention %K realtime decoding %K simultaneous decoding %X OBJECTIVE: Real-time monitoring of the brain is potentially valuable for performance monitoring, communication, training or rehabilitation. In natural situations, the brain performs a complex mix of various sensory, motor or cognitive functions. Thus, real-time brain monitoring would be most valuable if (a) it could decode information from multiple brain systems simultaneously, and (b) this decoding of each brain system were robust to variations in the activity of other (unrelated) brain systems. Previous studies showed that it is possible to decode some information from different brain systems in retrospect and/or in isolation. In our study, we set out to determine whether it is possible to simultaneously decode important information about a user from different brain systems in real time, and to evaluate the impact of concurrent activity in different brain systems on decoding performance. APPROACH: We study these questions using electrocorticographic signals recorded in humans. We first document procedures for generating stable decoding models given little training data, and then report their use for offline and for real-time decoding from 12 subjects (six for offline parameter optimization, six for online experimentation). The subjects engage in tasks that involve movement intention, movement execution and auditory functions, separately, and then simultaneously. Main Results: Our real-time results demonstrate that our system can identify intention and movement periods in single trials with an accuracy of 80.4% and 86.8%, respectively (where 50% would be expected by chance). Simultaneously, the decoding of the power envelope of an auditory stimulus resulted in an average correlation coefficient of 0.37 between the actual and decoded power envelopes. These decoders were trained separately and executed simultaneously in real time. SIGNIFICANCE: This study yielded the first demonstration that it is possible to decode simultaneously the functional activity of multiple independent brain systems. Our comparison of univariate and multivariate decoding strategies, and our analysis of the influence of their decoding parameters, provides benchmarks and guidelines for future research on this topic. %B Journal of Neural Engineering %8 10/2014 %G eng %U http://www.ncbi.nlm.nih.gov/pubmed/25080161 %R 10.1088/1741-2560/11/5/056001 %0 Journal Article %J NeuroImage %D 2014 %T Spatial and temporal relationships of electrocorticographic alpha and gamma activity during auditory processing. %A Potes, Cristhian %A Peter Brunner %A Gunduz, Aysegul %A Robert T. Knight %A Gerwin Schalk %K alpha and high gamma activity %K auditory processing %K electrocorticography (ECoG) %K functional connectivity %K granger causality %K thalamo-cortical interactions %X Neuroimaging approaches have implicated multiple brain sites in musical perception, including the posterior part of the superior temporal gyrus and adjacent perisylvian areas. However, the detailed spatial and temporal relationship of neural signals that support auditory processing is largely unknown. In this study, we applied a novel inter-subject analysis approach to electrophysiological signals recorded from the surface of the brain (electrocorticography (ECoG)) in ten human subjects. This approach allowed us to reliably identify those ECoG features that were related to the processing of a complex auditory stimulus (i.e., continuous piece of music) and to investigate their spatial, temporal, and causal relationships. Our results identified stimulus-related modulations in the alpha (8-12 Hz) and high gamma (70-110 Hz) bands at neuroanatomical locations implicated in auditory processing. Specifically, we identified stimulus-related ECoG modulations in the alpha band in areas adjacent to primary auditory cortex, which are known to receive afferent auditory projections from the thalamus (80 of a total of 15,107 tested sites). In contrast, we identified stimulus-related ECoG modulations in the high gamma band not only in areas close to primary auditory cortex but also in other perisylvian areas known to be involved in higher-order auditory processing, and in superior premotor cortex (412/15,107 sites). Across all implicated areas, modulations in the high gamma band preceded those in the alpha band by 280 ms, and activity in the high gamma band causally predicted alpha activity, but not vice versa (Granger causality, p<1e(-8)). Additionally, detailed analyses using Granger causality identified causal relationships of high gamma activity between distinct locations in early auditory pathways within superior temporal gyrus (STG) and posterior STG, between posterior STG and inferior frontal cortex, and between STG and premotor cortex. Evidence suggests that these relationships reflect direct cortico-cortical connections rather than common driving input from subcortical structures such as the thalamus. In summary, our inter-subject analyses defined the spatial and temporal relationships between music-related brain activity in the alpha and high gamma bands. They provide experimental evidence supporting current theories about the putative mechanisms of alpha and gamma activity, i.e., reflections of thalamo-cortical interactions and local cortical neural activity, respectively, and the results are also in agreement with existing functional models of auditory processing. %B NeuroImage %V 97 %P 188-95 %8 08/2014 %G eng %U http://www.ncbi.nlm.nih.gov/pubmed/24768933 %R 10.1016/j.neuroimage.2014.04.045 %0 Journal Article %J Epilepsy & Behavior %D 2013 %T Proceedings of the Fourth International Workshop on Advances in Electrocorticography. %A A L Ritaccio %A Peter Brunner %A Nathan E. Crone %A Gunduz, Aysegul %A Hirsch, Lawrence J. %A Kanwisher, Nancy %A Litt, Brian %A Kai J. Miller %A Morani, Daniel %A Parvizi, Josef %A Ramsey, Nick F %A Richner, Thomas J. %A Tandon, Niton %A Williams, Justin %A Gerwin Schalk %K Brain Mapping %K Brain–computer interface %K Electrocorticography %K Gamma-frequency electroencephalography %K High-frequency oscillations %K Neuroprosthetics %K Seizure detection %K Subdural grid %X The Fourth International Workshop on Advances in Electrocorticography (ECoG) convened in New Orleans, LA, on October 11–12, 2012. The proceedings of the workshop serves as an accurate record of the most contemporary clinical and experimental work on brain surface recording and represents the insights of a unique multidisciplinary ensemble of expert clinicians and scientists. Presentations covered a broad range of topics, including innovations in passive functional mapping, increased understanding of pathologic high-frequency oscillations, evolving sensor technologies, a human trial of ECoG-driven brain–machine interface, as well as fresh insights into brain electrical stimulation. %B Epilepsy & Behavior %V 29 %P 259–68 %8 11/2013 %G eng %U http://www.ncbi.nlm.nih.gov/pubmed/24034899 %N 2 %R 10.1016/j.yebeh.2013.08.012 %0 Journal Article %J PLoS ONE %D 2013 %T The Tracking of Speech Envelope in the Human Cortex. %A Kubanek, Jan %A Peter Brunner %A Gunduz, Aysegul %A Poeppel, David %A Gerwin Schalk %X Humans are highly adept at processing speech. Recently, it has been shown that slow temporal information in speech (i.e., the envelope of speech) is critical for speech comprehension. Furthermore, it has been found that evoked electric potentials in human cortex are correlated with the speech envelope. However, it has been unclear whether this essential linguistic feature is encoded differentially in specific regions, or whether it is represented throughout the auditory system. To answer this question, we recorded neural data with high temporal resolution directly from the cortex while human subjects listened to a spoken story. We found that the gamma activity in human auditory cortex robustly tracks the speech envelope. The effect is so marked that it is observed during a single presentation of the spoken story to each subject. The effect is stronger in regions situated relatively early in the auditory pathway (belt areas) compared to other regions involved in speech processing, including the superior temporal gyrus (STG) and the posterior inferior frontal gyrus (Broca's region). To further distinguish whether speech envelope is encoded in the auditory system as a phonological (speech-related), or instead as a more general acoustic feature, we also probed the auditory system with a melodic stimulus. We found that belt areas track melody envelope weakly, and as the only region considered. Together, our data provide the first direct electrophysiological evidence that the envelope of speech is robustly tracked in non-primary auditory cortex (belt areas in particular), and suggest that the considered higher-order regions (STG and Broca's region) partake in a more abstract linguistic analysis. %B PLoS ONE %V 8 %P e53398 - %8 01/2013 %G eng %U http://dx.doi.org/10.1371%2Fjournal.pone.0053398 %N 1 %R 10.1371/journal.pone.0053398 %0 Journal Article %J Neuroimage %D 2012 %T Decoding covert spatial attention using electrocorticographic (ECoG) signals in humans. %A Gunduz, Aysegul %A Peter Brunner %A Amy Daitch %A Leuthardt, E C %A A L Ritaccio %A Pesaran, Bijan %A Gerwin Schalk %K covert attention %K electrocorticography (ECoG) %K visual spatial attention %X

This study shows that electrocorticographic (ECoG) signals recorded from the surface of the brain provide detailed information about shifting of visual attention and its directional orientation in humans. ECoG allows for the identification of the cortical areas and time periods that hold the most information about covert attentional shifts. Our results suggest a transient distributed fronto-parietal mechanism for orienting of attention that is represented by different physiological processes. This neural mechanism encodes not only whether or not a subject shifts their attention to a location, but also the locus of attention. This work contributes to our understanding of the electrophysiological representation of attention in humans. It may also eventually lead to brain-computer interfaces (BCIs) that optimize user interaction with their surroundings or that allow people to communicate choices simply by shifting attention to them.

%B Neuroimage %V 60 %P 2285-93 %8 05/2012 %G eng %U http://www.ncbi.nlm.nih.gov/pubmed/22366333 %N 4 %R 10.1016/j.neuroimage.2012.02.017 %0 Journal Article %J Frontiers in Neuroengineering %D 2012 %T Decoding Onset and Direction of Movements using Electrocorticographic (ECoG) Signals in Humans. %A Wang, Z. %A Gunduz, Aysegul %A Peter Brunner %A A L Ritaccio %A Ji, Q %A Gerwin Schalk %K brain computer interface %K ECoG %K movement direction prediction %K movement onset prediction %K neurorehabilitation %K performance augmentation %X Communication of intent usually requires motor function. This requirement can be limiting when a person is engaged in a task, or prohibitive for some people suffering from neuromuscular disorders. Determining a person's intent, e.g., where and when to move, from brain signals rather than from muscles would have important applications in clinical or other domains. For example, detection of the onset and direction of intended movements may provide the basis for restoration of simple grasping function in people with chronic stroke, or could be used to optimize a user's interaction with the surrounding environment. Detecting the onset and direction of actual movements are a first step in this direction. In this study, we demonstrate that we can detect the onset of intended movements and their direction using electrocorticographic (ECoG) signals recorded from the surface of the cortex in humans. We also demonstrate in a simulation that the information encoded in ECoG about these movements may improve performance in a targeting task. In summary, the results in this paper suggest that detection of intended movement is possible, and may serve useful functions. %B Frontiers in Neuroengineering %V 5 %8 08/2012 %G eng %U http://www.ncbi.nlm.nih.gov/pubmed/22891058 %N 15 %R 10.3389/fneng.2012.00015 %0 Journal Article %J Neuroimage %D 2012 %T Dynamics of electrocorticographic (ECoG) activity in human temporal and frontal cortical areas during music listening. %A Potes, Cristhian %A Gunduz, Aysegul %A Peter Brunner %A Gerwin Schalk %K auditory processing %K electrocorticography (ECoG) %K high gamma activity %K sound intensity %X

Previous studies demonstrated that brain signals encode information about specific features of simple auditory stimuli or of general aspects of natural auditory stimuli. How brain signals represent the time course of specific features in natural auditory stimuli is not well understood. In this study, we show in eight human subjects that signals recorded from the surface of the brain (electrocorticography (ECoG)) encode information about the sound intensity of music. ECoG activity in the high gamma band recorded from the posterior part of the superior temporal gyrus as well as from an isolated area in the precentral gyrus was observed to be highly correlated with the sound intensity of music. These results not only confirm the role of auditory cortices in auditory processing but also point to an important role of premotor and motor cortices. They also encourage the use of ECoG activity to study more complex acoustic features of simple or natural auditory stimuli.

%B Neuroimage %V 61 %P 841-8 %8 07/2012 %G eng %U http://www.ncbi.nlm.nih.gov/pubmed/22537600 %N 4 %R 10.1016/j.neuroimage.2012.04.022 %0 Journal Article %J Epilepsy Behav %D 2012 %T Proceedings of the Third International Workshop on Advances in Electrocorticography. %A A L Ritaccio %A Beauchamp, Michael %A Bosman, Conrado %A Peter Brunner %A Chang, Edward %A Nathan E. Crone %A Gunduz, Aysegul %A Disha Gupta %A Robert T. Knight %A Leuthardt, Eric %A Litt, Brian %A Moran, Daniel %A Ojemann, Jeffrey %A Parvizi, Josef %A Ramsey, Nick %A Rieger, Jochem %A Viventi, Jonathan %A Voytek, Bradley %A Williams, Justin %A Gerwin Schalk %K Brain Mapping %K brain-computer interface %K Electrocorticography %K Gamma-frequency electroencephalography %K high-frequency oscillation %K Neuroprosthetics %K Seizure detection %K Subdural grid %X The Third International Workshop on Advances in Electrocorticography (ECoG) was convened in Washington, DC, on November 10-11, 2011. As in prior meetings, a true multidisciplinary fusion of clinicians, scientists, and engineers from many disciplines gathered to summarize contemporary experiences in brain surface recordings. The proceedings of this meeting serve as evidence of a very robust and transformative field but will yet again require revision to incorporate the advances that the following year will surely bring. %B Epilepsy Behav %V 25 %P 605-13 %8 12/2012 %G eng %U http://www.ncbi.nlm.nih.gov/pubmed/23160096 %N 4 %R 10.1016/j.yebeh.2012.09.016 %0 Journal Article %J J Vis Exp %D 2012 %T Recording Human Electrocorticographic (ECoG) Signals for Neuroscientific Research and Real-time Functional Cortical Mapping. %A Jeremy Jeremy Hill %A Disha Gupta %A Peter Brunner %A Gunduz, Aysegul %A Adamo, Matthew A %A A L Ritaccio %A Gerwin Schalk %K BCI2000 %K brain-computer interfacing %K Electrocorticography %K epilepsy monitoring %K functional brain mapping %K issue 64 %K Magnetic Resonance Imaging %K MRI %K neuroscience %K SIGFRIED %X

Neuroimaging studies of human cognitive, sensory, and motor processes are usually based on noninvasive techniques such as electroencephalography (EEG), magnetoencephalography or functional magnetic-resonance imaging. These techniques have either inherently low temporal or low spatial resolution, and suffer from low signal-to-noise ratio and/or poor high-frequency sensitivity. Thus, they are suboptimal for exploring the short-lived spatio-temporal dynamics of many of the underlying brain processes. In contrast, the invasive technique of electrocorticography (ECoG) provides brain signals that have an exceptionally high signal-to-noise ratio, less susceptibility to artifacts than EEG, and a high spatial and temporal resolution (i.e., <1 cm/<1 millisecond, respectively). ECoG involves measurement of electrical brain signals using electrodes that are implanted subdurally on the surface of the brain. Recent studies have shown that ECoG amplitudes in certain frequency bands carry substantial information about task-related activity, such as motor execution and planning, auditory processing and visual-spatial attention. Most of this information is captured in the high gamma range (around 70-110 Hz). Thus, gamma activity has been proposed as a robust and general indicator of local cortical function. ECoG can also reveal functional connectivity and resolve finer task-related spatial-temporal dynamics, thereby advancing our understanding of large-scale cortical processes. It has especially proven useful for advancing brain-computer interfacing (BCI) technology for decoding a user's intentions to enhance or improve communication and control. Nevertheless, human ECoG data are often hard to obtain because of the risks and limitations of the invasive procedures involved, and the need to record within the constraints of clinical settings. Still, clinical monitoring to localize epileptic foci offers a unique and valuable opportunity to collect human ECoG data. We describe our methods for collecting recording ECoG, and demonstrate how to use these signals for important real-time applications such as clinical mapping and brain-computer interfacing. Our example uses the BCI2000 software platform and the SIGFRIED method, an application for real-time mapping of brain functions. This procedure yields information that clinicians can subsequently use to guide the complex and laborious process of functional mapping by electrical stimulation. PREREQUISITES AND PLANNING: Patients with drug-resistant partial epilepsy may be candidates for resective surgery of an epileptic focus to minimize the frequency of seizures. Prior to resection, the patients undergo monitoring using subdural electrodes for two purposes: first, to localize the epileptic focus, and second, to identify nearby critical brain areas (i.e., eloquent cortex) where resection could result in long-term functional deficits. To implant electrodes, a craniotomy is performed to open the skull. Then, electrode grids and/or strips are placed on the cortex, usually beneath the dura. A typical grid has a set of 8 x 8 platinum-iridium electrodes of 4 mm diameter (2.3 mm exposed surface) embedded in silicon with an inter-electrode distance of 1cm. A strip typically contains 4 or 6 such electrodes in a single line. The locations for these grids/strips are planned by a team of neurologists and neurosurgeons, and are based on previous EEG monitoring, on a structural MRI of the patient's brain, and on relevant factors of the patient's history. Continuous recording over a period of 5-12 days serves to localize epileptic foci, and electrical stimulation via the implanted electrodes allows clinicians to map eloquent cortex. At the end of the monitoring period, explantation of the electrodes and therapeutic resection are performed together in one procedure. In addition to its primary clinical purpose, invasive monitoring also provides a unique opportunity to acquire human ECoG data for neuroscientific research. The decision to include a prospective patient in the research is based on the planned location of their electrodes, on the patient's performance scores on neuropsychological assessments, and on their informed consent, which is predicated on their understanding that participation in research is optional and is not related to their treatment. As with all research involving human subjects, the research protocol must be approved by the hospital's institutional review board. The decision to perform individual experimental tasks is made day-by-day, and is contingent on the patient's endurance and willingness to participate. Some or all of the experiments may be prevented by problems with the clinical state of the patient, such as post-operative facial swelling, temporary aphasia, frequent seizures, post-ictal fatigue and confusion, and more general pain or discomfort. At the Epilepsy Monitoring Unit at Albany Medical Center in Albany, New York, clinical monitoring is implemented around the clock using a 192-channel Nihon-Kohden Neurofax monitoring system. Research recordings are made in collaboration with the Wadsworth Center of the New York State Department of Health in Albany. Signals from the ECoG electrodes are fed simultaneously to the research and the clinical systems via splitter connectors. To ensure that the clinical and research systems do not interfere with each other, the two systems typically use separate grounds. In fact, an epidural strip of electrodes is sometimes implanted to provide a ground for the clinical system. Whether research or clinical recording system, the grounding electrode is chosen to be distant from the predicted epileptic focus and from cortical areas of interest for the research. Our research system consists of eight synchronized 16-channel g.USBamp amplifier/digitizer units (g.tec, Graz, Austria). These were chosen because they are safety-rated and FDA-approved for invasive recordings, they have a very low noise-floor in the high-frequency range in which the signals of interest are found, and they come with an SDK that allows them to be integrated with custom-written research software. In order to capture the high-gamma signal accurately, we acquire signals at 1200Hz sampling rate-considerably higher than that of the typical EEG experiment or that of many clinical monitoring systems. A built-in low-pass filter automatically prevents aliasing of signals higher than the digitizer can capture. The patient's eye gaze is tracked using a monitor with a built-in Tobii T-60 eye-tracking system (Tobii Tech., Stockholm, Sweden). Additional accessories such as joystick, bluetooth Wiimote (Nintendo Co.), data-glove (5(th) Dimension Technologies), keyboard, microphone, headphones, or video camera are connected depending on the requirements of the particular experiment. Data collection, stimulus presentation, synchronization with the different input/output accessories, and real-time analysis and visualization are accomplished using our BCI2000 software. BCI2000 is a freely available general-purpose software system for real-time biosignal data acquisition, processing and feedback. It includes an array of pre-built modules that can be flexibly configured for many different purposes, and that can be extended by researchers' own code in C++, MATLAB or Python. BCI2000 consists of four modules that communicate with each other via a network-capable protocol: a Source module that handles the acquisition of brain signals from one of 19 different hardware systems from different manufacturers; a Signal Processing module that extracts relevant ECoG features and translates them into output signals; an Application module that delivers stimuli and feedback to the subject; and the Operator module that provides a graphical interface to the investigator. A number of different experiments may be conducted with any given patient. The priority of experiments will be determined by the location of the particular patient's electrodes. However, we usually begin our experimentation using the SIGFRIED (SIGnal modeling For Realtime Identification and Event Detection) mapping method, which detects and displays significant task-related activity in real time. The resulting functional map allows us to further tailor subsequent experimental protocols and may also prove as a useful starting point for traditional mapping by electrocortical stimulation (ECS). Although ECS mapping remains the gold standard for predicting the clinical outcome of resection, the process of ECS mapping is time consuming and also has other problems, such as after-discharges or seizures. Thus, a passive functional mapping technique may prove valuable in providing an initial estimate of the locus of eloquent cortex, which may then be confirmed and refined by ECS. The results from our passive SIGFRIED mapping technique have been shown to exhibit substantial concurrence with the results derived using ECS mapping. The protocol described in this paper establishes a general methodology for gathering human ECoG data, before proceeding to illustrate how experiments can be initiated using the BCI2000 software platform. Finally, as a specific example, we describe how to perform passive functional mapping using the BCI2000-based SIGFRIED system.

%B J Vis Exp %8 05/2012 %G eng %U http://www.ncbi.nlm.nih.gov/pubmed/22782131 %N 64 %R 10.3791/3993 %0 Conference Proceedings %D 2011 %T Defense-related insights and solutions from neuroscience and neuroengineering %A Gunduz, Aysegul %A Gerwin Schalk %X Communication of intent usually requires motor function, which can be limiting during military missions. Determining a soldier's intent from brain signals rather than using muscles would have numerous applications for tactical combat. Brain-computer interfaces (BCIs) translate brain signals into machine readable form and could optimize a soldier's interaction with the surrounding environment. However, current BCI devices have largely remained laboratory curiosities, because current techniques either require extended training or do not have the requisite signal fidelity, because they are highly invasive and thus not safe or practical for use in humans, or because they rely on equipment (such as magnetic resonance imaging scanners) that do not allow for real-time applications and/or field deployment. The objective of our research program is to create a prototype of a system for communication and monitoring of orientation that uses brain signals to provide, in real time, an accurate assessment of the users intentional focus and imagined speech. We expect that our efforts will provide a prototype of the first intuitive brain-based communication and orientation system for human use. %8 06/2011 %G eng %R DOI: 10.1117/12.888189 %0 Conference Paper %D 2011 %T Defense-related insights and solutions from neuroscience and neuroengineering. %A Gunduz, Aysegul %A Gerwin Schalk %X Communication of intent usually requires motor function, which can be limiting during military missions. Determining a soldier's intent from brain signals rather than using muscles would have numerous applications for tactical combat. Brain-computer interfaces (BCIs) translate brain signals into machine readable form and could optimize a soldier's interaction with the surrounding environment. However, current BCI devices have largely remained laboratory curiosities, because current techniques either require extended training or do not have the requisite signal fidelity, because they are highly invasive and thus not safe or practical for use in humans, or because they rely on equipment (such as magnetic resonance imaging scanners) that do not allow for real-time applications and/or field deployment. The objective of our research program is to create a prototype of a system for communication and monitoring of orientation that uses brain signals to provide, in real time, an accurate assessment of the users intentional focus and imagined speech. We expect that our efforts will provide a prototype of the first intuitive brain-based communication and orientation system for human use. %8 06/2011 %G eng %U http://spie.org/Publications/Proceedings/Paper/10.1117/12.888189 %R 10.1117/12.888189 %0 Journal Article %J Front Hum Neurosci %D 2011 %T Neural Correlates of Covert Attention in Electrocorticographic (ECoG) Signals in Humans. %A Gunduz, Aysegul %A Peter Brunner %A Amy Daitch %A Leuthardt, E C %A A L Ritaccio %A Pesaran, Bijan %A Gerwin Schalk %K covert attention %K Electrocorticography %K intention %K motor response %K visual-spatial attention %X

Attention is a cognitive selection mechanism that allocates the limited processing resources of the brain to the sensory streams most relevant to our immediate goals, thereby enhancing responsiveness and behavioral performance. The underlying neural mechanisms of orienting attention are distributed across a widespread cortical network. While aspects of this network have been extensively studied, details about the electrophysiological dynamics of this network are scarce. In this study, we investigated attentional networks using electrocorticographic (ECoG) recordings from the surface of the brain, which combine broad spatial coverage with high temporal resolution, in five human subjects. ECoG was recorded when subjects covertly attended to a spatial location and responded to contrast changes in the presence of distractors in a modified Posner cueing task. ECoG amplitudes in the alpha, beta, and gamma bands identified neural changes associated with covert attention and motor preparation/execution in the different stages of the task. The results show that attentional engagement was primarily associated with ECoG activity in the visual, prefrontal, premotor, and parietal cortices. Motor preparation/execution was associated with ECoG activity in premotor/sensorimotor cortices. In summary, our results illustrate rich and distributed cortical dynamics that are associated with orienting attention and the subsequent motor preparation and execution. These findings are largely consistent with and expand on primate studies using intracortical recordings and human functional neuroimaging studies.

%B Front Hum Neurosci %V 5 %P 89 %8 09/2011 %G eng %U http://www.ncbi.nlm.nih.gov/pubmed/22046153 %R 10.3389/fnhum.2011.00089 %0 Journal Article %J Signal Processing %D 2009 %T Correntropy as a novel measure for nonlinearity tests. %A Gunduz, Aysegul %A Principe, Jose %K Correntropy %K kernel methods %K nonlinearity tests %K surrogate methods %X

Nonlinearity tests have become an essential step in system analysis and modeling due to the computational demands and complexity of analysis involved in nonlinear modeling. Standard nonlinear measures are either too complicated to estimate accurately (such as Lyapunov exponents and correlation dimension), or not able to capture sufficient but not necessary conditions of nonlinearity (such as time asymmetry). Correntropy is a kernel-based similarity measure which contains the information of both statistical and temporal structure of the underlying dataset. The capability of preserving nonlinear characteristics makes correntropy a suitable candidate as a measure for determining nonlinear dynamics. Moreover, since correntropy makes use of kernel methods, its estimation is computationally efficient. Using correntropy as the test statistic, nonlinearity tests based on the null hypothesis that signals of interest are realizations of linear Gaussian stochastic processes are carried out via surrogate data methods. Experiments performed on linear Gaussian, linear non-Gaussian, and nonlinear systems with varying in-band noise levels, data lengths, and kernel sizes confirm that correntropy can be employed as a discriminative measure for detecting nonlinear characteristics in time series. Results of tests performed on data collected from natural systems are in agreement with findings in time series analysis literature.

%B Signal Processing %V 89 %P 14 - 23 %8 01/2009 %G eng %U http://www.sciencedirect.com/science/article/pii/S0165168408002119 %R 10.1016/j.sigpro.2008.07.005 %0 Journal Article %J Neural Netw %D 2009 %T Mapping broadband electrocorticographic recordings to two-dimensional hand trajectories in humans Motor control features. %A Gunduz, Aysegul %A Sanchez, Justin C %A Carney, Paul R %A Principe, Jose %K Algorithms %K Brain %K Brain Mapping %K Electrodes, Implanted %K Electrodiagnosis %K Epilepsy %K Feasibility Studies %K Hand %K Humans %K Linear Models %K Motor Activity %K Neural Networks (Computer) %K Nonlinear Dynamics %K Signal Processing, Computer-Assisted %X

Brain-machine interfaces (BMIs) aim to translate the motor intent of locked-in patients into neuroprosthetic control commands. Electrocorticographical (ECoG) signals provide promising neural inputs to BMIs as shown in recent studies. In this paper, we utilize a broadband spectrum above the fast gamma ranges and systematically study the role of spectral resolution, in which the broadband is partitioned, on the reconstruction of the patients' hand trajectories. Traditionally, the power of ECoG rhythms (<200-300 Hz) has been computed in short duration bins and instantaneously and linearly mapped to cursor trajectories. Neither time embedding, nor nonlinear mappings have been previously implemented in ECoG neuroprosthesis. Herein, mapping of neural modulations to goal-oriented motor behavior is achieved via linear adaptive filters with embedded memory depths and as a novelty through echo state networks (ESNs), which provide nonlinear mappings without compromising training complexity or increasing the number of model parameters, with up to 85% correlation. Reconstructed hand trajectories are analyzed through spatial, spectral and temporal sensitivities. The superiority of nonlinear mappings in the cases of low spectral resolution and abundance of interictal activity is discussed.

%B Neural Netw %V 22 %P 1257-70 %8 11/2009 %G eng %U http://www.ncbi.nlm.nih.gov/pubmed/19647981 %N 9 %R 10.1016/j.neunet.2009.06.036 %0 Journal Article %J Conf Proc IEEE Eng Med Biol Soc %D 2008 %T Electrocorticographic interictal spike removal via denoising source separation for improved neuroprosthesis control. %A Gunduz, Aysegul %A Sanchez, Justin C %A Principe, Jose %K Algorithms %K Artifacts %K Diagnosis, Computer-Assisted %K Electroencephalography %K Epilepsy %K Evoked Potentials, Motor %K Motor Cortex %K Reproducibility of Results %K Sensitivity and Specificity %K User-Computer Interface %X

Electrocorticographic (ECoG) neuroprosthesis is a promising area of research that could provide channels of communication and control for patients who have lost their motor functions due to damage to the nervous system. However, implantation of subdural electrodes are clinically restricted to diagnostics of pre-surgical epileptic patients. Hence, interictal activity is present in the recordings across various areas of the sensorimotor cortex and suppresses the amplitude modulated features extracted to model hand trajectories. Denoising source separation is a recently introduced framework which extracts hidden structures of interest within the data through denoising the source estimates with filters designed around prior knowledge on the observations. Herein, we exploit the high amplitude quasiperiodic nature of the observed interictal spikes and show that removal of the interictal activity improves linear prediction of hand trajectories.

%B Conf Proc IEEE Eng Med Biol Soc %V 2008 %P 5224-7 %8 08/2008 %G eng %U http://www.ncbi.nlm.nih.gov/pubmed/19163895 %R 10.1109/IEMBS.2008.4650392 %0 Journal Article %J J Neurosci Methods %D 2008 %T Extraction and localization of mesoscopic motor control signals for human ECoG neuroprosthetics. %A Sanchez, Justin C %A Gunduz, Aysegul %A Carney, Paul R %A Principe, Jose %K Adolescent %K Biofeedback, Psychology %K Brain Mapping %K Cerebral Cortex %K Electroencephalography %K Epilepsies, Partial %K Female %K Hand %K Humans %K Magnetic Resonance Imaging %K Physical Therapy Modalities %K Psychomotor Performance %K Signal Processing, Computer-Assisted %K Spectrum Analysis %K User-Computer Interface %X

Electrocorticogram (ECoG) recordings for neuroprosthetics provide a mesoscopic level of abstraction of brain function between microwire single neuron recordings and the electroencephalogram (EEG). Single-trial ECoG neural interfaces require appropriate feature extraction and signal processing methods to identify and model in real-time signatures of motor events in spontaneous brain activity. Here, we develop the clinical experimental paradigm and analysis tools to record broadband (1Hz to 6kHz) ECoG from patients participating in a reaching and pointing task. Motivated by the significant role of amplitude modulated rate coding in extracellular spike based brain-machine interfaces (BMIs), we develop methods to quantify spatio-temporal intermittent increased ECoG voltages to determine if they provide viable control inputs for ECoG neural interfaces. This study seeks to explore preprocessing modalities that emphasize amplitude modulation across frequencies and channels in the ECoG above the level of noisy background fluctuations in order to derive the commands for complex, continuous control tasks. Preliminary experiments show that it is possible to derive online predictive models and spatially localize the generation of commands in the cortex for motor tasks using amplitude modulated ECoG.

%B J Neurosci Methods %V 167 %P 63-81 %8 01/2008 %G eng %U http://www.ncbi.nlm.nih.gov/pubmed/17582507 %N 1 %R 10.1016/j.jneumeth.2007.04.019 %0 Journal Article %J Conf Proc IEEE Eng Med Biol Soc %D 2006 %T Analysis of the correlation between local field potentials and neuronal firing rate in the motor cortex. %A Wang, Yiwen %A Sanchez, Justin C %A Principe, Jose %A Mitzelfelt, Jeremiah D %A Gunduz, Aysegul %K Action Potentials %K Animals %K Brain %K Brain Mapping %K Electric Stimulation %K Electrodes %K Evoked Potentials, Motor %K Male %K Models, Statistical %K Motor Cortex %K Neurons %K Rats %K Rats, Sprague-Dawley %K Signal Processing, Computer-Assisted %K Synaptic Transmission %X

Neuronal firing rate has been the signal of choice for invasive motor brain machine interfaces (BMI). The use of local field potentials (LFP) in BMI experiments may provide additional dendritic information about movement intent and may improve performance. Here we study the time-varying amplitude modulated relationship between local field potentials (LFP) and single unit activity (SUA) in the motor cortex. We record LFP and SUA in the primary motor cortex of rats trained to perform a lever pressing task, and evaluate the correlation between pairs of peri-event time histograms (PETH) and movement evoked local field potentials (mEP) at the same electrode. Three different correlation coefficients were calculated and compared between the neuronal PETH and the magnitude and power of the mEP. Correlation as high as 0.7 for some neurons occurred between the PETH and the mEP magnitude. As expected, the correlations between the single trial LFP and SUV are much lower due to the inherent variability of both signals.

%B Conf Proc IEEE Eng Med Biol Soc %V 1 %P 6185-8 %8 09/2006 %G eng %U http://www.ncbi.nlm.nih.gov/pubmed/17946745 %R 10.1109/IEMBS.2006.260516 %0 Journal Article %J Conf Proc IEEE Eng Med Biol Soc %D 2005 %T On-line Detection of Perceptual Signatures in Multichannel ECoG. %A Gunduz, Aysegul %A Principe, Jose %A Freeman, Walter %X

Neocortical ECoG studies have unveiled the presence of active states - spatial patterns of amplitude modulation- in the beta- gamma ranges in the presence of conditioned stimuli that resemble cinematographic frames. These sequences of active frames emerge with abrupt phase resettings, followed by resynchronization and stabilization over channels, and magnified intensity. An online pattern recognizer that captures the spatial and spectral characteristics of the active frames is presented. The results of detection are confirmed via high occurrences of pragmatic information, defined by the ratio of pattern intensity to pattern stability.

%B Conf Proc IEEE Eng Med Biol Soc %V 4 %P 3683-6 %8 09/2005 %G eng %U http://ieeexplore.ieee.org/xpl/freeabs_all.jsp?arnumber=1617281&abstractAccess=no&userType=inst %R 10.1109/IEMBS.2005.1617281