02272nas a2200277 4500008004100000245008600041210006900127260001200196520140600208653003301614653002901647653002001676653000901696653002501705653002401730653002001754653002201774100001401796700001401810700002401824700001501848700001901863700001901882700001601901856007701917 2015 eng d00aBrain-to-text: Decoding spoken sentences from phone representations in the brain.0 aBraintotext Decoding spoken sentences from phone representations c06/20153 aIt has long been speculated whether communication between humans and machines based on natural speech related cortical activity is possible. Over the past decade, studies have suggested that it is feasible to recognize isolated aspects of speech from neural signals, such as auditory features, phones or one of a few isolated words. However, until now it remained an unsolved challenge to decode continuously spoken speech from the neural substrate associated with speech and language processing. Here, we show for the first time that continuously spoken speech can be decoded into the expressed words from intracranial electrocorticographic (ECoG) recordings.Specifically, we implemented a system, which we call Brain-To-Text that models single phones, employs techniques from automatic speech recognition (ASR), and thereby transforms brain activity while speaking into the corresponding textual representation. Our results demonstrate that our system can achieve word error rates as low as 25% and phone error rates below 50%. Additionally, our approach contributes to the current understanding of the neural basis of continuous speech production by identifying those cortical regions that hold substantial information about individual phones. In conclusion, the Brain-To- Text system described in this paper represents an important step toward human-machine communication based on imagined speech.10aautomatic speech recognition10abrain-computer interface10abroadband gamma10aECoG10aElectrocorticography10apattern recognition10aspeech decoding10aspeech production1 aHerff, C.1 aHeger, D.1 ade Pesters, Adriana1 aTelaar, D.1 aBrunner, Peter1 aSchalk, Gerwin1 aSchultz, T. uhttp://journal.frontiersin.org/article/10.3389/fnins.2015.00217/abstract02766nas a2200205 4500008004100000245009500041210007100136260001200207490000700219520209000226653002902316653002102345653003002366653002102396653002702417100002502444700002402469700001902493856004802512 2014 eng d00aA general method for assessing brain–computer interface performance and its limitations.0 ageneral method for assessing brain–computer interface performanc c03/20140 v113 aObjective. When researchers evaluate brain–computer interface (BCI) systems, we want quantitative answers to questions such as: How good is the system's performance? How good does it need to be? and: Is it capable of reaching the desired level in future? In response to the current lack of objective, quantitative, study-independent approaches, we introduce methods that help to address such questions. We identified three challenges: (I) the need for efficient measurement techniques that adapt rapidly and reliably to capture a wide range of performance levels; (II) the need to express results in a way that allows comparison between similar but non-identical tasks; (III) the need to measure the extent to which certain components of a BCI system (e.g. the signal processing pipeline) not only support BCI performance, but also potentially restrict the maximum level it can reach. Approach. For challenge (I), we developed an automatic staircase method that adjusted task difficulty adaptively along a single abstract axis. For challenge (II), we used the rate of information gain between two Bernoulli distributions: one reflecting the observed success rate, the other reflecting chance performance estimated by a matched random-walk method. This measure includes Wolpaw's information transfer rate as a special case, but addresses the latter's limitations including its restriction to item-selection tasks. To validate our approach and address challenge (III), we compared four healthy subjects' performance using an EEG-based BCI, a 'Direct Controller' (a high-performance hardware input device), and a 'Pseudo-BCI Controller' (the same input device, but with control signals processed by the BCI signal processing pipeline). Main results. Our results confirm the repeatability and validity of our measures, and indicate that our BCI signal processing pipeline reduced attainable performance by about 33% (21 bits/min). Significance. Our approach provides a flexible basis for evaluating BCI performance and its limitations, across a wide range of tasks and task difficulties.10abrain-computer interface10ainformation gain10ainformation transfer rate10aNeuroprosthetics10aperformance evaluation1 aHill, Jeremy, Jeremy1 aHäuser, Ann-Katrin1 aSchalk, Gerwin uhttp://www.ncbi.nlm.nih.gov/pubmed/2465840601857nas a2200421 4500008004100000022001400041245008900055210006900144260001200213300001100225490000700236520059500243653001800838653002900856653003500885653002500920653002300945653004300968653003201011653002101043653002201064653001801086100001801104700001901122700002001141700001701161700002401178700001901202700002101221700002001242700002301262700002201285700001601307700002401323700002101347700001901368856004801387 2014 eng d a1525-506900aProceedings of the Fifth International Workshop on Advances in Electrocorticography.0 aProceedings of the Fifth International Workshop on Advances in E c12/2014 a183-920 v413 a
The Fifth International Workshop on Advances in Electrocorticography convened in San Diego, CA, on November 7-8, 2013. Advancements in methodology, implementation, and commercialization across both research and in the interval year since the last workshop were the focus of the gathering. Electrocorticography (ECoG) is now firmly established as a preferred signal source for advanced research in functional, cognitive, and neuroprosthetic domains. Published output in ECoG fields has increased tenfold in the past decade. These proceedings attempt to summarize the state of the art.
10aBrain Mapping10abrain-computer interface10aelectrical stimulation mapping10aElectrocorticography10afunctional mapping10aGamma-frequency electroencephalography10aHigh-frequency oscillations10aNeuroprosthetics10aSeizure detection10aSubdural grid1 aRitaccio, A L1 aBrunner, Peter1 aGunduz, Aysegul1 aHermes, Dora1 aHirsch, Lawrence, J1 aJacobs, Joshua1 aKamada, Kyousuke1 aKastner, Sabine1 aKnight, Robert, T.1 aLesser, Ronald, P1 aMiller, Kai1 aSejnowski, Terrence1 aWorrell, Gregory1 aSchalk, Gerwin uhttp://www.ncbi.nlm.nih.gov/pubmed/2546121302832nas a2200253 4500008004100000022001400041245012700055210006900182260001200251490000600263520200400269653002302273653003802296653002902334653002302363653001702386653001702403100002502420700002102445700002402466700002102490700001902511856004802530 2012 eng d a1662-453X00aCommunication and control by listening: towards optimal design of a two-class auditory streaming brain-computer interface.0 aCommunication and control by listening towards optimal design of c12/20120 v63 aMost brain-computer interface (BCI) systems require users to modulate brain signals in response to visual stimuli. Thus, they may not be useful to people with limited vision, such as those with severe paralysis. One important approach for overcoming this issue is auditory streaming, an approach whereby a BCI system is driven by shifts of attention between two simultaneously presented auditory stimulus streams. Motivated by the long-term goal of translating such a system into a reliable, simple yes-no interface for clinical usage, we aim to answer two main questions. First, we asked which of two previously published variants provides superior performance: a fixed-phase (FP) design in which the streams have equal period and opposite phase, or a drifting-phase (DP) design where the periods are unequal. We found FP to be superior to DP (p = 0.002): average performance levels were 80 and 72% correct, respectively. We were also able to show, in a pilot with one subject, that auditory streaming can support continuous control and neurofeedback applications: by shifting attention between ongoing left and right auditory streams, the subject was able to control the position of a paddle in a computer game. Second, we examined whether the system is dependent on eye movements, since it is known that eye movements and auditory attention may influence each other, and any dependence on the ability to move one’s eyes would be a barrier to translation to paralyzed users. We discovered that, despite instructions, some subjects did make eye movements that were indicative of the direction of attention. However, there was no correlation, across subjects, between the reliability of the eye movement signal and the reliability of the BCI system, indicating that our system was configured to work independently of eye movement. Together, these findings are an encouraging step forward toward BCIs that provide practical communication and control options for the most severely paralyzed users. 10aauditory attention10aauditory event-related potentials10abrain-computer interface10adichotic listening10aN1 potential10aP3 potential1 aHill, Jeremy, Jeremy1 aMoinuddin, Aisha1 aHäuser, Ann-Katrin1 aKienzle, Stephan1 aSchalk, Gerwin uhttp://www.ncbi.nlm.nih.gov/pubmed/2326731201889nas a2200469 4500008004100000022001400041245008900055210006900144260001200213300001100225490000700236520052900243653001800772653002900790653002500819653004300844653003100887653002100918653002200939653001800961100001800979700002300997700002001020700001901040700001801059700002201077700002001099700001701119700002301136700002001159700001601179700001801195700002101213700001901234700001701253700001901270700002201289700002001311700002101331700001901352856004801371 2012 eng d a1525-506900aProceedings of the Third International Workshop on Advances in Electrocorticography.0 aProceedings of the Third International Workshop on Advances in E c12/2012 a605-130 v253 aThe Third International Workshop on Advances in Electrocorticography (ECoG) was convened in Washington, DC, on November 10-11, 2011. As in prior meetings, a true multidisciplinary fusion of clinicians, scientists, and engineers from many disciplines gathered to summarize contemporary experiences in brain surface recordings. The proceedings of this meeting serve as evidence of a very robust and transformative field but will yet again require revision to incorporate the advances that the following year will surely bring.10aBrain Mapping10abrain-computer interface10aElectrocorticography10aGamma-frequency electroencephalography10ahigh-frequency oscillation10aNeuroprosthetics10aSeizure detection10aSubdural grid1 aRitaccio, A L1 aBeauchamp, Michael1 aBosman, Conrado1 aBrunner, Peter1 aChang, Edward1 aCrone, Nathan, E.1 aGunduz, Aysegul1 aGupta, Disha1 aKnight, Robert, T.1 aLeuthardt, Eric1 aLitt, Brian1 aMoran, Daniel1 aOjemann, Jeffrey1 aParvizi, Josef1 aRamsey, Nick1 aRieger, Jochem1 aViventi, Jonathan1 aVoytek, Bradley1 aWilliams, Justin1 aSchalk, Gerwin uhttp://www.ncbi.nlm.nih.gov/pubmed/2316009601769nas a2200373 4500008004100000245003800041210003700079260001200116300000900128490000600137520083700143653000800980653002900988653001601017100001801033700001601051700001501067700002101082700002101103700002101124700001201145700001501157700001601172700002001188700001301208700002101221700001501242700001901257700001601276700001601292700001501308700002401323856004801347 2012 eng d00aReview of the BCI Competition IV.0 aReview of the BCI Competition IV c07/2012 a1-310 v63 aThe BCI competition IV stands in the tradition of prior BCI competitions that aim to provide high quality neuroscientific data for open access to the scientific community. As experienced already in prior competitions not only scientists from the narrow field of BCI compete, but scholars with a broad variety of backgrounds and nationalities. They include high specialists as well as students. The goals of all BCI competitions have always been to challenge with respect to novel paradigms and complex data. We report on the following challenges: (1) asynchronous data, (2) synthetic, (3) multi-class continuous data, (4) session-to-session transfer, (5) directionally modulated MEG, (6) finger movements recorded by ECoG. As after past competitions, our hope is that winning entries may enhance the analysis methods of future BCIs.10aBCI10abrain-computer interface10acompetition1 aTangermann, M1 aMuller, K R1 aAertsen, A1 aBirbaumer, Niels1 aBraun, Christoph1 aBrunner, Clemens1 aLeeb, R1 aMehring, C1 aMiller, K J1 aMueller-Putz, G1 aNolte, G1 aPfurtscheller, G1 aPreissl, H1 aSchalk, Gerwin1 aSchlögl, A1 aVidaurre, C1 aWaldert, S1 aBlankertz, Benjamin uhttp://www.ncbi.nlm.nih.gov/pubmed/2281165702550nas a2200253 4500008004100000022001400041245009200055210006900147260001200216300000800228490000600236520180100242653002902043653002302072653002602095653001902121653002102140653002002161100001802181700001002199700002002209700001902229856004802248 2011 eng d a1662-453X00aPrior knowledge improves decoding of finger flexion from electrocorticographic signals.0 aPrior knowledge improves decoding of finger flexion from electro c11/2011 a1270 v53 aBrain-computer interfaces (BCIs) use brain signals to convey a user's intent. Some BCI approaches begin by decoding kinematic parameters of movements from brain signals, and then proceed to using these signals, in absence of movements, to allow a user to control an output. Recent results have shown that electrocorticographic (ECoG) recordings from the surface of the brain in humans can give information about kinematic parameters (e.g., hand velocity or finger flexion). The decoding approaches in these studies usually employed classical classification/regression algorithms that derive a linear mapping between brain signals and outputs. However, they typically only incorporate little prior information about the target movement parameter. In this paper, we incorporate prior knowledge using a Bayesian decoding method, and use it to decode finger flexion from ECoG signals. Specifically, we exploit the constraints that govern finger flexion and incorporate these constraints in the construction, structure, and the probabilistic functions of the prior model of a switched non-parametric dynamic system (SNDS). Given a measurement model resulting from a traditional linear regression method, we decoded finger flexion using posterior estimation that combined the prior and measurement models. Our results show that the application of the Bayesian decoding model, which incorporates prior knowledge, improves decoding performance compared to the application of a linear regression model, which does not incorporate prior knowledge. Thus, the results presented in this paper may ultimately lead to neurally controlled hand prostheses with full fine-grained finger articulation.
10abrain-computer interface10adecoding algorithm10aelectrocorticographic10afinger flexion10amachine learning10aprior knowledge1 aWang, Zuoguan1 aJi, Q1 aMiller, John, W1 aSchalk, Gerwin uhttp://www.ncbi.nlm.nih.gov/pubmed/2214494403409nas a2200253 4500008004100000022001400041245009700055210006900152260001200221300000600233490000600239520266600245653002902911653002502940653002802965653000902993653001203002100001903014700001803033700002203051700001503073700001903088856004803107 2011 eng d a1662-453X00aRapid Communication with a "P300" Matrix Speller Using Electrocorticographic Signals (ECoG).0 aRapid Communication with a P300 Matrix Speller Using Electrocort c02/2011 a50 v53 aA brain-computer interface (BCI) can provide a non-muscular communication channel to severely disabled people. One particular realization of a BCI is the P300 matrix speller that was originally described by Farwell and Donchin (1988). This speller uses event-related potentials (ERPs) that include the P300 ERP. All previous online studies of the P300 matrix speller used scalp-recorded electroencephalography (EEG) and were limited in their communication performance to only a few characters per minute. In our study, we investigated the feasibility of using electrocorticographic (ECoG) signals for online operation of the matrix speller, and determined associated spelling rates. We used the matrix speller that is implemented in the BCI2000 system. This speller used ECoG signals that were recorded from frontal, parietal, and occipital areas in one subject. This subject spelled a total of 444 characters in online experiments. The results showed that the subject sustained a rate of 17 characters/min (i.e., 69 bits/min), and achieved a peak rate of 22 characters/min (i.e., 113 bits/min). Detailed analysis of the results suggests that ERPs over visual areas (i.e., visual evoked potentials) contribute significantly to the performance of the matrix speller BCI system. Our results also point to potential reasons for the apparent advantages in spelling performance of ECoG compared to EEG. Thus, with additional verification in more subjects, these results may further extend the communication options for people with serious neuromuscular disabilities.
10abrain-computer interface10aElectrocorticography10aevent-related potential10aP30010aspeller1 aBrunner, Peter1 aRitaccio, A L1 aEmrich, Joseph, F1 aBischof, H1 aSchalk, Gerwin uhttp://www.ncbi.nlm.nih.gov/pubmed/2136935101207nas a2200181 4500008004100000020002200041245003200063210003000095260001900125520069300144653000800837653002900845653002300874653002200897100001900919700001900938856006800957 2009 eng d a978-3-642-02811-300aBrain-Computer Interaction.0 aBrainComputer Interaction bSpringerc20093 aDetection and automated interpretation of attention-related or intention-related brain activity carries significant promise for many military and civilian applications. This interpretation of brain activity could provide information about a person’s intended movements, imagined movements, or attentional focus, and thus could be valuable for optimizing or replacing traditional motor-based communication between a person and a computer or other output devices. We describe here the objective and preliminary results of our studies in this area.
10aBCI10abrain-computer interface10aneural engineering10aneural prosthesis1 aBrunner, Peter1 aSchalk, Gerwin uhttp://link.springer.com/chapter/10.1007%2F978-3-642-02812-0_8101351nas a2200409 4500008004100000020001800041245007500059210006900134260003300203653002700236653003000263653001900293653002900312653002500341653003300366653003400399653000800433653002700441653003900468653002100507653001700528653002000545653002900565653003000594653001500624653001800639653002300657653002600680653001100706653002300717653004000740100002400780700001900804700002600823700002100849856007100870 2005 eng d a0-7803-8710-400aTracking of the mu rhythm using an empirically derived matched filter.0 aTracking of the mu rhythm using an empirically derived matched f aArlington, VAbIEEEc03/200510abioelectric potentials10aBrain Computer Interfaces10abrain modeling10abrain-computer interface10acommunication device10acommunication system control10acortical mu rhythm modulation10aEEG10aElectroencephalography10aempirically derived matched filter10ahandicapped aids10alaboratories10amatched filters10amedical signal detection10amedical signal processing10amonitoring10amotor imagery10amu rhythm tracking10anoninvasive treatment10arhythm10asynchronous motors10atwo-dimensional cursor control data1 aKrusienski, Dean, J1 aSchalk, Gerwin1 aMcFarland, Dennis, J.1 aWolpaw, Jonathan uhttp://ieeexplore.ieee.org/xpl/articleDetails.jsp?arnumber=141955902685nas a2200433 4500008004100000022001400041245011000055210006900165260001200234300001600246490000700262520140200269653003101671653000801702653001601710653002901726653000801755653000801763653002801771653003601799653001401835653000901849653001901858653003201877653002901909100002401938700002601962700001901988700002402007700001902031700002102050700002002071700002002091700002402111700002402135700002302159700002102182856004802203 2004 eng d a0018-929400aThe BCI Competition 2003: progress and perspectives in detection and discrimination of EEG single trials.0 aBCI Competition 2003 progress and perspectives in detection and c06/2004 a1044–10510 v513 aInterest in developing a new method of man-to-machine communication–a brain-computer interface (BCI)–has grown steadily over the past few decades. BCIs create a new communication channel between the brain and an output device by bypassing conventional motor output pathways of nerves and muscles. These systems use signals recorded from the scalp, the surface of the cortex, or from inside the brain to enable users to control a variety of applications including simple word-processing software and orthotics. BCI technology could therefore provide a new communication and control option for individuals who cannot otherwise express their wishes to the outside world. Signal processing and classification methods are essential tools in the development of improved BCI technology. We organized the BCI Competition 2003 to evaluate the current state of the art of these tools. Four laboratories well versed in EEG-based BCI research provided six data sets in a documented format. We made these data sets (i.e., labeled training sets and unlabeled test sets) and their descriptions available on the Internet. The goal in the competition was to maximize the performance measure for the test labels. Researchers worldwide tested their algorithms and competed for the best classification results. This paper describes the six data sets and the results and function of the most successful algorithms.10aaugmentative communication10aBCI10abeta-rhythm10abrain-computer interface10aEEG10aERP10aimagined hand movements10alateralized readiness potential10amu-rhythm10aP30010aRehabilitation10asingle-trial classification10aslow cortical potentials1 aBlankertz, Benjamin1 aMüller, Klaus-Robert1 aCurio, Gabriel1 aVaughan, Theresa, M1 aSchalk, Gerwin1 aWolpaw, Jonathan1 aSchlögl, Alois1 aNeuper, Christa1 aPfurtscheller, Gert1 aHinterberger, Thilo1 aSchröder, Michael1 aBirbaumer, Niels uhttp://www.ncbi.nlm.nih.gov/pubmed/1518887601315nas a2200289 4500008004100000022001400041245006100055210005800116260001200174300001600186490000800202520045900210653003100669653002900700653002700729653002000756653002900776653002800805653001400833653001900847653002400866100001900890700002100909700002600930700002100956856004800977 2000 eng d a1388-245700aEEG-based communication: presence of an error potential.0 aEEGbased communication presence of an error potential c12/2000 a2138–21440 v1113 aEEG-based communication could be a valuable new augmentative communication technology for those with severe motor disabilities. Like all communication methods, it faces the problem of errors in transmission. In the Wadsworth EEG-based brain-computer interface (BCI) system, subjects learn to use mu or beta rhythm amplitude to move a cursor to targets on a computer screen. While cursor movement is highly accurate in trained subjects, it is not perfect.10aaugmentative communication10abrain-computer interface10aElectroencephalography10aerror potential10aerror related negativity10aevent related potential10amu rhythm10aRehabilitation10asensorimotor cortex1 aSchalk, Gerwin1 aWolpaw, Jonathan1 aMcFarland, Dennis, J.1 aPfurtscheller, G uhttp://www.ncbi.nlm.nih.gov/pubmed/11090763