TY - JOUR T1 - A practical, intuitive brain-computer interface for communicating 'yes' or 'no' by listening. JF - J Neural Eng Y1 - 2014 A1 - Jeremy Jeremy Hill A1 - Ricci, Erin A1 - Haider, Sameah A1 - McCane, Lynn M A1 - Susan M Heckman A1 - Jonathan Wolpaw A1 - Theresa M Vaughan KW - Adult KW - Aged KW - Algorithms KW - Auditory Perception KW - brain-computer interfaces KW - Communication Aids for Disabled KW - Electroencephalography KW - Equipment Design KW - Equipment Failure Analysis KW - Female KW - Humans KW - Male KW - Man-Machine Systems KW - Middle Aged KW - Quadriplegia KW - Treatment Outcome KW - User-Computer Interface AB - OBJECTIVE: Previous work has shown that it is possible to build an EEG-based binary brain-computer interface system (BCI) driven purely by shifts of attention to auditory stimuli. However, previous studies used abrupt, abstract stimuli that are often perceived as harsh and unpleasant, and whose lack of inherent meaning may make the interface unintuitive and difficult for beginners. We aimed to establish whether we could transition to a system based on more natural, intuitive stimuli (spoken words 'yes' and 'no') without loss of performance, and whether the system could be used by people in the locked-in state. APPROACH: We performed a counterbalanced, interleaved within-subject comparison between an auditory streaming BCI that used beep stimuli, and one that used word stimuli. Fourteen healthy volunteers performed two sessions each, on separate days. We also collected preliminary data from two subjects with advanced amyotrophic lateral sclerosis (ALS), who used the word-based system to answer a set of simple yes-no questions. MAIN RESULTS: The N1, N2 and P3 event-related potentials elicited by words varied more between subjects than those elicited by beeps. However, the difference between responses to attended and unattended stimuli was more consistent with words than beeps. Healthy subjects' performance with word stimuli (mean 77% ± 3.3 s.e.) was slightly but not significantly better than their performance with beep stimuli (mean 73% ± 2.8 s.e.). The two subjects with ALS used the word-based BCI to answer questions with a level of accuracy similar to that of the healthy subjects. SIGNIFICANCE: Since performance using word stimuli was at least as good as performance using beeps, we recommend that auditory streaming BCI systems be built with word stimuli to make the system more pleasant and intuitive. Our preliminary data show that word-based streaming BCI is a promising tool for communication by people who are locked in. VL - 11 UR - http://www.ncbi.nlm.nih.gov/pubmed/24838278 IS - 3 ER - TY - JOUR T1 - Silent Communication: toward using brain signals. JF - IEEE Pulse Y1 - 2012 A1 - Pei, Xiao-Mei A1 - Jeremy Jeremy Hill A1 - Gerwin Schalk KW - Animals KW - Brain KW - Brain Waves KW - Humans KW - Movement KW - User-Computer Interface AB -

From the 1980s movie Firefox to the more recent Avatar, popular science fiction has speculated about the possibility of a persons thoughts being read directly from his or her brain. Such braincomputer interfaces (BCIs) might allow people who are paralyzed to communicate with and control their environment, and there might also be applications in military situations wherever silent user-to-user communication is desirable. Previous studies have shown that BCI systems can use brain signals related to movements and movement imagery or attention-based character selection. Although these systems have successfully demonstrated the possibility to control devices using brain function, directly inferring which word a person intends to communicate has been elusive. A BCI using imagined speech might provide such a practical, intuitive device. Toward this goal, our studies to date addressed two scientific questions: (1) Can brain signals accurately characterize different aspects of speech? (2) Is it possible to predict spoken or imagined words or their components using brain signals?

VL - 3 UR - http://www.ncbi.nlm.nih.gov/pubmed/22344951 IS - 1 ER - TY - JOUR T1 - Causal influence of gamma oscillations on the sensorimotor rhythm. JF - Neuroimage Y1 - 2011 A1 - Grosse-Wentrup, Moritz A1 - Schölkopf, B A1 - Jeremy Jeremy Hill KW - Adult KW - Cerebral Cortex KW - Electroencephalography KW - Female KW - Humans KW - Imagination KW - Male KW - Signal Processing, Computer-Assisted KW - User-Computer Interface AB -

Gamma oscillations of the electromagnetic field of the brain are known to be involved in a variety of cognitive processes, and are believed to be fundamental for information processing within the brain. While gamma oscillations have been shown to be correlated with brain rhythms at different frequencies, to date no empirical evidence has been presented that supports a causal influence of gamma oscillations on other brain rhythms. In this work, we study the relation of gamma oscillations and the sensorimotor rhythm (SMR) in healthy human subjects using electroencephalography. We first demonstrate that modulation of the SMR, induced by motor imagery of either the left or right hand, is positively correlated with the power of frontal and occipital gamma oscillations, and negatively correlated with the power of centro-parietal gamma oscillations. We then demonstrate that the most simple causal structure, capable of explaining the observed correlation of gamma oscillations and the SMR, entails a causal influence of gamma oscillations on the SMR. This finding supports the fundamental role attributed to gamma oscillations for information processing within the brain, and is of particular importance for brain-computer interfaces (BCIs). As modulation of the SMR is typically used in BCIs to infer a subject's intention, our findings entail that gamma oscillations have a causal influence on a subject's capability to utilize a BCI for means of communication.

VL - 56 UR - http://www.ncbi.nlm.nih.gov/pubmed/20451626 IS - 2 ER - TY - JOUR T1 - Closing the sensorimotor loop: haptic feedback facilitates decoding of motor imagery. JF - J Neural Eng Y1 - 2011 A1 - Gomez-Rodriguez, M A1 - Peters, J A1 - Jeremy Jeremy Hill A1 - Schölkopf, B A1 - Gharabaghi, A A1 - Grosse-Wentrup, Moritz KW - Brain KW - Evoked Potentials, Motor KW - Evoked Potentials, Somatosensory KW - Feedback, Physiological KW - Female KW - Humans KW - Imagination KW - Male KW - Movement KW - Robotics KW - Touch KW - User-Computer Interface AB -

The combination of brain-computer interfaces (BCIs) with robot-assisted physical therapy constitutes a promising approach to neurorehabilitation of patients with severe hemiparetic syndromes caused by cerebrovascular brain damage (e.g. stroke) and other neurological conditions. In such a scenario, a key aspect is how to reestablish the disrupted sensorimotor feedback loop. However, to date it is an open question how artificially closing the sensorimotor feedback loop influences the decoding performance of a BCI. In this paper, we answer this issue by studying six healthy subjects and two stroke patients. We present empirical evidence that haptic feedback, provided by a seven degrees of freedom robotic arm, facilitates online decoding of arm movement intention. The results support the feasibility of future rehabilitative treatments based on the combination of robot-assisted physical therapy with BCIs.

VL - 8 UR - http://www.ncbi.nlm.nih.gov/pubmed/21474878 IS - 3 ER - TY - JOUR T1 - A graphical model framework for decoding in the visual ERP-based BCI speller. JF - Neural Comput Y1 - 2011 A1 - Martens, S M M A1 - Mooij, J M A1 - Jeremy Jeremy Hill A1 - Farquhar, Jason A1 - Schölkopf, B KW - Artificial Intelligence KW - Computer User Training KW - Discrimination Learning KW - Electroencephalography KW - Evoked Potentials KW - Evoked Potentials, Visual KW - Humans KW - Language KW - Models, Neurological KW - Models, Theoretical KW - Reading KW - Signal Processing, Computer-Assisted KW - User-Computer Interface KW - Visual Cortex KW - Visual Perception AB -

We present a graphical model framework for decoding in the visual ERP-based speller system. The proposed framework allows researchers to build generative models from which the decoding rules are obtained in a straightforward manner. We suggest two models for generating brain signals conditioned on the stimulus events. Both models incorporate letter frequency information but assume different dependencies between brain signals and stimulus events. For both models, we derive decoding rules and perform a discriminative training. We show on real visual speller data how decoding performance improves by incorporating letter frequency information and using a more realistic graphical model for the dependencies between the brain signals and the stimulus events. Furthermore, we discuss how the standard approach to decoding can be seen as a special case of the graphical model framework. The letter also gives more insight into the discriminative approach for decoding in the visual speller system.

VL - 23 UR - http://www.ncbi.nlm.nih.gov/pubmed/20964540 IS - 1 ER - TY - JOUR T1 - Transition from the locked in to the completely locked-in state: a physiological analysis. JF - Clin Neurophysiol Y1 - 2011 A1 - Murguialday, A Ramos A1 - Jeremy Jeremy Hill A1 - Bensch, M A1 - Martens, S M M A1 - S Halder A1 - Nijboer, F A1 - Schoelkopf, Bernhard A1 - Niels Birbaumer A1 - Gharabaghi, A KW - Adult KW - Amyotrophic Lateral Sclerosis KW - Area Under Curve KW - Brain KW - Communication Aids for Disabled KW - Disease Progression KW - Electroencephalography KW - Electromyography KW - Humans KW - Male KW - Signal Processing, Computer-Assisted KW - User-Computer Interface AB -

OBJECTIVE: 

To clarify the physiological and behavioral boundaries between locked-in (LIS) and the completely locked-in state (CLIS) (no voluntary eye movements, no communication possible) through electrophysiological data and to secure brain-computer-interface (BCI) communication.

METHODS: 

Electromyography from facial muscles, external anal sphincter (EAS), electrooculography and electrocorticographic data during different psychophysiological tests were acquired to define electrophysiological differences in an amyotrophic lateral sclerosis (ALS) patient with an intracranially implanted grid of 112 electrodes for nine months while the patient passed from the LIS to the CLIS.

RESULTS: 

At the very end of the LIS there was no facial muscle activity, nor external anal sphincter but eye control. Eye movements were slow and lasted for short periods only. During CLIS event related brainpotentials (ERP) to passive limb movements and auditory stimuli were recorded, vibrotactile stimulation of different body parts resulted in no ERP response.

CONCLUSIONS: 

The results presented contradict the commonly accepted assumption that the EAS is the last remaining muscle under voluntary control and demonstrate complete loss of eye movements in CLIS. The eye muscle was shown to be the last muscle group under voluntary control. The findings suggest ALS as a multisystem disorder, even affecting afferent sensory pathways.

SIGNIFICANCE: 

Auditory and proprioceptive brain-computer-interface (BCI) systems are the only remaining communication channels in CLIS.

VL - 122 UR - http://www.ncbi.nlm.nih.gov/pubmed/20888292 IS - 5 ER - TY - JOUR T1 - A note on ethical aspects of BCI. JF - Neural Netw Y1 - 2009 A1 - Haselager, Pim A1 - Vlek, Rutger A1 - Jeremy Jeremy Hill A1 - Nijboer, F KW - Bioethics KW - Brain KW - Communication KW - Communications Media KW - Cooperative Behavior KW - Humans KW - Informed Consent KW - Professional-Patient Relations KW - Quadriplegia KW - User-Computer Interface AB -

This paper focuses on ethical aspects of BCI, as a research and a clinical tool, that are challenging for practitioners currently working in the field. Specifically, the difficulties involved in acquiring informed consent from locked-in patients are investigated, in combination with an analysis of the shared moral responsibility in BCI teams, and the complications encountered in establishing effective communication with media.

VL - 22 UR - http://www.ncbi.nlm.nih.gov/pubmed/19616405 IS - 9 ER - TY - JOUR T1 - Overlap and refractory effects in a brain-computer interface speller based on the visual P300 event-related potential. JF - J Neural Eng Y1 - 2009 A1 - Martens, S M M A1 - Jeremy Jeremy Hill A1 - Farquhar, Jason A1 - Schölkopf, B KW - Algorithms KW - Brain KW - Cognition KW - Computer Simulation KW - Electroencephalography KW - Event-Related Potentials, P300 KW - Humans KW - Models, Neurological KW - Pattern Recognition, Automated KW - Photic Stimulation KW - Semantics KW - Signal Processing, Computer-Assisted KW - Task Performance and Analysis KW - User-Computer Interface KW - Writing AB -

We reveal the presence of refractory and overlap effects in the event-related potentials in visual P300 speller datasets, and we show their negative impact on the performance of the system. This finding has important implications for how to encode the letters that can be selected for communication. However, we show that such effects are dependent on stimulus parameters: an alternative stimulus type based on apparent motion suffers less from the refractory effects and leads to an improved letter prediction performance.

VL - 6 UR - http://www.ncbi.nlm.nih.gov/pubmed/19255462 IS - 2 ER - TY - JOUR T1 - Voluntary brain regulation and communication with electrocorticogram signals. JF - Epilepsy Behav Y1 - 2008 A1 - Hinterberger, T. A1 - Widman, Guido A1 - Lal, T.N A1 - Jeremy Jeremy Hill A1 - Tangermann, Michael A1 - Rosenstiel, W. A1 - Schölkopf, B A1 - Elger, Christian A1 - Niels Birbaumer KW - Adult KW - Biofeedback, Psychology KW - Cerebral Cortex KW - Communication Aids for Disabled KW - Dominance, Cerebral KW - Electroencephalography KW - Epilepsies, Partial KW - Female KW - Humans KW - Imagination KW - Male KW - Middle Aged KW - Motor Activity KW - Motor Cortex KW - Signal Processing, Computer-Assisted KW - Software KW - Somatosensory Cortex KW - Theta Rhythm KW - User-Computer Interface KW - Writing AB -

Brain-computer interfaces (BCIs) can be used for communication in writing without muscular activity or for learning to control seizures by voluntary regulation of brain signals such as the electroencephalogram (EEG). Three of five patients with epilepsy were able to spell their names with electrocorticogram (ECoG) signals derived from motor-related areas within only one or two training sessions. Imagery of finger or tongue movements was classified with support-vector classification of autoregressive coefficients derived from the ECoG signals. After training of the classifier, binary classification responses were used to select letters from a computer-generated menu. Offline analysis showed increased theta activity in the unsuccessful patients, whereas the successful patients exhibited dominant sensorimotor rhythms that they could control. The high spatial resolution and increased signal-to-noise ratio in ECoG signals, combined with short training periods, may offer an alternative for communication in complete paralysis, locked-in syndrome, and motor restoration.

VL - 13 UR - http://www.ncbi.nlm.nih.gov/pubmed/18495541 IS - 2 ER - TY - JOUR T1 - Classifying EEG and ECoG signals without subject training for fast BCI implementation: comparison of nonparalyzed and completely paralyzed subjects. JF - IEEE Trans Neural Syst Rehabil Eng Y1 - 2006 A1 - Jeremy Jeremy Hill A1 - Lal, T.N A1 - Schröder, Michael A1 - Hinterberger, T. A1 - Wilhelm, Barbara A1 - Nijboer, F A1 - Mochty, Ursula A1 - Widman, Guido A1 - Elger, Christian A1 - Schölkopf, B A1 - Kübler, A. A1 - Niels Birbaumer KW - Algorithms KW - Artificial Intelligence KW - Cluster Analysis KW - Computer User Training KW - Electroencephalography KW - Evoked Potentials KW - Female KW - Humans KW - Imagination KW - Male KW - Middle Aged KW - Paralysis KW - Pattern Recognition, Automated KW - User-Computer Interface AB -

We summarize results from a series of related studies that aim to develop a motor-imagery-based brain-computer interface using a single recording session of electroencephalogram (EEG) or electrocorticogram (ECoG) signals for each subject. We apply the same experimental and analytical methods to 11 nonparalysed subjects (eight EEG, three ECoG), and to five paralyzed subjects (four EEG, one ECoG) who had been unable to communicate for some time. While it was relatively easy to obtain classifiable signals quickly from most of the nonparalyzed subjects, it proved impossible to classify the signals obtained from the paralyzed patients by the same methods. This highlights the fact that though certain BCI paradigms may work well with healthy subjects, this does not necessarily indicate success with the target user group. We outline possible reasons for this failure to transfer.

VL - 14 UR - http://www.ncbi.nlm.nih.gov/pubmed/16792289 IS - 2 ER -