Real-time BCI with instantaneous feedback using EEG and fNIRS

Abstract

Brain-computer interfaces (BCIs) have become an important tool in human-computer interactions. The area of applications ranges from simple research to profound stroke therapy. In this thesis, a novel approach to combine electroencephalography (EEG) and functional near-infrared spectroscopy (fNIRS) is proposed to develop such an interface with real-time capability. The wireless g.Nautilus fNIRS (g.tec) with 16 channels of EEG, combined with 8 channels of fNIRS was used, acquiring optical densities and their corresponding oxygenated and deoxygenated hemoglobin concentrations. The system itself was validated on a motor imagery (MI) and Stroop experiment. In both experiments, 6 healthy subjects participated. The paradigm was divided into two runs, the calibration phase, and an evaluation phase. Per phase, a number of trials were performed with the same scheme: instruction, task, and rest. For the Stroop experiment, the conduction of a Stroop test against a resting state was classified. Basically, both runs were identical for the Stroop experiment. Left- and right- hand grasping imagination was performed for the MI experiment. During the calibration phase, visual instruction, avatar animation, and functional electrical stimulation (FES) were used to facilitate the imagination of the movement. For the online evaluation phase, positive FES and animation feedback were provided. EEG Features were extracted after preprocessing and spatial filtering. Several fNIRS features were extracted after the concentration levels were preprocessed. Linear discriminant analysis was utilized to classify the EEG and fNIRS features, respectively. The final prediction was performed by a meta classifier utilizing the scores of the individual classifiers. On average, the BCI achieved a mean accuracy of Average = 80%, Class0 = 73.3%, Class1 = 86.6% and Average = 83.6%, Class0 = 86.1%, Class1 = 81.0%, for the MI and Stroop experiment, respectively. The results indicate that fNIRS is applicable for the classification of online MI and to successfully determine the brain activity of the frontal lobe. Taking advantage of both modalities, EEG and fNIRS, significantly increased the robustness and accuracy in both experiments. In consequence, fNIRS is considered a simple, powerful, and affordable amendment for such applications. Based on the findings of the Stroop and MI experiment, the system might be expanded to a multi-class BCI in the future. Possible application areas for such a system are versatile and have yet to be assessed.