Comparative analysis of two-group supervised classification algorithms in the study of P300-based brain-computer interface

The main goal of the paper is to perform a comparative accuracy analysis of the two-group classification of EEG data collected during the P300-based brain-computer interface tests. The braincomputer interface is a technology that allows establishing communication between a human brain and external devices. BCIs may be applied in medicine to improve the life of disabled people and as well for entertainment. The P300 is an event-related potential (ERP) appearing about 300 ms after the occurrence of the stimulus of visual, auditory or sensory nature. It is based on the phenomenon observed in anticipation for a target event among non-target events. The 21-channel 201 Mitsar amplifier was used during the experiment to store EEG data from seven electrodes placed on the dedicated cap. The study was conducted on a group of five persons using P300 scenario available in OpenVibe software. The experiment was based on three steps the classifier learning process, comparison and averaging of the obtained result and the final test of the classifier. The comparative analysis was performed with the application of two supervised classification methods: Linear Discriminant Analysis (LDA) and Multi-layer Perceptron (MLP). The preliminary data analysis, extraction and feature selection was performed prior to the classification.


Introduction
Neuroimaging is associated with a group of research methods used to study the structure and function of the brain. Due to its remarkable time resolution, EEG is applied to study changes in brain activity over time and to analyse responses to external stimuli. On the basis of the signal that changes over time, one can locate and remove distortions, i.e. muscle artefacts. The EEG signal represents curves illustrating voltage changes in time occurring between the electrodes. The Brain-Computer Interface (BCI) is a system for communication with a computer using brain signals. By acquiring and translating brain signals to specific commands, the BCI system can be used as an alternative method of communication for people with severe neuromuscular disorders. Brain signals can be obtained by invasive or non-invasive methods.
Electroencephalography (EEG) is a method of recording the electrical activity of the brain from the human scalp using electrodes and conductive media. EEG focuses on measuring the electrical activity of the brain registered with electrodes placed on the scalp at specific locations. A variety of EEG electrodes arrangement standards are available on the market. Among them, the most popular is the 10-20 system consisting of 21 electrodes, which divides the head into proportional distances from visible landmarks such as the nasion, inion and ear sections [1].
EEG is treated as one of the most common methods of measuring brain function in the study of basic psychological and motor processes [2]. There are three main paradigms of BCI based on EEG techniques: P300, steady-state visual evoked potentials (SSVEP) and motor imagery (MI).
P300 paradigm is based on event-related potentials which appear as a reaction to a specific stimulus at a time between 300 and 600 ms after the demonstration of the expected stimulus [3]. SSVEP is related to the visual stimulus presented with the specific frequency [4]. The frequency of the stimulus might be observed in the EEG signal of an examined person. MI is associated with the movement of various parts of body, e.g. left/right hand, tongue or foot and it is based on event-related desynchronisation/synchronisation [5]. Each of the above-mentioned paradigms requires training to gain the expected classification results, however, the paradigms SSVEP and P300 may be used without earlier training as opposed to motor imagery.
Classification methods are divided into two groups: unsupervised learning and supervised learning. BCI usually employs the supervised learning based on the gathered dataset. The model is trained using a part of the dataset (usually 70-80%) and afterwards, the test and verification are performed on its remaining part. The following models are amongst the most widely employed in BCI classifiers: LDA (Linear Discriminant Analysis), SVM (Support Vector Machine), MLP (Multilayer Perceptron), KNN (K-nearest Neighbors) [7][8][9][10][11].
The aim of the presented works is to perform a comparative analysis of the accuracy of the two-group classification (LDA and MLP) of EEG signal data collected during the implementation of the braincomputer interface using the P300 paradigm. The comparative analysis included selected classification methods under supervision, the operation of which was preceded by preliminary data analysis, extraction and selection of features.

Related works
Originally, BCIs were developed as a communication channel with the disabled, who used the EEG technique for letter selection [12][13][14]. BCIs were based on several types of paradigms such as SSVEP [4], P300 [3,15] and MI [5]. First EEG studies focused on the phenomenon of evoked potential (P300), which showed that the responses to visual stimuli depend on their emotional significance for an individual person. Guided by this assumption, further attempts led to the appearance of a response in the form of a wave about 300 ms after the occurrence of the stimulus. It was first described in 1965 by a group of Sutton researchers and it is probably the best-studied ERP component in the field of cognitive research [16]. Stimuli took the form of displayed numbers.
BCIs based on P300 paradigm are constructed in such a way that GO (target) event is presented rarely among NO-GO (non-target) events. P300 allows for fast equipment calibration and easy use in many applications [17]. The most recent applications of BCIs have been implemented with the use of visual, auditory or vibrotactile stimuli [18,19]. The work [20] reported the development of BCI based on P300 signals generated by auditory stimuli which were characterised by different sound typologies and locations. In [21] vibro-tactile P300 was tested on a group of disabled and healthy people. In [22] authors applied P300 paradigm in BCI to improve Social Attention in Autistic Spectrum Disorder (ASD). The BCI based on P300 may be implemented in EmotivEpoc headset, which is promising and cheap device [23]. In [24] authors applied BCI based on P300 to classify the right/left-hand movements. A modified P300-based BCI speller is also used to operate a wheelchair. Classically, BCI is found in ambulatory and clinic applications [25].
However, P300 BCI is also applied in virtual reality environments for gaming. In [26] authors presented the review of hardware and simulation constraints required by BCI. According to the authors, the P300 BCI is promising for the video games industry. BCI might be used in hardware applications to control robotic devices which e.g. can be used for transportation and manipulation [27,28]. In [29] authors proposed the new approach of BCI based on P300 with no earlier training required.
Many laboratories report that during BCI research, regardless of the paradigm applied, about 20% of users cannot achieve the proper level of control [30]. To increase the efficiency of the study, BCI might be based on two paradigms: hybrid brain-computer -interface (hBCI) based on P300 and SSVEP are being more often developed and applied in research. One of the reasons why the hBCI appeared was illiteracy in patients and an attempt to eliminate the problem [31]. In [32] authors carried out a systematic comparison between the use of P300, SSVEP and hybrid paradigms of BCI. In [30] authors applied both, the P300 paradigm and the SSVEP potential in the study of cells of the board flashed with different frequencies, while the rows or columns flashed in the random order.

Experiment setup 3.1 Equipment
In the study, Mitsar 201 EEG amplifier was used to collect the signal. The gathered data were recorded in the EEG Studio program and transferred to the OpenVibe application via Lab Streaming Layer (LSL) library. LSL parameters, especially the value of sample count per sent block, were adjusted empirically considering errors in transmission (such as missed data) and delays. Fig. 1 shows that as the sampling rate increases, the number of blocks decreases -the larger the number of samples per buffer is, the longer is the block of data. In data received through LSL in OpenVibe, unnecessary values appear (repetitions providing no information) or a part of information is lost. In order to minimise errors resulting from transferring data with too large or insufficiently small blocks, the value of sample count per sent block was set to 32.

The experiment procedure
The research was carried out on a group of 5 men in the 22-25 age range. During the tests, participants used a P300 BCI interface consisting of an array of 12 characters (alphabet letters). At the beginning of the experiment, the target picture was presented to the users and afterwards, pictures from the entire array were randomly displayed in four series containing 12 flashes each. Depending on the stage of the research, the examined person was asked to focus on the target picture suggested earlier. The signal was recorded from seven electrodes (Cz, P3, P4, Pz, O1, O2, Oz) connected according to the international 10-20 system.
The experiment was based on the "P300 magic card" scenario available in the OpenVibe program. The scenario consists of three stages in which the classifier learns to recognise characters chosen by the user. The acquisition scenario was used to present stimuli and gather the data synchronised with stimuli. Users were asked to focus on the characters indicated by the scenario. The signal is then collected from seven selected electrodes, filtered and epoched, according to the stimuli. The data and stimulations are saved to a file and used in the second stage of the scenario which is learning the chosen supervised classifier. It is an offline procedure based on stimulation data, covering data, preprocessing and feature selection. The classifier learning results are saved in the form of classifier parameters.
Two classifiers were applied: linear discriminant analysis (LDA) and multi-layer perceptron (MLP). The third experiment stage was an online experiment with real-time signal processing and feedback for the end user. The scenario is displayed to the examined person, data processing and classification is performed online, based on the classifier training performed in the previous stage. The examined persons could see the results of BCI in the form of a displayed letter which was chosen by the scenario in a particular series (Fig. 2).

P300 paradigm
In the conducted research, the P300 paradigm was applied, which reflects the cognitive processing of events and is defined as a positive potential occurring about 300 ms after the presentation of an infrequent stimulus, i.e. the target stimulus, among all the presented stimuli. It is closely related to the response to a specific stimulus, visual, auditory and somatic. In the research, visual stimuli were used in the form of pictures presenting alphabet letters shown in the random order.
This method requires focusing on one, specified stimulus (for example a concrete letter) while different stimuli are displayed. Multiple repetitions are needed in order to achieve positive results. As a result of repeated repeating sequences of random highlights, subsequent characters are obtained.
The EEG signal processing system is the core of the BCI system. The general structure of the processing system is shown in Fig 3. It has three main components, i.e. pre-processing, extraction, and feature selection [33]. All data from the available EEG channels are given as input data. BCI collects brain signals and processes them in real time to detect certain patterns that reflect the intentions of the user. Pre-processing, designed to simplify processing operations with keeping information in data [34], covers channel selection, filtering in the range of 1-20 Hz and signal decimation. In the feature extraction step, different types of features can be obtained from the filtered signal. Features usually cover statistical characteristics or syntax descriptions. The feature extraction scheme aims at selecting the features or information that crucial to the classification process [35]. In this research, feature extraction is based on stimulation based epoching and epoch averaging.
The result of the extraction is the matrix of features representing the EEG signal. Feature selection eliminates unnecessary features prior to the classification process [33].
The role of the classification stage is to identify patterns in training data to facilitate the translation of EEG traits to control. The classification is performed to obtain results analysed with such criteria as accuracy, complexity and computational efficiency. Ultimately, the goal of the BCI classification stage is to ensure high accuracy, reliability and portability and real-time implementation. This stage of data analysis occurs under supervision, which means that the classifier must first learn to recognise patterns prior to applying them in practice.

Applied methods
The discussed study performs a comparative analysis of two classifiers, which are LDA and MLP.
Linear Discriminant Analysis (LDA) is a method applying a linear surface as the decision boundary. The goal of LDA is to use hyper-planes to separate data representing different classes. In the case of the twoclass problem, the class of the feature vector depends on which side of the hyperplane the vector is on. The LDA classifier imposes a very strong assumption on the basic division of data. The calculation of the discriminatory function is highly efficient, which is why LDA is a popular solution in the field of BCI [36].
Multi-layer perceptron (MLP) is a type of neural network that consists of several layers: the input layer, possibly one or several hidden layers and the output layer. Non-linear relationships can be presented with any accuracy using neural networks with an appropriate structure. MLP is a classifier sensitive to overtraining, especially for more active and non-stationary data, such as EEG. Therefore, careful selection of architecture is required [37].

Results
The data obtained from the classification is presented in the form of tables. The information about the classifier learning using the k-fold cross-validation algorithm, data recorded during each session, where k was set to 5. The results are presented in the tables below. The data sets of classifiers were divided into five sets. Each of them represents the efficiency of the learning algorithm, i.e. the degree to acquire knowledge at the stage of learning. The results for the LDA classifier are good. The average k-fold cross-validation for each subject exceeds 60%. The effects of MLP classification are much better. The average value exceeds 91%.

Classification
In this section, the results of classifier training and validation are presented. The experiment was conducted on five persons, T-able 3 shows the accuracy of the result obtained from the training and validation of each implemented classifier. It presents the accuracy of classifiers obtained for each class (target and non-target, represented respectively by numbers 1 and 2).
Classification results were obtained separately for each participant of the study based on the specified number of stimuli. LDA requirements were checked prior to the classification procedure directly in the OpenVibe software.

Discussion and conclusion
The paper presents the research of BCI based on P300 paradigm. The healthy users took part in the study in order to for used to assess the possibility of implementing a BCI based on P300. The study allowed analysing the concentration and focusing on target stimulus. The BCI based on P300 paradigm techniques applied to disabled patients might significantly improve their lives.   The results obtained during the analysis have shown that a higher number of trials should be conducted to gain more precise results. However, the results obtained after learning, comparing and averaging are satisfactory. Especially in the case of the MLP sorter which obtained results over 90%. In the obtained results MLP classifier occurred to be more effective than the LDA. However, these results were obtained separately for each participant and were calculated based on particular stimuli of a single participant.
The number of participant in the study is not sufficient to cover the full inter-participant statistical analysis of the whole dataset. That is why the presented results might be treated as preliminary results and further investigation needs to be performed.