Design of a Brain-Controlled Video Game based on a BCI System

. The Computer Brain Interface (BCIs) is a new type of user interface designed to recognize real-time user-specific intentions based on user brain mapping. Gradual advancing technology behind the BCI interface has made it possible to develop gaming applications that use directly brain input instead of well-known traditional control methods. This paper presents the Mental Pool Game, a brain-controlled computer game powered by Unity3D, which uses a commercial BCI device based on a network of electroencephalography (EEG) sensors placed on the scalp capable of creating brain patterns only by capturing conscious thoughts from a user that can be easily reproduced as mental actions to control strength and speed on which the user can project on the white ball to hit any of the eight balls on the pool table. Mental PoolGame, powered by Unity3D, is a platform that provides all tools necessary for a developer to develop three-dimensional games that has been designed so that the end user no longer uses classical commands that depend on the use of well-known peripheral input devices present on any PC, such as a keyboard, mouse and more recently a joystick to control the movement of space in an object in a virtual environment.


Introduction
Over the past decades, there was an exponential improvement in imaging technologies that allow researchers to assess cognitive workload, short-term memory and space / navigational behaviour in humans. Using new experimental paradigms and brain imaging devices, researchers get a deeper insight into neural correlations of emotion, knowledge and motor control.
Brain-Computer Interface (BCI) combines two fundamental research areas: Neuroscience and Human-Computer Interaction (HCI) that serve as a link bridge that allows users to interact with the computer based on commands transmitted directly from the brain to the standard input of the computer [1].
BCI systems use different neuroimaging sensors to retrieve evoked brain signals through a specific thinking process that corresponds to a certain mental state of the user at a time that can be converted into a computer command. In order to provide a general overview of sensory, motor and cognitive processes, the BCI system uses the electroencephalogram (EEG) principle to measure electrical activity of the brain, which is diffusely distributed across the scalp and that corresponds to the brainwaves alpha, beta and theta [2].

BCI related to games
Nowadays the increase in the accessibility of computer systems among the population and also the advancement in computing technology that is able to offer computers with more computing power and with an advanced 3d [3] computing graphics has pushed the video game industry to become a major industry that engage specialists from many subdisciplines to be able to provide an improved user experience in game play that became a new way of life able to improve task management [4], simulation and learning activities.
As far as brain-controlled video game technology is concerned, it can be used for both medical and non-medical purposes: to recover limb motricity in case of brain level trauma or in case of suffering by a neurologic disease such as stroke or severe neurodegenerative disease using generation of motor imagery based on BCI commands; as assistive technology for persons with mobility impairment such as spinal cord injuries who otherwise are not able to perform game controls; for relaxation and entertainment, in many cases used to measure the concentration level of a player [5,6,7].
The main concern in developing video games based on the BCI interface is to achieve a natural user interaction with the game environment capable to motivate the user both by implementing of some intuitive controls and by transmitting some entertaining and challenging experiences because the result of implementing a BCI system is dependent of user engagement.
Research in the field of video games, even in the case of BCI technology, has shown that rational playing of video games brings a benefit in the life of the player by acquiring new knowledge, experience and skill improvement given by certain cognitive processes that it involves during playing games such as: problem solving, remembering and training sessions [8].
Continuous growth of the video game industry driven by the development of Virtual Reality (VR) and the development of rapid game prototyping tools allowed the integration of the BCI interface into games development that can help the research environments like: Games as stimulus developed for studying human behavior; Games as means of training; Games able to evaluate performance metric of users [9].
A BCI interface uses EEG as an access point to detect and record the activity of neurons by creating a pattern of brainwaves that is generated by conscious thoughts that can then be analyzed by a computer and converted into brain-controlled commands applicable as inputs for a game.
Using the EEG principle in games designed to evaluate metric performances as a result of analyzing the data acquired for a particular user, there can be obtained measurements of his performance indices like: cognitive workload, level of stress, arousal, task engagement and level of relaxation [10].
With regard to the processing of EEG segments, this is not an easy task. A major common problem is that mental imaging operation used as entry for game control varies from one person to another on the one hand because the brainwave pattern obtained from the mental drive training session is unique for each person, requiring a training session for each user and on the other hand due to variables such as: attention, emotion, focusing on thought and hand grasping imagination [11]. In terms of brain activity analysis during video game play using the BCI interface, the major brainwave frequencies that present interest are: Alpha brainwave (7.5 -12 Hz) that reflects the relaxation level of the user and Beta brainwave (12 -30 Hz) that corresponds to the level of attention, concentration and vigilance of the player during play games. The research in gaming domain based on BCI interfaces has demonstrated that Beta rhythm magnitude it growing in amplitude when the player needs to navigate into game environment, because this activity it based on extra level of concentration from user. The same things are observed in the situation in which the player must avoid hazard moments or when it must focus to survive from enemy attacks [10,11,12].
Most of current games based on the BCI interface as game play control have been developed to measure the emotional state of the players such as "Sudden Attack", a firstperson shooter game type, developed in Unity3D gaming engine based on ThinkGear BCI plug-in to communicate with PC and that is able to measure four category of emotional states of the player: the attention/inattention and the meditation/uneasiness according to the beta brainwave level recorded that influence the changing of the sky lightening in gameplay environment due to human biological clock [13]. The game playing screen shot is presented in Fig. 1. The Mental War is another computer game controlled by the BCI interface that allows players to play using their own brainwaves to compete against other players. The game was designed to use the user's level of concentration and meditation as an input gained by a single EEG electrode placed on the forehead to generate a higher force than the opponent to pull it in the center of the screen to win the round. In terms of game play, the game was designed to be played in three ways: the single mode that allows the player to compete against the computer with three levels of difficulty; the collaborative mode in which a team works together to compete against another team where the level of attention of each player is added to those in their team. In this situation, the level of the group is affected by the way each player performs so that if a player performs poorly the others in the team must perform better to compensate; the competitive mode in which players compete against others. In this situation, the result is the resultant force obtained by subtraction of the average level of attention of each player [1].
In terms of game's engine, the game was developed within the Simple and Fast Multimedia Library software development kit for the rapid prototyping of gaming and multimedia applications using the C ++ object-oriented programming language to manage game's graphics, network and sound [1]. The graphical user interface (GUI) of the Mental War game is composed of 4 distinct elements: a focus counter for each player presented in area 1 of fig. 2; the blink boost for three volunteer blinks for each player to obtain a small force boost presented in area 2 of fig. 2 The LazyBrains represent a biofeedback BCI game developed by Drexel University in 2008 that uses a fNIR device instead of an EEG headset as controller. In terms of the functional point of view, the functional near-infrared imaging (fNIR) device is based on a neuro-imaging spectroscopic method to measure the level of neuronal activity of the brain at the neurovascular coupling level associated with metabolic activity and oxygenated hemoglobin in the blood vessels [14]. The game offers the user a play scenario that challenges him to lift a closed sewer grid or move a platform on raising and maintaining a high level of concentration. The game also offers a visual feedback using the indicator bar to indicate the user's level of concentration. If the level of concentration is reached, the task involves either moving a platform or concentration jump can be performed successfully. Regarding the game engine the LazyBrains was developed based on the Blade3D software development kit that provides a plug-in type architecture together with powerful real-time object database and a layered rendering system as game engine [14]. The game playing screen shot is presented in fig. 3.   Fig. 3. The game playing screen shot of the LazyBrains game [14]. 3 Software

Unity3D Gaming Engine
Unity is an open source cross-platform game engine developed by Unity Technologies. It is a developer-oriented platform that allows: the development of 2-dimensional and 3dimensional desktop applications for both Windows and Linux operating systems; mobile app development for Android OS and iOS operating system; development of console games [15]. The three-dimensional version of the Unity platform known as Unity3D includes a number of complex components useful to game developers such as: component mesh, component physics, rendering component, audio component and script component etc.
Unity3D also offers a wealth of resources from common ground scripts to common scripts to useful collision detection tools to create realistic games. As far as the implementation [16] of scripting functions is concerned, Unity3D supports both the C# programming language and the JavaScript programming language both designed to develop high performance multimedia applications and games [17].

Mental Pool Game design
In this paper we developed a 3D Mental Pool Game based on Unity3D game engine powered by C# programming language which has been designed for several target user groups, from healthy people to people with disabilities, so they will no longer have to use classical commands that depend on the use of peripheral input devices present on any PC such as a keyboard, mouse or joystick to control the movement of an object on a virtual environment, in our case to control the strength and speed that the user can project, using only his own brain, on a white ball in order to hit any of the eight balls placed on the pool table using a system based on a BCI interface as presented below:  The functional diagram of the BCI system as shown in fig. 4 describes the acquisition and analysis of EEG signals from the subject and the conversion of these signals into mental commands that can be used as inputs to send in game control commands directly from the brain.
The BCI headset used for acquiring EEG signals from the brain is a commercial version that provides five semi-dry EEG electrodes placed on the scalp level of user and that can detect the activity of the whole region of the brain.
Also, the BCI headset has a 3-axis gyroscope that is used to update the white ball position on the pool table within the game scene according to the user's position by changing the direction of the head movement.
In order to decode the EEG signals from the user of the game, the BCI headset uses a series of analogue to digital (ADC) converters to translate the electrical activity of the neurons into a binary form that can be recognized by any computer.
The EMOTIV BCI software development kit (SDK) is used to analyze and detect the user-generated binary commands generated by the game user using the BCI headset and, based on a script written in C # programming, it can convert them into key-press commands used as input to give to the white ball a specific movement speed on the table.
The logic diagram used to control the game as it was thought in the design phase is shown in fig. 5:   Fig. 5. Logic diagram for BCI system design. As can be seen in this figure, the video game user must first check whether there is a connection between the BCI headset and the computer. If there isn't, it means there is a hardware problem and the user must apply some of saline solution to rehydrate EEG sensors to improve their contact with the scalp that, otherwise, represents a common problem for non-invasive BCI systems based on semi-dry EEG electrodes.
Also to be able to play Mental Pool Game, the user must start the training session of two distinguish mental commands: Neutral mental command that corresponds to neutral state, which is used as a reference to distinguish between the conscious thoughts that form the brain pattern that corresponds to a particular action and the signals generated by noise sources such as head movement or due to bad contact of EEG electrode with scalp; Push mental command, which allows the game user to record the brain pattern on which he focus on an action imagining the forward movement of the white ball on the billiard table. The set speed of the white ball is variable depending on the cognitive power that the user allocates at certain moment to performing the mental task.
The graphical user interface (GUI) of the Mental Pool Game developed in the Unity3D platform game engine is shown in fig. 6:

Results & Discussions
Three healthy subjects were first familiarized with BCI headset on how mental patterns were formed based on the acquisition of EEG signals during a training session that involved a training of Neutral and Push mental commands within the software development kit Emotiv Xavier Control Panel. They had 30 seconds to train each mental command within the graphical user interface and in case of training of the Push mental command, during their training session they were asked to visualize the forward movement of a virtual cube, either directly or following a predefined animation and were able to track on top of the interface via a gauge indicator the own cognitive power that was dependent on the level of focus with which they performed the mental task [5]. After this, all of the subjects played one round of each game until the exhaustion of the balls on the pool table. During the experiment, it was found that the force with which the white ball was pushed onto the pool table to strike other balls is directly influenced by the level of attention and concentration that must be maintained throughout the mental load, with the power to concentrate depending on the fatigue of the subjects, the stress conditions, their emotional state, and the level of interest represented by the degree of attractiveness of the game in order to capture their attention.

Conclusions
This paper proposed a 3-dimensional game developed in the Unity3D game engine which has been implemented in accordance with physics laws to reproduce a realistic virtual version of the real game and whose controls can be driven directly by brain using the EEG signal caption from the user through the BCI interface that is able to convert electrical stimuli in binary form based on a script written in C# programming language that used a series of functions designed for mapping mental commands that are provided by Emotiv BCI software development kit in order to convert binary form of signals into keystroke sequences used as control inputs for game play. The Mental Pool Game has been designed so that it can be used both for purely recreational purposes by healthy people and for medical purposes especially in the case of people with permanent disabilities such as: severe neurodegenerative disease, spinal cord injuries or superior limbs amputation in which case the game based on BCI interface is used as assistive technology. Also, for medical purposes the game proposed can be used by people with temporary medical problems mainly for the purpose of faster recovery, such as stroke, people who have suffered brain damage or for treating children that suffer from Attention-Deficit Hyperactivity Disorder (ADHD).
Compared to the "Sudden Attack" shooter game, the Mental Pool Game offer is a strategy game offering the advantage of being able to be used by people suffering from photo sensorial epilepsy because it does not involve changing the sky lightening as gameplay environment.
Compared to the Mental War game that uses a BCI headset that has only one electroencephalogram electrode placed in the prefrontal cortex area, the BCI headset used as control interface for the Mental Pool game has the advantage of having a network consisting of five electroencephalogram electrodes placed according to the international system 10-20 that provides the placement of EEG electrodes on the scalp level and which ensures the measurement of activity from all cortical lobes of the brain. However, another advantage is that the BCI helmet has a multimodal interface based on a 3-axis gyroscope, an improvement in the lack of response of EEG electrodes due to their imperfect contact with the scalp of the user, requiring only one mental command to control the game play.
Compared to the LazyBrains game that not include network support our game is oriented to an online game play.
Based on simple user-driven mental commands, the game developed aims to improve the player's concentration power and level of attention by stimulating the brain to produce beta brainwave.
As a future development we propose: to adapt the game for mobile devices such as those running on the Android operating system as the Unity3D game development engine allows this, and commercial BCI systems now offer applications that provide the BCI interface for smartphones, smart devices especially for controlling other devices; to adapt the pool game in a manner that can be played in multiplayer mode by two players at the same time just like in the case of real pool game that should be in benefit of both players because the level of attention and concentration power increases when the two players are in direct competition to win the game; to export in cloud or save on computer the EEG data from gaming sessions for every player in part based on login data to be able to analyze them in offline mode using specialized software such as: BCI2000 that offers support for MatLab-Simulink development environment, EEGLAB a toolbox for MatLab or in Emotiv Xavier Test Bench, data that can be useful later in the medical field for diagnosis of brain disease or in patient recovery process.