1. Introduction

Brain Computer Interface (BCI) is a frontier research area in neural engineering that has gathered a great deal of attention from scientists and the general public. BCI technology allows communication to occur between the brain and an external machine[1], and its application can range from entertainment to assistive devices[2]. In a typical BCI system, the brain activities are recorded and processed by a computer system, which in turn, deciphers the mental or physical activities and creates commands to control external devices[3, 4].

One of the goals in BCI and neural engineering research is creating assistive devices for those with limited motor control. A successful BCI system is valuable in motor disable rehabilitation by allowing the subjects to perform physical practices[5]. This type of technology would drastically improve the quality of life for the patients by allowing these individuals to have better communication and more independent control[6, 7].

The operation of traditional electromyogram(EMG) based controlled prosthetics is based on the decoding of myoelectric signals of residual muscles[8, 9]. While these devices provide more basic control over the prostheses, certain limitations restrict their acceptability. Users with severely limited motor ability would require much effort to learn how to contract specific muscle groups in order to control the device. Therefore, the need to create a more intuitive control strategy based on the user’s naturally occurring brain signals is apparent.

The BCI technology today encompasses invasiveelectrocorticography (ECoG), implanted electrodes, or non-invasive electroencephalography (EEG)[10]. Current literature suggests that EEG is adequate to extract detailed information about precise movements of the upper limb[11]. For our study, non-invasive techniques based on EEG surface potentials appear to be a more sensible method for collecting and processing data[12-14] for neuroprosthetics applications with relatively few DoF. Real-time signal classification based on the activation and feature extraction from particular brain regions can allow for the control of the assistive devices.

When examining the neuronal activities for BCI applications, the signal intensity or signal power features has been commonly used for decoding user movement intents. The posterior parietal cortex (PPC) region is responsible for converting visual stimuli into motor movements[15] and is a vital location for decoding intended motor movement[12]. While most other research focused on discriminating EEG signals between left hand, right hand, toe, and tongue imagined movement[16, 17], our study endeavors to decode and classify EEG signals for the intended movement direction of the same limb, leading to a more realistic control of a single upper limb prosthetic arm[18]. Single-trial signal classification strategy was developed to evaluate the temporal, spatial, and spectral EEG features during the planning stages of motor movements in the left, right, and forward directions similar to[19-21]. Ultimately, such classification algorithm will be a part of a two-stage neuroprosthetics control strategy. In the first stage, the intended motor movement directions can be decoded using EEG signal features. The second stage is envisioned to be a motor imaginary classifier. In this paper, we shall focus our discussion on the classification of motor intention only.

Since the presentation of the visual cues does not influence the performance[22], we used “realistic” instead of “abstract” visual-cue in order to avoid a tedious calibration procedure. Furthermore, realistic visual image environment may enhance learning progress[23]. Ensemble Empirical Mode Decomposition (EEMD), where the signals are decomposed into intrinsic mode functions (IMF)[24], was utilized to isolate the frequency information in the training set.

We developed and validated the use of scalp EEG data and current density localization for intended movement direction analysis. Subsequently, we evaluate the feature classification strategy suitable for distinguishing the brain activity associated with the intended hand movement. Potential variations in electrode impedance at different recording locations and at different recording times may drastically impact the amplitude and signal-to-noise ratio (SNR) of the EEG signals. Finally, we proposed and evaluated a scaling factor based on the “signature” EEG signal after the presentation of the visual-cues. We hypothesize that such scaling factor is able to compensate the potential problems of electrode impedance differences between trials and across different locations. Our preliminary result indicated that the inclusion of such a scaling process would significantly improve the overall single-trial classification accuracy. In a two-class decoding scheme, the accuracy improved from approximately 60% to over 90% with the scaling factor. The implication of this work would have direct impact on the acceptability of the BCI neuroprosthetics application as the new device will function based on user’s intent, which can provide a more intuitive control paradigm, for simple device control with few DoF.

2. Methods

The experimental procedure for investigating the motor intention using targeted BCI is shown in Figure 1. It involves the design of a visual stimulation system, an EEG data acquisition system, a signal pre-processing unit, an artifact removal algorithm, a feature extraction method, and a signal classifier. The detail for each step is provided in the subsequent sections. The method for computing the visual-cue based scaling factor is proposed and described in the feature extraction section.

Figure 1. A flow chart describing overall experimental procedure for decoding the reaching tasks during the planning stage. The visual stimulation was provided to subjects for recording intended movement direction EEG signal. The suitable scaling factors which applied to the signal amplitude features in the test set were obtained after artifact removal. A 5x5-fold cross validation procedure was performed to determine the overall accuracy

2.1. Visual Stimulation and EEG Data Acquisition

2.1.1. Participants

Eight able-body participants with normal or corrected to normal eye sight (6 males and 2 females, age 19-29, all right handed users) were recruited in this study. All of the subjects had no prior experience with BCIs and no history of neurology disorders. The protocol has been approved by the Institutional Review Board for Human Use (IRB) at the Louisiana Tech University. All participants had read and signed an informed consent.

2.1.2. Task and Stimuli

Figure 2. The experimental setup is illustrated. Touch pads (circles) are placed at the base (resting) position and at the targets of the reaching tasks (left, right, and forward) to track whether the subject has performed the tasks correctly

The experimental setup is shown in Figure 2 where the subjects were seated in front of a computer screen and given visual cues from the computer monitor. Touch pad sensors were placed at the middle and to the sides of the monitor to track the subject responses. In order to obtain useful signals for the fast interpretation, the BCI task should be relatively easy to perform and require little effort from the users to prevent physical or mental fatigue[10]. In order to do so, a targeted delayed saccade/reach task was used in this study. A minimum of 450 trials were performed by each subject. The recording sessions were broken into blocks (90 trials / block), separated by 5 min breaks in between.

The sequence of each trial is shown in Figure 3. At the beginning of each trial, each subject was asked to relax the forearm and place the palm on the touch pad at the base position 40 cm away from the screen. Visual cues were provided using the E-Prime 2.0 system (Science Plus Group, Netherland) to inform the subjects of the proper movement to perform in a dark room. Two types of visual cues were provided. First, the “Effector cue” was displayed 500 ms after the beginning of each trial. It instructed which movement type the users should perform (imaginary movement with eyes closed, reach without eye movement, or saccade to target). The second visual cue, called the “Direction cue” was shown at the center of the screen 1000 ms after the “Effector cue”. It informed the user of the appropriate reaching directions (left, right, or forward). The subjects were asked to fixate on the center of the screen until the “Go cue” appears 700 ms after the “Direction cue”. They were then asked to perform the indicated actions as quickly as possible after the appearance of the “Go cue”. The nine different “Effector – Direction” combinations were evenly distributed and randomly provided over the whole experiment.

Figure 3. Time course of one trial is illustrated. The 700ms delay period between the presentation of the “Direction cue” and the “Go cue” is considered the period of directional movement planning. The EEG data within this time window is used for the analysis

2.1.3. Apparatus

The EEG evoked response potential (ERP) signals were recorded using a 128-channel HydroCel Geodesic Sensor Net (Electrical Geodesics Inc., Eugene, OR) with the Net-Station 5.3 software. Figure 4 shows the electrode placement as viewed from the top of the head and regions of interest around the PPC. All signals were anti-aliasing low-pass filtered at 100Hz, and digitized at a sample rate of 256Hz.

Figure 4. The channel map for 128 electrodes is shown with electrode number labeled. The illustration is observed from the top of the subject’s head with the front of the head pointing upward. The regions of interest near the PPC are circled

2.2. Signal Pre-processing and Artifact Removal

The EEG data was digitally filtered between 0.1~30 Hz. Since this project focused only on the analysis motor intention prior to any actual movements, all three different effectors were included and combined in the analysis. The data was separated into three groups (left, right, or forward) based on the “Direction cues”. Bad channels, as the results of poor skin contact, eye blink, eye movement, or muscle movement were detected based on their particular signal characteristics and abnormal amplitude information, were replaced by the averaged signals from neighboring channels using NetStation built-in functions. Only those artifact-free epochs (with amplitude <50μV) were used for further analysis. The data was also re-referenced to the average signal across all 128 electrodes. The 100ms before onset of each trial was used for baseline correction adjustments.

2.3. Offline Source Localization Validation

Source localization was performed offline as a way to validate that the activated brain regions of our recorded data is consistent with the literature. The process described in this subsection would not be needed in the real-time implementation of the motor intention decoder. Independent component analysis (ICA)[25] was first performed using the extended Infomax-ICA algorithm in the EEGLAB tools[26] to find the maximally temporally independent signals available[27]. Figure 5 illustrates the averaged signals across all the recording trials before and after the removal of eye motion artifact. The DIPFIT 2.0 algorithm was then used to estimate the dipole sources of the remaining independent component (IC) after spatial filtering[26]. The dipoles were projected onto the boundary element mode in EEGLAB then plotted on the average MNI (Montreal Neurological Institute) brain images[28]. The source locations were then specified using the Talairach coordinate system. Dipole locations from the source localization algorithm would not be used in the single-trial classification of arm movement direction since it is a time consuming process.

Figure 5. EEG signals before and after spatial filtering is shown. The signal for each channel is averaged across all trials prior to ICA spatial filtering. An example of eye artifact is seen in the EEG spatial map around 305ms. The averaged signal for each channel after ICA spatial filtering is shown at the bottom subplot

2.4. Ensemble Empirical Mode Decomposition

Ensemble empirical mode decomposition (EEMD) is a data-driven analysis method that separates the signal into a collection of intrinsic mode functions (IMFs). It is a powerful approach for analyzing nonlinear, non-stationary EEG signal since the method is only based on local characteristic time scale[29-31]. Unlike traditional bandpass filters, EEMD breaks down the signals in an empirical manner, which is strictly based on the signal characteristics without specifying any frequency bands[32]. Mode mixing problem that existed in the Empirical mode decomposition (EMD) method can be resolved by EEMD utilizing the uniformly distributed reference frame by the addition of white noise[33]. The procedure for EEMD has been described in great detailed in[29], and would not be repeated here.

2.5. Feature Extraction and Signal Classification

Signal classifiers were created using the Statistical Pattern Recognition Toolbox in Matlab[34] to decode the EEG signal features. A two-class analysis (left versus right) was performed using Fisher Linear Discrimination (FLD) binary classifier in a 5x5-fold cross validation procedure. Eighty percent (80%) of the data for each direction was randomly chosen to be the training set. The remaining 20% of the data was assigned to be the testing set. The “signature” signal was acquired in each region of interest (ROI) near the PPC region (see Figure 4) using the training set for each cross-validation study. In this study, the averaged ERP signal within 235ms after the presentation of the “Direction cue” would be considered the “signature” at each ROI. It has been observed that regardless of the intended reaching direction or the type of effectors requested of the subject, the averaged EEG signal within the first 235ms after the presentation of the “Direction cue” retains similar signal profile. Each “signature”, consisted of a dominant high delta (0 – 4Hz) and a low theta (4 – 8Hz) component, has been observed to have similar shapes, regardless of the intended movement direction. The local maximum and local minimum of the “signature” signal at each ROI were found and their difference was used as a scaling factor. The signal amplitude at each recording site was scaled accordingly with the following equation:

(1)

Where v_i(t) and v_i,scaled(t) denote the ERPs in the test set, at location i, before and after scaling. The values v_i,max and v_i,min are the maximum and minimum of the “signature” at the same location, found in the averaged training data set. Since this scaling process only involves multiplying the EEG recording by a different factor at each location, it is suitable for real-time applications. Figure 6 is the graphical illustration for scaling the EEG signals.

The impact of scaling was evaluated by investigating the features at different time delays after the “Direction cue”[35]. Once it has been established that using this cue-based “signature” can enhance the binary classification accuracy of the planned motor movement in two directions, we performed the second analysis. It involved the evaluation of the EEMD-based operation on the decoding accuracy. The high frequency noise in the EEG data was reduced through the elimination of IMF1 and IMF2[36], since the ERP difference of intended movement direction have been reported to be <12 Hz[37].

Figure 6. Graphical illustration of the scaling factor is shown. The scaling factor computed from the ROI channels in the training set is applied to the test set. The light color lines indicate the EEG signals from individual trials in the training set from one electrode; the dark bolded line indicates the average signal. The scaling factor is set to be the difference between the maximum and minimum values in the first 235ms of the averaged signal

3. Results

Using Talairach coordinate system, the dominant equivalent dipole source for each intended arm movement direction was observed near the PPC areas for all subjects. Figure 7 illustrates the result of the EEGLAB plug-in DIPFIT2.0 output for a particular subject where the coordinates for the left component[-20, -40, 24], the forward component[0, -33, 40], and the right component[28, -40, 23] are found. This is consistent with the results reported in the literature[12]. The effects of the parietal ICs were then back-projected onto the scalp for each subject after artifact removal (Figure 8).

Figure 7. Source reconstruction for three equivalent dipoles is illustrated. As a validation, estimated source dipole locations were found to be near the PPC regions, consistent with reported literature[2] with the residual variance for each dipole estimate found to be <6%

The evaluation of the EEMD-based operation on the decoding accuracy was performed by comparing the FLD decoder performance on IMF-removed data set. Since the average EEG signal amplitude in a 40ms window was signal the feature, removing IMF1 and IMF2 components did not significantly improve the decoder performance from 93.91±6.09% to 95.44±3.28% (p>0.4).

Figure 8. The independent component clusters of each subject are shown. The three independent component clusters that extracted from ten subjects’ ICA decomposition demonstrate activities at posterior parietal cortex region. The larger heads show the average projection across ten subjects. The smaller scalp maps are from individual subjects

As a preliminary evaluation of the proposed cue-based scaling strategy, a two-direction (left versus right) classifier was created. The averaged ERP data from the training set at each recording site was found, the scaling term was calculated to be the difference between the maximum and minimum values within 235ms after the presentation of the visual cue. Once the scaling factors were found, they were applied to the test set. Amplitude features at different time delays were evaluated and the improvement of classification accuracy after the scaling operation is shown in Figure 9. The highest classification accuracy (on the scaled data) was found to take place 271 – 310ms after the visual cues. Statistically significant improvement (p<0.01) in classification performance was found with scaling (accuracy 93.91±6.09%), than without (accuracy 60.11±9.02%). Table 1 summarizes the subject-by-subject result for the single-trial FLD using 5x5-fold cross validation.

Figure 9. EEG amplitude features obtained in the PPC regions can be used to classify the intended direction of reaching motion. Classification accuracy is at the highest using amplitude features 271 – 310ms after the presentation of the visual cues

Table 1. Single-trial binary classification of left versus right intended movement was performed using FLD. Statistically significant improvement in accuracy was found after cue-based “signature” scaling (p<0.01) Subject Without Scaling With Scaling Mean ± Stdev Mean ± Stdev A 66.40 ± 8.11% 99.33 ± 0.83% B 59.20 ± 4.05% 96.13 ± 3.87% C 72.80 ± 5.68% 91.60 ± 4.89% D 55.23 ± 5.48% 96.80 ± 2.25% E 68.54 ± 4.91% 95.33 ± 2.50% F 57.67 ± 5.49% 78.71 ± 7.03% G 54.75 ± 5.37% 94.79 ± 2.31% H 46.29 ± 5.54% 98.58 ± 1.43% Mean ± Stdev 60.11 ± 9.02% 93.91 ± 6.09%

4. Discussion

BCI technology enables people to interact with external devices in new and intuitive ways. As a prosthetic application, it helps people with limited muscle control (such as those suffering from spinal injury, stroke or cerebral palsy) regain some of the lost motor functions. Even though there is still debate over the best classification method for BCI, we developed and validated the use of surface EEG to distinguish the brain activity during planning of intended arm movements. EEG data was recorded from untrained subjects excluding feedback, where each individual subject was analyzed independently in this study. Subjects only instructed to perform the indicated reaching tasks (see Figure 3). In the framework of upper limb neuroprosthesis, this paradigm could be directly implemented as a part of the control strategy of the prosthetic arm for activity of daily living (ADL).

The spatial, temporal, and spectral features were extracted based on reported literature. We used the spatial information near the PPC regions as previously reported[15]. The temporal feature pertaining to the mean EEG signal amplitude 271 – 310ms after the presentation of the “Direction cue” visual-cue was found to have the most significant difference between the intended arm reaching directions, and the highest classification accuracy. A scaling strategy based on the EEG response to cue-based stimulus was proposed. The maxima and minima “signature” signal from 0 – 235ms after the presentation of the “Direction cue” was used as a scaling factor for subsequent single-trial analysis. The early synchronization in the delta (0 – 4Hz) and low theta (4 – 8Hz) bands is related to the “Direction cue”, which supports the idea of early component reflects the processing of the visual intention, where the alpha band (9 – 12Hz) is associated with the visual attention[38]. The “signature” signal around this frequency range can be found at different recording electrodes near the ROIs and the visual cortex during the delayed “Direction cue” period.

Our current study did not attempt to distinguish the three effectors. Recently, there have been many reported studies on the classification of saccade motor imagery versus motor execution tasks[21, 39-40]. The combination of motor planning and motor imagery for amputee subjects may be a more viable technique for controlling neuroprosthetics devices. The utility of the proposed cue-based “signature” scaling factor gave some promising results by improving the classification accuracy of intended motor directions. To test this scaling strategy in more realistic situations, it may be extended to non-visual cue based (voluntary movement) setup in the future. In these experiments, the subjects will decide the desired reaching destinations without the target-specific stimulation. The “signatures” in these situations would have been internally triggered, possibly dominated by a slightly different frequency component.

A non-invasive mobile prosthetic platform using wireless dry electrodes and wearable EEG systems would benefit in real world operational environments[41-42]. Before the implementation of a real-time BCI system, some hardware platforms and specific software need to be developed. Three main advantages of EEG recording system with a real-time signal processing platform are low-cost, easily customized and intuitive operation. Future development of specific software communication systems between EEG recording devices and signal processing platform must be designed and operated close to real-time. Other specification includes simple training protocol for rehabilitation purposes. More work is needed to understand how changes in attention and intention may impact EEG signals. Future study related to the angular direction decoding, instead of the current discrete directions, may be necessary. Participants withmotor-disabilities will be recruited to provide more conclusive results on the advantage of the proposed “signature” scaling and classification algorithms.

5. Conclusions

Although surface EEG signals have limit information about complex arm movements, we demonstrated EEG signal can be used to decode the direction of reaching tasks during the planning stage. Experiments were designed to provide visual-cues to guide the user imagery/arm movements. ICA and EEMD are efficient to remove artifact. A cue-based scaling strategy was developed to adjust the EEG signal amplitude near the PPC regions. Temporal information (271 – 310ms) after the presentation of the visual cues was found to hold the most discriminatory features. This work would have direct application based on the electroencephalographicalsignals of the user intent. In addition, motor intention combined with motor imagery paradigms also would provide more commends on the control of BCI. The overall single-trial classification accuracy of 93.91 ± 6.09% holds this paradigm promising for non-invasive BCI design in neuromotor prosthesis or wheelchair applications.

ACKNOWLEDGEMENTS

The project described was supported by Grant Number P20RR016456 from the National Institutes of Health (NIH) National Center for Research Resources (NCRR).