1. Introduction

The joystick is the main control interface for electric wheelchairs and has the advantage of detailed motion direction and speed control and also has disadvantages of requiring complex wrist movement that becomes difficult with age[1]. This may result in inadequate control leading to accidents. It is, therefore, useful to consider an interface that uses a part other than the wrist without the need for complex motion. Based on this concept, the interface of the electric wheelchairs have been presented, for example, such as voice input[2], direction instruction by tongue pressure sensor[3], EMG[4], head positioning and orientation[5], eye movement, and face inclination[6]. While such interfaces have controlled wheelchairs satisfactorily, many studies combine several body motions to operate a single wheelchair motion, e.g. forward, stop, and backward. As a result, these interfaces require training in logical control. We have developed an electric wheelchair, shown in Figure 1, having a Human Body Motion Interface (HBMI)[7,8].

In designing the HBMI, we checked chairseat or backrest pressure distribution as input to the interface. Preliminary experiments confirmed that a pressure change in the backrest showed body motion more clearly than chairseat pressure distribution. Based on this result, we made an prototype wheelchair for conducting experiments, using the Center Of Weight (C.O.W.) on the pressure distribution as the representative point for controlling the wheelchair shown in Figure.3. The user controls the electric wheelchair by leaning his/her upper body to the direction where user wants to go. Experiments have confirmed that the HBMI was applicable as an interface through the SD-method and factor analysis.

Figure 1. The electric wheelchair having Human Body Motion Interface

However the problem has been also remained. In some cases, the wheelchair did not move even if the user leaned his/her upper body. The velocities of each wheel have been generated in proportion to the difference between initial and present position of C.O.W.. This difference is generated by the inclination of the body. In some cases, while user leans his/her upper body, the difference doesn't increase due to the contact condition between body and backrest. In this case, the user cannot control the wheelchair arbitrarily in spite of leaning his/her body. In order to solve this problem, the gain which is proportion parameter between difference of C.O.W. and velocities of each wheel should be increased. However, too big gain lets the wheelchair cannot keep a stop, as detailed in Section 3. Therefore, the interface should measure the slight body motion and realize a secure stop. For solving this problem, following two subjects are important.

1. the sensitive measurement of pressure sensor.

2. the estimation of intention

In these topics, we can see several related researches. In the field of measurement, Lue et al. suggest that the calibration is needed for accurate and sensitive measurement for pressure sensor[9]. However, it is preferable to omit calibration for practical use. On the other hand, in the field of estimation of human intention, event-driven models based on graph theories, e.g. Hidden Markov Model [10] and fuzzy[11] are used. However the performance of these even-driven models depends on ability of the system designers[12].

From above, this paper consider following method to realize sensitive measurement and estimation of intention. First, we eliminate constant pressure data on the backrest when calculating C.O.W. to enlarge the difference of C.O.W.. Second, we consider extracting user's stop intention by using Self-Organizing Map (SOM)[13] to realize a secure stop. By these operations, the wheelchair can keep a stop while the user wants to keep a stop, and the velocities of each wheel are generated even if the difference of C.O.W. is small.

This paper is organized as follows: Section 2 outlines the wheelchair configuration using the HBMI. Section 3 discusses conventional wheelchair control scheme and its problem. Section 4 discusses the elimination of the constant pressure data for calculating C.O.W. and the classification of the body motion. Section 5 explains control taking into account motion classification and improved measurement through experiments. Section 6 summarizes conclusions.

2. Electric Wheelchair with HBMI

The outline of the electric wheelchair reviewed in the next section, see References[7,8].

2.1. System Configuration

This system uses a change of the C.O.W. in the pressure distribution occurring on the backrest that is a contact point between the user and the wheelchair to realize the control of the wheelchair. The system mainly consists of three elements: BPMS pressure sensor (Tekscan, Inc.), a motor drive system (JW-1, Yamaha), and a personal computer (PC), shown in Figure 2.

Figure 2. System Configuration

2.2. Control Scheme

Proposed system uses the C.O.W. shown in Figure 3 as a index of body motions. The interface for the JW-1 is composed of two voltages. We converted a change of the C.O.W. into two voltages to associate the body motion for the control of the wheelchair. This method is as follows, and depicted in Figure 4.

Figure 3. Coordinate on the backrest and C.O.W

Figure 4. Block diagram of control scheme

1. The initial position of C.O.W. P_g0 is recorded when the operation starts.

2. The instruction voltage is calculated by following equation.

(1)

Here

P_g : Present position of C.O.W.

P_g0 : Initial position of C.O.W.

A : Gain matrix from position to voltage

V : Instruction voltage

x_g, y_g : Position of C.O.W. in each axis.

v_fb : Instruction voltage (forward-back direction)

v_lr : Instruction voltage (right-left direction)

a_fb : Proportional gain (forward-back)

a_lr : Proportional gain (right-left)

3. The instruction voltage is sent to the motor drive unit.

A difference between initial C.O.W. P_g0 and present C.O.W. P_g is multiplied by Gain A and is transmitted to JW-1 as command voltage V to activate the wheelchair.

3. Dilemma between Increasing Gain A and Secure Stop

The experiment confirmed that the above mentioned HBMI has a potential as an interface by SD-method and a factor analysis using 10 subjects[8].

Figure 5. The C.O.W when leaning right/left

Figure 6. Change of C.O.W. vs body inclination

Figure 7. Inclinometer and coordinate definition

However, there is a problem. Figure 5 is one of the examples of this problem. This figure shows the pressure distribution while the user leans upper body to right and left with turning motion intention. A dark color is the strong pressure and bright color indicates the small pressure, the C.O.W. is also marked. In this case, the wheelchair did not move in spite of user's motion. Because, the change of C.O.W., which becomes instruction voltage by multiplying A in (1), is small even if the pressure distribution changes. Figure 6 shows the small change of C.O.W. in this case, with body inclination measured by inclinometer on the subject shown in Figure 7. In some cases, the C.O.W.s are almost same, although the pressure distribution shapes are changed by body motions. This makes deterioration of interface's usability because the user intention is not reflected to the motion of the wheelchair.

We can consider following idea as one of the solution, which is increasing gain A in order to generate big instruction voltage in the case of small change of C.O.W.. Although increasing gain A makes wheelchair move in small change of the C.O.W., slight body motion can also activate wheelchair motion. That is to say, it is difficult to stop the wheelchair. In order to keep a stop, the initial and present C.O.W.s should be same as (1) shows. However, it is difficult for human to precisely control body motion. Therefore a secure stop would be difficult due to big gain A. Here, there is a dilemma between increasing gain A and a secure stop.

4. The Solution to the Problem

We consider following two methods as the solution to the dilemma problem.

1. To enlarge the difference between initial and present C.O.W..

2. To extract user's stop intention from pressure data to make a secure stop.

The details are shown in the following section.

4.1. Enlargement of the Difference: P_g0 - P_g

The C.O.W. is calculated by (2).

(2)

Here, n_x means the number of sensor cells in x direction, n_y is the number of cells in y direction, F_ij is the pressure at the ij^th cell.

The C.O.W. is strongly affected by strong pressure area such as dark color cells in Figure 5. Dark color area is almost static in spite of change of body inclination, where body constantly contacts with the chair. The change of C.O.W. is dominated by this constant and static contact area, therefore the change of C.O.W. is small. We therefore eliminate the constant pressure area and calculate the C.O.W., namely we mask the constant pressure area and extract cells which show big pressure change.

In order to realize this, user swings his/her upper body right/left and forward/backward prior to control wheelchair. This preliminary motion is the calibration to detect constant contact area.

Figure 8. Extracted cells which shows big pressure change

The pressure distribution is recorded while this calibration. After subject’s calibrating motion, we calculate the C.O.W. by using only the cells which exceed the pressure threshold of the difference between maximum and minimum pressure. The cells under the threshold are masked, and we set their pressure data to zero. The red-colored cells in Figure 8 are bigger than threshold, and are used for calculating C.O.W.

Masked pressure distribution which is red-colored in Figure 8 eliminates constant pressure data, and extracts data which varies well. This masked pressure distribution is varied by even small body motion. Therefore the change of C.O.W. using this masked data shows slight body motion well.

4.2. Extraction of Stop Intention

Seven body motions are linked to wheelchair motion – (1) neutral, (2) forward, (3) leaning back, (4) leaning right, (5) leaning left, (6) twisting right, and (7) twisting left. By classifying body motion into these seven motions, the stop intention is extracted. Here, (1) neutral is linked to stop intention.

For this classification, we employ Self-Organizing Map (SOM)[13]. In this classification, the input to the SOM is 34×44-grid film pressure sensor output (1,496 pieces of data).

4.2.1. SOM Learning Results

The 1,496 cells pressure data is SOM input vector. For learning, we used 345 pieces of pressure data for a male subject 170 cm tall and weighing 60 kg in above mentioned seven motions. Table 1 lists SOM learning parameters. After the initial weight in the map is set up by random numbers, SOM learning is executed to obtain a weight vector at each cell on the output map.

Table 1. SOM parameters Number of Learning 500000 Learning rate 0.02 Radius 6 Map size 20×20

Figure 9. Similarity of each motion

Figure 10. The domain of each motion on the map

After the weight vector at each cell is obtained, we used pressure data when the subject conducted the seven body motions above as experimental data and calculated the weight vector of each cell and cosine similarity - called similarity here.

Figure 11. Control scheme including SOM and Data Masking

Figure 9 shows the weight vector of each cell on the output map and similarity to experimental data. For easy understanding, cells show 95% of maximum similarity for each motion. Different output patterns depend on individual motions. An output map corresponding to each motion was obtained from the similarity of each cell on the output map as shown in Figure 10.By this classification, we can extract user's stop intention.

5. Improvement Control Scheme

We improve the control scheme using two method described in section 4. SOM detects neutral position which is stop intention, and realizes a secure stop. Masking of pressure data eliminates the sensor cells which have small pressure change from the calculation of C.O.W., and realizes good response to slight body motion. These are shown in Figure 11.

The pressure data is sent to the SOM and masking process. SOM detects neutral position. Masking process eliminates the cells of small pressure change, and sends masked data to C.O.W. calculation process. If SOM does not detect neutral position, C.O.W. is calculated using masked data, and then the wheelchair moves. On the other hand, if SOM detects neutral position, SOM switches the initial C.O.W P_g0 to present C.O.W. P_g. For this, the difference of C.O.W (P_g0 - P_g) becomes zero, and then the wheelchair stops.

5.1. Experiment

Figure 12. The sequence of body motion

Figure 13. Trajectory of "Present C.O.W." in Figure 4 by inputting motion of Figure 12

Figure 14. Trajectory of "difference Pg0-Pg" in Figure 4 by inputting motion of Figure 12

Figure 15. Trajectory of "Present C.O.W." in Figure 11 (with masking process) by inputting motion of Figure 12

Figure 16. Trajectory of "difference Pg0-Pg" in Figure 11 (with masking process) by inputting motion of Figure 12

Figure 17. Trajectory of "difference Pg0-Pg" in Figure 11(with masking and SOM process) by inputting motion of Figure 12

We used the motion sequence shown in Fig. 12 to verify the improved control scheme:

1) The user takes a neutral position without putting arms on wheelchair armrests.

2) The user leans his body to right

3) The user keeps a neutral position and puts arms on the armrest.

4) The user removes arms from the armrest.

The intention of user control is to stop the wheelchair in motion 1), 3) and 4) and turn it to right in 3).

First, the results of conventional method (Figure 4) are shown. The change of "Present C.O.W." in Figure 4 is shown in Figure 13, and "difference P_g0 - P_g " is shown in Figure 14. In these figures, the numbers "1) to 4)" indicate the motion duration of Figure 12. From Figure 13 and 14, it is confirmed that the C.O.W. and difference of C.O.W. are changed due to sequence of body motion.

Next, the results including masking process shown in Figure 11 are depicted (not include SOM). The change of "Present C.O.W." in Figure 11 is shown in Figure 15, and "difference P_g0 - P_g " is shown in Figure 16. From these figures, it is confirmed that the change of C.O.W. is enlarged in comparison with Figure 13 by introducing masking process. The change of difference of C.O.W is also enlarged. Therefore, the body motion which shows small change of C.O.W. can be enlarged by adding masking process. However, the masking process also enlarges the change of C.O.W. in case of neutral position 3) in Figure 15and 16 which is duration of user's stop intention. The wheelchair cannot keep a stop due to the difference (P_g0 - P_g) caused by the swing up and down arms motion while the user intend to keep a stop.

Therefore, in order to realize a secure stop by extracting stop intention, the classification of body motion by SOM is included. The result is shown in Figure 17. When SOM detects the neutral position, the switch, which is connected SOM output in Figure 11, is changed so as to switch initial C.O.W. P_g0 to present C.O.W. P_g. In figure 17, the difference of C.O.W. is kept zero in the duration of 1), 3) and 4) by SOM switching. Therefore, the wheelchair can keep a secure stop.

From above things, improved control scheme has been able to control the wheelchair in small change of C.O.W. and realized a secure stop as long as detecting stop intention. Thus dilemma described in Section 3 has been solved.

6. Conclusions

This research focuses on the electric wheelchair controlled by Human Body Motion Interface (HBMI). HBMI uses the body motion which is caused by the voluntary motion. From our previous research, it has been confirmed that HBMI, which uses the center of weight on the pressure sensor attached on the backrest, has the ability of an interface. However the problem has also remained. The velocities of each wheel have been determined in proportion to the difference between initial and present position of the Center Of Weight(C.O.W.). This difference is generated by the inclination of the body. In some cases, while user leans his/her upper body, this difference doesn't increase due to the contact condition between body and backrest. In this case, the user cannot control the wheelchair arbitrarily in spite of leaning his/her body. For this problem, the gain which is proportion parameter between difference of C.O.W. and velocity of each wheel should be increased. However, too big gain lets the wheelchair cannot keep a stop. In order to solve this problem, in this paper, we first we eliminated constant pressure data on the backrest when calculating the C.O.W.. Second, considered extracting user's stop intention by using Self-Organizing Map (SOM). By these operations, the wheelchair can keep a secure stop while the user wants to keep a stop, and the velocities of each wheel are generated even if the difference of C.O.W. is small.