Sho Yokota 1, Hiroshi Hashimoto 2, Yasuhiro Ohyama 3, Daisuke Chugo 4, Jinhua She 3, Hisato Kobayashi 5
1Faculty of Science and Engineering, Setsunan University, Neyagawa, 572-8508, Osaka, Japan
2Master Program of Innovation for Design and Engineering, Advanced Institute of Industrial Technology, Shinagawa, Tokyo, Japan
3School of Computer Science, Tokyo University of Technology, Hachioji, Tokyo, Japan
4School of Science and Engineering, Kwansei Gakuin University, Sanda, Hyogo, Japan
5Faculty of Engineering and Design, Hosei University, Shinjuku, Tokyo, Japan
Correspondence to: Sho Yokota , Faculty of Science and Engineering, Setsunan University, Neyagawa, 572-8508, Osaka, Japan.
Email: | ![](email/10.5923.j.ajis.20120204.04.gif) |
Copyright © 2012 Scientific & Academic Publishing. All Rights Reserved.
Abstract
This research focuses on the electric wheelchair controlled by Human Body Motion Interface (HBMI). HBMI uses the body motion which is caused by the voluntary movement. From our previous research, it has been confirmed that HBMI, which uses the center of weight on the pressure sensor attached on the backrest, has the ability of an interface. However the problem has also remained. The velocities of each wheel have been determined in proportion to the difference between initial and present position of the center of weight. This difference is generated by the inclination of the body. In some cases, while user leans his/her upper body, the difference doesn't increase due to the contact condition between body and backrest. In this case, the user cannot control the wheelchair arbitrarily in spite of leaning his/her body. For this problem, the gain which is proportion parameter between difference of center of weight and velocity of each wheel should be increased. However, too big gain lets the wheelchair cannot keep a stop. In order to solve this problem, first, we consider extracting user's stop intention by using self-organizing map (SOM). Second, we eliminate constant pressure data on the backrest when calculating the center of weight. By these operations, the wheelchair can keep a stop while the user wants to keep a stop, and the velocities of each wheel are generated even if the difference of center of weight is small.
Keywords:
Self-Organizing Map, Pressure senor, Body Motion, Wheelchair, Intuitive, Interface
1. Introduction
The joystick is the main control interface for electric wheelchairs and has the advantage of detailed motion direction and speed control and also has disadvantages of requiring complex wrist movement that becomes difficult with age[1]. This may result in inadequate control leading to accidents. It is, therefore, useful to consider an interface that uses a part other than the wrist without the need for complex motion. Based on this concept, the interface of the electric wheelchairs have been presented, for example, such as voice input[2], direction instruction by tongue pressure sensor[3], EMG[4], head positioning and orientation[5], eye movement, and face inclination[6]. While such interfaces have controlled wheelchairs satisfactorily, many studies combine several body motions to operate a single wheelchair motion, e.g. forward, stop, and backward. As a result, these interfaces require training in logical control. We have developed an electric wheelchair, shown in Figure 1, having a Human Body Motion Interface (HBMI)[7,8].In designing the HBMI, we checked chairseat or backrest pressure distribution as input to the interface. Preliminary experiments confirmed that a pressure change in the backrest showed body motion more clearly than chairseat pressure distribution. Based on this result, we made an prototype wheelchair for conducting experiments, using the Center Of Weight (C.O.W.) on the pressure distribution as the representative point for controlling the wheelchair shown in Figure.3. The user controls the electric wheelchair by leaning his/her upper body to the direction where user wants to go. Experiments have confirmed that the HBMI was applicable as an interface through the SD-method and factor analysis.![](image/10.5923.j.ajis.20120204.04_001.gif) | Figure 1. The electric wheelchair having Human Body Motion Interface |
However the problem has been also remained. In some cases, the wheelchair did not move even if the user leaned his/her upper body. The velocities of each wheel have been generated in proportion to the difference between initial and present position of C.O.W.. This difference is generated by the inclination of the body. In some cases, while user leans his/her upper body, the difference doesn't increase due to the contact condition between body and backrest. In this case, the user cannot control the wheelchair arbitrarily in spite of leaning his/her body. In order to solve this problem, the gain which is proportion parameter between difference of C.O.W. and velocities of each wheel should be increased. However, too big gain lets the wheelchair cannot keep a stop, as detailed in Section 3. Therefore, the interface should measure the slight body motion and realize a secure stop. For solving this problem, following two subjects are important.1. the sensitive measurement of pressure sensor.2. the estimation of intentionIn these topics, we can see several related researches. In the field of measurement, Lue et al. suggest that the calibration is needed for accurate and sensitive measurement for pressure sensor[9]. However, it is preferable to omit calibration for practical use. On the other hand, in the field of estimation of human intention, event-driven models based on graph theories, e.g. Hidden Markov Model [10] and fuzzy[11] are used. However the performance of these even-driven models depends on ability of the system designers[12].From above, this paper consider following method to realize sensitive measurement and estimation of intention. First, we eliminate constant pressure data on the backrest when calculating C.O.W. to enlarge the difference of C.O.W.. Second, we consider extracting user's stop intention by using Self-Organizing Map (SOM)[13] to realize a secure stop. By these operations, the wheelchair can keep a stop while the user wants to keep a stop, and the velocities of each wheel are generated even if the difference of C.O.W. is small.This paper is organized as follows: Section 2 outlines the wheelchair configuration using the HBMI. Section 3 discusses conventional wheelchair control scheme and its problem. Section 4 discusses the elimination of the constant pressure data for calculating C.O.W. and the classification of the body motion. Section 5 explains control taking into account motion classification and improved measurement through experiments. Section 6 summarizes conclusions.
2. Electric Wheelchair with HBMI
The outline of the electric wheelchair reviewed in the next section, see References[7,8].
2.1. System Configuration
This system uses a change of the C.O.W. in the pressure distribution occurring on the backrest that is a contact point between the user and the wheelchair to realize the control of the wheelchair. The system mainly consists of three elements: BPMS pressure sensor (Tekscan, Inc.), a motor drive system (JW-1, Yamaha), and a personal computer (PC), shown in Figure 2.![](image/10.5923.j.ajis.20120204.04_002.gif) | Figure 2. System Configuration |
2.2. Control Scheme
Proposed system uses the C.O.W. shown in Figure 3 as a index of body motions. The interface for the JW-1 is composed of two voltages. We converted a change of the C.O.W. into two voltages to associate the body motion for the control of the wheelchair. This method is as follows, and depicted in Figure 4.![](image/10.5923.j.ajis.20120204.04_003.gif) | Figure 3. Coordinate on the backrest and C.O.W |
![](image/10.5923.j.ajis.20120204.04_004.gif) | Figure 4. Block diagram of control scheme |
1. The initial position of C.O.W. Pg0 is recorded when the operation starts.2. The instruction voltage is calculated by following equation.![](image/10.5923.j.ajis.20120204.04_005.gif) | (1) |
HerePg : Present position of C.O.W.Pg0 : Initial position of C.O.W.A : Gain matrix from position to voltageV : Instruction voltage
xg, yg : Position of C.O.W. in each axis.vfb : Instruction voltage (forward-back direction)vlr : Instruction voltage (right-left direction)afb : Proportional gain (forward-back)alr : Proportional gain (right-left)3. The instruction voltage is sent to the motor drive unit.A difference between initial C.O.W. Pg0 and present C.O.W. Pg is multiplied by Gain A and is transmitted to JW-1 as command voltage V to activate the wheelchair.
3. Dilemma between Increasing Gain A and Secure Stop
The experiment confirmed that the above mentioned HBMI has a potential as an interface by SD-method and a factor analysis using 10 subjects[8].![](image/10.5923.j.ajis.20120204.04_007.gif) | Figure 5. The C.O.W when leaning right/left |
![](image/10.5923.j.ajis.20120204.04_008.gif) | Figure 6. Change of C.O.W. vs body inclination |
![](image/10.5923.j.ajis.20120204.04_009.gif) | Figure 7. Inclinometer and coordinate definition |
However, there is a problem. Figure 5 is one of the examples of this problem. This figure shows the pressure distribution while the user leans upper body to right and left with turning motion intention. A dark color is the strong pressure and bright color indicates the small pressure, the C.O.W. is also marked. In this case, the wheelchair did not move in spite of user's motion. Because, the change of C.O.W., which becomes instruction voltage by multiplying A in (1), is small even if the pressure distribution changes. Figure 6 shows the small change of C.O.W. in this case, with body inclination measured by inclinometer on the subject shown in Figure 7. In some cases, the C.O.W.s are almost same, although the pressure distribution shapes are changed by body motions. This makes deterioration of interface's usability because the user intention is not reflected to the motion of the wheelchair.We can consider following idea as one of the solution, which is increasing gain A in order to generate big instruction voltage in the case of small change of C.O.W.. Although increasing gain A makes wheelchair move in small change of the C.O.W., slight body motion can also activate wheelchair motion. That is to say, it is difficult to stop the wheelchair. In order to keep a stop, the initial and present C.O.W.s should be same as (1) shows. However, it is difficult for human to precisely control body motion. Therefore a secure stop would be difficult due to big gain A. Here, there is a dilemma between increasing gain A and a secure stop.
4. The Solution to the Problem
We consider following two methods as the solution to the dilemma problem. 1. To enlarge the difference between initial and present C.O.W..2. To extract user's stop intention from pressure data to make a secure stop.The details are shown in the following section.
4.1. Enlargement of the Difference: Pg0 - Pg
The C.O.W. is calculated by (2).![](image/10.5923.j.ajis.20120204.04_010.gif) | (2) |
Here, nx means the number of sensor cells in x direction, ny is the number of cells in y direction, Fij is the pressure at the ijth cell.The C.O.W. is strongly affected by strong pressure area such as dark color cells in Figure 5. Dark color area is almost static in spite of change of body inclination, where body constantly contacts with the chair. The change of C.O.W. is dominated by this constant and static contact area, therefore the change of C.O.W. is small. We therefore eliminate the constant pressure area and calculate the C.O.W., namely we mask the constant pressure area and extract cells which show big pressure change.In order to realize this, user swings his/her upper body right/left and forward/backward prior to control wheelchair. This preliminary motion is the calibration to detect constant contact area.![](image/10.5923.j.ajis.20120204.04_011.gif) | Figure 8. Extracted cells which shows big pressure change |
The pressure distribution is recorded while this calibration. After subject’s calibrating motion, we calculate the C.O.W. by using only the cells which exceed the pressure threshold of the difference between maximum and minimum pressure. The cells under the threshold are masked, and we set their pressure data to zero. The red-colored cells in Figure 8 are bigger than threshold, and are used for calculating C.O.W.Masked pressure distribution which is red-colored in Figure 8 eliminates constant pressure data, and extracts data which varies well. This masked pressure distribution is varied by even small body motion. Therefore the change of C.O.W. using this masked data shows slight body motion well.
4.2. Extraction of Stop Intention
Seven body motions are linked to wheelchair motion – (1) neutral, (2) forward, (3) leaning back, (4) leaning right, (5) leaning left, (6) twisting right, and (7) twisting left. By classifying body motion into these seven motions, the stop intention is extracted. Here, (1) neutral is linked to stop intention.For this classification, we employ Self-Organizing Map (SOM)[13]. In this classification, the input to the SOM is 34×44-grid film pressure sensor output (1,496 pieces of data).
4.2.1. SOM Learning Results
The 1,496 cells pressure data is SOM input vector. For learning, we used 345 pieces of pressure data for a male subject 170 cm tall and weighing 60 kg in above mentioned seven motions. Table 1 lists SOM learning parameters. After the initial weight in the map is set up by random numbers, SOM learning is executed to obtain a weight vector at each cell on the output map. | Number of Learning | 500000 | Learning rate | 0.02 | Radius | 6 | Map size | 20×20 |
|
|
![](image/10.5923.j.ajis.20120204.04_012.gif) | Figure 9. Similarity of each motion |
![](image/10.5923.j.ajis.20120204.04_013.gif) | Figure 10. The domain of each motion on the map |
After the weight vector at each cell is obtained, we used pressure data when the subject conducted the seven body motions above as experimental data and calculated the weight vector of each cell and cosine similarity - called similarity here.![](image/10.5923.j.ajis.20120204.04_014.gif) | Figure 11. Control scheme including SOM and Data Masking |
Figure 9 shows the weight vector of each cell on the output map and similarity to experimental data. For easy understanding, cells show 95% of maximum similarity for each motion. Different output patterns depend on individual motions. An output map corresponding to each motion was obtained from the similarity of each cell on the output map as shown in Figure 10.By this classification, we can extract user's stop intention.
5. Improvement Control Scheme
We improve the control scheme using two method described in section 4. SOM detects neutral position which is stop intention, and realizes a secure stop. Masking of pressure data eliminates the sensor cells which have small pressure change from the calculation of C.O.W., and realizes good response to slight body motion. These are shown in Figure 11.The pressure data is sent to the SOM and masking process. SOM detects neutral position. Masking process eliminates the cells of small pressure change, and sends masked data to C.O.W. calculation process. If SOM does not detect neutral position, C.O.W. is calculated using masked data, and then the wheelchair moves. On the other hand, if SOM detects neutral position, SOM switches the initial C.O.W Pg0 to present C.O.W. Pg. For this, the difference of C.O.W (Pg0 - Pg) becomes zero, and then the wheelchair stops.
5.1. Experiment
![](image/10.5923.j.ajis.20120204.04_015.gif) | Figure 12. The sequence of body motion |
![](image/10.5923.j.ajis.20120204.04_016.gif) | Figure 13. Trajectory of "Present C.O.W." in Figure 4 by inputting motion of Figure 12 |
![](image/10.5923.j.ajis.20120204.04_017.gif) | Figure 14. Trajectory of "difference Pg0-Pg" in Figure 4 by inputting motion of Figure 12 |
![](image/10.5923.j.ajis.20120204.04_018.gif) | Figure 15. Trajectory of "Present C.O.W." in Figure 11 (with masking process) by inputting motion of Figure 12 |
![](image/10.5923.j.ajis.20120204.04_019.gif) | Figure 16. Trajectory of "difference Pg0-Pg" in Figure 11 (with masking process) by inputting motion of Figure 12 |
![](image/10.5923.j.ajis.20120204.04_020.gif) | Figure 17. Trajectory of "difference Pg0-Pg" in Figure 11(with masking and SOM process) by inputting motion of Figure 12 |
We used the motion sequence shown in Fig. 12 to verify the improved control scheme: 1) The user takes a neutral position without putting arms on wheelchair armrests.2) The user leans his body to right3) The user keeps a neutral position and puts arms on the armrest.4) The user removes arms from the armrest.The intention of user control is to stop the wheelchair in motion 1), 3) and 4) and turn it to right in 3).First, the results of conventional method (Figure 4) are shown. The change of "Present C.O.W." in Figure 4 is shown in Figure 13, and "difference Pg0 - Pg " is shown in Figure 14. In these figures, the numbers "1) to 4)" indicate the motion duration of Figure 12. From Figure 13 and 14, it is confirmed that the C.O.W. and difference of C.O.W. are changed due to sequence of body motion.Next, the results including masking process shown in Figure 11 are depicted (not include SOM). The change of "Present C.O.W." in Figure 11 is shown in Figure 15, and "difference Pg0 - Pg " is shown in Figure 16. From these figures, it is confirmed that the change of C.O.W. is enlarged in comparison with Figure 13 by introducing masking process. The change of difference of C.O.W is also enlarged. Therefore, the body motion which shows small change of C.O.W. can be enlarged by adding masking process. However, the masking process also enlarges the change of C.O.W. in case of neutral position 3) in Figure 15and 16 which is duration of user's stop intention. The wheelchair cannot keep a stop due to the difference (Pg0 - Pg) caused by the swing up and down arms motion while the user intend to keep a stop. Therefore, in order to realize a secure stop by extracting stop intention, the classification of body motion by SOM is included. The result is shown in Figure 17. When SOM detects the neutral position, the switch, which is connected SOM output in Figure 11, is changed so as to switch initial C.O.W. Pg0 to present C.O.W. Pg. In figure 17, the difference of C.O.W. is kept zero in the duration of 1), 3) and 4) by SOM switching. Therefore, the wheelchair can keep a secure stop.From above things, improved control scheme has been able to control the wheelchair in small change of C.O.W. and realized a secure stop as long as detecting stop intention. Thus dilemma described in Section 3 has been solved.
6. Conclusions
This research focuses on the electric wheelchair controlled by Human Body Motion Interface (HBMI). HBMI uses the body motion which is caused by the voluntary motion. From our previous research, it has been confirmed that HBMI, which uses the center of weight on the pressure sensor attached on the backrest, has the ability of an interface. However the problem has also remained. The velocities of each wheel have been determined in proportion to the difference between initial and present position of the Center Of Weight(C.O.W.). This difference is generated by the inclination of the body. In some cases, while user leans his/her upper body, this difference doesn't increase due to the contact condition between body and backrest. In this case, the user cannot control the wheelchair arbitrarily in spite of leaning his/her body. For this problem, the gain which is proportion parameter between difference of C.O.W. and velocity of each wheel should be increased. However, too big gain lets the wheelchair cannot keep a stop. In order to solve this problem, in this paper, we first we eliminated constant pressure data on the backrest when calculating the C.O.W.. Second, considered extracting user's stop intention by using Self-Organizing Map (SOM). By these operations, the wheelchair can keep a secure stop while the user wants to keep a stop, and the velocities of each wheel are generated even if the difference of C.O.W. is small.
References
[1] | Jien Kono, Joji Inada, "Special Characteristics of Tooth Brushing Movement in Elementary School and the Elderly", The Japanese Society for Dental Health, Vol.58, No.3(19950625),pp. g91-g92,1995 (in Japanese). |
[2] | R.C. Simpon and S. P. Levine, "Adaptive shared control of a smart wheelchair opeated by voice contol", Proc. of IEEE int. conf. Intellignet Robots and Systems, pp.622-626, 1997. |
[3] | Yutaka Ichinose, Masahiko Wakumoto, Kiyoshi Honda, Teruaki Azuma, Junji Sato,"Human Interface Using a Wireless Tongue-Palate Contact Pressure Sensor System and Its Application to the Control of an Electric Wheelchair",The transactions of the Institute of Electronics, Information and Communication Engineers, Vol.J86-D-II, No.2, pp. 364-367, 2003 (in Japanese). |
[4] | Inhyuk Moon, Myungjoon Lee, Junuk Chu, and Museong Mun, "Wearable EMG-based HCI for Electric-Powered Wheelchair Users with Motor Disabilities", Proceedings of the 2005 IEEE International Conference on Robotics and Automation, pp. 2649-2654, 2005. |
[5] | Minoru Kamata, Hiroaki Nishino, Hideki Yoshida, Takeshi Someya, Minoru Suzuki, "Development of a Wheelchair of Head Operation with Gyro Sensor",JSME Proc. of the Welfare Engineering Symposium 2001, pp. W420 (in Japanese). |
[6] | T. Kigoshi,et al., "Electric wheelchair control with eye position and face inclination",Technical repot of IEICE (ISSN:09135685),Vol. 107, NO. 72(20070518), pp.13-16, 2007. |
[7] | Sho Yokota, Yasuhiro Ohyama, Hiroshi Hashimoto, Jin-Hua She, "The Electric Wheelchair Controlled by Human Body Motion - Design of the prototype and basic experiment-", Proceedings of the 17th IEEE International Symposium on Robot and Human Interactive Communication RO-MAN2008, ISBN:987-1-42442213-5,pp. 303-308, 2008. |
[8] | Sho Yokota, Hiroshi Hashimoto, Yasuhiro Ohyama, Jin-Hua She, "Electric Wheelchair Controlled by Human Body Motion Interface", IEEJ Trans. EIS, Vol.129, No.10, pp. 1874-1880, 2009 (in Japanese). |
[9] | Zong-Ping Luo, Lawrence J. Berglund, Kai-Nan An, "Validation of F-Scan pressure sensor system: A technical not", Journal of rehabilitation Research and Development Vol. 35, No.2, pp. 186- 191, Jun3 1998. |
[10] | T. Suzuki, S. Sekizawa, et al., “Modeling and Recognition of Human Driving Behavior based on Stochastic Switched ARX model,” in Proc. of the 44th IEEE Conference on Decision and Control, and the European Control Conference 2005 (CDC/ECC2005), pp. 5095-5100, 2005. |
[11] | T. Inamura, Y. Nakamura, and I. Toshima, “Embodied symbol emergence based on mime-sis theory,” International Journal of Robotics Research, Vol.23, No. 4, pp. 363.377, 2004. |
[12] | Satoshi Suzuki, Fumio Harashima, "Bayesian intention estimator using Self-Organizing Map and its experimental verification", Proc. of 19th IEEE International Symposium on Robot and Human Interactive Communication (Ro-Man 2010), pp.270-275, 2010. |
[13] | Teuvo Kohonen, "Self-Organizing Maps", Springer, 1997 |