1. Introduction

Fall injuries are dangerous and often happen for older people. These injuries can be burdens to individuals, their families and healthcare systems [1]. According to the Canada’s Public Health Agency in 2014 [2], the rate of old people was hospitalized by fall increasingly, particularly there was 0.4% - 0.6% people at the age 65-69, who went to the hospital every year. It was rising 4.9% - 6.8% people at the age over 90. Fall and related fall injuries are some of causes leading to the death rate of people in the world. Therefore, the earlier fall recognition of elderly people is necessary and will increase their living opportunity. With the development of high technology, the automatic system with alarm is a necessary solution for fall recognition.

Many researches have used modern devices to monitor the human activities in recent years. These devices can recognize fall and then generate the alarm signals to relatives and medical staff for emergency support [2]. Based on features of fall, a method for detection and notification of fall has been constructed. The most importance of a system is accurate distinction between normal and fall activities in the series of daily activities of the elderly people in the indoor environment. The detection system consists of three main basic methods: a camera system [3, 4], environmental sensors [5] and wearable sensors [6, 7].

In a system, environmental sensors are often used to collect surrounding activities in the established environment and produce corresponding signals. In practice, there are types of sensors as infrared, vibration, laser, pressure and others are used in exploring changes of the environment. The passive system was designed using vibration sensors in seeking and recognizing changes of vibration in the floor [8, 9]. With this design, the system recognized the floor vibration and can distinguish between fall and other postures. Therefore, the systems with this type sensor were developed to find out the fall status of victims being fainted or strained based on spectrum.

Wearable sensors such as acceleration, gyroscope were usually used to install with human body for determination of tilt angles related to recognition of human postures. In order to perform recognition systems with these wearable sensors, two methods such as threshold method and machine learning algorithm were applied [10]. In particular, acceleration sensors were used in the system to measure problems related to chest of human. Therefore, signals obtained from the sensors were transmitted to computer for processing and then recognizing human motion states (fall or normal state) based on threshold method [11]. Experimental results of this research is that the accuracy with normal and fall cases is high, particularly the standing state is approximate of 100%, the going state is about 100% and two cases with under 95% accuracy are falling back without lying and falling left. The advantages of this system are small and easy to move.

With computer vision systems, cameras are often used to set up at some fixed positions in house or room to supervise activities of elderly people. Image processing is applied to calculate image data and then identify human postures. A Moving Average (MA) method in the system with the cameras was employed to detect fall states [12]. In particular, this approach allows detecting fall states base on depth map and normal color information [13]. Moreover, History Moving Image (HMI) was utilized to classify collected data to find moving state. The HMI features were extracted for searching output movements based on RGB depth maps. For recognition of fall, a Support Vector Machine (SVM) algorithm was applied to identify fall states using a camera system with RGB-Depth [14].

In recent years, machine learning method has been employed for recognition of many problem related to image data. In particular, for recognition of human gestures (standing, siting and lying), three methods such as the SVM, decision tree, Navie Bayes were utilized [15]. The results showed that the SVM method had the accuracy of 100%, the decision tree gave the 90% accuracy and the Navie Bayes was about 85%. Thus, the SVM algorithm is the most accuracy compared to the two remaining methods.

The Kinect camera system has been developed to use in many applications in recent years. In an application of fall detection, the system was installed with a Kinect camera and an accelerometer sensor to increase the accuracy of recognition. Therefore, methods of recognition such as the SVM, the Upper Fall Threshold (UFT), the Lower Fall Threshold (LFT) algorithm were applied. The final result showed that the recognition using the SVM with depth data combined to accelerate features has the higher accuracy than that using others [16].

The purpose of this research is to establish a system of fall recognition with a Kinect camera and warning for elderly people. Data obtained are fall and normal postures. For classification of postures, a Principle Component Analysis (PCA) algorithm will be applied to extract features of data. Thus, a Support Vector Machine (SVM) algorithm will be employed for classification of fall and non-fall. In addition, a warning system in the recognition will be designed to send a mobile message for warning to relatives. Experimental results will be shown so that the effectiveness of the proposed approach.

2. Data Acquisition and Materials

In this research, a fall recognition system for elderly people has three main parts: data acquisition from the Kinect camera, a computer for processing data and fall recognition and an alarm module as shown in Figure 1. This Kinect camera system, which has a camera and a laser sensor, can produce skeletal data with bone joints. It means that the skeletal data obtained have three axes, in which there is depth information. In particular, the depth axis can allow determining distance from the camera center to an object. In addition, this camera system produces the type of these skeletal data with three axes which can be easier for detection of human motions as well as fall recognition.

Figure 1. System of fall recognition and alert

In addition, the fall alert system is designed to recognize human postures and classify between fall and normal state as shown in Figure 2. In particular, skeletal data obtained from the camera system will be extracted features using a Principal Component Analysis (PCA) algorithm for training. Therefore, the trained data will be sampled to recognize human posture. For recognition of fall and non-fall, a SVM method will be applied to classify five pairs of fall and other activities such as (fall_non-fall, fall_stand, fall_sit, fal_-bend, fall_lie).

Figure 2. Block diagram of the recognition system

An alert system is an important part in the fall monitoring system for elderly people as shown in Figure 3. In particular, this system will recognize fall and then send an alert message to a mobile phone of their relatives. In this research, the alert system will be combined to the recognition system to create a smart recognition in the fall system. It means that when the recognition system has recognized falling, immediately the alert system will be activated to send one message to their relatives for emergency solutions.

Figure 3. Block diagram of an alarm system

3. Kinect Data

3.1. Skeleton of Kinect System

Figure 4. Diagram of twenty body- joint positions

In addition, each position of the skeletal image has the typical name, particularly the name of the 4^th position is “Head joint”. Therefore, all human postures will be affected by three types of these functional bones and human postures will be recognized based on these joint data.

3.2. Data Acquisition

In this paper, a Kinect camera system was installed at a fixed position in a room and three subjects, including two males, one female were introduced to perform experiments with different postures. The subjects invited have the age of 19-26 years old and the weights are between 51-75 kg. Moreover, the subjects were instructed to work out eight postures, including “fall front”, “fall back”, “fall right”, “fall left”, “stand”, “sit” on the chair, “bend”, lie down” on the floor.

In this recognition system, the Kinect camera is fixed at an appropriate position corresponding to the region of view with the 1.2 m height and the 3 m depth for collection of 3D skeletal data as shown in Figure 5. In particular, each 3D data of human bone joints obtained from the camera denotes with three coordinates (x, y, z), called (Horizontal, Vertical, Depth) for a joint position.

Figure 5. Schematic of the Kinect camera for collection of 3D image

In this study, each joint position I is described as the transpose matrix with three values in a coordinate (x, y, z) and expressed as follows:

(1)

where n describes the number of the joint points, n = 1, 2, …, 20.

In building 3D data, each image frame is determined to be 20 bone joints and each joint is in the 3D coordinate (x,y,z). Therefore, this frame can be described as a column vector I_m of 60 variables as shown in Fig. 6. This vector is arranged from matrices in Eq. (1) and each frame is expressed as follows:

(2)

Assume that one video clip has k frames, k=1,2, …, with human activities is defined as the following matrix, P:

(3)

This matrix P can is also expressed another way as follows:

(4)

The matrix, P is the set of database described as Figure. 6, in which each vector P_k, is described to be a column matrix with bone joint values of human postures in coordinates, (x,y,z).

Figure 6. Description of data, p, arranged to be a column vector

When a fall activity occurs, the human posture will change dramatically. It means that data obtained from the camera system will be the data including from normal state into fall on floor. In particular, the change of 3D data is determined based on bone joints in a very short time and the location of the first joints abruptly changes along the y vertical axis of the coordinate system as shown in Fig. 7.

Figure 7. Illustrating activities of human falls

The difference of “Head” joints at the y-axis of between two activities of fall and non-fall is introduced in Figure 7. It means that data of “Head” joint before falling is different from that of after falling. While, in normal activities such as stand, sit on chairs, bend, lie down on the floor, values in the coordinates of the “Head” joints have no change nearly. However, during fall activities, values in the y coordinates of the first joint change dramatically in a short time and then these values have no change nearly in a certain period of time as shown in Figure 8. For example, considering the data in the range of from the 11^th frame to the 20^th one, four lines of data represented in different colors of dark blue, green, blue and red corresponding to postures of “fall front”, “fall back”, “fall left”, “fall right” are suddenly reduced, while values in the y-axis with colors of purple, yellow, black, blue corresponding to the states of “stand”, “sit” on chairs, “bend”, “lie” down on the floor have no change. Therefore, the data, which are circled with red color, are the basic so that one can perform data analysis for estimation and fall recognition of human postures.

Figure 8. Y-axis coordinates of the head joints between fall and non-fall states

For calculation of recognition, the obtained data of eight postures from the camera system are processed to convert into a matrix size of 60x2400 which is called column data as shown in Table 1. In addition, the data are separated to distinguish each posture corresponding to a data cluster, so it has eight data clusters.

Table 1. Column data cluster corresponding to human postures

In this research, the process of collecting data plays an important role for training and recognition of postures. The data collected by the Kinect camera system will be extracted using a PCA algorithm and then classified based on a SVM method. Experimental results of 120 samples performed on three people corresponding to 2400 image frames for recognition of human falling.

4. Methodology

4.1. Feature Extraction using PCA Algorithm

A Principal Component Analysis (PCA) algorithm is applied to extract feature components of fall data. In particular, skeletal data of fall and non-fall postures are pre-processed to produce the data matrices. Thus, the average value , with i=1,2,…,k is determined from the row components of the matrix P in Eq. (4) and this equation is expressed as follows:

(5)

For similarity, one can determine the remaining average components, .

Next step of this PCA algorithm is that the Standard Deviation (SD) will allow to calculate vectors as follows:

(6)

For similarity, the remaining vectors will be determined, .

From the set of the vectors, , a matrix A is built as follows:

(6)

After determining the matrix A, a covariance matrix of the normalized data needs to be found for description of correlation of each vector to the vector space. Therefore, this covariance is expressed as the following equation:

(7)

In this PCA algorithm, determining eigenvectors based on eigenvalues of data sets is important. Thus, in order to calculate eigenvalues, one needs to calculate determinant of a matrix using the following equation:

(8)

where C is a covariance matrix; λ is the eigenvalues and P is the matrix of Eq. (4).

After determining the eigenvalues, the eigenvector V can be calculated using the following equation:

(9)

In this paper, the PCA method is employed for extraction of fall data. The main purpose of this method is that the principal features of eight postures are chosen and the weak components in the set of data are removed. Therefore, that the principal features of eight postures will be trained using the SVM algorithm for fall classification. In this research, with eight postures of three persons, the set of data for training is very large after the PCA, particularly a coefficient matrix of 2400 x 2400 is created corresponding to feature vectors k.

4.2. Support Vector Machine Algorithm

In this paper, a SVM method is utilized in the system of fall recognition to classify between two postures of fall and normal activities of elderly people. From this classification, one can recognize fall state.

After feature data of postures are determined using the PCA, the SVM algorithm is applied to train the PCA feature data and classify states of fall and non-fall. In the SVM algorithm, the linear hyperplane is an area to divide the data set into two subsets collection according to the linear hyperplane. Assume that pattern elements (datasets after the PCA) are expressed as follows:

(10)

where while is subclass of x_{i .}

Therefore, to identify the hyperplane, one needs to find a plane making Euclidean distance between two layers as shown in Figure 9. In particular, a vector with the distance values close to the plane is called the support vector, in which, the positive value is y = +1 or H1 and the negative one is in the region of y = -1 or H2.

Figure 9. Description of the SVM algorithm with hyperplane

The SVM algorithm is to find an optimal hyperplane for classification of two classes and expressed as follows:

(11)

Assume that the equation of hyperplane is , where w is the vector with perpendicular points to the separating hyperplane and with is distance from this hyper plane to the origin, and ||w|| is the magnitude of w.

Moreover, d₊ (d_-) is the shortest distance from the hyper-boundary to positive (negative) samples. Therefore, margin of plane is d₊ + (d_-) in the linear case and the support vector looks for the separating hyper plane with the largest margin using the primal Lagrangian. Suppose that all training data satisfy the following constraints:

(12)

(13)

From two equations (10) and (11), one has:

(14)

According to the positive Lagrange multiplier, with i = 1, 2, ..., m, the Lagrangian function is represented as follows:

(15)

where is the Lagrange multiplier. Taking the derivation w of L_p, the equation is calculated as follows:

(16)

or it can be re-written as follows:

(17)

The final equation is expressed as follows:

(18a)

(18b)

In classification, each training sample corresponds to x_i and Lagrange coefficients. After training, is the support vectors and located on one of the two hyperplanes, H1 and H2. If the result of data classified has the value x*, its subclass is of x* (-1 or +1) and is described through the following formula:

(19)

In this study, 2400 samples of eight postures on three people were chosen for training, in which it includes 1200 falls (fall front, fall back, fall left, fall right) and 1200 non-falls (stand, sit on chairs, bend, lie down on the floor).

All paragraphs must be indented. All paragraphs must be justified alignment. With justified alignment, both sides of the paragraph are straight.

5. Results and Discussion

In the experiments, offline databases were used for feature extraction using the PCA algorithm and then these features were trained for classification using the SVM. Experimental results were shown to illustrate the effectiveness of the proposed method in the fall recognition system.

5.1. Representation of Fall and Non-Fall Postures

In this research, three subjects were invited to attend experiments and introduced to well understand the operation of the fall recognition system. Therefore, every subject was instructed to perform different activities and one activity was worked out ten times of eight postures. In particular, these subjects were also instructed to move their postures such as “Stand”, “Sit”, “Bend”, “lying” and “fall”, in which the fall state has four falling postures (forward, backward, left and right). Therefore, the data were collected on three subjects with different postures from the Kinect camera, in which they can be arranged into four types of states: “Stand”, “Bend”, “Lying” and “Sit”.

Figure 10. Images of non-fall postures: (a1), (a2), (a3)_ “Stand”; (b1), (b2), (b3)_“Sit on a chair”; (c1), (c2), (c3)_ “Bend”; (d1), (d2), (d3)_ “Lie down on the floor”

Every subject invited to perform eight postures “Stand”, “Sit”, “Bend”, “Lying”, “Fall forward”, “Fall backward”, “Fall left”, “Fall right” and each posture is ten times. Thus, the total samples obtained from three people are 240, in which each collected image has from 50 to 60 frames. In addition, the collected data have 60 columns of 20 bone joints represented in three coordinates (x, y, z). In practice, postures of daily activities of three people (two males and one female) were recorded as shown in Figure 10 and Figure 11.

Figure 11. Images of fall postures: (a1), (a2), (a3)_“Fall front”; (b1), (b2), (b3)_“ Fall back”; (c1), (c2), (c3)_ “Fall left”; (d1), (d2), (d3)_ “Fall right”

In these cases, the 4^th joint, called the head joint (in Fig. 4), has the largest change compared between falls and non-falls for classification. Hence, data at head joints in each case are important in analyzing to be able to produce difference compared to joints at the remaining joints.

Figure 12 describes fall state of a subject, in which three lines (red, blue and green) represent coordinates (x, y, z). It shows the distribution of joints information when people fall. In particular, the vertical axis is the value of the distance and the horizontal axis represents the number of image frame reviewed. The signal lines of red, blue, green illustrate the coordinate value, change of the position of the head joints corresponding to the three axes x, y, z in a certain period time.

Figure 12. Representation of four fall states related to head joints: (a) Fall forward. (b) Fall backward. (c) Fall to left. (d) Fall to right

In Figure 12 with four cases of falls, the first 10 frames of 60 frames show a state of rest. However, from the 11^th to 20^th frames, green and blue data lines dramatically decrease in a very short time at axes-y and -x. Therefore, it is an important change for fall recognition. While these green and blue lines in Figure 13 nearly have no change at the vertical axis-y (green color) as circled with red during 60 data frames.

Figure 13. Representation of four non-fall states related to head joints: (a) Stand. (b) Sit on a chair. (c) Bend. (d) Lie down on the floor

During these experiments, one can be based on changes of data lines of fall and non-fall at the x and y axes at short time for fall recognition. In the set of fall and non-fall data, the fall data represents sudden change compared to change of the normal data at the same vertical axis-y. In practice, it is dependent on levels of fall and non-fall states in very short time for calculation of fall classification. Therefore, in order to have more accurate recognition, the SVM algorithm was applied to classify states of fall or non-fall for fall recognition and alert.

5.2. Fall Recognition Using the SVM

This research represents experimental results corresponding to human postures for recognition of fall and non-fall as shown in Figure 14. In particular, this figure shows two postures, one for fall on floor and another one for standing state corresponding to three skeletal data lines (red, green and blue).

Figure 14. Representation posture images corresponding to data lines: (a) Fall posture and simulation result; (b) Non-fall posture and simulation result

The recognition results using the SVM algorithm were performed on 2400 trained experimental data and 1200 test samples. In the trained samples including 1200 falls and 1200 non-falls, the average accuracy is about 82.2%. Another test is that with the test samples of 600 falls and 600 non-falls, the 82.7% accuracy is obtained as shown in Table II. In particular, in 1200 training samples, the system correctly recognized 976 fall states and it is the 81.8% accuracy. Similarly, other types of activities produced the results as shown in Table 2.

Table 2. Recognition results of fall and non-fall activities

In this system, the SVM algorithm was employed to classify between a fall state and a normal activity. In addition, in order to perform this classification, 600 fall samples and 300 samples of other postures were used and the accuracy was calculated as shown in Table 3. In this research, there were three subjects who were invited to work out fall and normal activities with many times for taking samples. In particular, these activities are that stand, sit on a chair, bend, lie down on the floor is trained. Cases of identity include: fall - stand, fall - sit in a chair, fall - bend, fall – lie down on the floor. Therefore, the SVM was applied to classify each pair of fall and a normal activity.

Table 3. Recognition results of fall and other activities

5.3. Alert System for Falling

In this research, an alert system was designed for transferring message to subject’s relatives when it recognizes a fall of an elderly person as shown in Figure 15. In particular, the notification messages were installed available in the system for sending to any a mobile phone for an announcement and a SMS. Thus, the mobile phone can receive the message with the content of "Falling. Emergency! Help".

Figure 15. Description of the fall system with notification message and calls

In order to enhance help from people close to house, a buzzer module connected to a computer was designed to create sound to inform relatives. In particular, when an elderly person is fallen, the fall system recognizes fall posture and then send a message to mobile phone of relatives.

The Kinect camera system has been used popularly in many different fields such as computer games, smart wheelchairs, detection of arm movements and others. In an application of the Kinect camera for determination of different speeds (fast, medium and slow) of arm movements of 27 people with the average age of 29. The result showed that the overall error of the interclass classification was 5.43% [17]. The Kinect camera system was used to recognize human pose postures in parts from depth single images and then predicted 3D positions of body joints with high accuracy [18].

In this paper, the fall recognition system using the Kinect camera system to obtain fall and non-fall postures on three subjects. Skeletal data of eight postures are divided into sample data for extraction of features. In addition, the SVM algorithm was utilized to train the feature data and then to classify fall and non-fall states for fall recognition and alert with the average accuracy of 81.8%. In particular, an alert system in the fall recognition will send a message as an emergency contact to user mobile phones for help. It is obvious that this is one important real application in recognizing early fall for elderly people for help and treatment. Moreover it remains the trusted problem in future research as well development of the better fall recognition.

6. Conclusions

In this research, the skeletal data of three subjects with eight postures obtained from a Kinect system to be converted into images. These image data were extracted features using a PCA method. For fall posture classification, a SVM algorithm was employed for training the image data set and then for classifying fall and non-fall postures. Experimental results showed that, the average accuracy between fall and non-fall is 81.8% and 83.5%, respectively. In addition, the fall recognition could send message to relatives through mobile phones. With these effective results of classification using the PCA-SVM algorithm, it will be suitable to remain for future relevant studies.

ACKNOWLEDGEMENTS

The authors would like to acknowledge the support of HCMC University of Technology and Education, Vietnam, with Grand No. T2016 – 44TĐ. Moreover, we would send to thank students and colleagues for this research.