1. Introduction

The face is a principal tool for humans to recognise one another. In addition, cognisance of the skin tone will also aid in recognising humans in a natural way. Likewise, skin detection is a great tool in automated face detection systems. It was reported that an HP webcam which had facial tracking software could not recognise dark-skinned faces [1]. Microsoft’s Kinect game was also reported to have problems recognising dark people despite its face recognition abilities [2]. Afterwards, Tech-micron reported the inability of a sensor based soap dispenser to dispense soap to dark skinned people [3] and recently it was reported that many facial recognitions featured cameras most of the time did not recognise dark people [4]. These are clear indications of the need for local input in the development of skin based technologies and this has necessitated optimal detection of the dark skin which is presented in this paper.

An illumination invariant skin detection algorithm using a hybrid skin segmentation system was developed by [5]. The RGB (Red Green Blue) images were converted to the YCrCb (luminance Chromatic Red Chromatic Blue) colour space and utilised the fast skin segmentation algorithm as developed by [6] and the fine adaptive single Gaussian distribution modelling threshold method. This hybrid method was used to segment the skin. The system was tested on the dark face database which contains coloured images of dark faces captured under real life situations. The detection rate obtained by the fast skin segmentation model was 66.18% unlike that of [6] which was tested on a white, light and dark skin database which had a detection rate of 92.35%. This is an indication that the same algorithm that works for light skin do not work well for dark skin. This has necessitated the characterisation of the black skin in this research. This informed the development of the Explicit-rule skin segmentation algorithm in this paper to characterise the dark skin. A similar research was carried out by [7] to characterise the Indian sub-continental skin and reported a 91.1% true positive detection rate using the HSV (Hue, Saturation, Value) Colour model.

A heuristic thresholding of YCbCr colour space is used to detect skin tone in [8]. This utilized explicit rules with thresholds in the color space to enhance skin ton detection. It successfully classified the pixels of images of single and groups as skin and non-skin. It obtained a high quality detection rate of 93% and low false positive rates. A human skin detection algorithm that uses RGB, HSV and YCbCr models for improvement of skin pixel recognition is presented in [9]. The algorithm uses explicit rules defined by values in a range of predefined thresholds for each parameter combining them with the bitwise AND operation to determine skin and non-skin pixels. A precision of 89.3% and accuracy of 94.43% was obtained from a subset of images from the test. This Paper presents a hybrid skin coloured based face detection system that optimally detect black faces. This utilises the Gaussian distribution model hybridised with an explicit rule based algorithm that typically detects black skin in a face recognition algorithm.

2. Methodology

In this experiment, images were acquired using a Close Circuit Television (CCTV) colour camera to capture video stream digital image, which is fed into the face pre-processing segment of the system. The RGB images are converted to YCrCb colour space which is more suitable for skin modelling [10, 12]. The face is detected, segmented and cropped using the skin detection and projection methods.

2.1. Skin Detection

In this paper skin detection is carried out by collecting skin samples from 300 images from [11] for training and the mean, variance, and covariance of the training dataset were estimated. The Gaussian model is suitable parametric method to model human skin [12], it is modelled to discriminate non skin pixels by producing an output of a greyscale image which shows skin likelihood regions. The greyscale image is transformed into a binary skin map image showing skin and non-skin areas using a thresholding technique [13, 14]. It has the advantage of generalizing well with small amount of training set, low cost of computation and requires low storage [15, 12].

The colour distribution of the skin pixel is defined by Gaussian probability density function. Hence, the probability of a pixel being a skin is given by:

(1)

where X is the colour vector, is the mean vector, is the covariance, and is the determinant of the covariance.

(2)

(3)

The parameters and were estimated from a training data set of 27000 pixels of skin samples, where N is the total number of skin colour samples X_i. The P (X|skin) is obtained by the Mahalanobis distance from the X colour vector to the mean vector given the covariance . The pixels were segmented as skin (SkinA) and non-skin pixels by adaptive thresholding method [5] which classify each pixel as skin or non-skin region in a binary skin map. The skin regions are classified as 1 and the non-skin region as 0.

Human skin is also modelled by explicitly defined rules derived from skin locus in 2D or 3D colour spaces [12], rules are proposed that define thresholds (boundaries) with which pixels are classified into skin or non-skin regions [5, 6, 8, 9, 12]. Explicit colour base thresholding is the most basic approach to skin tone classification [8]. In this paper the CbCr image was passed through the Explicit-rule skin segmentation algorithm to obtain the skin region for black skin tones.

The algorithm for the Explicit-rule is as follows:

(4)

The output of the two algorithms (Gaussian skin probability model and Explicit-rule) were fused using pixel based logic rule combination, by the logical OR or AND to get the actual skin region.

(5)

The skin detection process is an improvement of [5] where skins were detected by a fusion of the Gaussian model and Fast-skin segmentation algorithm [6]. When the skin region is obtained, the segmented region may have holes. In order to remove these holes two types of morphological operations, erosion and dilation were applied. It is after these operations that the skin image is segmented for the face mask.

2.2. Eye and Face Detection

To detect face, RGB images were converted to grey scale, while binarization was achieved by Canny Edge Detector [16]. Location of the eyes was used for pose normalisation or image de-skewing.

The left and right eyes pupil centres were located using horizontal and vertical projection as stated in equation 6.

(6)

where and are the coordinates of the minimum value of the summation of intensity values of the pixels horizontally and vertically in the edge map . These coordinates were used to de-skew and align the face. Coordinates obtained in equation 6 were used in equation 7 to de-skew the face for alignment of the eyes in the image. De-skewing was done so as to make the line joining the two eyes to be horizontal by rotating the face image by angle ϴ, which is the angle of elevation or depression of the line joining the two eyes. This alignment is required to make tracking algorithm work efficiently.

(7)

where are the coordinates of the left eye and are the coordinates of the right eye. Thereafter the eyes inter distance was used to normalise the face as illustrated Figure 1. After alignment, the distance between the two eyes is labelled as ‘d’ which was used as a reference to draw the face segmentation box around the face region and an eye region segmentation around the eye region using anthropological measures as shown in Figure 1.

Figure 1. Dimension of Normalised Face and Eye Templates Box Using Anthropological Measurements

3. Results

The system was implemented and tested using videos from the Black Eye-blink and Face Corpora (BEFC) [11]. The system was evaluated using some basic matrices by which classifications are assessed. Detection rate is the percentage of correctly detected faces. False Positives (FP) are non-skin pixels incorrectly classified as skin pixels, True Positives (TP) are the number of skin pixels correctly identified as skin, True Negative (TN) is the number of non-skin pixels correctly identified as non-skin and the False Negative (FN) are the skin pixels incorrectly identified as non-skin. The False Positive Rate (FPR) and False Negative Rate (FNR) are adopted as the quantitative performance in the skin detection algorithm. The F-measure, specificity and precision as in [8] are also measured. These are calculated using the equations;

(8)

(9)

(10)

(11)

(12)

(13)

The performance measures are given in table 1.

Skin samples from 300 images were used to obtain the Gaussian probability distribution function. A skin probability image from which the skin area is then mapped out using a threshold value obtained by experimentation. The results of the Gaussian probability map are shown in Figures 2. Figure 2.a shows the original images, 2.b parts indicated as probable skin regions by the Gaussian distribution and 2.c, the skin regions segmented after a threshold (0< threshold<=1) has been applied to the Gaussian probability distribution. Portions of the image that are white were segmented as skin region while black portions were classified as non-skin region. Of the 90 videos used as test data, 79 skins were correctly detected, which gives an 87% detection accuracy, with a 17% false positive detections.

Figure 2. (a) Original images (b) Gaussian Skin Probability Images. (c) Skin Maps after Threshold

The Explicit-rule skin segmentation model was also used to segment skin map. This experiment was carried out by plotting the histogram of Cb and Cr colour space of dark skin. Colour distribution of a human face in chromatic colour space was estimated from 27000 pixels of skin samples from 300 colour images. Figure 3 (a) and (b) show the histogram of the Cb and Cr colour spaces respectively. From these, thresholds of dark coloured skins were obtained, the lower limit threshold for Cr was 85 and the upper limit threshold 200, that is (85<Cr<200) and the lower limit threshold for the Cb was 60 while the upper limit was 130, that is (60<Cb<130).

Table 1. Performance Measure of Proposed Algorithm

Figure 3. (a): Histogram of the Cb colour space of black skin, (b): Histogram of the Cr colour space of black skin

The boundaries of the thresholds were picked using the histogram valley method. The lower limit was chosen as the value of the valley of the histogram and the upper limit as the end of the histogram. Pixels whose value fell within these boundaries were segmented as skin pixels. The dark skin colour, unlike the light skin colour, is a multi-modal (has multiple modal points in the histogram because of the variety of tones of the dark skin) distribution as indicated in the histograms, unlike the white skin which is uni-modal (has a single modal point in the histogram) in distribution. Sample results of the Explicit-rule skin map is presented in Figure 4. It also gave a good skin and non-skin classification. This is to increase the accuracy of the skin detection. The Explicit-rule skin detection algorithm gave a detection accuracy of 71% with 64 correctly detected skin maps of the 90 face images and a false positive of about 41%.

Figure 4. An Explicit-rule Skin Maps

The Explicit-rule skin map was fused with the Gaussian probability skin map by a pixel based fussing combination using logical AND or OR to get a more accurate skin map. Samples of hybrid skin maps are shown in Figure 5, 89% of the AND hybrid came out accurate while 55% of the OR Hybrid came out accurately. From hybrid skin map, the skin image was masked on the original RGB image to segment the skin from the image. This was used for further processing to segment the face region. Typical skin segmentations obtained from images are shown in Figure 6.

Figure 5. Hybrid Images of Skin Segmentation Map.

Figure 6. Face Images Segmented by Skin Map

3.1. Discussion of Results

A major contribution of this research work is the characterization of the dark skin. The fast skin segmentation algorithm based on the CrCb colour space for skin detection was used in [6]. The fast-skin detection algorithm was later used by [5] in the detection of dark skin and the study recorded a lower accuracy (66.18%). The proposed Explicit-rule dark skin algorithm in this research was used to detect the dark skin and it reported a higher rate (71%) than that obtained by the fast-skin segmentation [6] as reported by [5] algorithm on the dark skin. Table 1 presents the results obtained.

A detection rate of 92.3% was reported by [6] using fast-skin segmentation algorithm, while [5] reported 66.18% when the same algorithm was implemented on Black faces captured under varying illumination conditions. This buttresses the fact that skin colour models that model the light skin accurately do not model dark skins accurately. Using the dark skin Explicit-rule model in this research, a detection rate of 71% was obtained which is an improvement on [5]. Analysis is presented in Table 2.

Table 2. Analysis of Result Obtained from Different Skin Detection Algorithms

The result obtained from the fast skin segmentation. The hybrid skin model by [5] recorded a detection rate of 83.3 % and the Hybrid model in this study recorded a detection rate of 89%. These reports buttress the fact that the characterisation of the dark skin is necessary for improved detection rates in skin based modelling for dark skin.

4. Conclusions

This paper has presented a hybrid skin colour based face detection system, which utilised both the Gaussian probability density function and the Explicit-rule algorithm to detect dark skin optimally. It achieved a detection rate of 89% which is an improvement on [5] that had a detection rate of 83.3%. This system was further utilised in an eye-blink communication system enabled by face recognition to trigger safety and security alerts.