1. Introduction

Crying is the first sign of life at birth. It is an innate behavior that plays a fundamental role in the survival, health and development of the child. It is also the least explicit expression of distress or pain. Tests for evaluating the health of the newborn, such as Apgar score (Appearance, Pulse, Grimace, Activity, and Respiration) are now routine during birth. However, these tests do not incorporate quantitative measurement of acoustic characteristics of cries, providing only a qualitative summary based on observation. Our goal is therefore to utilise cries in order to improve monitoring of the first days of infant life[1, 2, 3].

Cry analysis allows identification of enfant with medical syndromes when no symptoms are present[2, 4]. Early diagnosis before the onset of clinical symptoms will increase the likelihood of successful intervention before the illness has serious impact on the health of the infant. In the case of pathologies that are not detectable without in-depth examination and specialized tests, if medical treatment is begun soon enough, enfant can heal completely.

Early diagnosis of various pathologies that can afflict newborn using the spontaneous cries during the first four weeks of life is crucial because from 4 weeks, infants acquired a voluntary control of these vocal tracts[2].

The cry itself contains a wealth of information. Its acoustic properties reflect the conditions that caused it to be produced[4]. In order to evaluate infant cries, acoustic parameters must be measured quantitatively and deviations from “normality” must be interpreted.

The classification of the cries is a basic techniques used to design a newborn’s PCIS (pathological cry identification system). It falls within the automation of a natural perception of cries and its modelling using advanced signal processing techniques. It contains two main parts namely a cry signal characterization and modelling.

There have been numerous fundamental studies related to this research area. The thrust of one of these lines of study has been to analyze infant cries in order to catalogue acoustic manifestations of pathological symptoms[1, 4, 5, 6, 7]. Our study is based on several hypotheses that these researchers have formulated. These hypotheses assert essentially that infant cries stem from physical and psychological status as well as both internal and external stimuli[4]. Both temporal and acoustic characteristics contain biological alarms. By observing spectrograms and spectra of audio signals of cries, associations have been made with pathological conditions. This visual approach is applied after the fact. To our best knowledge, there is no software tool that alerts health-care workers to the development of pathologies in newborns by direct monitoring of cries. Therefore, we propose to conduct a basic experimental research through the following:

● Review the most pertinent characteristics of infant cries, based on scientific consensus.

● Identify the most promising characteristics and formally define them.

● Associate these characteristics with pathologies of interest.

● Automatic measurement of these characteristics.

● Test the influence of use the studied characteristics in addition to other features in performance of PCIS.

One of the aims of this research is therefore to define as precisely as possible these characteristics, to create a tool for estimating them without human intervention and then quantify them for each pathological condition examined. The principal advantage of this approach is that once the tool is developed, it will evolve through updates as other pathologies are characterized. Our main contribution is therefore the quantification of acoustic characteristics associated with pathologies and ultimately provides a basis for alerting health-care workers to intervene. Of course the developed tool and system is not being proposed to replace health-care personnel. However, it could be highly appreciated for alerting personnel, especially in the hectic hospital setting. Once developed, the operation of the system is simple and does not require many resources. It can be set up quickly and at minimal cost.

This paper is divided into seven sections. In section 2, we present a succinct review of the scientific literature providing acoustic definitions of infant cries and their characteristics and modes. We describe in section 3 known associations between infant medical conditions and cry characteristics, with focus on healthy infants, premature births and certain health problems. Section 4 presents the database used in this study and discusses our adopted methodology for qualitative and quantitative characterization of cries with pathological implications and cries without such implications among the healthy infants studied. Section 5 is devoted to the analysis of the results obtained for an initial quantitative characterization of healthy and pathological cries. The next section presents experimental validation of use the proposed characteristics concatenated with MFCC coefficients (Mel Frequency Cepstral Coefficients) in PNN classifier (Probabilistic Neural Network). We conclude this paper with a conclusion.

2. Acoustic Definition of Cries

Infant crying consists primarily of rhythmic alternation of phonation and breaths. The cry coincides with exhaling, the phonation being produced by the larynx[4]. Each infant cry is acoustically unique. In most children, the fundamental frequency F0 varies from 250 Hz to 450 Hz, with a first formant at a frequency F1 of 1100 Hz and a second formant at a frequency F2 of 3100 Hz[4]. Lester et al. define three identifiable modes of cries due to vocal cord vibrations: a). Basic cry or phonation with F0 (350–750) Hz, b) High pitched cry F0 (750–1000Hz) or Hyper-phonation F0 (1000–2000Hz) and c) Noisy, turbulent or dysphonic cries. A certain terminology has been developed in various studies to describe the acoustic characteristics of cries[2, 4, 5, 8]. The generally accepted terms are shown in Table1.

3. The Characteristics of Cries under Certain Medical Conditions

3.1. The Cries of Healthy Newborns

During the first months of life of an infant, cries are short with simple melodic forms and become longer and more complex with age[9]. At the age of one year, cries of infants born healthy are characterized by F0 varying from 400 to 600 H z with the average value of 450 Hz, often with a decreasing or increasing-decreasing melody shape[8], with super- imposed symmetric harmonics and an average duration of 1–1.5s[6]. The variation of F0 is regular. A shift is frequent at the beginning of the signal in the case of cries produced by pain and spontaneous cries, but rare during the first days of the newborn[10].

Table 1. Cry characteristics

3.2. The Cries of Premature Newborns

The differences in the characteristics of these cries are proportional to the number of weeks by which the birth was premature, the more premature, the higher fundamental frequency[1]. Some cries are short and piercing, while others may be of duration and F0 similar to the cries of full-term newborns. This can be due to some disorders not detected at birth[1, 8].

3.3. Cries of Newborns with Various Pathologies

The cries of a newborn with a pathological condition are persistent with little punctuation, reflecting high irritability and poor physiological stability. When a central nervous system (CNS) disorder is involved, the cry exhibits auditory abnormalities. However, its F0 is high and its melodic contour is irregular[2]. In spite of differences in measurement procedures, all cry studies have shown that high F0 is an indicator of a neurological problem[4]. Other markers also associated with neonatal risk include hyper-phonic cries, very high-pitched cries, noisy or dysphonic cries as well as changing mode between phonation, dysphonic cries and variability of F0 and F1[11]. Spectrographic analysis studies carried out on the cries of infants with disorders such as meningitis, hydrocephaly, congenital abnormalities, chromosomal aberrations or metabolic dysfunction indicate abnormal cry features[8]. In the syndrome known as cat cry, the cry is intense and monotone with a flat melodic contour, while in infants with a hypothyroid condition, the cries are of weak intensity. In newborns with herpetic encephalitis, the cry is significantly dysphonic and marked by increased noise concentration[4].

4. Methodology for Qualitative and Quantitative Characterization

Our approach for characterization of healthy and pathologic cries is represented on the simplified blocks diagram illustrated in Figure 1. It consists of four steps: 1) choosing of the pathologies to be studied and recording of cries of newborns in order to build an experimental database, 2) qualitative characterization of the cries, based on identification of important characteristics in studied cries, 3) quantitative characterization based on estimation of selected characteristics and 4) establishing quantitative relationships between these characteristics and the pathologies studied.

4.1. Cry Database

The database used contains 3146 cry samples of 1s duration from 66 newborn babies aged 1 day to 4 weeks. 1774 cry samples from 31 healthy newborn (among them 764 are premature) and 1372 from 35 newborn who present some diseases (among them 829 are premature). The Table 2 shows the pathologies studied by categories of diseases. These cries were collected with the aid of medical collaborators of neonatology department at Saint-Justine Hospital in Montreal. Each infant is recorded two or three times, with at least one hour between each recording over a period of not more than ten days. The date and time of recording, gender identification, birth date, diagnosis, and ethnicity are noted for each crying episode.

Figure 1. Newborn Cries Characterization

Table 2. The pathologies studied

The recording of cries is done using a small recorder, placed at a distance of 10 cm of babies’ mouth with a sampling rate of 44.1 kHz.

The conditions in which the cries are recorded are: hunger, sampling blood and change of diapers. In order to confirm the health status of the infant or to determine that an undiagnosed health problem was in fact present during recording; we consult the follow-up examination conducted six months after recording of the cries.

In view of the spectacular increase in the numbers of pre-mature births recorded throughout much of the world, we decided to compare this category of cries to the cries of full-term newborns. Pre-mature newborns face major risks associated with organ functional immaturity. They are also more vulnerable to infection and face an increased risk of cerebral lesion when a significant jaundice is present. Early diagnosis of the various pathologies that can afflict this category of infant is crucial.

4.2. Qualitative Characterization of Cries

The principal aim of this part of the work was to identify the acoustic characteristics that best serve to characterize the cries studied and to establish qualitative and relationships between these characteristics and the various pathologies studied. This work is completed using PRAAT, a freeware program for the analysis and the reconstruction of acoustic speech signals[12].

The spectrographic analysis based on observation of pathological cry signals as well as cries of healthy infants to identify acoustic characteristics of these cries, allows us to establish the initial qualitative relationships between acoustic characteristics of cries and pathologies shown in Table 3. This analysis leads us to the conclusion that cries of infants with pathological conditions are indeed quite different from those of healthy infants.

Examples of spectrogram and estimated F0 for full-term, premature, healthy and sick newborns are shown in Figure 2. We note raising-falling pitch contour for healthy newborn’s cries (preterm and full-term) and lower spectral intensity, height and irregular pitch contour in pathologic cries.

4.3. Quantitative Characterization of Cries

The quantitative characterization of cries contains an automated measurement of acoustic characteristics of cries that make major contributions to the differentiation between pathological cries and healthy cries.

In order to establish quantitative relationships between cry characteristics and the pathologies, we began by estimating or measuring certain acoustic characteristics cited in the literature in association with severe medical conditions, such as F0, the presence of hyper-phonation and irregularity of F0.

The principal means used by researchers to analyze infant cries involve software initially dedicated to adult voice analysis. Since the adult and infant vocal passages differ in shape, these tools should be used with precaution[4]. We developed our own measuring tool using Matlab. This tool estimates F0 and percentage of hyper-phonic segment as well as the F0 irregularity, as de fined in Table 4.

The following approach was taken when estimating these characteristics:

● Noise filtering and segmenting recordings into useful and non-useful portions.

● Estimation of F0 in short segments typically of 20ms.

● Identification of hyper-phonic segments, as well as irregularity of F0 as defined in Table 4.

● Calculation of average percentage of hyper-phonic segments () and irregularity of F0 ().

This has been performed: 1) for healthy and pathologic cries, 2) by category of pathologies and finally 3) by pathology and gestational age. We used the following formulas:

,

Where , and are the total number of segments, the total number of hyper-phonic segments and the total number of F0 irregularity respectively.

● Utilization of standard deviation for estimated characteristics by category of pathologies and application of ANOVA variance analysis for healthy and pathologic cries to carry out the analysis of the estimated characteristics by gestational age (Full-term, Preterm).

Table 3. Characteristics of cries examined by spectrographic analysis

Figure 2. Waveform and spectrogram of healthy and pathologic cries

4.4. Fundamental Frequency Estimation

Since F0 is one of the most widely used characteristics for distinguishing cries and measurement of most other characteristics stems from it, precise measurement of F0 and its variations over time is viewed as an essential component of reliable information on the health status of newborns and its variations over time is viewed as an essential component of reliable information on the health status of newborns.

In this paper, the modified SIFT algorithm (Simple Inverse Filtering Tracking) is used for estimating the fundamental frequency. It was demonstrated that this algorithm includes the autocorrelation properties and the ceptral pitch analysis technique[13]. In addition, the performance of this algorithm has been tested on a real newborn’s cry database[6, 14].

The main steps of this algorithm are:

● Division of the signal into overlapping frames of 20ms and N samples with 10ms recovering and multiplying each frame by Hamming window.

● Performing glottal inverse filtering to attenuate the influence of vocal tract.

● Estimation of the autocorrelation sequence.

● Performing a peak picking and decision algorithm in which the peak value is compared to voiced threshold.

● The fundamental frequency is estimated using:

T0= 1/F0 = argmaxηr_i (η), where r_i (η) is the autocorrelation sequence with i=[1,..,N].

● Smoothing of the result fundamental frequency contour using median filter.

Details about the SIFT algorithm can be found in[13, 14].

Table 4. Measured characteristics of cries

5. Results Analysis

Examples of estimated F0 for full-term, premature, healthy and sick newborns are shown in Figure 3.

Figure 3(a) shows the estimation of F0 for the cry of a healthy, full-term newborn. The frequency range is 400–500 Hz and no hyper-phonic segments, shifts or gliding are noted. Figure 3(b) shows the estimated F0 for the cry of a full-term newborn suffering from undiagnosed vena cava thrombosis during the days following birth. The measured values characterize a pathological cry. The pathological condition was confirmed six months after birth. Variation of F0 in this case is irregular. We note the presence of long hyper-phonic segments with gliding and shifts of F0 within the 700–2200 Hz range. Figure 3(c) shows the estimated F0 for the cry of a premature but otherwise apparently healthy newborn. We did not note the presence of hyper-phonic segments. F0 varies somewhat more and with less regularity than in the case of the full-term healthy infant. Figure 3(d) shows the estimated F0 for the cry of a premature newborn suffering from Gastroschisis.

As in case (b), the measured values correspond to a cry with pathological implications. The presence of several hyper-phonic segments is noted. The F0 variations are less regular than are those of the healthy premature infant cries.

The estimated percentages of hyper-phonic segments as well as the irregularity of F0 are shown in Figure 4. The recordings used for this estimation are of premature infants (p) and full-term infants (t), including healthy infants and infants suffering from pathologies listed in Table 2. Figure 4(a) indicates that the estimated characteristics allow well an initial distinction between a healthy newborn and a sick newborn, because the average percentage of the hyper-phonation and the irregularity of F0 are clearly higher for sick babies compared to healthy ones.

These results show that the cries of healthy newborns (full-term or preterm) contain around 6.5% of hyper-phonic segments and around 6.5% of irregularity of F0. On the other hand, the cries of sick newborns contain around 11% of hyper-phonic segments and around 9% of irregularity of F0.

These preliminary results led us to estimate the characteristics by category of pathologies. The estimated characteristics according to the category of pathologies are presented in Figure 4(b). We notice that the average percentage of the hyper-phonation and the irregularity of F0 are higher for all categories compared to that of healthy babies. Similarly, the highest average percentage of the hyper-phonation and the irregularity of F0 are found in the category of the newborn with neurological problems.

As shown in Figure.5, the calculation of the standard deviation for estimated characteristics by category of pathologies shows a large dispersion of measured characteristics for Heart Defect and Neurological categories in both case of average percentage of hyper-phonic segments and irregularity of F0.

The estimation of studied characteristics by pathologies and gestational age indicates a better variation of the newborns cries characteristics. This estimation is presented in Figure 4(c). It shows clearly that the average percentages of hyper-phonic segments in the cry samples are similar for both healthy premature infants and healthy full-term newborns. It also shows that the average percentages of irregularity of F0 are slightly more important for premature infants than healthy full-term newborns. This result is consistent with spectrographic studies of crying newborns[2, 8].

The results shown in Figure 4(c) also indicate that the average percentages of hyper-phonic segments and irregularity in the cry samples from infants with a pathological condition are clearly higher compared to healthy newborns. These percentages vary from one pathological condition to another and are dependant to level of prematurity. We thus are inferring that the characteristics of the cries do not vary according to the categories of diseases but the pathology itself. For example, in the case of the category Heart Defect, where the percentages of the characteristics estimated for pathology Thrombosis in the Vena Cava is much higher compared to the percentages of the characteristics for pathology Tetralogy of Fallot.

The calculation of the standard deviation for estimated characteristics by pathologies is shown in Figure 6. The results indicate a large dispersion for measured hyper-phonic segments in the case of Lingual frenum and Gastroschisis. They also indicate a best estimation of average percentage of hyper-phonic segment in the case of healthy preterm newborn, thrombosis in the vena cava, asphyxia, RSD and Tetralogy of Fallot diseases.

The large dispersion for measured irregularity of F0 is found in asphyxia, IUGR – microcephaly, Gastroschisis and, IUGR – asphyxia diseases. The best estimation for this characteristic is found in full term and preterm newborn, Lingual frenum, RSD, Tetralogy of Fallot diseases.

With this study, we should also be able to demonstrate that some pathological conditions do not manifest themselves in cries and are therefore not detectable using the characteristics studied such as Tetralogy of Fallot. More characteristics may be necessary to better distinguish and/or detect these pathologies.

Figure 3. Estimation of F0 using the modified SIFT algorithm

Figure 4. The average percentage of hyper-phonic segments and irregularity of F0 by (a): Pathologic and healthy cries, (b): Category of pathologies, (c): Pathologies and gestational age

Figure 5. The mean percentage and standard deviation of (a): hyper-phonic segments and (b): irregularity of F0 by category of pathologies

Figure 6. The mean percentage and standard deviation of (a): hyper-phonic segments and (b): irregularity of F0 by pathologies

We performed ANOVA variance analysis for healthy and pathologic cries to compare behavior of the estimated characteristics(percentage of hyper-phonic segments and irregularity of F0) with respect to gestational age (preterm, full term). We notice from these results in Table 5, significant statistical differences (F=7.27, p<0.05) between average percentage of hyper-phonic segment of healthy premature cries and full-term healthy cries. For all other cases no significant difference were found by gestational age.

Table 5. Analysis of variance (ANOVA) according to gestational age

Hence, the percentage of hyper-phonic segments is dependent on gestational age for healthy cries. For pathologic cries, this characteristic does not depend on gestational age but it’s depending on pathology itself. The percentage of irregularity of F0 does not dependent on gestational age in both cases of healthy and pathologic cries.

6. Experimental Validation

The adopted methodology to test the influence of use the studied characteristics in addition to other features in performance of PCIS is represented on the simplified diagram blocks illustrated in Figure 7. Matlab is used for the development of all blocks of the system.

In this context, we used studied characteristics (AP_hyp, APirrg) concatenated with the most used features in the most recent speaker recognition systems to characterize acoustic features of cries signals, namely MFCC (Mel Frequency Cepstral Coefficients) witch based on Fourier analysis and filter bank on a Mel scale. These characteristics once obtained are used as input of PNN classifier (Probabilistic Neural Network). PNN is generally used for classification problems in the medical domain[15, 16].

The use of the PNN classifier is motivated by its speed and simplicity of the training process[17]. It is obtained using function newpnn () in Matlab.

6.1. Features Extraction

MFCC coefficients are extracted using 20ms interlaced frame with 10ms recovering, providing 99 windows for one second cry samples. We extracted 12 MFCC parameters for each 20ms of a cry signal. We obtained an MFCC matrix of 12 lines ×99 columns for each sample of 1second. Thereafter we calculate the average percentages of hyper-phonic segments and irregularity for each sample of 1second according to the methodology explained in section 4. The two sets of characteristics once obtained are used as an input of the PNN classifier.

The structure of this input consists of converting each matrix of MFCC coefficients specific to a cry sample into only one vector of 12* 99 = 1188 dimension for one second sample. Thereafter we concatenate each vector with his corresponding percentages of hyper-phonic segments and irregularity of F0.

Figure 7. Experimental validation

6.2. Experimental Results

To evaluate the efficiency of proposed PCIS using studied characteristics, two experiments were performed in case of full term and preterm newborns.

The first experiment consists of testing the proposed PCIS to separate pathologic cries from healthy ones, with or without studied characteristics (AP_hyp, AP_irrg). The samples of cries are distributed in two classes, healthy cries and pathological cries, 120 samples for each class.

In the case of pathologic cries of full term newborns, we used follow pathologic cries: (20 samples of asphyxia, 20 samples of lingual frenum, 20 samples of meningitis and 60 samples of healthy cries) for training and (20 samples of samples peritonitis, 40 samples of thrombosis in the vena cava and 60 samples of healthy cries) for tests.

In the case of pathologic preterm newborns cries, we used follow pathologic cries: (20 samples of IUGR – asphyxia, 20 samples of gastroschisis, 20 samples of Tetralogy of Fallot and 60 samples of healthy cries) for training and (40 samples of RDS, 20 samples of IUGR- microcephaly and 60 samples of healthy cries) for tests.

The second experiment consists of testing the proposed PCIS to separate cries by pathologies from healthy ones with or without studied characteristics (AP_hyp, AP_irrg). The samples of cries are distributed in five classes, 70 samples for each class. In case of full term newborn: healthy cries, asphyxia cries, lingual frenum cries, meningitis cries, thrombosis in in the vena cava cries are used. In the case of preterm newborns cries: healthy cries, IUGR–asphyxia cries, IUGR-microcephaly cries, gastroschisis cries, RDS cries are used. For the test of proposed PCIS, 1/5 of each subset is used for test and 4/5 for training with cross validation method.

It is important to note that the samples of healthy cries used for training are different from once used for testing for both experiment. The results of the correct identification rate (Overall accuracy) for all experiments are shown in Table 6 and Table 7. The results obtained from the experimental PCIS using studied characteristics concatenating to MFCC coefficients as input of PNN classifier, indicate that the use of AP_hyp and AP_irrg improve performance of a PCIS particularly for identification system by pathologies in second experiment.

Table 6. Results of first experiment

Table 7. Results of second experiment

7. Conclusions

The acoustic characteristics of infant cries reflect CNS integrity. The potential for their use in the early diagnosis of various pathologies in newborns remains largely undeveloped. Our aim in this study was to analyse and characterize the healthy and pathologic cries of newborns. This characterization is important for the development of a robust system of early diagnosis for the identification of selected pathological conditions.

In this paper, we have presented the three principal steps of our research project. We have specified the methodology followed for characterizing the cry. We have also identified and quantified the acoustic characteristics that appear the most apt for betraying the pathological conditions such as percentage of hyper-phonic modes (AP_hyp) and irregularity of F0 (AP_irrg). And finally we have tested the performance of proposed PCIS using MFCC coefficients and studied characteristics (AP_hyp, AP_irrg).

The results obtained are very encouraging and even exceed our expectations. They showed that these characteristics are depending on pathology itself not on gestational age. In fact, the use of AP_hyp, AP_irrg improve performance of the proposed PCIS to separate cries by pathologies from healthy ones. We conclude that the studied characteristics are very effective for the development of a pathological cry detection system.

We also expect to improve the results by using other classifier and other acoustic characteristics in addition to AP_hyp, AP_irrg .

For further characterization of the pathological cries studied, we shall proceed with measurement and quantification of other relevant characteristics. In order to achieve a reliable evaluation of characterization performances, we are still recording a larger database with a greater variety of pathologies and more subjects for each pathology.

ACKNOWLEDGEMENTS

We would like to thank Dr. Barrington and members of neonatology group of Saint-Justine Hospital in Montreal (QC) for their dedication of the collection of the Infant's cry database. This research work has been funded by a grant from the Bill & Melinda Gates Foundation through the Grand Challenges Explorations Initiative.”