1. Introduction

Forecasting is of vital importance in determining efficient strategies for the future. For this reason, a number of scientific studies that aim to forecast time series have been put forward. Traditional time series approaches with probabilistic models require some assumptions such as number of observations, normal distribution, and linearity. Thus, these approaches can lead to misleading forecasting results when these assumptions are not satisfied. Therefore, non-probabilistic approaches have been put forward as an alternative to probabilistic time series forecasting models. In recent years, fuzzy time series approach has become prominent among non-probabilistic time series forecasting methods.

Zadeh [1] formed a basis with fuzzy set theory which was put forward by own for fuzzy time series approaches which was firstly proposed by Song and Chissom [2]. Song and Chissom [2] introduced fuzzy time series in two parts as time-variant and time-invariant. In this study, it has been focused on time-invariant fuzzy time series. Fuzzy time series forecasting models consist of three stages. All these stages play an active role on the forecasting performance. These stages are; fuzzification, determination of fuzzy relations and defuzzification, respectively.

In fuzzification stage, despite the use of partition of the universe of discourse, more systematic approaches have been put forward in recent years. The first studies in this stage, equal intervals lengths were determined arbitrary in the studies of Song and Chissom [2-4] and Chen [5, 6]. Huarng [7] proposed two different approaches based on mean and distribution. These approaches have shown the role of interval lengths on the forecasting performance. Egrioglu et al. [8, 9] determined equal length intervals based on optimization, in this stage. Besides, Huarng and Yu [10] suggested an idea that the partition of universe of discourse was determined with changing of sub-intervals based on ratio. Yolcu et al. [11] proposed an approach to optimize this ratio.

Separately to determine the dynamic length of interval, Davari et al. [12], Kuo et al. [13,14], Park et al. [15], Hsu et al. [16], Fu et al. [17] and Huang et al. [18] utilized particle swarm optimization technique, Chen and Chung [19] and Lee et al. [20, 21] also utilized genetic algorithm. All of these approaches examined carefully it is clearly seen that a more systematic structure occurs gradually in fuzzification stage.

However, the membership values of fuzzy observations were determined subjectively in first studies. In order to eliminate this problem, Cheng et al. [22] and Li et al. [23] used fuzzy C-means (FCM) clustering technique; Egrioglu et al. [24] used Gustafson-Kessel fuzzy clustering technique in fuzzification stage.

The stage of determination of fuzzy relation is a stage that the inner-relation of fuzzy time series is determined in this stage. Therefore, this stage is highly important because of finding the proper model and thus this stage provides a major contribution to obtain superior forecasting performance. Song and Chissom [2-4] used fuzzy logic relation matrix in this stage. Sullivan and Woodall [25] used transition matrices based on Markov chain instead of using fuzzy logic relation matrix. Chen [5] proposed a simplified approach with the idea of matrix calculations are based on complex operations. This approach used fuzzy logic group relationships tables to determine the fuzzy relations. Huarng and Yu [26] proposed a first order fuzzy time series approach using feed forward artificial neural network (FFANN) in this stage. Besides, Aladag et al. [27] developed the method of Huarng and Yu [26] and proposed a high order fuzzy time series forecasting model. During the determination of fuzzy relations, the membership values have been ignored and index numbers of fuzzy sets have been considered in all of these approaches.

This situation may cause lack of information and it may affect the forecasting performance, negatively. In order to eliminate this problem, Yu and Huarng [28] proposed an approach considering all memberships to determine the fuzzy relations. But Yu and Huarng [28] also determined these membership values subjectively. To solve this problem Yolcu et al. [29] proposed an approach. According to this approach all memberships were determined by FCM technique instead of the study of Yu and Huarng [28]. Despite their superior properties of these two methods, they contain only first order fuzzy time series forecasting model. However, it will be more suitable using high order models instead of first order models for forecasting of fuzzy time series with a more complex structure relation. Therefore, the methods proposed by Yu and Huarng [28] and Yolcu et al. [29] can be transformed into a high order model structure. But this situation will lead to a problem that the number of input of artificial neural network will increase too much. In such a case, the training of the network takes a long time, the training of network becomes more difficult and also its superior forecasting performance may affected, negatively. In defuzzification stage, many researchers such as Chen [5], Huarng [7] and Huarng and Yu [10] used centroid method. Song and Chissom [4] used ANN in this stage and Cheng et al. [22] and Aladag et al. [30] preferred to use adaptive expectation method. Yolcu et al. [29] proposed an approach considering all membership values in this stage.

In this study, a high order fuzzy time series approach using fuzzy c-means clustering method in fuzzification stage and considering all the membership values in the stages of determination of fuzzy relations and defuzzification stage has been proposed. Thus, a more systematic approach that uses more information has been proposed. In addition, in the stage of determination of fuzzy relations the problem of the too much input number of artificial neural network with multiple inputs and multiple outputs has been solved by using operation of union. The proposed method has been applied to different time series used in the literature and the results have compared with some other fuzzy time series methods.

2. Overview of Related Techniques

2.1. The FCM Clustering Technique

FCM clustering technique has been used especially in recent years in fuzzification stage of the fuzzy time series forecasting methods. The FCM clustering technique was firstly introduced by Bezdek [31]. This clustering technique is the most widely used clustering algorithm. In this technique, fuzzy clustering is conducted by minimizing the least squared errors within groups. Let u_ij be the membership values, v_i be the center of cluster, n be the number of variables, and c be the number of clusters. Then the objective function, which is tried to be minimized in fuzzy clustering, is

(1)

where β is a constant (β > 1) and called the fuzzy index. d(x_j,v_i) is a similarity measure between an observation and the center of corresponding fuzzy cluster. The objective function J_β is minimized subject to constraints given below.

(2)

In FCM clustering method, to solve the minimization problem given above, an iterative algorithm is used. In each iteration, the values of v_i and u_ij are updated by using the formulas given in (3) and (4), respectively.

(3)

(4)

2.2. Artificial Neural Network

Artificial neural networks can be defined as the mathematical algorithm that is inspired by the biological neural networks. Artificial neural networks, defined as the mathematical algorithms, are the algorithms that can learn from the examples and that can generalize what is learnt. Network presentation is the graphical expression of mathematical algorithms. Artificial neural networks are much more different than biological ones in terms of their structure and ability [32]. Artificial neural networks compose of a mathematical model [33].

Three basic components that direct the operation of artificial neural networks are as follows;

Architecture Structure: The architecture structure of multilayer feed forward artificial neural network consists of input layer, hidden layer(s) and output layer (see Figure. 1). Each layer consists of neurons. The architecture structure is determined based on deciding the number of neuron in each layer. These neurons are linked each other by weights. There is no link among the neurons in the same layer.

Figure 1. Architecture of multilayer feed forward neural network

Learning Algorithm: Although, there are many learning algorithms used in the determination of weights in artificial neural networks, the most widely used one is Back Propagation algorithm which updates weights based on the difference between available data and the output of the network. Learning parameter which is used in back propagation algorithm and which can be taken fixedly or updated in the algorithm dynamically, plays an important role in reaching optimal results.

Activation Function: The proper selection of activation function that enables curvilinear matching between input and output units, significantly affect the performance of the network. When the activation function, generally selected as unipolar, bipolar or linear, is not linear, slope parameter should be determined. Slope parameter is another factor that plays an important role in reaching the appropriate conclusion.

2.3. Basic Definitions and Concepts Related to Fuzzy Time Series

The fuzzy time series was firstly defined by Song and Chissom [2]. The basic definitions of fuzzy time series, and time variant and time invariant fuzzy time series definitions are given below.

Definition 1 Let Y(t) (t = …, 0, 1, 2, …), a subset of real numbers, be the universe of discourse on which fuzzy sets f_j(t) are defined. If F(t) is a collection of f₁(t), f₂(t), … then F(t) is called a fuzzy time series defined on Y(t).

Definition 2 Suppose F(t) is caused by F(t – 1) only, i.e., F(t – 1) → F(t). Then this relation can be expressed as F(t) = F(t – 1) R(t, t – 1) where R(t, t – 1) is the fuzzy relationship between F(t – 1) and F(t), and F(t) = F(t – 1) R(t, t – 1) is called the first order model of F(t). " " represents max-min composition of fuzzy sets.

Definition 3 Suppose R(t, t – 1) is a first order model of F(t). If for any t, R(t, t – 1) is independent of t , i.e., for any t , R(t, t – 1) = R(t – 1, t – 2), then F(t) is called a time invariant fuzzy time series otherwise it is called a time variant fuzzy time series.

Song and Chissom [2] firstly introduced an algorithm based on the first order model for forecasting time invariant F(t). In Song and Chissom’s [3] labour, the fuzzy relationship matrix R(t, t – 1) = R is obtained by many matrix operations. The fuzzy forecasts are obtained based on max-min composition as below:

(5)

The dimension of R matrix depends on the number of fuzzy sets. The number of fuzzy sets equals to the number of intervals that compose of universe of discourse. The more fuzzy sets are used, the more matrix operations are needed for obtain R matrix. When the number of fuzzy set is high, using the method proposed by Song and Chissom [3] considerably increases the computational cost. In this situation, the method suggested by Song and Chissom [3] is given as below.

Definition 4 Let F(t) be a time invariant fuzzy time series. If F(t) is caused by F(t – 2), F(t – 1), … , and F(t – n) then this fuzzy logical relationship is represented by

(6)

and it is called the n^th order fuzzy time series forecasting model.

It is possible to use union operation to obtain fuzzy forecasts for high order model. This situation can be given by the equation

(7)

In equation (7), union operation means that F(t) is affected by F(t – 1) or F(t – 2) or … or F(t – n).

3. The Proposed Method

Each stage of fuzzy time series approach is highly effective on the forecasting performance of the method. Therefore, there are many studies on these stages in the literature. The main purpose of researchers is to make the methods more systematic and to present more suitable approaches for the fuzzy set theory. For this purpose, Cheng et al. [34] and Li et al. [23] used FCM clustering technique; Egrioglu et al. [24] used Gustafson-Kessel fuzzy clustering technique in the fuzzification stage. In these methods, the partition of universe of discourse was not used in fuzzification stage and the membership values were not determined with a systematic approach.

The stage of determination of fuzzy relation can be considered as the most important determinant of the forecasting performance of the model since the inner-relations of fuzzy time series are determined in this stage. In almost all studies available in the literature, the membership values are ignored in this stage. This situation may affect the forecasting performance negatively since ignorance of membership values cause lack of information. Yu and Huarng [28] proposed an approach that considers all membership values to determine the relations with FFANN but these membership values were determined subjectively.

In order to eliminate this problem Yolcu et al. [29] proposed a new approach. In this approach, the membership values were determined with FCM technique and all membership values were taken into consideration. Although fuzzy time series forecasting models which were proposed by

Yu and Huarng [28] and Yolcu et al. [29] have important advantages, these models were only first order forecasting models. On the other hand, it is usually necessary to use high order models since many time series encountered in real life contains complex relations. To forecast such time series, high-order fuzzy time series forecasting models should be utilized. In high order models, the increment of the input number of FFANN is a crucial problem. In the first order models, the number of FFANN inputs is number of fuzzy sets while the number of FFANN inputs is product of number of fuzzy sets and model order in the high order models. Therefore, the number of inputs can be too much in the high order models. In such a case, training of a network will take a quite long time as well as gets difficult so the performance of forecasting will be affected negatively.

By the method proposed in this study, it is aim to forecast of time series which contains high order fuzzy relations. The proposed method avoids subjective decisions by using FCM technique in fuzzification stage and the proposed method also presents an approach using FFANN in the stage of determination of fuzzy relations and takes all membership values into account in defuzzification stage.

The proposed method has some important advantages which can be summarized as follows:

• There is no require to partition of universe of discourse in fuzzification stage and memberships are determined based on a systematic method by using FCM.

• Operation of union has been applied to all fuzzy lagged variables to overcome the problem that is too much input for artificial neural network models.

• The proposed approach is in accordance with the fuzzy set theory since this approach takes all membership values into consideration in the stages of determination of fuzzy relations and defuzzification. Therefore, information loss is avoided and an increase in the explanation power of the model is provided.

• The proposed method is considerably suitable for forecasting of time series with complex relations since it is based on high order fuzzy time series forecasting model.

• As a result of the advantages of the proposed method given above, it is clearly expected that the method has high forecasting accuracy.

The algorithm of the proposed method is given below step by step.

Step 1 Fuzzify observations of time series by using FCM clustering method.

Let c be the number of fuzzy sets, such that where n is the number of observations. FCM clustering algorithm in which the number of fuzzy sets is c is applied to the time series consists of crisp values. Then, after the center of each fuzzy set is determined, the degrees for each observation, which denote a degree of belonging to a fuzzy set for that observation, are calculated with respect to the obtained center values of fuzzy sets. Finally, ordered fuzzy sets L_r (r=1,2,…,c) are obtained according to the ascending ordered centers, which are denoted by v_r (r=1,2,…,c).

Step 2 Define the fuzzy relationship with FFANN.

The number of neurons in the input and output layers of FFANN is equal to the number of fuzzy sets, c. The numbers of neurons in the hidden layer are decided via trial and error. In this process, the point to take into consideration is to avoid a possible loss in ability of generalization of the feed forward neural network. The inputs of FFANN consist of membership values and these membership values are obtained by applying operation of union to the vector of membership values of observations of each lagged time series. For example, we consider three order fuzzy time series forecasting model. Let the number of fuzzy sets be three. The membership values are given in Table 1.

Table 1. An example of memberships of observations

The fuzzy forecast for t=4 can be calculated as below:

and then;

The architecture of the network is shown in Figure 2 with 3 fuzzy sets and 2 neurons in the hidden layer. In Figure 2, denotes the membership degree of belonging to i^th fuzzy set of related observation of time series X(t). Then, the target values of FFANN are every membership degrees of belonging to c fuzzy sets of the observation of time series at t while the inputs of the networks are every membership degrees of belonging to c fuzzy sets of the observation of lagged time series at t .

Figure 2. The architecture of the feed forward network with 3 fuzzy sets

The following logistic activation function is used in all layers of FFANN, of whose architecture is explained as follows:

(8)

In the FFANN with this architectural structure optimal weights are obtained by training with learning algorithm Levenberg-Marquardt. Thus, the trained FFANN have learned the relationship among the membership degrees of the subsequent observations of time series.

For example, suppose that we consider the time series given in Table 1. When we defined the architectural structure as given in Figure 2, the input and the targets of ANN would be as Table 2.

Table 2. An example of determine fuzzy relation

Step 3 Defuzzification.

When it is desired to get the fuzzy forecast for the observation at t, firstly the membership degrees of the observation of lagged time series at associated with the centers of the fuzzy sets which are defined by the method of FCM clustering are obtained. Then, these membership degrees are given as input of FFANN and the output is generated. These outputs are actually the membership degrees for the fuzzy forecasting value of the observation at t. It should be here noted that the sum of the degrees of membership is not equal to 1, to the contrary FCM method. In defuzzification stage, the degrees of memberships of the fuzzy forecast are put in Equation (9) given below and are transformed into weights. And finally, the defuzzified forecast of the observation at t is calculated as in Equation (10).

(9)

(10)

, i=1,2,…,c denotes the membership degrees for the fuzzy forecasting value of the observation at t obtained from the output of FFANN, are weights used for defuzzifying forecasts.

Referring back to Table 1, consider that we have got the output of ANN for third observation (t=4) like 0.87, 0.23, 0.02. In this case, we obtained

4. Applications

In order to evaluate the performance of the proposed method, three different data sets which were used in previous studies were analyzed with both the proposed method and some other methods in the literature. Besides, the all of the obtained results were evaluated as comparatively over the training sets and the test sets, together. In the comparison, mean square error (MSE) and root mean square error (RMSE) given in (11) are being calculated.

(11)

where and , T represent crisp time series, defuzzified forecasts, and the number of forecasts, respectively. The algorithm of the proposed method is coded in MATLAB version 7.9.

4.1. The Enrollment Data of University of Alabama

First of all we aim to evaluate the forecasting performance of the proposed method over the training data set. For this purpose, the proposed method was applied to the enrollment data of University of Alabama between the years 1971 and 1992. The enrollment data is given in Table 3.

Table 3. The Enrollment data

The center values of fuzzy sets and the membership values obtained from FCM clustering algorithm are presented in Table 4 and Table 5, respectively.

Table 4. The center values of fuzzy sets obtained from the FCM

Table 5. The membership values

The fuzzy and defuzzified forecasts of proposed method are given in Table 6. When Table 6 is examined it is clearly seen that the proposed method has superior forecasting performance.

Table 6. The forecasts produced by the proposed method

Besides, MSE values obtained from the proposed method and the various methods are presented in Table 7. When the Table 7 is examined it is clearly seen that the proposed method gives the most accurate forecasts in terms of MSE for the enrollment data. The graph of the forecasts obtained from the proposed method and the observations of enrollment data is given in Figure 3.

Figure 3. The graph of forecasts and observations

Table 7. The comparison of the obtained results

4.2. Index 100 in Stock Market of Istanbul

In order to evaluate the performance of the proposed method, the other fuzzy time series data is index 100 in stock market of Istanbul (BIST). Data set includes observations between 01.10.2010 – 23.12.2010. The graph of the data set is shown in Figure 4. The last fifteen observations of data set are used for test set.

Figure 4. The data set of BIST

The results obtained for this data set were evaluated with the results of other methods. In Chen [5], interval lengths were taken between 200 and 1000 with increment 100. The interval length in the methods given in Huarng [7, 36] was determined when the method was progressing. The number of fuzzy sets was experienced between 5 and 15 in the proposed method and the paper Cheng et al. [34], Song and Chissom [3] and Yolcu et al. [29]. The forecasts of the best situations obtained from proposed method and the other methods in the literature are given in Table 8.

Table 8. The obtained results for the test data of data set 3

The best forecasts obtained from proposed method is found when the number of the fuzzy sets was 12, the number of the neurons in the hidden layer was one and the order of model was two for the proposed method.

4.3. Taiwan Futures Exchange (TAIFEX) Data

Finally, the proposed forecasting approach is applied to TAIFEX data presented in Table 9. The time series has 47 daily observations between 03.08.1998 and 30.09.1998. The last 16 observations between 10.09.1998 and 30.09.1998 are used for test set.

Table 9. TAIFEX data

The forecasting performance of the proposed method is evaluated over the test set, again. The number of the fuzzy sets was 7, the number of the neurons in the hidden layer was one and the order of model was five for the best forecasts obtained from proposed method. Then, the forecasts obtained from all methods are compared with each other and the all obtained results are summarized in Table 10.

Table 10. The obtained results for TAIFEX data

When the Table 10 examined the superior forecasting performance of the proposed method is remarkable.

5. Discussion and Conclusions

Although fuzzy time series approaches provide some advantages, they also have some disadvantages. Some of them require subjective decisions especially in fuzzification stage and ignore the membership values. These drawbacks directly affect the forecasting performance. In this study, a new high order fuzzy time series forecasting model is proposed to deal with these problems. The proposed model determines the membership values systematically by using FCM technique in the fuzzification stage. These membership values were used in the stage of determination of fuzzy relations and so better forecasting results can be obtained by using more information.

Moreover, in the proposed method, using the operation of union overcome the problem that is too much input for artificial neural network models. In the implementation, the proposed method has been applied to various time series and the forecasting performance has been evaluated for both training and test sets. The obtained results show that the proposed method has superior forecasting performance besides its all advantages.