1. Introduction

Conventional time series methods utilize probability theory to model uncertainty. Although future values are not known, the intervals that could include these future values can be predicted with a specified probability. In real world time series, uncertainty in future values of the series can be expressed by probability. In addition to this, there is an uncertainty in representing observation values of the time series. For instance, if time series consists of daily temperature values, using only one temperature value for each day causes an uncertainty. The reason is that temperature is continuously changing in a day. Even in a same day, temperature takes infinitely many values. When a day is represented only a temperature value, which is real, and this series is used to forecast, this uncertainty cannot be modelled and misleading results are obtained. In such cases, it would be wiser to represent the observations of time series by linguistic values or fuzzy sets. Time series whose observations are represented by linguistic values or fuzzy sets are called fuzzy time series. To analyze such time series, fuzzy time series approaches are used instead of conventional ones. In fuzzy time series methods, a probabilistic approach is not used to forecast the future values. Instead of this, future values is tried to be forecast by utilizing fuzzy logic theory.

A wide literature on fuzzy time series has been produced in recent years. Fuzzy time series was firstly introduced by Song and Chissom (1993a). Song and Chissom (1993a) is divided fuzzy time series into two classes that are time variant and time invariant. Song and Chissom (1993b) proposed an algorithm to analyze time invariant fuzzy time series. Like other fuzzy inference systems, fuzzy time series consists of three phases such as fuzzification, determination of fuzzy relations, and defuzzification. It is a well-known fact that all of these phases directly affect the forecasting performance of the method. In the literature, there are many studies in which there is a contribution to each of these phases. In the most preferred approach for fuzzification phase, the universe of discourse is defined and it is partitioned into equal or unequal intervals by a proper technique. According to these intervals, fuzzy sets are defined by determining the membership values. Then, observations are mapped into these fuzzy sets by a predefined rule.

Huarng (2001) examined the effect of length of interval on forecasting results. Huarng (2001) also suggested two methods which are based on average and distribution in order to determine the length of interval. Egrioglu et al. (2010, 2011) utilized single variable constrained optimization to determine the interval length for first and high order fuzzy time series models. Huarng and Yu (2006a) suggested using a dynamic length of interval based on a given proportion. In this approach, the universe of discourse is partitioned by changing interval lengths instead of fixed interval lengths. Some of the next studies have been inspired from the idea of using a dynamic length of interval. Yolcu et al. (2009) improved the approach proposed by Huarng and Yu (2006a) by using optimization for determining the best value of the proportion. Davari et al. (2009), Kuo et al. (2009; 2010), Park et al. (2010), and Hsu et al. (2010) determined the dynamic lengths of interval using particle swarm optimization (PSO). It was observed that using PSO method considerably increases forecasting accuracy. Chen and Chung (2006) and Lee et al. (2007; 2008) utilized genetic algorithms in the fuzzification phase. Cheng et al. (2008) employed fuzzy c-means method for fuzzification.

Another important phase that has a significant effect on forecasting performance is determination of fuzzy relations between observations. For this phase, Song and Chissom (1993b) proposed a method based on fuzzy logic relationships. They used various matrix operations to produce these relationships. Chen (1996) suggested a method based on fuzzy logic group relation tables. The method proposed by Chen (1996) is easier than Song and Chissom’s (1993b) method. In the literature, the most preferred way to define fuzzy relations is using fuzzy logic group relation tables. As an alternative to these two methods, Huarng and Yu (2006b) suggested using artificial neural networks to establish fuzzy relations. Aladag et al. (2009; 2010) and Aladag (2012) proposed high order fuzzy time series forecasting approaches and employed feed forward neural networks to define fuzzy relations in their methods. Egrioglu et al. (2009a, b, c) suggested bivariate and multivariate models and utilized feed forward neural networks. Alpaslan et al. (2011; 2012) and Yolcu et al. (2013) proposed fuzzy time series approaches using membership values and exploited feed forward neural networks for determination of fuzzy relations.

In the literature, fuzzification and determination of fuzzy relations phases have been considered as two separated process. In recent years, intelligent optimization techniques, which does not require subjective decisions, have been used for the both phases. For these two phases, using two different optimization methods produces two different errors. If one optimization method can be used instead of two ones, it is expected that total error can be decreased. This is the motivation of this study so we suggest a novel hybrid fuzzy time series forecasting approach which produces better forecast by decreasing the total error. In the proposed approach, the both fuzzification and determination of fuzzy relations phases are performed in same optimization process is proposed. In the proposed method, end points of intervals used in fuzzification phase and weights of multiplicative neuron model are determined in same optimization process. And, PSO algorithm is utilized in this process. To evaluate the performance of the proposed hybrid method, it was applied to real time series and obtained results are compared to those obtained from other fuzzy time series methods available in the literature. It was observed that the proposed approach has high performance accuracy.

The rest of the paper is structured as follows. Basic definitions of fuzzy time series are given in the succeeding section. PSO method and multiplicative neuron model are summarized in Section 3 and 4, respectively. Section 5 introduces the hybrid fuzzy time series approach proposed in this study. The results obtained from the implementation of the proposed method are presented in Section 6. Finally, the last section provides brief discussion and concludes the paper.

2. Fuzzy Time Series

The fuzzy time series was firstly introduced in Song and Chissom (1993a). The fuzzy time series, time variant and time invariant fuzzy time series definitions are given below Song and Chissom (1993a).

Definition 1 Let Y(t) (t=…, 0, 1, 2, …), a subset of real numbers, be the universe of discourse on which fuzzy sets f_j(t) are defined. If F(t) is a collection of f₁(t), f₂(t), … then F(t)is called a fuzzy time series defined on Y(t).

Definition 2 Suppose F(t) is caused by F(t – 1) only, i.e., F(t – 1) → F(t). Then this relation can be expressed as F(t) = F(t – 1) R(t, t – 1) where R(t, t – 1) is the fuzzy relationship between F(t – 1) and F(t), and F(t) = F(t – 1) R(t, t – 1) is called the first order model of F(t). " " represents max-min composition of fuzzy sets.

Definition 3 Suppose R(t, t – 1) is a first order model of F(t). If for any t, R(t, t – 1) is independent of t, i.e., for any t, R(t, t – 1) = R(t – 1, t – 2), then F(t) is called a time invariant fuzzy time series otherwise it is called a time variant fuzzy time series.

Song and Chissom (1993b) firstly introduced an algorithm based on the first order model for forecasting time invariant F(t). In Song and Chissom (1993b) the fuzzy relationship matrix R(t, t – 1) = R is obtained by many matrix operations. The fuzzy forecasts are obtained based on max-min composition as below:

(1)

The dimension of R matrix is dependent number of fuzzy sets which are partition number of universe and discourse. If we want using more fuzzy sets, we need different matrix operations for obtain R matrix.

Definition 4 Let F(t) be a time invariant fuzzy time series. If F(t) is caused by F(t – 2), F(t – 1), … , and F(t – n) then this fuzzy logical relationship is represented by

(2)

and it is called the n^th order fuzzy time series forecasting model.

3. Particle Swarm Optimization (PSO)

Particle swarm optimization, which is a population based heuristic algorithm, was firstly proposed by Kennedy and Eberhart (1995). Distinguishing feature of this heuristic algorithm is that it simultaneously examines different points in different regions of the solution space to find the global optimum solution. Local optimum traps can be avoided because of this feature.

In the literature, it was shown that using some time varying parameters can increase the convergence speed of the algorithm. Ma et al. (2006) employed time varying acceleration coefficient in standard particle swarm optimization method. In another study, Shi and Eberhart (1999) used time varying inertia weight. In the modified particle swarm optimization, this time varying constituents are used together. This is the only difference between standard and modified particle swarm optimization methods.

Algorithm 1: The modified particle swarm optimization

Step 1 Positions of each k^th (k = 1, 2, …, pn) particles’ positions are randomly determined and kept in a vector X_k given as follows:

(3)

where represents i^th position of k^th particle. pn and d represents the number of particles in a swarm and positions, respectively.

Step 2 Velocities are randomly determined and stored in a vector V_k given below.

(4)

Step 3 According to the evaluation function, pbest and gbest particles given in (5) and (6), respectively, are determined.

(5)

(6)

where pbest is a vector stores the positions corresponding to the k^th particle’s best individual performance, and gbest represents the best particle, which has the best evaluation function value, found so far.

Step 4 Let c₁ and c₂ represents cognitive and social coefficients, respectively, and w is the inertia parameter. Let (c_1i, c_1f), (c_2i, c_2f), and (w₁, w₂) be the intervals which includes possible values for c₁, c₂ and w, respectively. At each iteration, these parameters are calculated by using the formulas given in (7), (8) and (9).

(7)

(8)

(9)

where maxt and t represent maximum iteration number and current iteration number, respectively.

Step 5 Values of velocities and positions are updated by using the formulas given in (10) and (11), respectively.

(10)

(11)

where rand₁ and rand₂ are random values from the interval [0, 1].

Step 6 Steps 3 to 5 are repeated until a predetermined maximum iteration number (maxt) is reached.

4. Multiplicative Neuron Model

Input signal for a neuron of a general feed forward neural network model is calculated by using an additive aggregation function. On the other hand, input signal of a multiplicative neuron model is multiplicatively computed by using a multiplicative aggregation function. Yadav et al. (2007) firstly proposed single multiplicative neuron model. Also, he showed that using a single multiplicative neuron model to forecast time series produces satisfactory forecasting results. Zhao and Yang (2009) proposed to use cooperative particle swarm optimization instead of back propagation algorithm that was modified for multiplicative neuron model in Yadav et al. (2007). The structure of a single multiplicative neuron model with five inputs is depicted in Figure 1.

Figure 1. A single multiplicative neuron model

As seen in the figure above, this model includes only one neuron and differently from feed forward neural networks, instead of using an additive aggregation function, a multiplicative aggregation function is utilized to calculate input signal of the neuron. function is a multiplicative form of weighted inputs. The model with five inputs given in Figure 1 includes ten weights. Five of them correspond to five inputs and the others correspond to bias of these five inputs. Activation function can be in a logistic form and let a function given below is used as the activation function.

(12)

Thus, activation value (net) can be computed as follows:

(13)

Then, the output of the model is y=f(net). While the multiplicative neuron model is trained by particle swarm optimization, sum of square errors (SSE) is used as the fitness function. SSE value is computed as follow:

(14)

where d_i and y_i are i^th target value and i^th output of the model, respectively.

5. The Proposed Hybrid Approach

In existing fuzzy time approaches, fuzzification and defining fuzzy relations phases are performed separately. For these phases, two separate optimization processes are performed. This cause two errors from two optimization processes. In the proposed hybrid fuzzy time series approach, both the optimal intervals used in the partition of universe of discourse and the optimal weight values of artificial neural network model used to define fuzzy relations are determined using the same particle swarm optimization algorithm. In the proposed approach, positions of a particle in the particle swarm optimization algorithm are composed of beginning and ending points of intervals used in the partition of universe of discourse and weights of the multiplicative neuron model. The proposed hybrid forecasting approach has important advantages. These are as follows:

● The length of interval is not arbitrarily determined in the proposed hybrid method. Instead of this, interval length is systematically determined by using PSO method.

● It is very hard to use fuzzy logic group relationship tables for high order fuzzy time series model. It is easy to utilize the proposed approach since it is not necessary to use fuzzy logic group relationship tables.

● In the proposed hybrid approach, multiplicative neuron model is used to establish fuzzy relations so the proposed method has flexible modeling ability of artificial neural networks.

● The determination of the number of neurons in hidden layer is an important issue in artificial neural networks. However, the proposed hybrid method does not have this problem since multiplicative neuron model is used.

● The proposed hybrid approach is the first method in the literature that performs partition of universe of discourse and defines fuzzy relations in same optimization process.

The algorithm of the proposed hybrid approach is presented as follows:

Step 1 Lower and upper bounds (min and max) for universe of discourse (U), the order of fuzzy time series forecasting model, the parameters (w₁, w₂, c_1i, c_2i, c_1f, c_2f) of particle swarm optimization method (Aladag et al., 2012), the number of particles (pn) and the number of iterations (tmax) are determined.

Step 2 For variables which will be optimized based on the particle swarm optimization, initial positions and speeds are determined.

The universe of discourse and the partition of it according to variables which will be optimized can be given as follows:

where max and min are maximum and minimum values of time series respectively, represent particles positions in PSO, and N represents the number of intervals. For example, if N=4 and pn=4, three variables which are bound values of intervals like given above, will be optimized by PSO algorithm. These variables’ values are needed for fuzzification phase. Also, the weights of the single multiplicative neuron model constructed based on the model order used in determination of fuzzy relations will be optimized by PSO method. If a second order fuzzy time series forecasting model is employed, four weights will be optimized. In this case, positions and velocities are presented in Table 1. In this table, first three positions of a particle represent boundaries of intervals that will be used in the fuzzification phase. The last four positions represent the weights of the single multiplicative neuron model. In other words, it can be given as follows:

for first three positions,

for the last four positions.

Table 1. Particles in PSO Algorithm

Step 3 Fitness values for all particles are calculated. When fitness values are computed, steps 3.1 to 3.4 are followed.

Step 3.1 Time series is fuzzified based on the subintervals obtained from the particle.

For instance, according to the first particle shown in Table 1, partition of universe of discourse is obtained as follows.

Thus, fuzzy sets can be obtained in the following ways:

Fuzzy time series is generated by mapping each observation into a fuzzy set which has the maximum membership value in the interval including this observation.

Step 3.2 Fuzzy relations between observations are determined based on a single multiplicative neuron model. The weights of the single multiplicative neuron model are obtained from positions of the particle. For example, the weights of the single multiplicative neuron model are obtained from the last four positions of Particle 1 given in Table 1 . The inputs of the single multiplicative neuron model are index number of fuzzy sets compose of lagged fuzzy time series. As mentioned before, these lagged series are obtained based on the model order. The output of the model is index number of fuzzy set for time t which is an observations of fuzzy time series. Input value is rounded to the nearest integer since input is an index value. Thus, fuzzy forecasts are obtained.

Step 3.3 The fuzzy forecasts are defuzzified. For each fuzzy forecast, middle point of the corresponding interval which has the maximum membership value is the defuzzified forecast.

Step 3.4 Fitness value, which is mean square error (MSE) value, is calculated by using the formula given below.

(15)

Step 4 Let Pbest_k is a vector stores the positions corresponding to the k^th particle's best individual performance, and Gbest represents the best particle, which has the best evaluation function value, found so far. Pbest and Gbest are calculated based on MSE values obtained in the previous step. New positions and speeds of particles are calculated by using the formulas given in (11) and (10), respectively.

Step 5 If maximum iteration number is reached, Step 3 is repeated. Otherwise, the algorithm is continued from Step 6.

Step 6 gbest is taken as the optimal solution. gbest was explained in Section 3.

6. Application

The proposed hybrid approach is applied to four different data sets. These are Data Set l, Data Set 2, Data Set 3 obtained from index 100 for the stocks and bonds exchange market of Istanbul (IMKB). Observations of Data Set 1, 2, and 3 were recorded in October 3, 2008 - December 31, 2008, October 1, 2009 - December 23, 2009, and October 1, 2010 - December 23, 2010, respectively. When IMKB is analyzed, the best cases of all methods are tried to be determined. For doing this, interval lengths are changed between 200 and 1000 with increment 100, and number of fuzzy sets are changed between 5 and 15. Then, among for all possible cases, the best cases for methods are determined. The fourth time series Data Set 4 is Taiwan Futures Exchange-TAIFEX data. Different test sets were used in the analysis. The proposed method was applied to all data sets. And, obtained results are compared to those produced by other alternative methods available in the literature. Evaluation of all methods are performed by utilizing root mean square error (RMSE) and mean absolute percentage error (MAPE). The related formulas of these performance measures are given below. In the formulas, Actual_t and Forecast_t represent observation value and corresponding forecast value for time t, respectively.

(16)

(17)

6.1. Data Set 1

The graph of Data Set 1 is depicted in Figure 2. Two lengths for the test set are used. These lengths of the test set are 7 and 15. For the test set including the last seven observations of Data Set 1, the obtained results are presented in Table 2.

Figure 2. Graph of Data Set 1

Table 2. The Obtained Results When Length of the Test Set is 7

After practicing, the forecasts obtained from the case where the best result was obtained for the test data and the error criteria related to those forecasts are presented in Table 2. The best results given in Table 2 are obtained when

● The method proposed by Song and Chissom (1993b) is applied with 12 fuzzy sets;

● The interval length is 1200 for the method of Chen (1996);

● The interval length is 800 for the distribution based method (Huarng, 2001);

● The interval length is 200 for the average based method (Huarng, 2001);

● The ratio sample percentile is 0.5 for the ratio based method (Huarng and Yu, 2006);

● The number of fuzzy sets is 5 for the method of Cheng et al. (2008);

● The number of fuzzy sets is 11 and the number of the neurons in the hidden layer is 5 for the method of Yolcu et al. (2013);

● The proposed hybrid approach produced the best results when third order model is used and the number of intervals is 10.

When Table 2 is examined, it is seen that the proposed hybrid method is superior to the other methods in terms of RMSE criterion. In terms of MAPE measure, its performance is also very good.

In a similar way, all obtained results for 15 length of the test set are summarized in Table 3.

● The method proposed by Song and Chissom (1993b) is applied with 15 fuzzy sets;

● The interval length is 1200 for the method of Chen (1996);

● The interval length is 800 for the distribution based method (Huarng, 2001);

● The interval length is 200 for the average based method (Huarng, 2001);

● The ratio sample percentile is 0.5 for the ratio based method (Huarng and Yu, 2006);

● The number of fuzzy sets is 15 for the method of Cheng et al. (2008);

● The number of fuzzy sets is 12 and the number of the neurons in the hidden layer is 2 for the method of Yolcu et al. (2013);

● The proposed hybrid approach produced the best results when 5^th order model is used and the number of intervals is 6.

According to Table 3, the proposed hybrid method is superior to the other methods in terms of the both performance measures RMSE and MAPE.

Table 3. The Obtained Results When Length of the Test Set is 15

6.2. Data Set 2

Secondly, Data Set 2 whose graph is given in Figure 3 is analyzed. Two lengths for the test set are used. These lengths of the test set are 7 and 15. After practicing, the forecasts obtained from the case where the best result was obtained for the test data and the error criteria related to those forecasts are presented in Table 4 and 5.

Figure 3. Graph of Data Set 2

The best results given in Table 4 are obtained when

● The method proposed by Song and Chissom (1993b) is applied with 9 fuzzy sets;

● The interval length is 1300 for the method of Chen (1996);

● The interval length is 800 for the distribution based method (Huarng, 2001);

● The interval length is 200 for the average based method (Huarng, 2001);

● The ratio sample percentile is 0.5 for the ratio based method (Huarng and Yu, 2006);

● The number of fuzzy sets is 15 for the method of Cheng et al. (2008);

● The number of fuzzy sets is 13 and the number of the neurons in the hidden layer is 7 for the method of Yolcu et al. (2013);

● The proposed hybrid approach produced the best results when 5^th order model is used and the number of intervals is 11.

According to Table 4, the proposed method produces the most accurate forecasts in terms of the both performance measures RMSE and MAPE.

Table 4. The Obtained Results When Length of the Test Set Is 7

The best results given in Table 5 are obtained when

● The method proposed by Song and Chissom (1993b) is applied with 9 fuzzy sets;

● The interval length is 1500 for the method of Chen (1996);

● The interval length is 800 for the distribution based method (Huarng, 2001);

● The interval length is 200 for the average based method (Huarng, 2001);

● The ratio sample percentile is 0.5 for the ratio based method (Huarng and Yu, 2006);

● The number of fuzzy sets is 6 for the method of Cheng et al. (2008);

● The number of fuzzy sets is 7 and the number of the neurons in the hidden layer is 3 for the method of Yolcu et al. (2013);

● The proposed hybrid approach produced the best results when 5^th order model is used and the number of intervals is 11.

Table 5. The Obtained Results When Length of the Test Set is 15

When Table 5 is examined, it is seen that the proposed hybrid method is superior to the other methods in terms of the both performance measures.

Table 6. The Obtained Results When Length of the Test Set is 7

6.3. Data Set 3

Thirdly, Data Set 3 whose graph is given in Figure 4 is analyzed. Two lengths for the test set are used. These lengths of the test set are 7 and 15. After practicing, the forecasts obtained from the case where the best result was obtained for the test data and the error criteria related to those forecasts are presented in Table 6 and 7.

Figure 4. Graph of Data Set 3

Table 7. The Obtained Results When Length of the Test Set is 15

Table 8. The Obtained Results When Length of the Test Set is 7

The best results given in Table 6 are obtained when

● The method proposed by Song and Chissom (1993b) is applied with 9 fuzzy sets;

● The interval length is 1100 for the method of Chen (1996);

● The interval length is 1000 for the distribution based method (Huarng, 2001);

● The interval length is 200 for the average based method (Huarng, 2001);

● The ratio sample percentile is 0.5 for the ratio based method (Huarng and Yu, 2006);

● The number of fuzzy sets is 9 for the method of Cheng et al. (2008);

● The number of fuzzy sets is 7 and the number of the neurons in the hidden layer is 6 for the method of Yolcu et al. (2013);

● The proposed approach produced the best results when 5^th order model is used and the number of intervals is 14.

According to Table 6, the proposed hybrid approach is superior to the other methods in terms of the both performance measures RMSE and MAPE.

The best results given in Table 7 are obtained when

● The method proposed by Song and Chissom (1993b) is applied with 8 fuzzy sets;

● The interval length is 1100 for the method of Chen (1996);

● The interval length is 1000 for the distribution based method (Huarng, 2001);

● The interval length is 200 for the average based method (Huarng, 2001);

● The ratio sample percentile is 0.5 for the ratio based method (Huarng and Yu, 2006);

● The number of fuzzy sets is 10 for the method of Cheng et al. (2008);

● The number of fuzzy sets is 7 and the number of the neurons in the hidden layer is 7 for the method of Yolcu et al. (2013);

The proposed approach produced the best results when second order model is used and the number of intervals is 12.

According to Table 7, it is clearly seen that the proposed method gives the most accurate forecasts in terms of both performance measures RMSE and MAPE.

6.4. Data Set 4 (TAIFEX)

Finally, Data Set 4 whose graph is given in Figure 5 is analyzed. When this time series is analyzed, the last 16 observations are employed for the test set. The proposed hybrid method and some other methods are applied to Data Set 4. All obtained forecasting results are summarized in Table 8.

Figure 5. Graph of Data Set 4

The best case for the proposed hybrid approach was obtained when 5^th order model is used and the number of intervals is 7. When Table 8 is examined, it is obvious that the proposed fuzzy time series forecasting method is superior to the other methods in terms of the both performance measures RMSE and MAPE.

7. Conclusions and Discussion

There have been many studies about fuzzy time series in recent years. Fuzzy time series consists of three main phases such as fuzzification, determination of' fuzzy relation, and defuzzification. Especially, fuzzification and determination of fuzzy relation stages have an important effect on forecasting accuracy. Therefore, there have been many studies in which various methods including intelligent optimization techniques are employed for these two phases. Fuzzification and determination of fuzzy relation have been performed in separate optimization processes so two errors produced by these two processes arise. It is expected that this causes increase in total error of the system. In this study, to reach high forecasting accuracy, we propose a novel hybrid fuzzy time series forecasting method in which the both fuzzification and determination of fuzzy relations are performed in same optimization process. In the literature, the proposed approach is the first one includes only one optimization process for the both fuzzification and determination of fuzzy relations. Multiplicative neuron model is utilized to establish fuzzy relations in the proposed method. In the optimization process, end points of intervals used in fuzzification phase and weights of multiplicative neuron model are determined by using PSO algorithm. In other words, the both phase fuzzification and determination of fuzzy relations are performed in same optimization process. And, PSO algorithm is utilized in this optimization process.

In order to evaluate the performance of the proposed fuzzy time series forecasting approach, an experimental study is performed by using 4 real world time series. The proposed approach is applied to these series. These series are also forecasted by some other approaches available in the literature and obtained forecasting results are compared. As a result of the comparison, it is clearly observed that the proposed fuzzy time series approach produces very accurate forecasting results for real world time series. In addition to high forecasting performance, the proposed method provides important advantages. In the fuzzification phase, the length of interval is systematically determined by using PSO algorithm. It is easy to use the proposed method since it is not necessary to use fuzzy logic group relationship tables. The proposed method has flexible modeling ability of artificial neural networks since multiplicative neuron model is employed for defining fuzzy relations between observations. Besides, the proposed method does not have the problem of determination of the best neuron number in the hidden layer since multiplicative neuron model is preferred.

ACKNOWLEDGEMENTS

This work was supported by “The Scientific and Technological Research Council of Turkey (TUBITAK)”, Turkey, under project number 210T150.