1. Introduction

Ann is a simplified mathematical model of the human brain. It can be implemented by both electric elements and computer software. It is a parallel distributed processor with large numbers of connections, it is an information processing system that has certain performance characters in common with biological neural networks. Ann has been developed as generalizations of mathematical models of human cognition or neural biology, based on the assumptions that:

1- Information processing occurs at many simple elements called neurons that are fundamental to the operation of Ann's.

2- Signals are passed between neurons over connection links.

3- Each connection link has an associated weight which, in a typical neural net, multiplies the signal transmitted.

4- Each neuron applies an action function (usually nonlinear) to its net input (sum of weighted input signals) to determine its output signal [16].

The units in a network are organized into a given topology by a set of connections, or weights.

Ann is characterized by [31]:

1- Architecture: its pattern of connections between the neurons.

2- Training Algorithm: its method of determining the weights on the connections.

3- Activation function.

Ann are often classified as single layer or multilayer. In determining the number of layers, the input units are not counted as a layer, because they perform no computation. Equivalently, the number of layers in the net can be defined to be the number of layers of weighted interconnects links between the slabs of neurons [47].

1.1. Multilayer Feed Forward Architecture [23]

In a layered neural network the neurons are organized in the form of layers. We have at least two layers: an input and an output layer. The layers between the input and the output layer (if any) are called hidden layers, whose computation nodes are correspondingly called hidden neurons or hidden units. Extra hidden neurons raise the network’s ability to extract higher-order statistics from (input) data.

The Ann is said to be fully connected in the sense that every node in each layer of the network is connected to every other node in the adjacent forward layer; otherwise the network is called partially connected. Each layer consists of a certain number of neurons; each neuron is connected to other neurons of the previous layer through adaptable synaptic weights w and biases b.

1.2. Literature Review

Pan and Duraisamy in 2018 [34] studied the use of feedforward neural networks (FNN) to develop models of non-linear dynamical systems from data. Emphasis is placed on predictions at long times, with limited data availability. Inspired by global stability analysis, and the observation of strong correlation between the local error and the maximal singular value of the Jacobian of the ANN, they introduced Jacobian regularization in the loss function. This regularization suppresses the sensitivity of the prediction to the local error and is shown to improve accuracy and robustness. Comparison between the proposed approach and sparse polynomial regression is presented in numerical examples ranging from simple ODE systems to nonlinear PDE systems including vortex shedding behind a cylinder, and instability-driven buoyant mixing ow. Furthermore, limitations of feedforward neural networks are highlighted, especially when the training data does not include a low dimensional attractor. The need to model dynamical behavior from data is pervasive across science and engineering. Applications are found in diverse fields such as in control systems [44], time series modeling [40], and describing the evolution of coherent structures [13]. While data-driven modeling of dynamical systems can be broadly classified as a special case of system identification [24], it is important to note certain distinguishing qualities: the learning process may be performed off-line, physical systems may involve very high dimensions, and the goal may involve the prediction of long-time behavior from limited training data. Artificial neural networks (ANN) have attracted considerable attention in recent years in domains such as image recognition in computer vision [19, 38] and in control applications [13]. The success of ANNs arises from their ability to effectively learn low-dimensional representations from complex data and in building relationships between features and outputs. Neural networks with a single hidden layer and nonlinear activation function are guaranteed to be able to predict any Borel measurable function to any degree of accuracy on a compact domain [18]. The idea of leveraging neural networks to model dynamical systems has been explored since the 1990s. ANNs are prevalent in the system identification and time series modeling community [21, 29, 30, 36], where the mapping between inputs and outputs is of prime interest. Billings et al. [6] explored connections between neural networks and the nonlinear autoregressive moving average model (NARMAX) with exogenous inputs. It was shown that neural networks with one hidden layer and sigmoid activation function represent an infinite series consisting of polynomials of the input and state units. Elanayar and Shin [14] proposed the approximation of nonlinear stochastic dynamical systems using radial basis feedforward neural networks. Early work using neural networks to forecast multivariate time series of commodity prices [10] demonstrated its ability to model stochastic systems without knowledge of the underlying governing equations. Tsung and Cottrell [45] proposed learning the dynamics in phase space using a feedforward neural network with time-delayed coordinates. Paez and Urbina [32, 33, 46] modeled a nonlinear hardening oscillator using a neural network-based model combined with dimension reduction using canonical variate analysis (CVA). Smaoui [41, 42, 43] pioneered the use of neural networks to predict fluid dynamic systems such as the unstable manifold model for bursting behavior in the 2-D Navier-Stokes and the Kuramoto-Sivashinsky equations. The dimensionality of the original PDE system is reduced by considering a small number of proper orthogonal decomposition (POD) coefficients [5]. Interestingly, similar ideas of using principal component analysis for dimension reduction can be traced back to work in cognitive science by Elman [15]. Elman also showed that knowledge of the intrinsic dimensions of the system can be very helpful in determining the structure of the neural network. However, in the majority of the results [41, 42, 43], the neural network model is only evaluated a few time steps from the training set, which might not be a stringent performance test if longer time predictions are of interest. ANNs have also been applied to chaotic nonlinear systems that are challenging from a data-driven modeling perspective, especially if long time predictions are desired. Instead of minimizing the pointwise prediction error, Bakker et al. [4] satisfied the Diks’criterion in learning the chaotic attractor. Later, Lin et al. [22] demonstrated that even the simplest feedforward neural network for nonlinear chaotic hydrodynamics can show consistency in the time-averaged characteristics, power spectra, and Lyapunov exponent between the measurements and the model. A major difficulty in modeling dynamical systems is the issue of memory. It is known that even for a Markovian system, the corresponding reduced-dimensional system could be non-Markovian [11, 35]. In general, there are two main ways of introducing memory effects in neural networks. First, a simple workaround for feedforward neural networks (FNN) is to introduce time delayed states in the inputs [12]. However, the drawback is that this could potentially lead to an unnecessarily large number of parameters [20]. To mitigate this, Bakker [4] considered following Broomhead and King [7] in reducing the dimension of the delay vector using weighted principal component analysis (PCA). The second approach uses output or hidden units as additional feedback. As an example, Elman’s network [20] is a recurrent neural network (RNN) that incorporates memory in a dynamic fashion. Miyoshi et al. [26] demonstrated that recurrent RBF networks have the ability to reconstruct simple chaotic dynamics. Sato and Nagaya [39] showed that evolutionary algorithms can be used to train recurrent neural networks to capture the Lorenz system. Bailer-Jones et al. [3] used a standard RNN to predict the time derivative in discrete or continuous form for simple dynamical systems; this can be considered an RNN extension to Tsung’s phase space learning [45]. Wang et al. [48] proposed a framework combining POD for dimension reduction and long-short-term memory (LSTM) recurrent neural networks and applied it to a fluid dynamic system.

1.3. Fast Fourier Transform

The first to propose the techniques that we now call the fast Fourier transform (FFT) for calculating the coefficients in a trigonometric expansion of an asteroid’s orbit in 1805 [9]. However, Fast Fourier transform is an algorithm that calculates the value of the discret Fourier transform in faster. The speed this algorithm is due to the fact that it does not calculate the parts that are equal to zero. The algorithm is discovered by James W. Cooley and John W. Tukey who published the algorithm in 1965 [12].

As know today

(1)

The coefficients can be datermain as follows:

(2)

(3)

(4)

The discrete Fourier transform (DFT) is one of the most powerful tools in Digital signal processing. The DFT enables us to conveniently analyze and Design systems in frequency domain [1] and the formal as:

(5)

2. Noise Asymmetric Map Solution

In this section we will explain how this approach can be used to find the approximate solution of the Asymmetric map.

NA(x) is the solution to be computed. Let y_t(x, p) denotes a trial solution with adjustable parameters p.

In the proposed approach, the trial solution y_t employs a FFNN and the parameters p corresponding to the weights and biases of the neural architecture. We choose a form for the trial function y_t(x) such that y_t(x,p) = N(x, p) where N(x, p) is a single-output FFNN with parameters (weights) p and n input units fed with the input vector x.

2.1. Computation of the Gradient

The error corresponding to each input vector x_i is the value E (x_i) which has to force near zero. Computation of this error value involves not only the FFNN output but also the derivatives of the output with respect to any of its inputs. Therefore, for computation the gradient of the error with respect to the network weights, consider a multilayer FFNN with n input units (where n is the dimensions of the domain), two hidden layer with H sigmoid units, q hidden layer and a linear output unit.

For a given input vector x ( x₁, x₂, …, x_n ) the output of the FFNN is:

(6)

w_ij denotes the weight connecting the input unit j to the hidden unit i

denotes the weight connecting the hidden unit i to the hidden unit k

v_k denotes the weight connecting the hidden unit k to the out put unit,

b_i denotes the bias of hidden unit i,

b_ik denotes the bias of hidden unit i to the hidden unit k, and

σ is the transfer function

The gradient of suggest FFNN, with respect to the coefficients of the FFNN can be computed as:

(7)

(8)

(9)

(10)

(11)

The derivative of the error performance with respect to the FFNN coefficients can be defined and then it is easy to find minimization solution.

3. Suggested Networks

It is well known that a multilayer FFNN consist one hidden layer can approximate any function to any accuracy [28], but dynamical maps they have more completed behavior than other functions, thus, we suggest FFNN contains two hidden layer, one input and one output to estimate a solution for dynamic maps.

The suggested network divided the inputs in to two parts 60% for training and 40% for testing. The error quantity to be minimized is given by:

(12)

Where x_i ∈ [0, 1]. It is easy to evaluate the gradient of the performance with respect to the coefficient. Using (7) – (11). The training progress algorithm of FFNN with supervises training and BFGS algorithm is given as follows. Assume that there are one nodes in the First layer (input layer), the second layer (hidden layer) consist with ten nodes, the third layer (hidden layer) consist with five nodes and the fourth layer (output layer) consist with one node.

Following the steps of the technique as an algorithm,

Step 0: input and target.

Insert the input (x: x₁, x₂, x₃,…., x_n) and the target.

Step 1: allocating inputs.

Each input goes to each neurons with a hidden layer.

Step 2: initialization weights.

We initial weights and bias from the uniform distribution respectively for all connection in the neural network.

Figure 1. Flowchart for training algorithm with BFGS

Step 3: select the following.

- parameter epoch

- parameter goal

- performance function (MSE)

Step 4: calculations in each node in the first hidden layer.

In each node in hidden layer, computing the sum of the product of weights and inputs and adding the result to the bias.

Step 5: compute the output of each node for the first hidden layer.

Take the active function for sum value in step4, then its output is sent to the second hidden layer as input.

Step 6: calculations in each node in the second layer.

In each node in second hidden layer, computing the sum of the product of weights and inputs and adding the result to the bias.

Step 7: compute the output of each node for the second hidden layer.

Take the active function for sum value in step6, then its output is sent to the output layer as input.

Step 8: calculations in output layer.

There is only one neuron (node) in the output layer. The node sum is the product of weights by inputs.

Step 9: compute the output of node in output layer

The value of active function for node output is also considered as the output of overall network.

Step 10: compute the mean square error (MSE).

The mean square error is computed as follows

the MSE is a measure of performance.

Step 11: The checking.

When such that is small value close to zero, then stop the training and the bias and weights are sent. Otherwise training process goes to the next step.

Step 12: when select the training rule, the low for update weights and bias between the hidden layer and the output layer are calculated

Step 13: the update weights and bias in output layer.

At end for each iteration, the weights and bias are updating as follows:

When (new) means the current iteration and (old) means the previous iteration,

represent the gradient for weights and bias, is the parameter selected to minimize the performance function along the search direction, represent the invers hessian matrix, v is the weight in the output layer and b is the bias.

Step 14: the update of weights and bias in the first hidden layer.

Each hidden node in the first hidden layer updates the weights and bias as follow:

Where w is the weight of hidden layer and b is the bias.

Step 15: the update of weights and bias in the second hidden layer as follow.

Where s is the weight of hidden layer and b is the bias.

Step 16: return to step2 for next iteration.

4. Asymmetric Logistic Map (ALM) [2]

The dynamical system for asymmetric logistic map can be defined as follows

(13)

Where and

4.1. Description of Training Process for Noisy Asymmetric Logistic Map (NALM)

We use the suggested network with tansig, logsig and FFT transfer functions to train the data for normal NALM. It is suitable to choose the maximum number of epochs to reach to the high performance. The variances used in this case are (0.05, 0.5, 15).

In this case, we will train with a=0.5with 4 cases values of bifurcation parameter where the first and second parts of ALM are as follow,

i- The two parts are noisy deterministic.

ii- The first part is noisy deterministic and the second part is noisy chaotic.

iii- The first part is noisy chaotic and the second part is noisy deterministic.

iv- The two parts are noisy chaotic.

It is worth to mention that the run size is k=1000.

Table from (1) to (9) contains the results.

Table (1). Time and MSE of approximate solution after many training trials with normal noise by using logsig transfer function when the variance equal to 0.05

Table (2). Time and MSE of approximate solution after many training trials with normal noise by using tansig transfer function when the variance equal to 0.05

Table (3). Time and MSE of approximate solution after many training trials with normal noise by using FFT transfer function when the variance equal to 0.05

Table (4). Time and MSE of approximate solution after many training trials with normal noise by using logsig transfer function when the variance equal to 0.5

Table (5). Time and MSE of approximate solution after many training trials with normal noise by using tansig transfer function when the variance equal to 0.5

Table (6). Time and MSE of approximate solution after many training trials with normal noise by using FFT transfer function when the variance equal to 0.5

Table (7). Time and MSE of approximate solution after many training trials with normal noise by using logsig transfer function when the variance equal to 15

Table (8). Time and MSE of approximate solution after many training trials with normal noise by using tansig transfer function when the variance equal to 15

Table (9). Time and MSE of approximate solution after many training trials with normal noise by using FFT transfer function when the variance equal to 15

4.2. Normal Distribution [37]

Let x be a normally distributed random variable with mean and variance , where with probability density function and cumulative distribution function are respectively,

(14)

(15)

One can generate values from Normal random variable as follows,

(16)

Where q is represent the number point and v is represent the variance.

4.3. Results Discussion

The experimental results in tables from (1) to (9) show that FFT transfer function has the best MSE results for all (i) to (iv) cases, with normal noisy.

The logsig and tansig transfer functions have good performance only in case (i) when two parts of NALM are deterministic with additive noise, but the performance of them is much less than that of FFT transfer function.

5. Tent Map

One dimensional tent map is expressed through the following equation:

(17)

Where x0 is the initial value, xn ∈ [0, 1] and r ∈ [0,2], [17]. A Tent map is an iterated function of a dynamical system that exhibits chaotic behaviours (orbits) and is governed by equation 3. It has a similar shape to the logistic map shape with a corner. It is one-dimensional map generating periodic chaotic behaviour similar to a logistic map. [25]

Figure 2. Bifurcation diagram for the tent map [27]

Figure 3. When the deferent value [27]

6. Asymmetric Logistic-tent Map (ALTM) [2]

The dynamical system for the asymmetric logistic-tent map can be defined as follows:

(18)

Where and

6.1. Discretion of Training Process for Noisy Asymmetric Logistic – tent Map (NALTM)

We use the suggested network with tansig, logsig and FFT transfer functions to train the data for normal NALTM., it is suitable to choose the maximum number of epochs to reach to the high performance, the variances used in this case are (0.05, 0.5, 15). It is worth to mention that the run size is k=1000.

Tables from (10) to (18) contain the results.

Table (10). Time and MSE of approximate solution after many training trials with normal noise by using logsig transfer function when the variance equal to 0.05

Table (11). Time and MSE of approximate solution after many training trials with normal noise by using tansig transfer function when the variance equal to 0.05

Table (12). Time and MSE of approximate solution after many training trials with normal noise by using FFT transfer function when the variance equal to 0.05

Table (13). Time and MSE of approximate solution after many training trials with normal noise by using logsig transfer function when the variance equal to 0.5

Table (14). Time and MSE of approximate solution after many training trials with normal noise by using tansig transfer function when the variance equal to 0.5

Table (15). Time and MSE of approximate solution after many training trials with normal noise by using FFT transfer function when the variance equal to 0.5

Table (16). Time and MSE of approximate solution after many training trials with normal noise by using logsig transfer function when the variance equal to 15

Table (17). Time and MSE of approximate solution after many training trials with normal noise by using tansig transfer function when the variance equal to 15

Table (18). Time and MSE of approximate solution after many training trials with normal noise by using FFT transfer function when the variance equal to 15

6.2. Results Discussion

The performance results proved that the use of FFT as transfer function is much superior compared with using tansig and logsig as transfer functions.

We see that the results of logsig when the variance (0.05 and 0.5) given in tables (10 and 13) and results of tansig transfer function in tables (11 and 14) are near from each other in performance. When the variance increase to value 15 the performance for this transfer functions is very bad.

As effective conclusion, we can say that the performance of proposed algorithm is best from others in all cases from Both sides, speed and accuracy.

7. Asymmetric Tent-logistic Map (ATLM)

The dynamical system for the asymmetric tent- logistic map can be defined as follows:

(19)

Where and

7.1. Description of Training Process for Noisy Asymmetric Tent-logistic Map (NATLM)

We use the suggested network with tansig, logsig and FFT transfer functions to train the data for normal NATLM., it is suitable to choose the maximum number of epochs to reach to the high performance, the variances used in this case are (0.05, 0.5, 15). It is worth to mention that the run size is k=1000.

Tables from (19) to (27) contain the results.

Table (19). Time and MSE of approximate solution after many training trials with normal noise by using logsig transfer function when the variance equal to 0.05

Table (20). Time and MSE of approximate solution after many training trials with normal noise by using tansig transfer function when the variance equal to 0.05

Table (21). Time and MSE of approximate solution after many training trials with normal noise by using FFT transfer function when the variance equal to 0.05

Table (22). Time and MSE of approximate solution after many training trials with normal noise by using logsig transfer function when the variance equal to 0.5

Table (23). Time and MSE of approximate solution after many training trials with normal noise by using tansig transfer function when the variance equal to 0.5

Table (24). Time and MSE of approximate solution after many training trials with normal noise by using FFT transfer function when the variance equal to 0.5

Table (25). Time and MSE of approximate solution after many training trials with normal noise by using logsig transfer function when the variance equal to 15

Table (26). Time and MSE of approximate solution after many training trials with normal noise by using tansig transfer function when the variance equal to 15

Table (27). Time and MSE of approximate solution after many training trials with normal noise by using FFT transfer function when the variance equal to 15

7.2. Results Discussion

From the tables of results, it is clear when the noise is normal with all considered variances values, that the FFT transfer function is the best among all others transfer functions.

We see that the results of logsig when the variance (0.05) given in table (19) are better than of results of tansig transfer function in table (20), when the variance (0.5) we see from the table (23) the results tansig are better than the results logsig in table (22) and When the variance increase to value 15 the performance for this transfer functions is very bad but the results.

8. Summary

From the above cases, it is clear that suggested FFT as transfer function in artificial neural network gives the good results and good accuracy in noisy deterministic and noisy chaotic comparison with usual transfer functions in artificial neural network (see Tables (1) – (27)). Therefore, we can conclude that the FFNN with FFT which we proposed can handle effectively of the noisy asymmetric dynamical maps and provide accurate approximate solution throughout the whole domain and not only at the training set.

The estimation of noisy asymmetric dynamical maps obtained by trained Ann's offer some advantages, such as:

1- Complexity of computations increases with the increase of the number of sampling points in noisy asymmetric dynamical maps (NALM, NALTM and NATLM).

2- The FFNNs with FFT transfer function provides a solution with very good performance function in compare with Logsig and Tagsig traditional transfer functions.

3- The proposed FFNNs with FFT transfer function can be applied to noisy asymmetric dynamical maps.

4- The proposed transfer function FFT gave best rustles especially when the noisy asymmetric dynamical maps is chaotic and high chaotic.

5- In general, the experimental results show that the FFNN side by side FFT transfer function which proposed can handle effectively noisy asymmetric dynamical maps and provide accurate approximate solution throughout the whole domain, because three points the first point neural network computations is parallel the second point FFT analysis of data and computations are parallel and third point FFT transfer function returns differences in data to sources original related homogeneity data and sectors of the work.

Some future works can be recommended. These works is as follows

1- Using networks with three or more hidden layers.

2- Using the architecture feedback neural network.

3- Increase the neurons in each hidden layer.

4- Use anther random variables as noise on asymmetric dynamical maps.

5- Choose initial weights to be distributed as different random variables.