American Journal of Signal Processing

p-ISSN: 2165-9354    e-ISSN: 2165-9362

2015;  5(2A): 1-8

doi:10.5923/s.ajsp.201501.01

An Adaptive Diffusion Algorithm Based on Augmented QLMS for Distributed Filtering of Hypercomplex Processes

Azam Khalili, Reza G. Rahmati, Amir Rastegarnia

Department of Electrical Engineering, Malayer University, Malayer, Iran

Correspondence to: Azam Khalili, Department of Electrical Engineering, Malayer University, Malayer, Iran.

Email:

Copyright © 2015 Scientific & Academic Publishing. All Rights Reserved.

Abstract

An adaptive network consists of a set of adaptive filters that cooperate with each other to solve an optimization problem using a predefined cost function. In this paper we propose an adaptive diffusion algorithm based on augmented quaternion least mean square (AQLMS) for processing of three and four-dimensional processes collaboratively. We provide mean stability of proposed algorithm and show that the distributed widely linear (equivalent to augmented term) model outperforms the distributed standard linear model and non-cooperative case. Simulations in the prediction setting on benchmark 4D and 3D signals show the results.

Keywords: Multidimensional adaptive filters, -properness, Widely linear model, Quadrivariate processes, Cooperation, Prediction, Diffusion, Adaptive network

Cite this paper: Azam Khalili, Reza G. Rahmati, Amir Rastegarnia, An Adaptive Diffusion Algorithm Based on Augmented QLMS for Distributed Filtering of Hypercomplex Processes, American Journal of Signal Processing, Vol. 5 No. 2A, 2015, pp. 1-8. doi: 10.5923/s.ajsp.201501.01.

1. Introduction

Network is the science related to issues such as assemblage, processing, cooperation, and distribution information over graphs connecting a multitude of nodes, in which collaboration among nodes can result in superior adaptation and learning performance over graphs [1]. The applications include the areas of intrusion detection, spectrum sensing, online dictionary learning, sparse data recovery, target localization, and biological networks. In the first scenario, each node acquires data and processes it independently of the other nodes. We refer to this mode of processing as noncooperative mode. In the second scenario, the nodes transmit their own data to a fusion center for processing. We refer to this mode of processing as centralized or batch mode. although centralized solution is powerful, but it suffers from a number of drawbacks such as fusion center failures and communication burden between the nodes and the fusion center.
There are several distributed strategies to solve these problems that are based on incremental (e.g., [2-6]), consensus (e.g., [7-9]), and diffusion (e.g., [10-14]) techniques. although the incremental method requires low communication traffic between nodes with each node receiving data from one preceding node and sharing data with one subsequent node, but this mode of cooperation suffers from a number of limitation such as node and link failures, whilst determining a cyclic path that visits all nodes is generally an NP-hard problem. We can motivate two other distributed techniques based on consensus and diffusion strategies that do not suffer from these limitations. Despite its high communication in analogy to incremental strategy, diffusion strategy has robustness to link and node failures.
A superior result in augmented complex statistics is the work by Neeser and Massey [15], who have devised a comprehensive description of the concept of properness (second order circularity) and rotation invariant probability distribution for processing of the complex-valued (two dimensional) signals. They showed that the covariance matrix of a complex random vector x alone is not adequate to describe second order statistical information completely for general complex-valued signals and thus the pseudocovariance or complementary covariance matrix also needs to be taken into account. These fundamentals have been successfully used to design widely linear algorithms for processing of the improper complex-valued signals optimally in adaptive signal processing [16], that is, for an optimal linear estimator, the 'augmented' input must be used, leading to the widely linear model given by with h and g denoting filter coefficients.
In order to deal with three and four dimensional signals (such as bodysensor measurements [17], color images [18], wind and renewable energy [19]), Quaternion domain is introduced and can be regarded as a non-commutative extension of complex domain [20]. It is therefore natural to extend the developments in so-called augmented complex statistics and widely linear modeling in the complex domain to the quaternion domain, in order to provide enhanced accuracy especially for processes with of different powers in data channels, such as in wind modeling.
Some of adaptive filtering algorithms introduced in quaternion domain include widely linear quaternion least mean square (WLQLMS) [21], widely linear involution quaternion LMS (WLIQLMS) [22], augmented QLMS (AQLMS) [23].
Adaptive networks based on the widely linear models have been considered in some recent works such as [24-28]. In this paper, we propose a diffusion based adaptive network for distributed adaptive filtering of three- and four-dimensional processes, such as those observed in atmospheric modeling (wind, vector fields). The proposed algorithm has the advantage that exploits spatiotemporal diversity to improve the filtering performance. Moreover, it accounts for complex nonlinear dynamics and coupling between the dimensions, which are the main problems of mentioned multidimensional processes. The proposed algorithm employs both the covariance and pseudocovariance terms within its update and can therefore cater for noncircularly symmetric quaternion data. We derive the required condition for mean stability of the proposed algorithm. We evaluate the performance of the proposed algorithm with some simulation results, where the results show the superior performance of the proposed algorithm.
The organization of this paper is as follows: in section 2, we introduce the quaternion domain. In section 3, we derive the proposed distributed algorithm. In section 4, we analyze the mean stability of proposed algorithm. In section 5, the simulations will be presented and finally in section 6, we conclude our work.
A summary of notations is given in Table 1.
Table 1. Summary of notations
     

2. Quaternion Domain

The quaternion domain is a non-commutative extension of the complex domain, and devises an adequate framework to process three and four-dimensional signals. A quaternion variable constitutes a real part and a vector part also known as a pure quaternion, consisting of three imaginary components, and can be expressed as
(1)
the orthogonal unit vectors are related as follows
(2)
given , the noncommutativity property of the quaternion product is described as follows
with
(3)
where , with symbol denoting the vector product. The quaternion conjugate is also defined as
(4)
given , It follows that
(5)

2.1. Augmented Basis Vector

The real and imaginary parts of a complex variable can be extracted as and . It is therefore natural to consider ‘augmented’ basis vector as to describe the elements of the corresponding bivariate signal in . In order to employ such manipulation in the quaternion domain, three perpendicular quaternion involutions (self-inverse mappings) is defined as follows
(6)
and therefore the four components of the quaternion variable can now be expressed as [29]
(7)
notice that the quaternion conjugate operator is also an involution, and is expressed as
(8)
based on (6)-(8), we can use a combination of the vector to define an augmented quaternion vector in quaternion domain to relate the components of quaternion signal to its real valued quadrivariate counterpart in . In this paper, we use the vector as augmented basis vector for widely linear modeling.

2.2. Second Order Circularity in and -Properness

Second order circularity is established for a quaternion variable when its probability distribution be rotation-invariant with respect to the six pairs of axes: and , with ‘1’ denoting the real axis and the corresponding imaginary axes. In other words, for a quaternion random variable to be -proper, it should satisfy the properties summarised in Table 2 [30].
Table 2. Properties of a
     -proper random variable
     
The first property P1 indicates that all of the components have same power, second property P2 states that all of the components are two-by-two uncorrelated, third property P3 implies that, in contrast to the complex domain, the pseudocovariance matrix does not vanish for -proper random variables. Finally, the fourth property P4 denotes that the covariance of a quaternion random variable is a sum of the covariances of the variable components.

3. Derivation of D-AQLMS Algorithm

In order to cater for both circular and noncircular quaternion processes, a quaternion-valued widely linear model based on the augmented basis vector as is introduced in [23] and is given by
(9)
by expressing the weight vectors and into the compact formand regarding the non-commutativity of the quaternion products, weight update of the Augmented QLMS (AQLMS) is then given by
(10)
where the error with denoting the desired response. In order to investigate this learning algorithm in a distributed manner by diffusion strategy, consider a network with N distributed nodes (as Figure 1) with each node k equipped with an AQLMS filter which has access the realizations at time n. we denote the desired signal by and input regressor vector with length M by . Each node k also can have access to data from the set of nodes defined as neighborhood so that they are directly linked to node k. by Combine-Then-Adapt (CTA) form of diffusion protocol, at first, each node k combine the weight vectors from the neighborhood including itself as follows
Figure 1. Diffusion mode of cooperation structure
(11)
where denotes the combinational weight vector in node k which is computed by non-negative real combiner coefficients satisfying following conditions
(12)
where is a matrix with entries and denotes the vector with unit entries. Then, in Adapt step, the local weight update at node k can be expressed as
(13)
where the error is computed as .

4. Mean Stability Analysis of D-AQLMS Algorithm

In order to analyze the convergence in the mean of D-AQLMS algorithm, we rewrite the combinational weight vector (11) and weight update (13), by regarding the property (5), in the global form as follows
(14)
(15)
where
where diag{.} operator denotes the diagonal matrix and the identity matrix with M representing the filter order. We also consider a teaching signal in the global form as a linear model for data as follows
(16)
where represents the optimal (and unknown) local augmented weight vector, and the quadruply white Gaussian noise with zero mean and variance , which assumed to be uncorrelated with . By substituting the signal (16) into the weight update (15), we have
(17)
by multiplying the both sides of (17) by -1 and then adding we obtain
(18)
where the weight error vectors and . by substituting the relation (14) into the (18) and then applying the statistical expectation operator we obtain
(19)
where and . We have equivalently
(20)
with and . Where denote the involutions of B and we indeed have used the following property for a given quaternion variable
(21)
Thus for the algorithm to be converged in the mean, i.e., as , or equivalently
(22)
due to the fact that (because the rows and columns add to 1), we must have
(23)
for the case when all nodes have the same stepsize , we can obtain the following lower bounds for the stepsize
(24)
where denotes the dominant eigenvalue. Finally, we can say
(25)

5. Simulations

In this section, we will compare the performance of proposed algorithm D-AQLMS with its standard linear counterpart (D-QLMS) and non-cooperative case in one step ahead prediction setting. The network include N = 6 nodes, and filter order is set to M = 5. Among the several static combination rules [10], by using network topology in Figure 2, we employ the Metropolis rule described as follows
(26)
Figure 2. Diffusion network topology used in simulations
For evaluating the algorithms performance, we use two signals including an improper three-dimensional process known as the Lorenz attractor used originally to model atmospheric turbulence, but also to model lasers, dynamos, and the motion of waterwheel [31]. Mathematically, the Lorenz signal can be expressed as a nonlinear system of coupled differential equations:
(27)
where . The other signal is generated by a linear AR(4) model as follows
(28)
where denotes the circular quadruply white Gaussian noise. Figure 3 illustrates the evaluation of mean square error (MSE) of proposed algorithm in analogy to non-cooperative case for one step ahead prediction of the mentioned signals. Observe that as expected the proposed cooperative algorithm outperforms the non-cooperative case. Figure 4 shows the evaluation of MSE for one step ahead prediction of the noncircular Lorenz signal for comparing the proposed algorithm with its standard linear counterpart. It is worth noting that due to non-circular nature of the Lorenz signal, the widely linear distributed algorithm has better performance than standard linear distributed one.
Figure 3. Comparison of MSEs of non-cooperative AQLMS and Diffusion AQLMS algorithms for prediction of (a) circular 4D AR(4) process with, and (b) noncircular 3D Lorenz signal with
Figure 4. Comparison of MSEs of D-AQLMS and D-QLMS for prediction of noncircular 3D Lorenz signal with

6. Conclusions

We have proposed a Diffusion Augmented QLMS (D-AQLMS) algorithm for distributed processing of both circular and noncircular quaternion-valued signals. The advantage of the widely linear diffusion algorithm over standard linear diffusion and noncooperative case in terms of steady state performance has been shown in simulations on synthetic signals.

References

[1]  A. H. Sayed, Adaptive networks, Proceedings of the IEEE, vol. 102, no. 4, pp. 460-497, April 2014.
[2]  C. G. Lopes and A. H. Sayed, ‘‘Incremental adaptive strategies over distributed networks,’’ IEEE Trans. Signal Process., vol. 55, no. 8, pp. 4064–4077, Aug. 2007.
[3]  L. Li, C. G. Lopes, J. Chambers, and A. H. Sayed, ‘‘Distributed estimation over an adaptive incremental network based on the affine projection algorithm,’’ IEEE Trans. Signal Process., vol. 58, no. 1, pp. 151–164, Jan. 2010.
[4]  A. Rastegarnia, M. A. Tinati, and A. Khalili, Performance analysis of quantized incremental LMS algorithm for distributed adaptive estimation, Signal Processing, vol. 90, no. 8, pp. 2621 – 2627, 2010.
[5]  M.A. Tinati, a. Rastegarnia, and a. Khalili, “An incremental least-mean square algorithm with adaptive combiner,” IET 3rd International Conference on Wireless, Mobile and Multimedia Networks (ICWMMN 2010), pp. 266–269, 2010.
[6]  F. Cattivelli and A. H. Sayed, ‘‘Analysis of spatial and incremental LMS processing for distributed estimation,’’ IEEE Trans. Signal Process., vol. 59, no. 4, pp. 1465–1480, Apr. 2011.
[7]  B. Johansson, T. Keviczky, M. Johansson, and K. Johansson, ‘‘Subgradient methods and consensus algorithms for solving convex optimization problems,’’ in Proc. IEEE Conf. Decision Control, Cancun, Mexico, Dec. 2008, pp. 4185–4190.
[8]  R. Olfati-Saber and J. Shamma, ‘‘Consensus filters for sensor networks and distributed sensor fusion,’’ in Proc. 44th IEEE Conf. Decision Control, Seville, Spain, Dec. 2005, pp. 6698–6703.
[9]  S. Sardellitti, M. Giona, and S. Barbarossa, ‘‘Fast distributed average consensus algorithms based on advection-diffusion processes,’’ IEEE Trans. Signal Process., vol. 58, no. 2, pp. 826–842, Feb. 2010.
[10]  J. Chen and A. H. Sayed, ‘‘Diffusion adaptation strategies for distributed optimization and learning over networks,’’ IEEE Trans. Signal Process., vol. 60, no. 8, pp. 4289–4305, Aug. 2012.
[11]  A. H. Sayed, S.-Y. Tu, J. Chen, X. Zhao, and Z. Towfic, ‘‘Diffusion strategies for adaptation and learning over networks,’’ IEEE Signal Process. Mag., vol. 30, no. 3, pp. 155–171, May 2013.
[12]  C. G. Lopes and A. H. Sayed, ‘‘Diffusion least-mean squares over adaptive networks: Formulation and performance analysis,’’ IEEE Trans. Signal Process., vol. 56, no. 7, pp. 3122–3136, Jul. 2008.
[13]  F. S. Cattivelli and A. H. Sayed, ‘‘Diffusion LMS strategies for distributed estimation,’’ IEEE Trans. Signal Process., vol. 58, no. 3, pp. 1035–1048, Mar. 2010.
[14]  A. H. Sayed, Adaptation, learning, and optimization over networks, Foundations and Trends in Machine Learning, vol. 7, issue 4-5, pp. 311-801, NOW Publishers, Boston-Delft, 2014.
[15]  F. Neeser, J. Massey, Proper complex random processes with applications to information theory, IEEE Transactions on Information Theory 39 (4) (1993) 1293–1302.
[16]  D.P. Mandic, V.S.L. Goh, Complex Valued Nonlinear Adaptive Filters: Noncircularity, Widely Linear and Neural Models, John Wiley and Sons Ltd, 2009.
[17]  N. Rehman and D. P. Mandic, "Empirical mode decomposition for trivariate signals," IEEE Transactions on Signal Processing, (in print), 2010.
[18]  S.-C. P Ei and C.-M. Cheng, "Color image processing by using binary quaternion-moment-preserving thresholding technique," IEEE Transactions on Image Processing, vol. 8, no. 5, pp. 614-628, 1999.
[19]  C. Cheong-Took, D. P. Mandic, and K. Aihara, "Quaternion-valued short term forecasting of wind profile," in Proc. of The World Congress on Computational Intelligence, 2010.
[20]  Cheong Took C, Mandic D. Augmented second-order statistics of quaternion signals. Signal Processing 2011; 91(2):214e24.
[21]  C. Took, D. Mandic, A quaternion widely linear adaptive filter, IEEE Transactions on Signal Processing, 58 (2010) 4427-4431.
[22]  C. Took, C. Jahanchahi, D. Mandic, A unifying framework for the analysis of quaternion valued adaptive filters, in: Signals, Systems and Computers (ASILOMAR), 2011 Conference Record of the Forty Fifth Asilomar Conference on, pp. 1771-1774.
[23]  C. Took, D. Mandic, The quaternion LMS algorithm for adaptive filtering of hypercomplex processes, IEEE Transactions on Signal Processing, 57 (2009) 1316-1327.
[24]  A. Khalili, A. Rastegarnia, and W. Bazzi, “A collaborative adaptive algorithm for the filtering of noncircular complex signals,” in Telecommunications (IST), 2014 7th International Symposium on, Sept 2014, pp. 96–99.
[25]  A. Khalili, A. Rastegarnia, W. M. Bazzi, and Z. Yang, “Derivation and analysis of incremental augmented complex LMS algorithm,” IET Signal Processing, 9 (4), 312-319.
[26]  A Khalili, A Rastegarnia, WM Bazzi, Incremental augmented affine projection algorithm for collaborative processing of complex signals, 2015 International Conference on Information and Communication Technology Research (ICTRC), 60-63, 2015.
[27]  Azam Khalili, Wael Bazzi, Amir Rastegarnia, Analysis of incremental augmented affine projection algorithm for distributed estimation of complex signals, available at http://arxiv.org/abs/1410.4477.
[28]  Y. Xia, D. Mandic, and A. Sayed, “An adaptive diffusion augmented CLMS algorithm for distributed filtering of noncircular complex signals,” Signal Processing Letters, IEEE, vol. 18, no. 11, pp. 659–662, Nov 2011.
[29]  T.A. Ell, S.J. Sangwine, Quaternion involutions and anti-involutions, Computers & Mathematics with Applications 53 (1) (2007) 137–143.
[30]  N.N. Vakhania, Random vectors with values in quaternion Hilbert spaces, Theory of Probability and its Applications 43 (1) (1999) 99–115.
[31]  S. H. Strogatz, Nonlinear dynamics and chaos: With applications to physics, biology, chemistry and engineering (studies in nonlinearity), 1st ed. Boulder, CO: Westview Press (2001).