1. Introduction

Least squares regression is sensitive to outlier points. It has dominated the statistical literature for a long time. This dominance and popularity of the least squares regression can be ascribed, at least partially to the fact that the theory is simple, well developed and documented. The computer packages are also easily available.

The least squares regression is optimal and in the maximum likelihood estimators of the unknown parameters of the model if the errors are independent and follow a normal distribution with mean zero and a common (though unknown) variance .

The least squares regression is very far from the optimal in many non-Gaussian situations, especially when the errors follow distributions with longer tails. For the regression problems Huber (1973) [7] stated “just a single grossly outlying observation may spoil the least squares estimate, and, moreover, outliers are much harder to spot in the regression than in the simple location case”. The outliers occurring with extreme values of the regressor variables can be especially disruptive. Andrews (1974) [3] noted that even when the errors follow a normal distribution, alternatives to least squares may be required; especially if the form of the model is not exactly known. Further, least squares are not very satisfactory if the quadraticloss function is not a satisfactory measure of the loss. Loss denotes the seriousness of the nonzero prediction error to the investigator, where prediction error is the difference between the predicted and the observed value of the response variable. Meyer & Glauber (1964) [9] stated that for at least certain economic problems absolute error may be amore satisfactory measure of loss than the squared error. The least absolute deviation errors regression (or for brevity, absolute errors regression) overcomes the aforementioned drawbacks of the least squares regression and provides an attractive alternative. It is less sensitive than least squares regression to the extreme errors and assumes absolute error loss function. Because of its resistance to outliers, it provides a better starting point than the least squares regression for certain robust regression procedures. Unlike, other robust regression procedures, it does not require (a rejection parameter). It may be noted that the absolute errors estimates are maximum likelihood and hence asymptotically efficient when the errors follow the Laplace distribution.

Many quantitative models are utilized to diagnose and evaluate the response to studies of their regression analysis is an important statistical tool that is routinely applied in most applied sciences. To ease the model formulation and computation, some desired assumptions such as normality of the response variable are made on the regression structure. Out of many possible regression techniques for fitting the model, the ordinary least squares (OLS) method has been traditionally adopted due to the ease of computation. However, there is presently a widespread awareness of the dangers posed by the occurrence of outliers in the OLS estimates (Rousseuw and Leroy, 2003) [11]. The robustness method is considered as an alternative to a least squares method, especially if the regression model does not meet the fundamental assumptions or there are violations of it in the least square method. The estimation of significance becomes impaired; furthermore the prediction and estimation of the model may become biased.

2. Least Square or L₂-norm Method (OLS)

Utilizing the Ordinary Least Squares (OLS) method, the estimator is found byminimizing the sum of squared residuals:

This gives the OLS estimator for as:

(1)

The OLS estimate is optimal when the error distribution is assumed to be normal (Hampel 1974 [6] and Mosteller & Tukey, 1977 [10]) in the presence of influential observations, robust regression is a suitable alternative to the OLS , Robust procedures have been the focus of many studies recently, all of which triggered by the ideas of Hampel 1974 [6] .

2.1. Mean Square Error for Model

(2)

2.2. Mean Square Error for Estimator

(3)

2.3. Mean Absolute Error

(4)

3. Least Absolute Deviation (or L₁-norm) Method (LAD)

This estimator obtains a higher efficiency than OLS through minimizing the sum of the absolute errors:

Once LAD estimation is justified and its edge over the OLS estimation (in an appropriate condition) is established, an efficient algorithm to obtain LAD estimates has a practical significance. A progress in this direction was made by Abdelmalek (1971, 1974) [1, 2], Fair (1974) [5], Schlossmacher (1973) [13] and Spyropoulos, Kiountouzis & Young (1973) [14], They also proposed an improved algorithm for L1 estimation that is very similar to iterative weighted least squares.

Even though calculus cannot be used to obtain an explicit formula for the solution to the L1-regression problem, Robert (2001) [12], it can be used to obtain an iterative procedure when properly initialized, converges to the solution of the L1-regression problem. The resulting iterative process is called iteratively Reweighted least squares. In this section, we briefly discuss this method. We start by considering the objective function for L1-regression, Robert (2001) [12]:

(5)

(6)

Differentiating this objective function is a problem, since it involves absolute values However, the absolute value function:

is differentiable everywhere except at one point: z = 0. Furthermore, we can use the following simple formula for the derivative, where it exists

Using this formula to differentiate f with respect to each variable, and setting the derivatives to zero, we get the following equations for critical points

(7)

Where r =1, 2, …., m

Can rewrite (7) as:

(8)

Let denote the diagonal matrix, DasGupta & Mishra (2004) [4], where:

We can write these equations in matrix notation as follows:

This equation can’t be solved for x as we were able to do in L2-regression because of the dependence of the diagonal matrix on But let us rearrange this system ofequations by multiplying both sides by of:

(9)

This formula suggests an iterative scheme that hopefully converges to a solution. Indeed, we start by initializing arbitrarily and then use the above formula to successively compute new approximations. If we let denote the approximation at the k^th iteration, then the update formula can be expressed as:

(10)

Assuming only that the matrix inverse exists at every iteration, one can show that this iteration scheme converges to a solution to the L1-regression problem. LAD regression estimates are obtainable from the function L1 fit in the computer language(S-PLUS) and from the robust regression package ROBSYS, Levent, Soner, Gokhan & Hulya (2006) [8].

3.1. Mean Square Error for Model

(11)

3.2. Mean Square Error for Estimator

(12)

3.3. Mean Absolute Error

(13)

4. The Simulation Study

The following model was used

(14)

Now, was obtained from a set numbers (1, 1.1, 1.2, 1.3 ...), where, distribution of errors (normal, Contaminant, Laplace, Cauchy). And size of random: n=20, n=50, n=100 and n=200.

5. Concluding Remarks

Table (1). OLS& IRWLS estimators

Through the above table:

1. It has been shown that the LAD (L₁-norm) is more efficient in using IRWLS approach to estimate the parameters in all cases of distributions of error for the model, especially in the case of following normal distribution in the LS method in all sample sizes, and this was confirmed by all standards of comparison used in the research.

2. The method of LAD style is very suitable and efficient for estimating the parameters and regression analysis, for using the absolute deviations of error in the estimate.