1. Introduction

The Jackknife is a resampling technique use for estimating the bias and standard error of an estimator and provides an approximate confidence interval for the parameter of interest. The principle behind jackknife method lies in systematically recomputing the statistic leaving out one or more observation(s) at a time from the sample set thereby generating n separate samples each of size n-1 or n-d respectively. From this new set of replicates of the statistic, an estimate for bias and the variance of the statistic can be calculated[3].[5] used linear regression analysis to examine the relationship between the Fish Age (FA) as response variable, Total Length (TL) and the Otolith Length (OL) as predictor variables. They examined dependence of FA on TL and OL using bootstrap and Jackknife algorithm.[5] used bootstrap to study linear regression model of the form with error term being independent. In[5], the bootstrap errors are drawn with replacement from the set of estimated residuals. Response values are constructed as the sum of an initial estimate forand .[6] used bootstrap method to investigate the effects of sparsity of data for the binary regression model. He discovered that the bootstrap method provided robust (accurate) result for the sparse data.[4] considers regression and correlation models, and obtain the bootstrap approximation to the distribution of least squares estimates.[2] provides algorithm for data analysis and bootstrap for the construction of confidence sets and tests in classical models which involves exact or asymptotic distribution.[14] proposes a class of weighted jackknife variance estimators for the ordinary least squares estimator by deleting any fixed number of observations at a time. He observed that the weighted jackknife variance estimators are unbiased for homoscedastic errors.[11] proposes an unbiased ridge estimator using the jackknife procedure of bias reduction. They demonstrated that the jackknife estimator had smaller bias than the generalized ridge estimator (GRE).[1] proposes Modified jackknife ridge regression estimator (MJR) by combining the ideas of GRR and JRR estimators. In their article, they proposed a new estimator named generalized jackknife ridge regression estimator (GJR) by generalizing the MJR. Their result showed that the new proposed estimator (GJR) is superior in the mean square error (MSE) than the generalized ridge regression estimator in regression analysis.

2. Materials and Method

Given a model of the form

(2.1)

where the are the parameters, are the predictor variables and the error term independently identically distributed and are uncorrelated.

Equation (2.1) is assumed to be intrinsically nonlinear. Suppose we have a sample of n observations on the and , then, we can write

(2.2)

The n-equation can be written compactly in a matrix notation as

(2.3)

where

and

The error sum of squares for the nonlinear model is defined as

(2.4)

Let the least square estimates of , these estimates minimize the . The least square estimates of are obtained by differentiating (2.4) with respect to , equate to zero and solve for , this results in J normal equations:

(2.5)

In estimating the parameters of nonlinear regression model, we use the Guass-Newton method based on Taylor’s series to approximate equation (2.3). Now, considering the functionwhich is the deterministic component of

(2.6)

Let be the initial approximate value of . Adopting Taylor’s series expansion of about , we have the linear approximation

(2.7)

Substituting expressions (2.7) in (2.6) we obtain

(2.8)

Equation (2.8) may be viewed as a linear approximation in a neighborhood of the starting value

Let

Hence, equation (2.8) becomes

(2.9)

(2.10)

In a matrix form, we have

(2.11)

Compactly, equation (2.11) becomes

(2.12)

where

We obtain the Sum of squares error

(2.13)

Hence,

(2.14)

Therefore, the least square estimates of is

(2.15)

Thus, minimizes the error sum of squares,

(2.16)

Now, the estimates of parameters of non-linear regression (2.1) are

(2.17)

Iteratively, equation (2.17) reduces to

Thus

(2.18)

where are the least squares estimates of obtained at the

iterations. The iterative process continues until

where is the error tolerance [12][7]

After each iteration, is evaluated to check if a reduction in its value has actually been achieved. At the end of the iteration, we have

(2.19)

and iteration is stopped if convergence is achieved. The final estimates of the parameters at the end of the iteration are: .

2.1. Jackknife Delete-One Algorithm for the Estimation of Non-linear Regression Parameters

Let vector denotes the values associated with observation sets. The steps of the delete-one jackknife regression are as follows.

Given randomly drawn sample of size n from a population and label the elements of the vector as the vector be the response variables, is the matrix of dimension for the predictor variables, where

1. Omit first row of the vector and label remaining observation sets and as the first delete-one Jackknife sample

2. Calculate the least square estimates for nonlinear regression coefficient from the first jackknife sample; .

3. Compute using the Gauss-Newton method, the value is treated as the initial value in the first approximated linear model.

4. We return to the second step and again compute .At each iteration, new represent increments that are added to the estimates from the previous iteration according to step 3 and eventually find , which is up to .

5. Stopping Rule; this iteration process continues until, where , for the values of from the first delete-one Jackknife estimates .

6. Then, omit second row of the vector and label remaining n-1 sized observation sets and and repeat steps 2 to 5 above for the estimate of regression coefficients. Similarly, omit each one of the n observation sets and estimate the non linear regression coefficients as in the step 2 to 5 above for alternately, where is Jackknife regression coefficient vector estimated after deleting of observation set from .

7. Obtain the probability distribution of Jackknife estimates .

8. Calculate the jackknife regression coefficient estimate which is the mean of the distribution[9] as;

(2.21)

2.2. Jackknife Delete-d Algorithm for Estimation of Non-Linear Regression

Let vector denotes the values associated with observation sets. Draw a random sample of size n from the observation set (population) and label the elements of each vector as the vector be the response variables, and be the matrix of dimension n × k for the predictor variables, where j=1, 2, …, k and i = 1, 2 , …, n.

Step 1: Divide the sample into “s” independent group of size d.

Step 2: Omit first d observation set from full sample at a time and estimate the nonlinear regression parameter from (n - d) remaining observation set using the least square estimate for the nonlinear regression parameter from the first delete-d sample; .

Step 3: Compute using the Gauss-Newton method, the value is assumed as the initial value in the first approximation.

Step 4: Repeat the second step and again compute. At each iteration, new represent increments that are added to the estimates from the previous iteration according to step 3 and eventually obtain up to and consequently

Step 5: Stopping Rule; the iteration process continues until , (whereis the tolerance magnitude) and the parameters are computed from delete-d samples

Step 6: Omit second d observation set from full sample at a time and estimate the nonlinear regression parameters from remaining observation set based on the delete-d sample; and repeat step 3 to step 5 for the second delete-d sample.

Step 7: Alternately omit each d of the n observation set and estimate the parameters as where is the jackknife regression parameter vector estimated after deletion of d observation set from full sample, for k =1,2,…,s; where , and ; where d is an integer.

Step 8: Obtain the probability distribution of nonlinear regression parameter estimates .

Step 9: Calculate the nonlinear regression parameter estimate

(2.22)

(see[9])

Standard error for the nonlinear regression parameters is

Where

(2.23)

The computer program in R for Jackknife

#x is the vector of independent variable

#theta is the vector of parameters of the model

#This function calculates the matrix of partial derivatives

F=function(x,theta)

{

output=matrix(0,ncol=2,nrow=length(x))

for(i in 1:length(x))output[i,]=c(exp(theta[2]*x[i]),theta[1]*x[i]*exp(theta[2]*x[i]))

output

}

#This function calculates the regression coefficients using the Gauss-Newton Method

gaussnewton=function(y,x,initial,tol)

{

theta=initial

count=0

eps=y-(theta[1]*exp(theta[2]*x))

SS=sum(eps**2)

diff=1

while(tol{

S=SS

ff=F(x,theta)

theta=c(theta+solve(t(ff)%*%ff)%*%t(ff)%*%eps)

eps=y-(theta[1]*exp(theta[2]*x))

SS=sum(eps**2)

diff=abs(SS-S)

count=count+1

if(count==100) break

pp=c(theta,SS)

#at each iteration

}

pp

}

#This part of the code does the Delete d jacknife

jack=function(data,p,d,initial)

#p is the no of cols in the data.

#For ex., if p=4 then there is 1 dept. var and 3 indept vars

{

n=length(data[,1]) #the sample size

z=matrix(0,ncol=p,nrow=n)

u=combn(n,d)

output=matrix(0,ncol=p+1,nrow=ncol(u))

y=data[,1]

x=data[,2:p]

for (i in 1:(ncol(u))) # is the number of iterations

{

dd=c(u[,i])

yn=y[-dd]

xn=x[-dd]

logreg=gaussnewton(yn,xn,initial,tol)

coef=logreg

output[i,]=c(coef) #store the regression coefficients

}

output

}

#Then to run the code use the following

y <- c(data)

x <- c(data)

data=cbind(y,x)

initial=c(initial)

expo=jack(data,p,d,initial)

#Run the following to view the jacknife results

theta_0=mean(expo[,1])

theta_0

theta_1=mean(expo[,2])

theta_1

SSE=mean(expo[,3])

SSE

Problem:[8]

A hospital administrator wished to develop a regression model for the predicting the degree of long-term recovery after discharge from the hospital for fifteen severely injured patients. The predictor variable to be utilized is number of days of hospitalization (x), and the dependent variable is a prognosis index values. Data collected are shown in the Table below.

Table 1. Data for Severely Injured Patients Example[8] Patient 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 X 2 5 7 10 14 19 26 31 34 38 45 52 53 60 65 Y 54 50 45 37 35 25 20 16 18 13 8 11 8 4 6

Figure 1. Scatter Plot and Fitted Nonlinear Regression Function - Severely Injured Patients Example

3. Results and Discussion

Table shows the result for the parameters estimates and the error sum of squares obtained in each iteration.

Table 2. Analytical Result for the Estimated Parameters (      ) for data in Table 1 ITERATIONS θ0 θ1 ssε 0 56.08298 1 49.46378 2 49.4593 3 49.4593

Table 3. Summary results of the Analytical, Jackknife delete- 1 and Jackknife delete-d techniques and their Least Squares Criterion Measure Analytic Delete-1 Delete-2 Delete-3 Delete-4 Delete-5 θ1 58.6065 58.599 58.5892 58.5735 58.5501 58.5147 θ2 -0.0395 -0.03958 -0.03957 -0.03956 -0.03955 -0.03953 ssε 49.4593 45.7369 42.007 38.2684 34.5174 30.75165

Table 3. Summary results of the Analytical, Jackknife delete- 1 and Jackknife delete-d techniques and their Least Squares Criterion Measure (continue) Delete-6 Delete-7 Delete-8 Delete-9 Delete-10 Delete-11 θ1 58.46 58.37 58.2415 58.04314 58.7724 57.61277 θ2 -0.03949 -0.03944 -0.03936 -0.03924 -0.03924 -0.03888 ssε 26.9677 23.16296 19.33635 15.49097 11.63559 7.78429

Discussion

The least squares criterion measure for the starting values has been reduced in the first iteration and also further reduced in the second, third iterations respectively. The third iteration led to no change in either the estimates of the coefficient or the least squarescriterion measure. Hence, convergence is achieved, and the iterations end. Table 3 shows the results of the analytical and the Jackknifes computation. The fitted regression functions for both analytical and Jackknifes delete -1 computation are:

and respectively.

The sums of squares error for the analytical and Jackknifes computation are also shown in the table 3. Also, as the number of d observations deleted in each resampling stage increases, the error sum of squares reduces minimally.

4. Conclusions

We have described the Jackknife algorithm in estimation of the parameters of nonlinear regression model implementation in exponential regression model. The results obtained as shown in Tables 2 and 3 indicate that the Jackknife methods produced a minimum error sum of squares than the analytical method. We also observe that as the number of d observations deleted in each resampling stage increases, the error sum of squares reduces minimally. Hence, the Jackknife techniques yielded approximately the same inference as the analytical method with a better reduced error sum of squares.

Appendix

Delete One Jackknife Result for the Estimates of Parameters

y <- c(54,50,45,37,35,25,20,16,18,13,8,11,8,4,6)

x <- c(2,5,7,10,14,19,26,31,34,38,45,52,53,60,65)

data=cbind(y,x)

initial=c(56.66,-0.03797)

expo=jack(data,2,1,initial)

#Run the following to view the jacknife results

theta_0=mean(expo[,1])

theta_0

[1] 58.59964

theta_1=mean(expo[,2])

theta_1

[1] -0.03958257

SSE=mean(expo[,3])

SSE

[1] 45.73693

> expo

[,1] [,2] [,3]

[1,] 58.72515 -0.03967516 49.42362

[2,] 57.69838 -0.03906341 44.59006

[3,] 58.40287 -0.03950187 49.05152

[4,] 59.16565 -0.03961937 42.57768

[5,] 58.45181 -0.03973249 47.48378

[6,] 58.66917 -0.03904419 41.73934

[7,] 58.54728 -0.03931089 48.45668

[8,] 58.48971 -0.03921048 47.87337

[9,] 58.92722 -0.04049873 40.86874

[10,] 58.60389 -0.03957958 49.45880

[11,] 58.36243 -0.03902207 45.56932

[12,] 59.04047 -0.04053060 35.99040

[13,] 58.70595 -0.03980039 48.74806

[14,] 58.44419 -0.03925252 47.21911

[15,] 58.76037 -0.03989685 47.00351

Deleted (= 5) E Result for the Estimates of Parameters

y <- c(54,50,45,37,35,25,20,16,18,13,8,11,8,4,6)

x <- c(2,5,7,10,14,19,26,31,34,38,45,52,53,60,65)

data=cbind(y,x)

initial=c(56.66,-0.03797)

expo=jack(data,2,5,initial)

#Run the following to view the jacknife results

theta_0=mean(expo[,1])

theta_0

[1] 58.51474

theta_1=mean(expo[,2])

theta_1

[1] -0.03953024

SSE=mean(expo[,3])

SSE

[1] 30.75165

[1,] 58.67728 -0.03959126 27.73554

[2,] 58.17681 -0.03846691 38.39546

[3,] 59.16003 -0.04070913 29.55899

[4,] 59.68595 -0.04223456 20.55436

[5,] 59.23983 -0.04121757 22.01381

[6,] 59.70170 -0.04224374 18.53772

[7,] 58.67701 -0.03992907 35.55294

[8,] 59.17548 -0.04102820 33.88313

[9,] 58.75452 -0.04008201 33.77203

[10,] 59.75422 -0.04241509 20.84062

[2994,] 59.78918 -0.04239372 16.85996

[2995,] 59.39368 -0.04150764 18.60413

[2996,] 58.89694 -0.04038925 32.77863

[2997,] 59.84530 -0.04254249 17.11752

[2998,] 58.80620 -0.04000943 30.68688

[2999,] 59.26868 -0.04100269 29.02210

[3000,] 58.87835 -0.04014695 28.89213

[3001,] 58.37072 -0.03902467 40.60791

[3002,] 59.35796 -0.04123589 29.70771

[3003,] 59.01265 -0.04042930 27.88792