NONLINEAR REGRESSION MODEL
The nonlinear regression model is defined by (13.4.1) yt = /Др) + ut, t = 1, 2, . . . , T,
where /,(•) is a known function, P is a ^vector of unknown parameters, and {ut} are i. i.d. with Eut = 0 and Vut = a2. In practice we often specify /*(P) = /(x(, P), where x, is a vector of exogenous variables which, unlike the linear regression model, may not necessarily be of the same dimension as p.
An example of the nonlinear regression model is the CobbDouglas production function with an additive error term,
(13.4.2) d = + Щ,
where Q, K, and L denote output, capital input, and labor input, respectively. Another example is the CES production function (see Arrow et al., 1961):
(13.4.3) Qt = + (1 – р2)АГРзГр4/Рз + щ.
We can write (13.4.1) in vector notation as
(13.4.4) у = f(p) + u,
where y, f, and u are Tvectors having yt, ft, and ut, respectively, for the fth element.
The nonlinear least squares (NLLS) estimator of P is defined as the value of P that minimizes
T
(13.4.5) Sr(P) = X^/((P)]2
(=1
Denoting the NLLS estimator by P, we can estimate a2 by
(13.4.6) d2 = ^ ST(P).
The estimators P and d2 can be shown to be the maximum likelihood estimators if {ut} are assumed to be jointly normal. The derivation is analogous to the linear case given in Section 12.2.5.
The minimization of Sj(P) must generally be done by an iterative method. The NewtonRaphson method described in Section 7.3.3 can be used for this purpose. Another iterative method, the GaussNewton method, is specifically designed for the nonlinear regression model. Let Pi be the initial value, be it an estimator or a mere guess. Expand/,(p) in a Taylor series around P = as
. (Э – Pi), p,
where dft / Эр’ is a Xdimensional row vector whose 7th element is the derivative of ft with respect to the jth element of p. Note that (13.4.7) holds approximately because the derivatives are evaluated by Pj. Inserting
(13.4.7)
into the righthand side of (13.4.1) and rearranging terms, we obtain:
The secondround estimator of the iteration, P2, is obtained as the LS estimator applied to (13.4.8), treating the entire lefthand side as the dependent variable and dft/dp as the vector of regressors. The iteration
is repeated until it converges. It is simpler than the NewtonRaphson method because it requires computation of only the first derivatives of ft, whereas NewtonRaphson requires the second derivatives as well.
We can show that under general assumptions the NLLS estimator p is consistent and
The above result is analogous to the asymptotic normality of the LS estimator given in Theorem 12.2.4. Note that df/ЭР’ above is just like X in Theorem 12.2.4. The difference is that3f/3p’ depends on the unknown parameter P and hence is unknown, whereas X is assumed to be known. The practical implication of (13.4.9) is that
A „ „ 
( л 9 
Гэг 
df 
!л 

~ N 
P, CT 
эр 
эр’ 0 
0 – 
J 
(13.4.10) p 
The asymptotic variancecovariance matrix above is comparable to formula (12.2.22) for the LS estimator. We can test hypotheses about P in the nonlinear regression model by the methods presented in Section 12.4, provided that we use df/dfi’p for X.
Leave a reply