An Alternative Derivation of the Constrained Least Squares Estimator

Define а К X (К — q) matrix R such that the matrix (Q, R) is nonsingular and R’ Q = 0. Such a matrix can always be found; it is not unique, and any matrix
that satisfies these conditions will do. Then, defining A = (Q, R)’, у = АД and Z = XA-1, we can rewrite the basic model (1.1.4) as

y = Xfi+u (1.4.7)

= XA-‘A/H-ii = Zy + u.

If we partition у = (yj, У2)’, where Уі = Q’fi and y2 = R’fi, we see that the constraints (1.4.1) specify y, and leave y2 unspecified. The vector y2 hasK— q elements; thus we have reduced the problem of estimating К parameters subject to q constraints to the problem of estimating К — q free parameters. Using y, = c and

A"1 = [Q(Q’Q)_I, R(R’R)_1], (1.4.8)

we have from (1.4.7)

у – XCKQ’Qr’c = XR(R’R)-1y2 + u. (1.4.9)

Let y2 be the least squares estimator of y2 in (1.4.9):

y2 – R’R(R, X’XR)~1R, X,[y – XQfQ’Q)"^]. (1.4.10)

Now, transforming from у back to fi by the relationship fi — A-1y, we obtain the CLS estimator of fi:

Подпись: (1.4.11)fi= RfR’X’XRj-‘R’X’y

+ [I – R(R, X’XR)_1R’X’X]Q(Q’Q)_1c.

Note that (1.4.11) is different from (1.4.5). Equation (1.4.5) is valid only if X’X is nonsingular, whereas (1.4.11) can be defined even if X’X is singular provided that R’ X ’ XR is nonsingular. We can show that if X’ X is nonsingu­lar, (1.4.11) is unique and equal to (1.4.5). Denote the right-hand sides of (1.4.5) and (1.4.11) by Д and fi2, respectively. Then it is easy to show

[Rq,,X](A-&) = 0. (1.4.12)

Therefore fii = fi2 if the matrix in the square bracket above is nonsingular. But we have

Подпись:R’X’X"]|"_ _."l _ TR’X’XR R’X’XQ]

where the matrix on the right-hand side is clearly nonsingular because non­
singularity of X’X implies nonsingularity of R’X’XR. Because the matrix [R, Q] is nonsingular, it follows that the matrix

is nonsingular, as we desired.

Leave a reply

You may use these HTML tags and attributes: <a href="" title=""> <abbr title=""> <acronym title=""> <b> <blockquote cite=""> <cite> <code> <del datetime=""> <em> <i> <q cite=""> <s> <strike> <strong>