Constrained Least Squares Estimator (CLS)
The constrained least squares estimator (CLS) of /?, denoted by Д is defined to be the value of P that minimizes the sum of squared residuals
S(P) = (y-XP)'(y-XP) (1A2)
under the constraints (1.4.1). In Section 1.2.1 we showed that (1.4.2)Js minimized without constraint at the least squares estimator fi. Writing S(fi) for the sum of squares of the least squares residuals, we can rewrite (1.4.2) as
S(P) = S(P) + (P-P)’X’X(P-P). (1.4.3)
Instead of directly minimizing (1.4.2) under (1.4.1), we minimize (1.4.3) under ^1.4.1), which is mathematically simpler.
Put p — P = 6 and Q’ji—c = y. Then, because S(P) does not depend on P, the problem is equivalent to the minimization of S’X’XS under Q’S = y. Equating the derivatives of ‘ X’ X<H – 2A’ (Q’ ^ — y) with respect to S and the ^-vector of Lagrange multipliers A to zero, we obtain the solution
S = (X’ X)- *Q[Q’ (X’ X)- ’Q]- ‘y. (1.4.4)
Transforming from S and у to the original variables, we can write the minimizing value P of S(P) as
p = p-(X’X)~lQ[Q’ (X’X)-!Q]-l(Q’P – с). (1.4.5)
The corresponding estimator of a2 can be defined as
It is easy to show that the P and a2 are the constrained maximum likelihood estimators if we assume normality of u in Model 1.