4. Linear Regression with Exogenous Variables

Let \((Y,\mathbf X,u)\) be as defined in Chapter 3, and assume \(\mathbf X\) has no perfect collinearity, \(\mathbb E[\mathbf X\mathbf X']\) exists, \(\mathbb E[Y^2]<\infty\), and \(\mathbb E[\mathbf Xu]=\mathbf 0\) (exogeneity).

4.1.1 Solving for \(\boldsymbol\beta\).

From \(\mathbb E[\mathbf Xu]=\mathbf 0\) with \(u=Y-\mathbf X'\boldsymbol\beta\):

$$\mathbf 0=\mathbb E[\mathbf Xu]=\mathbb E[\mathbf X(Y-\mathbf X'\boldsymbol\beta)]=\mathbb E[\mathbf XY]-\mathbb E[\mathbf X\mathbf X']\boldsymbol\beta\Rightarrow\mathbb E[\mathbf X\mathbf X']\boldsymbol\beta=\mathbb E[\mathbf XY] \tag{4.1}$$

$$\Rightarrow\boldsymbol\beta=\mathbb E[\mathbf X\mathbf X']^{-1}\mathbb E[\mathbf XY] \tag{4.2}$$

4.1.2 Solving for a subvector of \(\boldsymbol\beta\) (Frisch-Waugh-Lovell). Partition \(\mathbf X\) into \((\mathbf X_1)_{k_1\times1}\), \((\mathbf X_2)_{k_2\times1}\) and \(\boldsymbol\beta\) into \(\boldsymbol\beta_1,\boldsymbol\beta_2\),

$$Y=\mathbf X'\boldsymbol\beta+u=\mathbf X_1'\boldsymbol\beta_1+\mathbf X_2'\boldsymbol\beta_2+u \tag{4.3}$$

From \(\mathbb E[\mathbf Xu]=\mathbf 0\), \(\mathbb E[\mathbf X_1u]=\mathbf 0_{k_1\times1}\), \(\mathbb E[\mathbf X_2u]=\mathbf 0_{k_2\times1}\) (4.4). To recover only \(\boldsymbol\beta_1\), in three steps:

1. Take \(Y\)'s residual \(\tilde Y=Y-\mathrm{BLP}(Y\mid\mathbf X_2)\);
2. Take \(\mathbf X_1\)'s residual \(\tilde{\mathbf X}_1=\mathbf X_1-\mathrm{BLP}(\mathbf X_1\mid\mathbf X_2)\);
3. Regress \(\tilde Y=\tilde{\mathbf X}_1'\boldsymbol\beta_1+\tilde u\), giving \(\tilde{\boldsymbol\beta}=\mathbb E[\tilde{\mathbf X}_1\tilde{\mathbf X}_1']^{-1}\mathbb E[\tilde{\mathbf X}_1\tilde Y]\) (4.5).

Corollary 4.1 (\(\mathbf X_2\) a constant). If \(\mathbf X_2=1\), then \(\mathrm{BLP}(Y\mid1)=\mathbb E[Y]\), \(\mathrm{BLP}(\mathbf X_1\mid1)=\mathbb E[\mathbf X_1]\), so \(\tilde Y=Y-\mathbb E[Y]\), \(\tilde{\mathbf X}_1=\mathbf X_1-\mathbb E[\mathbf X_1]\) (4.6); substituting into (4.5):

$$\boldsymbol\beta_1=\mathrm{Var}(\mathbf X_1)^{-1}\mathrm{Cov}(\mathbf X_1,Y) \tag{4.7}$$

i.e., after removing the constant, the slope coefficient equals "covariance over variance."

4.1.3 Omitted-variable bias (OVB). True model \(Y=\beta_0+\mathbf X_1'\boldsymbol\beta_1+\mathbf X_2'\boldsymbol\beta_2+u\); if we omit \(\mathbf X_2\) and wrongly estimate \(Y=\beta_0^*+\mathbf X_1'\boldsymbol\beta_1^*+u^*\), then by (4.7):

$$\boldsymbol\beta_1^*=\boldsymbol\beta_1+\underbrace{\mathrm{Var}(\mathbf X_1)^{-1}\mathrm{Cov}(\mathbf X_1,\mathbf X_2)\boldsymbol\beta_2}_{\text{omitted variable bias}}$$

The bias = "the regression coefficient of \(\mathbf X_1\) on \(\mathbf X_2\)" times "the true effect \(\boldsymbol\beta_2\) of \(\mathbf X_2\)." No bias when \(\mathbf X_1\perp\mathbf X_2\) (\(\mathrm{Cov}=0\)) or \(\boldsymbol\beta_2=0\).

4.1.4 Measurement error (attenuation bias). Model \(Y=\beta_0+\mathbf X_1'\boldsymbol\beta_1+u\) (4.8), but \(\mathbf X_1\) is unobservable and we observe only \(\hat{\mathbf X}_1=\mathbf X_1+\mathbf v\). Classical measurement-error assumptions: \(\mathbb E[\mathbf v]=\mathbf 0\), \(\mathrm{Cov}(u,\mathbf v)=\mathbf 0\), \(\mathrm{Cov}(\mathbf X_1,\mathbf v)=\mathbf 0\). Regressing \(Y=\beta_0^*+\hat{\mathbf X}_1'\boldsymbol\beta_1^*+u^*\) with \(\hat{\mathbf X}_1\) in place of \(\mathbf X_1\), by Corollary 4.1:

$$\boldsymbol\beta_1^*=\mathrm{Var}(\hat{\mathbf X}_1)^{-1}\mathrm{Cov}(\hat{\mathbf X}_1,Y)=(\mathrm{Var}(\mathbf X_1)+\mathrm{Var}(\mathbf v))^{-1}\mathrm{Var}(\mathbf X_1)\boldsymbol\beta_1$$

In one dimension \(\beta_1^*=\frac{\mathrm{Var}(X_1)}{\mathrm{Var}(X_1)+\mathrm{Var}(v)}\beta_1\), where the factor \(\frac{\mathrm{Var}(X_1)}{\mathrm{Var}(X_1)+\mathrm{Var}(v)}\in(0,1)\), so \(\beta_1^*\) is shrunk toward zero — called attenuation bias.

Example 4.2 More generally, partition \(\mathbf X\) into \(X_0=1\), \(X_1\in\mathbb R\) (mismeasured), \(\mathbf X_2\in\mathbb R^{k_2}\) (measured exactly), \(Y=\beta_0+\beta_1X_1+\mathbf X_2'\boldsymbol\beta_2+u\); using FWL to partial out \(\mathbf X_2\) first (\(\tilde{\hat X}_1=\hat X_1-\mathrm{BLP}(\hat X_1\mid1,\mathbf X_2)\), \(\tilde Y=Y-\mathrm{BLP}(Y\mid1,\mathbf X_2)\)), one shows \(\tilde{\hat X}_1=\tilde X_1+v\), and the attenuation conclusion still holds: \(\beta_1^*=\frac{\mathrm{Var}(\tilde X_1)}{\mathrm{Var}(\tilde X_1)+\mathrm{Var}(v)}\beta_1\).

4.2.1 Estimating \(\boldsymbol\beta\). Under the population properties (\(\mathbb E[\mathbf Xu]=0\), \(\mathbb E[\mathbf X\mathbf X']<\infty\), no perfect collinearity, \((Y,\mathbf X)\sim P\)) and an i.i.d. sample, the natural sample analog of \(\boldsymbol\beta=\mathbb E[\mathbf X\mathbf X']^{-1}\mathbb E[\mathbf XY]\) (4.9) is

$$\hat{\boldsymbol\beta}_n=\Big(\frac1n\sum_{i=1}^n\mathbf X_i\mathbf X_i'\Big)^{-1}\Big(\frac1n\sum_{i=1}^n\mathbf X_iY_i\Big) \tag{4.10}$$

the ordinary least squares (OLS) estimator — it minimizes the sum of squared distances between the true \(Y_i\) and the fitted \(\hat Y_i=\mathbf X_i'\hat{\boldsymbol\beta}_n\): \(\hat{\boldsymbol\beta}_n=\arg\min_{\mathbf b}\frac1n\sum(Y_i-\mathbf X_i'\mathbf b)^2\). The first-order condition \(\frac1n\sum\mathbf X_i(Y_i-\mathbf X_i'\hat{\boldsymbol\beta}_n)=\mathbf 0\) (4.11) gives (4.12) (same as (4.10)); by CMT, \(\det(\frac1n\sum\mathbf X_i\mathbf X_i')\xrightarrow{p}\det(\mathbb E[\mathbf X\mathbf X'])\ne0\), so it is almost surely invertible for large \(n\). The residual \(\hat u_i=Y_i-\hat Y_i\) satisfies \(\sum\mathbf X_i\hat u_i=\mathbf 0\) (4.13); since \(X_{i,0}=1\), the first-order condition for \(\beta_0\) gives \(\sum\hat u_i=0\) (4.14).

4.2.2 OLS as a projection. In matrix notation: \(\mathbb X_{n\times(k+1)}=(\mathbf X_1,\dots,\mathbf X_n)'\) (each row is one observation transposed), \(\mathbb Y=(Y_1,\dots,Y_n)'\), \(\mathbb U=(u_1,\dots,u_n)'\), \(\hat{\mathbb Y}=\mathbb X\hat{\boldsymbol\beta}_n\), \(\hat{\mathbb U}=\mathbb Y-\hat{\mathbb Y}\). Then

$$\hat{\boldsymbol\beta}_n=(\mathbb X'\mathbb X)^{-1}\mathbb X'\mathbb Y \tag{4.15}$$

solves \(\min_{\mathbf b}|\mathbb Y-\mathbb X\mathbf b|^2\). The column space of \(\mathbb X\) is \(\mathrm{Col}[\mathbb X]=\{\mathbb X\mathbf b:\mathbf b\in\mathbb R^{k+1}\}\). Define the projection matrix

$$\mathbb P=\mathbb X(\mathbb X'\mathbb X)^{-1}\mathbb X',\qquad\mathbb P\mathbb Y=\mathbb X\hat{\boldsymbol\beta}_n=\hat{\mathbb Y}$$

\(\mathbb P\mathbb Y\in\mathrm{Col}[\mathbb X]\), so \(\mathbb P\) projects \(\mathbb Y\) onto \(\mathrm{Col}[\mathbb X]\). The residual-maker matrix \(\mathbb M=\mathbb I-\mathbb P=\mathbb I-\mathbb X(\mathbb X'\mathbb X)^{-1}\mathbb X'\) projects \(\mathbb Y\) onto the space orthogonal to \(\mathrm{Col}[\mathbb X]\): \(\mathbb M\mathbb X=\mathbf 0\), \(\mathbb M\mathbb Y=\hat{\mathbb U}\), \(\mathbb M\mathbb U=\hat{\mathbb U}\). Both \(\mathbb P\) and \(\mathbb M\) are idempotent (\(\mathbb P^2=\mathbb P\), \(\mathbb M^2=\mathbb M\)) and symmetric (\(\mathbb P'=\mathbb P\), \(\mathbb M'=\mathbb M\)).

4.2.3 Estimating a subvector \(\hat{\boldsymbol\beta}_{1,n}\) (sample version of FWL). Partition \(\mathbb X\) into \(\mathbb X_1,\mathbb X_2\), \(\mathbb P\) into \(\mathbb P_1,\mathbb P_2\), \(\mathbb M\) into \(\mathbb M_1,\mathbb M_2\). The sample analog \(\mathbb Y=\mathbb X_1\hat{\boldsymbol\beta}_{1,n}+\mathbb X_2\hat{\boldsymbol\beta}_{2,n}+\hat{\mathbb U}\). Left-multiply by \(\mathbb M_2\) to remove \(\mathbb X_2\) (\(\mathbb M_2\mathbb X_2=0\)), with \(\mathbb M_2\hat{\mathbb U}=\hat{\mathbb U}\): \(\mathbb M_2\mathbb Y=\mathbb M_2\mathbb X_1\hat{\boldsymbol\beta}_{1,n}+\hat{\mathbb U}\). Then left-multiply by \((\mathbb M_2\mathbb X_1)'\) and use \(\mathbb X_1'\hat{\mathbb U}=0\):

$$\hat{\boldsymbol\beta}_{1,n}=[(\mathbb M_2\mathbb X_1)'(\mathbb M_2\mathbb X_1)]^{-1}(\mathbb M_2\mathbb X_1)'(\mathbb M_2\mathbb Y) \tag{4.17}$$

the Frisch-Waugh-Lovell decomposition. By idempotency and symmetry of \(\mathbb M_2\), it simplifies to \(\hat{\boldsymbol\beta}_{1,n}=[\mathbb X_1'\mathbb M_2\mathbb X_1]^{-1}\mathbb X_1'\mathbb M_2\mathbb Y\).

4.2.4 Measures of fit. The \(R\)-square \(R^2\equiv\frac{\mathrm{ESS}}{\mathrm{TSS}}=1-\frac{\mathrm{SSR}}{\mathrm{TSS}}\) (4.18), where the estimated sum of squares \(\mathrm{ESS}=\sum(\hat Y_i-\bar Y_n)^2\), sum of squared residuals \(\mathrm{SSR}=\sum\hat u_i^2\), total sum of squares \(\mathrm{TSS}=\sum(Y_i-\bar Y_n)^2\). Since \(\sum\hat u_i(\hat Y_i-\bar Y_n)=0\), \(\mathrm{TSS}=\mathrm{ESS}+\mathrm{SSR}\) (4.19), hence \(0\le R^2\le1\). \(R^2=1\) is a perfect fit (\(\hat u_i=0\)), \(R^2=0\) means no predictive power. Adding more variables never decreases \(R^2\) (more degrees of freedom), so introduce the adjusted \(R^2\) as a penalty:

$$\bar R^2=1-\frac{\frac1{n-(k+1)}\sum\hat u_i^2}{\frac1{n-1}\sum(Y_i-\bar Y_n)^2}=1-\frac{n-1}{n-k-1}\frac{\mathrm{SSR}}{\mathrm{TSS}}$$

\(\bar R^2\) can be negative. Neither \(R^2\) nor \(\bar R^2\) has any causal basis — they are descriptions of fit, not causal interpretations.

All below are under the §4.2.1 baseline assumptions; subsections add extra assumptions.

4.3.1 Unbiasedness.

4.3.2 Gauss-Markov Theorem.

4.3.3 Consistency.

4.3.4 Limiting distribution.

4.3.5 Estimating \(\Omega\). We do not know the true \(\Omega\) and must construct a consistent estimator. Under homoskedasticity \(\Omega=\sigma^2\mathbb E[\mathbf X\mathbf X']^{-1}\), with the natural estimator \(\hat\Omega_n=(\frac1n\sum\mathbf X_i\mathbf X_i')^{-1}\hat\sigma_n^2\) (\(\hat\sigma_n^2=\frac1n\sum\hat u_i^2\)). In general (heteroskedasticity-robust):

$$\hat\Omega_n=\Big(\tfrac1n\sum\mathbf X_i\mathbf X_i'\Big)^{-1}\Big(\tfrac1n\sum\mathbf X_i\mathbf X_i'\hat u_i^2\Big)\Big(\tfrac1n\sum\mathbf X_i\mathbf X_i'\Big)^{-1} \tag{4.25}$$

We must show the middle term \(\frac1n\sum\mathbf X_i\mathbf X_i'\hat u_i^2\xrightarrow{p}\mathrm{Var}(\mathbf Xu)\). Split \(\hat u_i^2=u_i^2+(\hat u_i^2-u_i^2)\) (4.26): the first term by WLLN \(\frac1n\sum\mathbf X_i\mathbf X_i'u_i^2\xrightarrow{p}\mathbb E[\mathbf X\mathbf X'u^2]\); the second uses \(\hat u_i^2-u_i^2=-2u_i\mathbf X_i'(\hat{\boldsymbol\beta}_n-\boldsymbol\beta)+[\mathbf X_i'(\hat{\boldsymbol\beta}_n-\boldsymbol\beta)]^2\) together with the lemma below to show it \(\xrightarrow{p}0\).

Finally \(\hat\Omega_n\xrightarrow{p}\Omega\).

4.4.1 A single linear restriction (\(t\) test, \(p=1\)). Test \(H_0:\mathbf r'\boldsymbol\beta=c\) vs \(H_1:\mathbf r'\boldsymbol\beta\ne c\) (\(\mathbf r\in\mathbb R^{k+1}\), \(c\in\mathbb R\)). From \(\sqrt n(\hat{\boldsymbol\beta}_n-\boldsymbol\beta)\xrightarrow{d}N(0,\Omega)\) and CMT: \(\sqrt n(\mathbf r'\hat{\boldsymbol\beta}_n-\mathbf r'\boldsymbol\beta)\xrightarrow{d}N(0,\mathbf r'\Omega\mathbf r)\). Take the statistic

$$T_n=\frac{\sqrt n(\mathbf r'\hat{\boldsymbol\beta}_n-c)}{\sqrt{\mathbf r'\hat\Omega_n\mathbf r}}\xrightarrow{d}N(0,1)\quad(\text{under }H_0)$$

and \(\phi_n=\mathbf 1\{|T_n|>z_{1-\frac\alpha2}\}\) is consistent in level.

4.4.2 Multiple linear restrictions (Wald test). Test \(H_0:\mathbf R\boldsymbol\beta=\mathbf c\) vs \(H_1:\mathbf R\boldsymbol\beta\ne\mathbf c\), with \(\mathbf R\) a \(p\times(k+1)\) matrix of linearly independent rows (so \(\mathbf R\Omega\mathbf R'\) is invertible). From \(\sqrt n(\mathbf R\hat{\boldsymbol\beta}_n-\mathbf R\boldsymbol\beta)\xrightarrow{d}N(0,\mathbf R\Omega\mathbf R')\) and the fact \(\mathbf x\sim N(0,\Sigma)\Rightarrow\mathbf x'\Sigma^{-1}\mathbf x\sim\chi^2\):

$$T_n=n(\mathbf R\hat{\boldsymbol\beta}_n-\mathbf c)'(\mathbf R\hat\Omega_n\mathbf R')^{-1}(\mathbf R\hat{\boldsymbol\beta}_n-\mathbf c)\xrightarrow{d}\chi^2_p\quad(\text{under }H_0)$$

and \(\phi_n=\mathbf 1\{T_n>c_{p,1-\alpha}\}\) (the \(1-\alpha\) quantile of \(\chi^2_p\)).

4.4.3 Lagrange multiplier (LM) test. The constrained least squares (CLS) estimator \(\tilde{\boldsymbol\beta}_n\) solves \(\min_{\boldsymbol\beta:\mathbf R\boldsymbol\beta=\mathbf c}\frac1n\sum(Y_i-\mathbf X_i'\boldsymbol\beta)^2\). The Lagrangian

$$\mathcal L(\boldsymbol\beta,\boldsymbol\lambda)=\frac1{2n}\sum(Y_i-\mathbf X_i'\boldsymbol\beta)^2+\boldsymbol\lambda'(\mathbf R\boldsymbol\beta-\mathbf c)$$

(the \(\frac12\) cancels a constant). The first-order conditions \(\frac{\partial\mathcal L}{\partial\boldsymbol\beta}=-\frac1n\sum\mathbf X_i(Y_i-\mathbf X_i'\tilde{\boldsymbol\beta}_n)+\mathbf R'\tilde{\boldsymbol\lambda}_n=\mathbf 0\), \(\frac{\partial\mathcal L}{\partial\boldsymbol\lambda}=\mathbf R\tilde{\boldsymbol\beta}_n-\mathbf c=\mathbf 0\). Solving for the multiplier

$$\tilde{\boldsymbol\lambda}_n=\Big(\mathbf R\big(\tfrac1n\sum\mathbf X_i\mathbf X_i'\big)^{-1}\mathbf R'\Big)^{-1}(\mathbf R\hat{\boldsymbol\beta}_n-\mathbf c)$$

Under \(H_0\) (\(\mathbf R\boldsymbol\beta=\mathbf c\)), \(\tilde{\boldsymbol\lambda}_n\xrightarrow{p}0\). Its asymptotic distribution \(\sqrt n\tilde{\boldsymbol\lambda}_n\xrightarrow{d}N(0,\Pi)\) (with \(\Pi=(\mathbf R\mathbb E[\mathbf X\mathbf X']^{-1}\mathbf R')^{-1}\mathbf R\mathbb E[\mathbf X\mathbf X']^{-1}\mathrm{Var}(\mathbf Xu)\mathbb E[\mathbf X\mathbf X']^{-1}\mathbf R'(\mathbf R\mathbb E[\mathbf X\mathbf X']^{-1}\mathbf R')^{-1}\)). The test \(T_n=n\tilde{\boldsymbol\lambda}_n'\hat\Pi^{-1}\tilde{\boldsymbol\lambda}_n\xrightarrow{d}\chi^2_p\), \(\phi_n=\mathbf 1\{T_n>\chi^2_{p,1-\alpha}\}\).

4.4.4 Nonlinear restrictions (delta method). Test \(H_0:f(\boldsymbol\beta)=\mathbf c\) vs \(H_1:f(\boldsymbol\beta)\ne\mathbf c\), where \(f:\mathbb R^{k+1}\to\mathbb R^p\) is continuously differentiable at \(\boldsymbol\beta\) with full-row-rank Jacobian \(D_\beta f(\boldsymbol\beta)=\frac{\partial f(\boldsymbol\beta)}{\partial\boldsymbol\beta'}\) (\(p\times(k+1)\), \(p\le k+1\)). By the delta method \(\sqrt n(f(\hat{\boldsymbol\beta}_n)-f(\boldsymbol\beta))\xrightarrow{d}N(0,D_\beta f(\boldsymbol\beta)\Omega D_\beta f(\boldsymbol\beta)')\), so

$$T_n=n(f(\hat{\boldsymbol\beta}_n)-\mathbf c)'\big(D_\beta f(\hat{\boldsymbol\beta}_n)\hat\Omega_n D_\beta f(\hat{\boldsymbol\beta}_n)'\big)^{-1}(f(\hat{\boldsymbol\beta}_n)-\mathbf c)\xrightarrow{d}\chi^2_p$$

and \(\phi_n=\mathbf 1\{T_n>c_{p,1-\alpha}\}\).

The Gauss-Markov theorem shows OLS is BLUE under homoskedasticity; but under heteroskedasticity OLS is no longer efficient, and generalized least squares (GLS) is more efficient. Let \((Y_i,\mathbf X_i)\) be i.i.d., \(Y_i\in\mathbb R\), \(\mathbf X_i\in\mathbb R^{k+1}\), \(\mathbb E[Y_i\mid\mathbf X_i]=\mathbf X_i'\boldsymbol\beta\), \(\mathrm{Var}(Y_i\mid\mathbf X_i)=\sigma^2(\mathbf X_i)\) (known, $>0$). Let \(\mathbb D=\mathrm{diag}(\sigma^2(\mathbf X_1),\dots,\sigma^2(\mathbf X_n))\), with \(\mathbb X\)'s columns linearly independent.

4.5.1 Unbiasedness. Estimator \(\tilde{\boldsymbol\beta}_n=\mathbb A'\mathbb Y\); GLS takes \(\mathbb A'=(\mathbb X'\mathbb D^{-1}\mathbb X)^{-1}\mathbb X'\mathbb D^{-1}\). \(\mathbb X'\mathbb D^{-1}\mathbb X\) is positive definite (hence invertible): for \(\mathbf c\ne\mathbf 0\), \(\mathbf c'\mathbb X'\mathbb D^{-1}\mathbb X\mathbf c=\sum_i\frac{m_i^2}{\sigma^2(\mathbf X_i)}>0\) (\(\mathbb X\mathbf c=(m_1,\dots,m_n)'\ne\mathbf 0\)). GLS is unbiased: \(\mathbb A'\mathbb X=\mathbb I\).

4.5.2 Conditional variance-covariance matrix.

$$\mathrm{Var}(\tilde{\boldsymbol\beta}_n\mid\mathbb X)=\mathbb A'\mathbb D\mathbb A=(\mathbb X'\mathbb D^{-1}\mathbb X)^{-1} \tag{4.28}$$

(substitute \(\mathrm{Var}(\mathbb Y\mid\mathbb X)=\mathbb D\) and \(\mathbb A'=(\mathbb X'\mathbb D^{-1}\mathbb X)^{-1}\mathbb X'\mathbb D^{-1}\) and simplify).

4.5.3 GLS is BLUE under heteroskedasticity. Among all linear unbiased estimators, GLS has the smallest conditional variance.

4.5.4 General case. Regression \(Y_i=\mathbf X_i'\boldsymbol\beta+\varepsilon_i\), stacked \(\mathbb Y=\mathbb X\boldsymbol\beta+\boldsymbol\varepsilon\), with conditional variance-covariance \(\mathrm{Cov}(\boldsymbol\varepsilon\mid\mathbb X)=\boldsymbol\Sigma\) (not required to be diagonal). GLS solves

$$\min_{\boldsymbol\beta}(\mathbb Y-\mathbb X\boldsymbol\beta)'\boldsymbol\Sigma^{-1}(\mathbb Y-\mathbb X\boldsymbol\beta)$$

The first-order condition \(-2(\mathbb Y-\mathbb X\boldsymbol\beta)'\boldsymbol\Sigma^{-1}\mathbb X=\mathbf 0_{1\times K}\) gives

$$\boldsymbol\beta=(\mathbb X'\boldsymbol\Sigma^{-1}\mathbb X)(\mathbb X'\boldsymbol\Sigma^{-1}\mathbb Y)$$

i.e. \(\boldsymbol\beta=(\mathbb X'\boldsymbol\Sigma^{-1}\mathbb X)^{-1}\mathbb X'\boldsymbol\Sigma^{-1}\mathbb Y\). This is the general case, since \(\boldsymbol\Sigma\) need not be diagonal (it may include serial correlation, etc.).