13. Generalized Method of Moments (GMM) Estimation

Likelihood methods have a shortcoming in that you "have to do everything to do something." That is, to use likelihood methods you need to fully specify your model and likelihood function. The benefit of GMM is that you can work with only partially specified models. This statement holds in the sense that for GMM we can specify moment restrictions and then estimate the parameters of our model in order to best fit these restrictions.

• \(\beta\) is a \(k\times1\) vector of the true parameters of interest, which is unknown.
• Moment restrictions function \(F(\mathbf x,\mathbf b)\): a function whose image is a \(r\times1\) vector. \(\mathbf x\) is in general a vector of some dimension (according to our data), and \(\mathbf b\) is a \(k\times1\) vector corresponding to the dimension of our parameter \(\beta\) to be estimated.
• Plug in \(\mathbf x=\mathbf X_t\) and \(\mathbf b=\beta\), and take expectation; then it will be zero, i.e. $$\mathbb E[F(\mathbf X_t,\beta)]=\mathbf 0\tag{13.1}$$
• The expectation (i.e. moments) is w.r.t. the stochastic process \(\{\mathbf X_t\}\); these \(r\) moment restrictions come from our economic model.
• In general, we could write the expectation as conditional on some partition \(\mathcal I\) of invariant events, i.e. \(\mathbb E[F(\mathbf X_t,\beta)\mid\mathcal I]=\mathbf 0\); but if \(\{\mathbf X_t\}\) is ergodic (\(\mathbb P\) assigns 0 or 1 to each invariant event), the expectation becomes unconditional, which is (13.1).
• This is called moment restrictions, and we have \(r\) moment restrictions in total, forming a system of \(r\) equations with \(k\) unknowns.

Selection matrix. \(\mathbf A^i\) is a \(r\times1\) column vector, and $$\mathbf A=\begin{bmatrix}\mid&\mid&\cdots&\mid\\\mathbf A^1&\mathbf A^2&\cdots&\mathbf A^k\\\mid&\mid&\cdots&\mid\end{bmatrix}$$ is a \(r\times k\) matrix called the selection matrix (\(r=\) number of moment restrictions $=$ number of moments to match, \(k=\) number of parameters to estimate). By (13.1), \(\mathbf A\) automatically satisfies $$\mathbf A'\mathbb E[F(\mathbf X_t,\beta)]=\mathbf 0\quad\text{or}\quad\mathbb E[\mathbf A'F(\mathbf X_t,\beta)]=\mathbf 0$$ Equivalently, for each column vector \(\mathbf A^i\) (\(i=1,\dots,k\)) of \(\mathbf A\), $$\mathbb E[\mathbf A^i\cdot F(\mathbf X_t,\beta)]=\mathbf 0,\quad i=1,\dots,k\tag{13.2}$$ (where \(\cdot\) is the dot product of two column vectors, i.e. \(\mathbf A^i\cdot F=(\mathbf A^i)'F\).) This is a system of \(k\) equations with \(k\) unknowns. We assume the only solution to $$\mathbf A'\mathbb E[F(\mathbf X_t,\mathbf b)]=\mathbf 0\tag{13.3}$$ is \(\mathbf b=\beta\). Starting from a system of \(r\) equations given by (13.1), and using the selection matrix \(\mathbf A\) to linearly transform those \(r\) equations into \(k\) equations: when \(r\ge k\), we have at least as many restrictions as unknowns, called just-identification or over-identification (we can assume a unique solution \(\mathbf b=\beta\) to (13.3)); when \(runder-identification (there might be multiple solutions to (13.3)).

GMM estimator. The GMM estimator \(\mathbf b_N\) of \(\beta\) is solved by the asymptotic analog of the moment restriction (13.2): $$\frac1N\sum_{t=1}^N(\mathbf A^i)'F(\mathbf X_t,\mathbf b_N)=\mathbf 0,\quad i=1,2,\dots,k$$ or the stacked version $$\frac1N\sum_{t=1}^N\mathbf A'F(\mathbf X_t,\mathbf b_N)=\mathbf 0\tag{13.4}$$

Period-specific selection matrix \(\mathbf A_t\). There are several possible reasons to specify a selection matrix with subscript \(t\): - For estimation purposes, we may place higher weights on some restrictions in some periods, $$\frac1N\sum_{t=1}^N\mathbf A_t'F(\mathbf X_t,\mathbf b_N)=\mathbf 0\tag{13.5}$$ allowing us to accurately control our estimation priorities. In this case \(\mathbb E[F(\mathbf X_t,\beta)]=\mathbf 0\) is still true without introducing \(\mathbf A_t\). - It could also be that our economic model tells us we have different moment restrictions in different periods based on the revealed information at that time; we want to put all possible restrictions in \(F(\mathbf X_t,\beta)\) and activate some of restrictions and shut down other restrictions for period \(t\) (see §13.2.3 on conditional moment restrictions). In this case \(\mathbb E[F(\mathbf X_t,\beta)]=\mathbf 0\) is not necessarily true without introducing \(\mathbf A_t\); instead the moment restrictions become \(\mathbf A_t'\mathbb E[F(\mathbf X_t,\beta)]=\mathbf 0\) or \(\mathbb E[\mathbf A_t'F(\mathbf X_t,\beta)]=\mathbf 0\).

Suppose our function \(F\) is separable and, in particular, we can write $$F(\mathbf X_t,\mathbf b)=F_1(\mathbf X_t)-F_2(\mathbf b)$$ then $$\mathbb E[F(\mathbf X_t,\beta)]=\mathbf 0\ \Rightarrow\ \mathbb E[F_1(\mathbf X_t)]=F_2(\beta)$$ In this formulation, \(F_1\) defines the moments to be matched, and \(F_2(\beta)\), based on the underlying economic model, gives the predicted moments as a function of the true parameter vector \(\beta\).

Consider a typical IV set-up where we have $$\mathbb E[\mathbf X_t u_t]\ne\mathbf 0$$ where \(\mathbf X_t\) is contained in \(\mathbf Y_t\) which contains both the dependent variable and some independent variables for a regression, i.e. $$\mathbf Y_t=\begin{bmatrix}Y_t\\X_{1,t}\\X_{2,t}\\\vdots\\X_{k,t}\end{bmatrix}\quad\text{and}\quad Y_t=\alpha_1 X_{1,t}+\alpha_2 X_{2,t}+\dots+\alpha_k X_{k,t}+u_t$$ with \(u_t\) the error term in that regression. Let $$\alpha=\begin{bmatrix}1\\-\alpha_1\\-\alpha_2\\\vdots\\-\alpha_k\end{bmatrix}\ \Rightarrow\ \mathbf Y_t'\alpha=u_t$$ Suppose further we have some instrument variables \(\mathbf Z_t\) (an \(r\times1\) vector) that satisfy \(\mathbb E[\mathbf Z_t u_t]=\mathbf 0\); then $$\mathbb E[\mathbf Z_t u_t]=\mathbf 0\ \Leftrightarrow\ \mathbb E[\mathbf Z_t\mathbf Y_t'\alpha]=\mathbf 0$$ This construction is similar to our general GMM set-up: \(\alpha\) is analogous to our parameters of interest \(\beta\), the term \(\mathbb E[\mathbf Z_t\mathbf Y_t'\alpha]=\mathbf 0\) is like a moment restriction, with moment restriction function \(F(\mathbf Y_t,\alpha)=\mathbf Z_t\mathbf Y_t'\alpha\), satisfying \(\mathbb E[F(\mathbf Y_t,\alpha)]=\mathbb E[\mathbf Z_t\mathbf Y_t'\alpha]=\mathbf 0\).

We can also consider conditional moment restrictions of the form $$\mathbf A_t'\mathbb E[F(\mathbf X_t,\beta)\mid\mathcal F_t]=\mathbf 0\quad\text{or}\quad\mathbb E[\mathbf A_t'F(\mathbf X_t,\beta)\mid\mathcal F_t]=\mathbf 0$$ where we index the selection matrix \(\mathbf A_t\) by time and denote the information set at period \(t\) by \(\mathcal F_t\). Notice that by the law of iterated expectations, conditional moment restrictions imply unconditional moment restrictions as $$\mathbb E[\mathbf A_t'F(\mathbf X_t,\beta)]=\mathbb E[\mathbf A_t'\mathbb E[F(\mathbf X_t,\beta)\mid\mathcal F_t]]=\mathbf 0\quad\text{or}\quad\mathbf A_t'\mathbb E[F(\mathbf X_t,\beta)]=\mathbf 0$$

• Consider the stochastic consumption process \(\{c_t\}_{t=0}^\infty\). Suppose \((c_0,c_1,\dots,c_\tau,\dots)\) is on some indifference curve.
• Let \((c_0-P_0(r),c_1,\dots,c_\tau+r\xi_\tau,\dots)\) be on the same indifference curve as \((c_0,c_1,\dots,c_\tau,\dots)\), where: \(\xi_\tau\) is the stochastic payoff of an asset that has a price on date 0 and only has payoff on date \(\tau\); \(r\) is the amount of such asset bought on date 0; \(P_0(r)\) is the total cost of buying \(r\) units of that asset on date 0.
• Define (at \(r=0\)) $$\pi_0^\tau(\xi_\tau)=\frac{dP_0(r)}{dr}\Big|_{r=0}$$
• Since \((c_0-P_0(r),c_1,\dots,c_\tau+r\xi_\tau,\dots)\) and \((c_0,c_1,\dots,c_\tau,\dots)\) are on the same indifference curve, \(\pi_0^\tau(\xi_\tau)\) is the units of consumption that the consumer would be indifferent to give up in exchange for \(\xi_\tau\) units of consumption at date \(\tau\) when the consumer doesn't own any such asset.
• So \(\pi_0^\tau(\xi_\tau)\) can be interpreted as the shadow price of such asset on date 0 in terms of real consumption good.
• Denote the stochastic discounting factor of date \(\tau\) by \(\frac{S_\tau}{S_0}\); then we have that $$\pi_0^\tau(\xi_\tau)=\mathbb E\Big[\frac{S_\tau}{S_0}\xi_\tau\mid\mathcal F_0\Big]\tag{13.6}$$
• Since the discounting \(\frac{S_\tau}{S_0}\) is evaluated in the perspective of date 0, the expectation is based on the information set \(\mathcal F_0\) at date 0.
• The classic f.o.c. analysis gives that the consumer would optimally choose the consumption sequence such that $$\frac{p_\tau^0}{p_0^0}=\frac{MU(c_\tau^0)}{MU(c_0^0)}$$ where \(MU\) is the marginal utility, \(c_\tau^0\) is consumption at date \(\tau\) from the perspective of date 0, \(c_0^0\) is consumption at date 0, \(p_\tau^0\) is the present-value price of one unit of consumption at \(\tau\) evaluated at date 0, and \(p_0^0\) likewise for date 0. By construction of the discounting factor \(\frac{S_\tau}{S_0}=\frac{p_\tau^0}{p_0^0}\), then \(\frac{S_\tau}{S_0}=\frac{MU(c_\tau^0)}{MU(c_0^0)}\).

Suppose the consumer has the time-additively-separable utility function $$u(\{c_t\}_{t=0}^\infty)=\mathbb E\Big[\sum_{t=0}^\infty\exp(-\delta t)u(c_t)\mid\mathcal F_t\Big]$$ Then \(MU(c_\tau^0)=\exp(-\delta\tau)u'(c_\tau)\), \(MU(c_0^0)=u'(c_0)\), so $$\frac{S_\tau}{S_0}=\exp(-\delta\tau)\frac{u'(c_\tau)}{u'(c_0)}$$ By (13.6), $$\pi_0^\tau(\xi_\tau)=\mathbb E\Big[\exp(-\delta\tau)\frac{u'(c_\tau)}{u'(c_0)}\xi_\tau\mid\mathcal F_0\Big]\tag{13.7}$$ where \(\pi_0^\tau(\xi_\tau)\) is the observed price of such asset and \(\delta\) is the consumer's subjective discount rate from the perspective of date 0.

Further assumptions to solve the model: normalize the observed price to be 1, i.e. \(\pi_0^\tau(\xi_\tau)=1\). Then \(\xi_\tau\) is the stochastic gross return of such asset. Suppose \((\xi_\tau,\frac{S_\tau}{S_0})\) is jointly log-normal (by Definition 21.2 in the appendix); we can write $$\log\xi_\tau=\mu_0^\xi+\sigma^\xi\cdot\mathbf W_\tau$$ $$\log S_\tau-\log S_0=\mu_0^S+\sigma^S\cdot\mathbf W_\tau$$ where \(\mathbf W_\tau\) is a \(l\times1\) i.i.d. shock vector that follows \(\mathbf W_\tau\sim N(\mathbf 0,\mathbf I)\), and \(\sigma^\xi,\sigma^S\) are the \(l\times1\) exposure vectors.

Since \(\big(\mu_0^\xi+\frac{|\sigma^\xi|^2}2\big)=\log\mathbb E[\xi_\tau\mid\mathcal F_0]\), we can interpret it as the expected (log) return of such asset. If we let \(\sigma^\xi=\mathbf 0\), then such asset has no exposure to risk at all, which makes it a risk-free asset. In that case (13.8) still holds, i.e. \(\big(\mu_0^\xi+\frac{|\sigma^\xi|^2}2\big)+\big(\mu_0^S+\frac{|\sigma^S|^2}2\big)=0\Rightarrow\big(\mu_0^\xi+\frac{|\sigma^\xi|^2}2\big)=-\big(\mu_0^S+\frac{|\sigma^S|^2}2\big)\), so \(-\big(\mu_0^S+\frac{|\sigma^S|^2}2\big)\) is the expected (log) return of the risk-free asset. Therefore \(\big(\mu_0^\xi+\frac{|\sigma^\xi|^2}2\big)+\big(\mu_0^S+\frac{|\sigma^S|^2}2\big)\) (the LHS of (13.8)) is the expected risk premium of our asset of interest, while the RHS of (13.8) is $$\sum_{j=1}^l\sigma_j^\xi(-\sigma_j^S)$$ Since \(\sigma_j^\xi\) is the risk exposure of the asset to \(W_{j,\tau}\), we can say that the vector \(-\sigma^S\) is the risk compensation. In this case the asset has premium with linear loading of risk exposures.

Assume we have the CLT hold such that $$\frac1{\sqrt N}\sum_{t=1}^N F(\mathbf X_t,\beta)\xrightarrow{d}N(\mathbf 0,\mathbf V)\tag{13.9}$$ and LLN hold such that $$\frac1N\sum_{t=1}^N F(\mathbf X_t,\mathbf b_N)\xrightarrow{p}\mathbb E[F(\mathbf X_t,\mathbf b_N)]=\mathbf 0\ \Rightarrow\ \mathbf b_N\xrightarrow{p}\beta$$ Then, for the \(j\)-th column \(\mathbf A^j\) of the selection matrix \(\mathbf A\), we can use the Mean Value Theorem: $$\begin{aligned}\frac1{\sqrt N}\mathbf A^j\cdot\sum_{t=1}^N F(\mathbf X_t,\mathbf b_N)&=\frac1{\sqrt N}\mathbf A^j\cdot\sum_{t=1}^N F(\mathbf X_t,\beta)+\frac1{\sqrt N}\mathbf A^j\cdot\sum_{t=1}^N\frac{\partial F(\mathbf X_t,\overline{\mathbf b}_N)}{\partial\beta'}(\mathbf b_N-\beta)\\&=\frac1{\sqrt N}\mathbf A^j\cdot\sum_{t=1}^N F(\mathbf X_t,\beta)+\frac1N\mathbf A^j\cdot\sum_{t=1}^N\frac{\partial F(\mathbf X_t,\overline{\mathbf b}_N)}{\partial\beta'}\sqrt N(\mathbf b_N-\beta)\end{aligned}\tag{13.10}$$ where each \(j\)-th element in \(\overline{\mathbf b}_N\) lies somewhere on the line segment between the \(j\)-th element in \(\mathbf b_N\) and the \(j\)-th element in \(\beta\). Since \(\mathbf b_N\to\beta\), we also have \(\overline{\mathbf b}_N\to\beta\). Again, by LLN, $$\frac1N\sum_{t=1}^N\frac{\partial F(\mathbf X_t,\overline{\mathbf b}_N)}{\partial\beta'}\xrightarrow{p}\underbrace{\mathbb E\Big[\frac{\partial F(\mathbf X_t,\beta)}{\partial\beta'}\Big]}_{\equiv\mathbf D}\tag{13.11}$$ Stacking \(\mathbf A^j\)'s and using (13.10), (13.11): $$\frac1{\sqrt N}\mathbf A'\sum_{t=1}^N F(\mathbf X_t,\mathbf b_N)\xrightarrow{p}\frac1{\sqrt N}\mathbf A'\sum_{t=1}^N F(\mathbf X_t,\beta)+\mathbf A'\mathbf D\sqrt N(\mathbf b_N-\beta)$$ By construction \(\frac1{\sqrt N}\mathbf A'\sum F(\mathbf X_t,\mathbf b_N)=\mathbf 0\), so $$\frac1{\sqrt N}\mathbf A'\sum_{t=1}^N F(\mathbf X_t,\beta)+\mathbf A'\mathbf D\sqrt N(\mathbf b_N-\beta)\xrightarrow{p}\mathbf 0$$ i.e. $$\mathbf A'\mathbf D\sqrt N(\mathbf b_N-\beta)\xrightarrow{p}-\Big(\frac1{\sqrt N}\mathbf A'\sum_{t=1}^N F(\mathbf X_t,\beta)\Big)\tag{13.12}$$ By (13.9), \(\frac1{\sqrt N}\mathbf A'\sum F(\mathbf X_t,\beta)\xrightarrow{d}N(\mathbf 0,\mathbf A'\mathbf V\mathbf A)\), so $$\mathbf A'\mathbf D\sqrt N(\mathbf b_N-\beta)\xrightarrow{d}-N(\mathbf 0,\mathbf A'\mathbf V\mathbf A)\overset{d}{=}N(\mathbf 0,\mathbf A'\mathbf V\mathbf A)$$ $$\Rightarrow\sqrt N(\mathbf b_N-\beta)\xrightarrow{d}N(\mathbf 0,\operatorname{Cov}(\mathbf A))\tag{13.13}$$ where $$\operatorname{Cov}(\mathbf A)=(\mathbf A'\mathbf D)^{-1}\mathbf A'\mathbf V\mathbf A\big((\mathbf A'\mathbf D)^{-1}\big)'=(\mathbf A'\mathbf D)^{-1}\mathbf A'\mathbf V\mathbf A(\mathbf D'\mathbf A)^{-1}\tag{13.14}$$

From the previous part, \(\sqrt N(\mathbf b_N-\beta)\xrightarrow{d}N(\mathbf 0,\operatorname{Cov}(\mathbf A))\), \(\operatorname{Cov}(\mathbf A)=(\mathbf A'\mathbf D)^{-1}\mathbf A'\mathbf V\mathbf A(\mathbf D'\mathbf A)^{-1}\). We want to find a greatest lower bound for \(\operatorname{Cov}(\mathbf A)\) provided there exists \(\mathbf A\) s.t. \(\mathbf A'\mathbf D\) is non-singular. (Notice that if \(\mathbf D'\mathbf A\) is singular, then \(\operatorname{Cov}(\mathbf A)\) is not well-defined.) The following steps.

1. WLOG impose \(\mathbf D'\mathbf A=\mathbf I\). Why is this without loss of generality? - (a) For a non-singular \(k\times k\) matrix \(\mathbf F\), \(\operatorname{Cov}(\mathbf A)=\operatorname{Cov}(\mathbf{AF})\): $$\begin{aligned}\operatorname{Cov}(\mathbf{AF})&=\big((\mathbf{AF})'\mathbf D\big)^{-1}(\mathbf{AF})'\mathbf V(\mathbf{AF})\big(\mathbf D'(\mathbf{AF})\big)^{-1}\\&=(\mathbf F'\mathbf A'\mathbf D)^{-1}\mathbf F'\mathbf A'\mathbf V\mathbf A\mathbf F\big((\mathbf D'\mathbf A)\mathbf F\big)^{-1}\\&=(\mathbf A'\mathbf D)^{-1}(\mathbf F')^{-1}\mathbf F'\mathbf A'\mathbf V\mathbf A\mathbf F\mathbf F^{-1}(\mathbf D'\mathbf A)^{-1}\\&=(\mathbf A'\mathbf D)^{-1}\mathbf A'\mathbf V\mathbf A(\mathbf D'\mathbf A)^{-1}=\operatorname{Cov}(\mathbf A)\end{aligned}$$ - (b) Suppose \(\mathbf D'\mathbf A\ne\mathbf I\); post-multiply \(\mathbf A\) by \((\mathbf D'\mathbf A)^{-1}\), and denote \(\tilde{\mathbf A}=\mathbf A(\mathbf D'\mathbf A)^{-1}\). Then \(\mathbf D'\tilde{\mathbf A}=\mathbf D'\mathbf A(\mathbf D'\mathbf A)^{-1}=\mathbf I\). By (a) (with \(\mathbf F=(\mathbf D'\mathbf A)^{-1}\)), $$\operatorname{Cov}(\mathbf A)=\operatorname{Cov}(\tilde{\mathbf A})\tag{13.16}$$ - (c) Also \(\mathbf A'\frac1{\sqrt N}\sum F(\mathbf X_t,\mathbf b_N)=\mathbf 0\Rightarrow(\mathbf A(\mathbf D'\mathbf A)^{-1})'\frac1{\sqrt N}\sum F(\mathbf X_t,\mathbf b_N)=\mathbf 0\Rightarrow\tilde{\mathbf A}'\frac1{\sqrt N}\sum F(\mathbf X_t,\mathbf b_N)=\mathbf 0\) (13.17). - (d) Since \(\mathbf b_N\) is uniquely solved, \(\mathbf A\) and \(\tilde{\mathbf A}\) give the same GMM estimator and the same limiting distribution, so they are equivalent. \(\tilde{\mathbf A}\) has the property \(\mathbf D'\tilde{\mathbf A}=\mathbf I\), so WLOG we say \(\mathbf A\) has the property \(\mathbf D'\mathbf A=\mathbf I\).

2. Find \(\tilde{\mathbf A}\) s.t. \(\mathbf A'\mathbf V\tilde{\mathbf A}=\mathbf A'\mathbf D\) for all \(\mathbf A\): $$\tilde{\mathbf A}=\mathbf V^{-1}\mathbf D\tag{13.18}$$

3. Define $$\mathbf A^\star=\tilde{\mathbf A}(\mathbf D'\tilde{\mathbf A})^{-1}=\mathbf V^{-1}\mathbf D(\mathbf D'\mathbf V^{-1}\mathbf D)^{-1}$$

4. Show \(\operatorname{Cov}(\mathbf A)\ge\operatorname{Cov}(\mathbf A^\star)=\operatorname{Cov}(\tilde{\mathbf A})\). - (a) By Step 1, \(\operatorname{Cov}(\mathbf A^\star)=\operatorname{Cov}(\tilde{\mathbf A}(\mathbf D'\tilde{\mathbf A})^{-1})=\operatorname{Cov}(\tilde{\mathbf A})\). - (b) Notice that (under \(\mathbf A'\mathbf D=\mathbf I\), i.e. \((\mathbf A^\star)'\mathbf D=\mathbf I\)) $$\mathbf A'\mathbf V\mathbf A^\star=\mathbf A'\mathbf V\mathbf V^{-1}\mathbf D(\mathbf D'\mathbf V^{-1}\mathbf D)^{-1}=\mathbf A'\mathbf D(\mathbf D'\mathbf V^{-1}\mathbf D)^{-1}=(\mathbf D'\mathbf V^{-1}\mathbf D)^{-1}$$ similarly \((\mathbf A^\star)'\mathbf V\mathbf A=\mathbf A'\mathbf V\mathbf A^\star\), and \((\mathbf A^\star)'\mathbf V\mathbf A^\star=(\mathbf D'\mathbf V^{-1}\mathbf D)^{-1}=\mathbf A'\mathbf V\mathbf A^\star\). - (c) Then $$\begin{aligned}\underbrace{(\mathbf A-\mathbf A^\star)'\mathbf V(\mathbf A-\mathbf A^\star)}_{\text{PSD}}&=\mathbf A'\mathbf V\mathbf A\underbrace{-(\mathbf A^\star)'\mathbf V\mathbf A-\mathbf A'\mathbf V\mathbf A^\star+(\mathbf A^\star)'\mathbf V\mathbf A^\star}_{\text{three equal terms}}\\&=\mathbf A'\mathbf V\mathbf A-(\mathbf A^\star)'\mathbf V\mathbf A=\operatorname{Cov}(\mathbf A)-\operatorname{Cov}(\tilde{\mathbf A})\end{aligned}$$ where the LHS is positive semi-definite (PSD), and the three terms are equal so two cancel. This implies $$\operatorname{Cov}(\mathbf A)\ge\operatorname{Cov}(\tilde{\mathbf A})=\operatorname{Cov}(\mathbf A^\star)$$ i.e. \(\operatorname{Cov}(\mathbf A^\star)\) gives the smallest possible asymptotic variance of any choice of \(\mathbf A\) — the efficiency bound is $$\operatorname{Cov}(\mathbf A^\star)=\operatorname{Cov}(\tilde{\mathbf A})=(\mathbf D'\mathbf V^{-1}\mathbf D)^{-1}$$

Consider the following model $$\alpha'\mathbb E[\mathbf Y_t\mid\mathcal F_{t-l}]=0\tag{13.19}$$ with \(l\ge1\) and \(\mathbf z_{t-l}\in\mathcal F_{t-l}\). Note \(\alpha\) is \(k\times1\), \(\mathbb E[\mathbf Y_t\mid\mathcal F_{t-l}]\) is \(k\times1\), and \(\mathbf z_{t-l}\) is \(r\times1\). This conditional moment restriction (13.19) implies the unconditional moment restriction (13.20): $$\mathbb E[\mathbf z_{t-l}\mathbf Y_t']\alpha=\mathbf 0\tag{13.20}$$ because (by LIE) $$\begin{aligned}\mathbb E[\mathbf z_{t-l}\mathbf Y_t']\alpha&=\mathbb E[\mathbb E[\mathbf z_{t-l}\mathbf Y_t'\mid\mathcal F_{t-l}]]\alpha=\mathbb E[\mathbf z_{t-l}\mathbb E[\mathbf Y_t'\mid\mathcal F_{t-l}]\alpha]\\&=\mathbb E[\mathbf z_{t-l}(\alpha'\mathbb E[\mathbf Y_t\mid\mathcal F_{t-l}])']=\mathbb E[\mathbf z_{t-l}\cdot0]=\mathbf 0\end{aligned}$$ where the parameter of interest \(\beta\) is \(\alpha\), the moment restriction function is \(F(\mathbf Y_t,\alpha)=\mathbf z_{t-l}\mathbf Y_t'\alpha\), and again the unconditional moment restriction is \(\mathbb E[F(\mathbf Y_t,\alpha)]=\mathbb E[\mathbf z_{t-l}\mathbf Y_t']\alpha=\mathbf 0\). (See §13.2.2 for how to understand these notations in a consistent way with our usual set-up for IV regression.)

Suppose that \(\mathbf Y_t\) and \(\alpha\) are both \(2\times1\) to illustrate the basic idea in a simpler way. Then \(\mathbb E[F(\mathbf Y_t,\alpha)]\) is \(r\times1\), \(\mathbb E[\mathbf z_{t-l}\mathbf Y_t']\) is \(r\times2\), and \(\operatorname{rank}(\mathbb E[\mathbf z_{t-l}\mathbf Y_t'])\le2\). - If \(\operatorname{rank}(\mathbb E[\mathbf z_{t-l}\mathbf Y_t'])=0\), the model is under-identified and \(\alpha\) can take any value. - If \(\operatorname{rank}(\mathbb E[\mathbf z_{t-l}\mathbf Y_t'])=2\), the model is over-identified and \(\alpha\) can only be \(\begin{bmatrix}0\\0\end{bmatrix}\). - If \(\operatorname{rank}(\mathbb E[\mathbf z_{t-l}\mathbf Y_t'])=1\), the model is just-identified (in general where \(\mathbf Y_t\) is \(k\)-dimensional, just-identification means \(\operatorname{rank}=k-1\)), and \(\alpha\) can be identified up to a scalar, i.e. $$\alpha=\begin{bmatrix}\beta\\1\end{bmatrix}\quad\text{or}\quad\alpha=\begin{bmatrix}1\\\beta\end{bmatrix}$$ depending on which element of the \(2\times1\) vector we choose to normalize.

Pre-testing. Suppose \(\mathbf Y_t=\begin{bmatrix}Y_t^1\\Y_t^2\end{bmatrix}\), and then we can regress \(Y_t^1\) on \(\mathbf z_{t-l}\) and \(Y_t^2\) on \(\mathbf z_{t-l}\). If both of these regression coefficients are not significant, then we won't be able to identify \(\alpha\), i.e. \(\operatorname{rank}(\mathbb E[\mathbf z_{t-l}\mathbf Y_t'])=0\), in which case \(\mathbb E[\mathbf z_{t-l}\mathbf Y_t']\alpha=\mathbf 0\) holds for any \(\alpha\). Such a procedure of regressing \(Y_t^1\) on \(\mathbf z_{t-l}\) and \(Y_t^2\) on \(\mathbf z_{t-l}\) to test instrument relevance is called pre-testing.

Recall that previously in (13.9) we defined matrix \(\mathbf V\) by \(\frac1{\sqrt N}\sum F(\mathbf X_t,\beta)\xrightarrow{d}N(\mathbf 0,\mathbf V)\). Here in this case, $$\mathbf V=\lim_{N\to\infty}\mathbb E\Big[\Big(\frac1{\sqrt N}\sum_{t=1}^N F(\mathbf Y_t,\alpha)\Big)\Big(\frac1{\sqrt N}\sum_{t=1}^N F(\mathbf Y_t,\alpha)\Big)'\Big]=\lim_{N\to\infty}\mathbb E\Big[\Big(\frac1{\sqrt N}\sum_{t=1}^N\mathbf z_{t-l}\mathbf Y_t'\alpha\Big)\Big(\frac1{\sqrt N}\sum_{t=1}^N\mathbf z_{t-l}\mathbf Y_t'\alpha\Big)'\Big]$$ Now define the \(r\times1\) vector \(\mathbf G_t=F(\mathbf Y_t,\alpha)=\mathbf z_{t-l}\underbrace{\mathbf Y_t'\alpha}_{\equiv u_t}=\mathbf z_{t-l}u_t\). By the law of iterated expectations, when \(j\ge2\) we have \(\mathbf G_{t-j}=\mathbf z_{t-l-j}u_{t-j}\in\mathcal F_{t-l}\), and then $$\begin{aligned}\mathbb E[\mathbf G_t\mathbf G_{t-j}']&=\mathbb E[\mathbb E[\mathbf G_t\mathbf G_{t-j}'\mid\mathcal F_{t-l}]]=\mathbb E[\mathbb E[\mathbf G_t\mid\mathcal F_{t-l}]\mathbf G_{t-j}']\\&=\mathbb E[\mathbb E[\mathbf z_{t-l}u_t\mid\mathcal F_{t-l}]\mathbf G_{t-j}']=\mathbb E[\mathbf z_{t-l}\underbrace{\mathbb E[\mathbf Y_t'\alpha\mid\mathcal F_{t-l}]}_{=0}\mathbf G_{t-j}']=\mathbf 0\end{aligned}\tag{13.21}$$ From this result plus the stationarity assumption of \(\mathbf z_{t-l}\) and \(\mathbf Y_t\), we have that $$\mathbf V=\mathbb E[\mathbf G_t\mathbf G_t']+\mathbb E[\mathbf G_t\mathbf G_{t-1}']+\mathbb E[\mathbf G_t\mathbf G_{t+1}']$$ (the cross-product terms for \(j\ge2\) drop out because of (13.21).)

Recall we defined the \(r\times k\) matrix \(\mathbf D=\mathbb E\big[\frac{\partial F(\mathbf X_t,\beta)}{\partial\beta'}\big]\). Notice that our expression for \(\mathbf D\) depends on which element of the vector \(\alpha\) we normalized. If \(\alpha\) takes the form \(\alpha=\begin{bmatrix}\beta\\1\end{bmatrix}\), then since the second element is known by construction, there is only one parameter of interest \(\beta\), and then $$\mathbf D=\mathbb E\Big[\frac{\partial F(\mathbf Y_t,\beta)}{\partial\beta}\Big]=\mathbb E\Big[\frac{\partial(\mathbf z_{t-l}\mathbf Y_t'\alpha)}{\partial\beta}\Big]=\mathbb E\Big[\frac{\partial[\mathbf z_{t-l}(Y_t^1\beta+Y_t^2)]}{\partial\beta}\Big]=\mathbb E[\mathbf z_{t-l}Y_t^1]$$ which is a \(r\times1\) vector. If instead we start by imposing \(\alpha=\begin{bmatrix}1\\\beta\end{bmatrix}\), then similarly \(\mathbf D=\mathbb E[\mathbf z_{t-l}Y_t^2]\), also a \(r\times1\) vector. So our \(\mathbf D\) matrix depends on our choice of normalization.

In the general case we established that with \(\tilde{\mathbf A}=\mathbf V^{-1}\mathbf D\), the efficiency bound is \(\operatorname{Cov}(\tilde{\mathbf A})=(\mathbf D'\mathbf V^{-1}\mathbf D)^{-1}\). In our example (with \(\alpha=\begin{bmatrix}\beta\\1\end{bmatrix}\), \(\mathbf D=\mathbb E[\mathbf z_{t-l}Y_t^1]\), \(l=2\)), the efficiency bound is $$\operatorname{Cov}(\tilde{\mathbf A})=\Big(\big(\mathbb E[\mathbf z_{t-l}Y_t^1]\big)'\big(\mathbb E[\mathbf G_t\mathbf G_t']+\mathbb E[\mathbf G_t\mathbf G_{t-1}']+\mathbb E[\mathbf G_t\mathbf G_{t+1}']\big)^{-1}\mathbb E[\mathbf z_{t-l}Y_t^1]\Big)^{-1}$$

For the TSLS estimator to attain the GMM efficiency bound, we need to impose two more assumptions: 1. \(\mathbf G_t\) is i.i.d. and so \(l=1\). 2. Homoskedasticity, i.e. \(\mathbb E[u_t^2\mid\mathcal F_{t-l}]=\sigma^2\).

Then under these additional assumptions, the TSLS estimator attains the GMM efficiency bound.

In other words, under i.i.d. and homoskedasticity assumptions, the TSLS estimator attains the GMM efficiency bound.

From the Law of Large Numbers we know \(\frac1N\sum F(\mathbf X_t,\mathbf b_N)\xrightarrow{p}\mathbb E[F(\mathbf X_t,\mathbf b_N)]\); from the Central Limit Theorem we know \(\frac1{\sqrt N}\sum F(\mathbf X_t,\beta)\xrightarrow{d}N(\mathbf 0,\mathbf V)\). But in order to carry out a test for \(\beta\) based on \(\mathbf b_N\), we need the limiting distribution of \(\frac1{\sqrt N}\sum F(\mathbf X_t,\mathbf b_N)\) instead of \(\frac1{\sqrt N}\sum F(\mathbf X_t,\beta)\). Their limiting distributions are different because \(\mathbf b_N\) is an estimation of \(\beta\) and the estimation has some distribution incurred by the sampling distribution. The adjustment result is $$\frac1{\sqrt N}\sum_{t=1}^N F(\mathbf X_t,\mathbf b_N)\xrightarrow{p}\Big[\mathbf I-\mathbf D(\mathbf A'\mathbf D)^{-1}\mathbf A'\Big]\frac1{\sqrt N}\sum_{t=1}^N F(\mathbf X_t,\beta)\tag{13.22}$$ Notice \(\mathbf A'\big(\mathbf I-\mathbf D(\mathbf A'\mathbf D)^{-1}\mathbf A'\big)=\mathbf 0\). This makes sense because we solved \(\mathbf b_N\) by \(\mathbf A'\frac1{\sqrt N}\sum F(\mathbf X_t,\mathbf b_N)=\mathbf 0\); so if (13.22) is true, it must be that \(\mathbf A'\big[\mathbf I-\mathbf D(\mathbf A'\mathbf D)^{-1}\mathbf A'\big]\frac1{\sqrt N}\sum F(\mathbf X_t,\beta)=\mathbf 0\) with probability 1.

Since \(\mathbf A'\big[\mathbf I-\mathbf D(\mathbf A'\mathbf D)^{-1}\mathbf A'\big]=\mathbf 0\), the \(r\times r\) matrix \(\big[\mathbf I-\mathbf D(\mathbf A'\mathbf D)^{-1}\mathbf A'\big]\) has rank strictly less than \(r\), and thus the variance-covariance matrix of \(\frac1{\sqrt N}\sum F(\mathbf X_t,\mathbf b_N)\) is singular (non-invertible). So we cannot directly construct a chi-square estimator (which requires invertibility of the variance-covariance matrix); instead we will be using the invertible part of that variance-covariance matrix to carry out the test $$H_0:\ \mathbb E[F(\mathbf X_t,\mathbf b_N)]=\mathbf 0\ (\text{or }\mathbf b_N=\beta)$$ $$H_1:\ \mathbb E[F(\mathbf X_t,\mathbf b_N)]\ne\mathbf 0\ (\text{or }\mathbf b_N\ne\beta)$$

Way 1: Calibration and Verification. Calibrate (estimate) the model using only \(k\) coordinates of \(F\), and verify the remaining \(r-k\) restrictions. Partition \(\mathbb E[F(\mathbf X_t,\beta)]=\mathbf 0\) into the first \(k\) rows \(\mathbb E[F_1(\mathbf X_t,\beta)]_{r\times1}=\mathbf 0\) and the last \(r-k\) rows \(\mathbb E[F_2(\mathbf X_t,\beta)]_{r\times1}=\mathbf 0\). 1. WLOG select the first \(k\) rows, \(\mathbf A'=[\mathbf I_{k\times k}\ \mathbf 0_{k\times(r-k)}]\) (or, more efficiently, \(\mathbf A'=[(\mathbf V_1^{-1}\mathbf D_1)'\ \mathbf 0_{k\times(r-k)}]\), where \(\mathbf V_1^{-1}\) is the upper-left \(k\times k\) submatrix of \(\mathbf V\) and \(\mathbf D_1\) is the first \(k\) rows of \(\mathbf D\)). \(F_1(\mathbf X_t,\beta)\) has \(r\) entries, the first \(k\) the same as \(F(\mathbf X_t,\beta)\) and the last \(r-k\) set to 0; \(F_2(\mathbf X_t,\beta)\) has the first \(k\) set to 0 and the last \(r-k\) the same as \(F(\mathbf X_t,\beta)\). 2. Estimate \(\mathbf b_N^{F_1}\) using \(\mathbb E[F_1(\mathbf X_t,\beta)]_{r\times1}=\mathbf 0\) with \(\mathbf A'=[\mathbf I_{k\times k}\ \mathbf 0]\), solving \(\frac1N\sum F_1(\mathbf X_t,\mathbf b_N^{F_1})=\mathbf 0\) or \(\frac1N\sum\mathbf A'F(\mathbf X_t,\mathbf b_N^{F_1})=\mathbf 0\). 3. Test \(H_0\) using \(\mathbb E[F_2(\mathbf X_t,\beta)]_{r\times1}=\mathbf 0\) with \(\mathbf A=[\mathbf I_{k\times k}\ \mathbf 0_{k\times(r-k)}]'\): - (a) Use the fact that \(\big[\frac1{\sqrt N}\sum F_2(\mathbf X_t,\mathbf b_N^{F_1})\big]_{r\times1}\xrightarrow{p}\mathbf B\big(\mathbf I-\mathbf D(\mathbf A'\mathbf D)^{-1}\mathbf A'\big)\big(\frac1{\sqrt N}\sum F(\mathbf X_t,\beta)\big)\) (13.23), \(\mathbf B=[\mathbf 0_{r\times k}\ \mathbf I_{r\times(r-k)}]\), partition \(\mathbf D=\begin{bmatrix}(\mathbf D_1)_{k\times k}\\(\mathbf D_2)_{(r-k)\times k}\end{bmatrix}\). - (b) Rewrite (13.23): \(\big[\frac1{\sqrt N}\sum F_2(\mathbf X_t,\mathbf b_N^{F_1})\big]\xrightarrow{p}\big[-(\mathbf D_2(\mathbf D_1)^{-1})\ \mathbf I_{r\times(r-k)}\big]\big(\frac1{\sqrt N}\sum F(\mathbf X_t,\beta)\big)\). - (c) \(\big[\frac1{\sqrt N}\sum F_2(\mathbf X_t,\mathbf b_N^{F_1})\big]\xrightarrow{d}N(\mathbf 0,\Gamma)\) (13.24), \(\Gamma=\big[-(\mathbf D_2(\mathbf D_1)^{-1})\ \mathbf I_{r\times(r-k)}\big]\mathbf V\begin{bmatrix}-((\mathbf D_1')^{-1}\mathbf D_2')\\\mathbf I_{(r-k)\times r}\end{bmatrix}\). - (d) Construct a Wald test statistic $$\Big[\frac1{\sqrt N}\sum_{t=1}^N F_2(\mathbf X_t,\mathbf b_N^{F_1})\Big]'\Gamma^{-1}\Big[\frac1{\sqrt N}\sum_{t=1}^N F_2(\mathbf X_t,\mathbf b_N^{F_1})\Big]\xrightarrow{d}\chi^2_{r-k}$$ (Using the GMM-efficient selection matrix \(\mathbf A_1'=(\mathbf V_1^{-1}\mathbf D_1)'\) to solve \(\mathbf b_{e,N}^{F_1}\), one similarly gets the limit \(N(\mathbf 0,\Omega)\) (13.25), and the Wald statistic again \(\xrightarrow{d}\chi^2_{r-k}\).)

Way 2: Testing with Efficient Estimation. From \(\frac1{\sqrt N}\sum F(\mathbf X_t,\beta)\xrightarrow{d}N(\mathbf 0,\mathbf V)\), let \(\mathbf V^{-1}=\Lambda'\Lambda\) (\(\Lambda\) invertible); then \(\frac1{\sqrt N}\sum\Lambda F(\mathbf X_t,\beta)\xrightarrow{d}N(\mathbf 0,\Lambda\mathbf V\Lambda')=N(\mathbf 0,\mathbf I)\) (13.26). Recall \(\frac1{\sqrt N}\sum F(\mathbf X_t,\mathbf b_N)\xrightarrow{p}(\mathbf I-\mathbf D(\mathbf A'\mathbf D)^{-1}\mathbf A')\frac1{\sqrt N}\sum F(\mathbf X_t,\beta)\) (13.27). Take the efficient \(\mathbf A=\mathbf V^{-1}\mathbf D\) (\(\mathbf A'=\mathbf D'\mathbf V^{-1}\)); pre-multiply both sides of (13.27) by \(\Lambda\): $$\frac1{\sqrt N}\sum\Lambda F(\mathbf X_t,\mathbf b_N)\xrightarrow{p}\Lambda\big(\mathbf I-\mathbf D(\mathbf D'\mathbf V^{-1}\mathbf D)^{-1}\mathbf D'\mathbf V^{-1}\big)\frac1{\sqrt N}\sum F(\mathbf X_t,\beta)\tag{13.28}$$ Define \(\Delta_{r\times r}=\Lambda\mathbf D(\mathbf D'\Lambda'\Lambda\mathbf D)^{-1}\mathbf D'\Lambda'\); then both \(\Delta\) and \(\mathbf I-\Delta\) are idempotent (\(\Delta\Delta=\Delta\), \((\mathbf I-\Delta)(\mathbf I-\Delta)=\mathbf I-\Delta\)), with \(\Delta\) of rank \(k\), \((\mathbf I-\Delta)\) of rank \(r-k\), complementary. Rewrite (13.28): \(\frac1{\sqrt N}\sum\Lambda F(\mathbf X_t,\mathbf b_N)\xrightarrow{p}(\mathbf I-\Delta)\frac1{\sqrt N}\sum\Lambda F(\mathbf X_t,\beta)\) (13.29), \(\frac1{\sqrt N}\sum\Lambda F(\mathbf X_t,\mathbf b_N)\xrightarrow{d}N(\mathbf 0,\Phi)\) (13.30), \(\Phi=(\mathbf I-\Delta)\). Three chi-squares: $$\Big[\tfrac1{\sqrt N}\sum\Lambda F(\mathbf X_t,\beta)\Big]'\Big[\tfrac1{\sqrt N}\sum\Lambda F(\mathbf X_t,\beta)\Big]\xrightarrow{d}\chi^2_r\tag{13.31}$$ $$\Big[(\mathbf I-\Delta)\tfrac1{\sqrt N}\sum\Lambda F(\mathbf X_t,\beta)\Big]'\Big[(\mathbf I-\Delta)\tfrac1{\sqrt N}\sum\Lambda F(\mathbf X_t,\beta)\Big]\xrightarrow{d}\chi^2_{r-k}\tag{13.32}$$ $$\Big[\Delta\tfrac1{\sqrt N}\sum\Lambda F(\mathbf X_t,\beta)\Big]'\Big[\Delta\tfrac1{\sqrt N}\sum\Lambda F(\mathbf X_t,\beta)\Big]\xrightarrow{d}\chi^2_k\tag{13.33}$$ Two quadratic forms: - Form 1: by (13.32) and (13.29), $$\Big[\frac1{\sqrt N}\sum F(\mathbf X_t,\mathbf b_N)\Big]'\mathbf V^{-1}\Big[\frac1{\sqrt N}\sum F(\mathbf X_t,\mathbf b_N)\Big]\xrightarrow{d}\chi^2_{r-k}\tag{13.34}$$ - Form 2: from \(\big[\frac1{\sqrt N}\sum F(\mathbf X_t,\beta)\big]'\mathbf V^{-1}\big[\cdots\big]\xrightarrow{d}\chi^2_r\) and (13.34), $$\Big[\frac1{\sqrt N}\sum F(\mathbf X_t,\beta)\Big]'\mathbf V^{-1}\Big[\frac1{\sqrt N}\sum F(\mathbf X_t,\beta)\Big]-\Big[\frac1{\sqrt N}\sum F(\mathbf X_t,\mathbf b_N)\Big]'\mathbf V^{-1}\Big[\frac1{\sqrt N}\sum F(\mathbf X_t,\mathbf b_N)\Big]\xrightarrow{d}\chi^2_k\tag{13.35}$$

Way 3: Simultaneous Estimation and Testing. Estimate \(\beta\) and simultaneously test \(H_0\). Consider $$\min_{\mathbf b}\Big[\frac1{\sqrt N}\sum_{t=1}^N F(\mathbf X_t,\mathbf b)\Big]'\mathbf V^{-1}\Big[\frac1{\sqrt N}\sum_{t=1}^N F(\mathbf X_t,\mathbf b)\Big]\tag{13.36}$$ If \(H_0\) is true (\(\mathbf b=\beta\)), the objective function should not be too big; we are actually estimating \(\beta\) by finding the \(\mathbf b\) s.t. the test statistic reaches its minimum. - If in this case we still have to reject \(H_0\) (which is not supposed to happen), we need to consider maybe the moment restrictions themselves are wrong. - In most cases, we would reach a solution \(\mathbf b\) such that \(H_0\) is not rejected (by construction of \(\mathbf b\)), which makes \(\mathbf b\) a reasonable estimator for \(\beta\).

The f.o.c. for (13.36): denote the objective function \(\mathcal L=\mathbf F'\mathbf V^{-1}\mathbf F\) (\(\mathbf F=\frac1{\sqrt N}\sum F(\mathbf X_t,\mathbf b)\)); then $$\begin{aligned}\mathbf 0=\frac{\partial\mathcal L}{\partial\mathbf b_N'}&=\frac{\partial\mathcal L}{\partial\mathbf F'}\frac{\partial\mathbf F}{\partial\mathbf b_N'}=2\mathbf F'\mathbf V^{-1}\frac{\partial\mathbf F}{\partial\mathbf b_N'}=2\Big[\frac1{\sqrt N}\sum F(\mathbf X_t,\mathbf b_N)\Big]'\mathbf V^{-1}\underbrace{\frac1N\sum\frac{\partial F(\mathbf X_t,\mathbf b_N)}{\partial\mathbf b_N'}}_{\equiv\mathbf D_N}\\&\Rightarrow\underbrace{\mathbf D_N'\mathbf V^{-1}}_{\equiv\mathbf A_N'}\Big[\frac1{\sqrt N}\sum F(\mathbf X_t,\mathbf b_N)\Big]=\mathbf 0\end{aligned}$$ and thus \(\mathbf A_N'=\mathbf D_N'\mathbf V^{-1}=\tilde{\mathbf A}_N'\), i.e. the GMM estimator solved in this way attains its efficiency bound.

Till now we pretended that we know \(\mathbf V\), which is not true. So we have to estimate it in order to carry out all the tests discussed above. Three ways of estimating \(\mathbf V\): 1. Two-step method: (a) First, estimate \(\beta\) based on a selection matrix \(\mathbf A\), which in general is not \(\mathbf V^{-1}\mathbf D\) since we don't know \(\mathbf V\); at this step the estimator might not attain its efficiency bound; the selection matrix \(\mathbf A\) here can be anything since we have no idea what \(\mathbf V\) is, and typically we set \(\mathbf A=\mathbf D\) (i.e. replace \(\mathbf V^{-1}\) with the identity matrix \(\mathbf I\)). (b) Second, use the estimated \(\hat\beta\) to estimate \(\mathbf V\) by the sample analog of the definition of \(\mathbf V\). 2. Interactive: Continue to do the two-step method and hope it would converge to a fixed point (which is not necessarily always the case). 3. Continuous updating: Solve $$\min_{\mathbf b}\Big[\frac1{\sqrt N}\sum_{t=1}^N F(\mathbf X_t,\mathbf b)\Big]'(\mathbf V(\mathbf b))^{-1}\Big[\frac1{\sqrt N}\sum_{t=1}^N F(\mathbf X_t,\mathbf b)\Big]\tag{13.37}$$ where \(\mathbf V\) is estimated at each \(\mathbf b\). This construction in (13.37) gives more flexibility than (13.36) by allowing \(\mathbf V\) to move with different choices of \(\mathbf b\). So, using this method, we are achieving the real minimized value of the objective function (which is also the chi-square test statistic where \(\mathbf V(\mathbf b)\) agrees with \(\mathbf b\) at each point of \(\mathbf b\)). This method is used more often in practice.