12. Stochastic Processes

• \(\omega\in\Omega\) is a sample point in the sample space \(\Omega\).
• \(\mathcal F\) is a \(\sigma\)-algebra over \(\Omega\).
• \((\Omega,\mathcal F)\) is a measurable space, with elements of \(\mathcal F\) called measurable sets.
• A function \(\mathbb P:\mathcal F\to[0,1]\) is a probability measure with the properties:
• Non-negativity: \(\forall E\in\mathcal F\), \(\mu(E)\ge0\).
• Null set: \(\mu(\varnothing)=0\).
• Countable additivity (\(\sigma\)-additivity): for any countable collection \(\{E_i\}_{i=1}^\infty\) of pairwise disjoint sets in \(\mathcal F\), \(\mu\big(\bigcup_{k=1}^\infty E_k\big)=\sum_{k=1}^\infty\mu(E_k)\).
• Unit normalization: \(\mu(\Omega)=1\).
• \((\Omega,\mathcal F,\mathbb P)\) is a measure space with a probability measure, also called a probability space. (See Appendix 23.)

• Let \(\omega=(\mathbf r_0,\mathbf r_1,\dots)_{k\times1}\) be a sample point in the sample space \(\Omega\subseteq\mathbb R^{k\times n}\) (\(\mathbf r_j\in\mathbb R^n\) for all \(j=1,2,\dots\)).
• \(\mathbb S:\Omega\to\Omega\) is a measurable transformation s.t. for \(\omega=(\mathbf r_0,\mathbf r_1,\dots)_{k\times1}\), $$\mathbb S(\omega)\equiv(\mathbf r_1,\mathbf r_2,\dots)_{k\times1},\quad\mathbb S^2(\omega)\equiv(\mathbf r_2,\mathbf r_3,\dots)_{k\times1},\quad\dots,\quad\mathbb S^l(\omega)\equiv(\mathbf r_l,\mathbf r_{l+1},\dots)_{k\times1}$$ i.e. \(\mathbb S\) is a "shifting-one-step-ahead" mapping.
• \(X:\Omega\to\mathbb R^n\) is a measurement function whose image is the first element in the sample point vector: $$X_0(\omega)=X(\omega)=\mathbf r_0,\quad X_1(\omega)=X(\mathbb S(\omega))=\mathbf r_1,\quad\dots,\quad X_l(\omega)=X(\mathbb S^l(\omega))=\mathbf r_l$$ Then we can define \(\{X_t\}_{t=0}^\infty\) as a stochastic process.

Let \(\{\Lambda_j\}_{j=1}^\infty\) denote a countable partition of \(\Omega\) where each \(\Lambda_j\) is an invariant event. Let \(\mathcal I=\{\Lambda_1,\Lambda_2,\dots\}\) be a collection of all the elements in the partition \(\{\Lambda_j\}_{j=1}^\infty\). Then the expectation of a random vector \(X\) conditional on \(\omega\in\Lambda_j\) is $$\mathbb E[X\mid\Lambda_j]=\int_{\Lambda_j}\frac{X(\omega)\,d\mathbb P}{\mathbb P(\{\Lambda_j\})}$$ and the expectation of \(X\) conditional on the partition \(\mathcal I\) is a function of \(\omega\): $$\mathbb E[X\mid\mathcal I](\omega)=\int_{\Lambda_j}\frac{X(\omega)\,d\mathbb P}{\mathbb P(\{\Lambda_j\})}\quad\text{if }\omega\in\Lambda_j$$

(The partition satisfies \(\Lambda_i\cap\Lambda_j=\varnothing\) (\(i\ne j\)), \(\bigcup_{j=1}^\infty\Lambda_j=\Omega\), and each \(\Lambda_j\) is non-empty. Since \(\Omega\) and \(\varnothing\) are always invariant events, such a partition can always be obtained.)

Exercise 12.1 (Finite-state Markov chain). How is \((\mathbb S,\mathbb P)\) defined for a finite-state (first-order) Markov chain?

• First consider the sample point \(\omega\) and sample space \(\Omega\). Suppose there are \(k\) states \(S\equiv\{s_1,s_2,\dots,s_k\}\). Let \(\omega_i=(s_{i_1},s_{i_2},s_{i_3},\dots)\) be a sample point (a state sequence with infinite terms); for \(j=1,2,\dots\), \(i_j\in\{1,2,\dots,k\}\), i.e. \(s_{i_j}\in S=\{s_1,\dots,s_k\}\). The sample space is \(\Omega=\{\omega_i=(s_{i_1},s_{i_2},\dots):s_{i_j}\in S\}\); the diversity of elements in \(\Omega\) comes from the arbitrary choice of \(s_{i_j}\) from \(S\) for each position.
• Definition of \(\mathbb S\): \(\mathbb S:\Omega\to\Omega\) is a measurable transformation s.t. for \(\omega_i=(s_{i_1},s_{i_2},s_{i_3},\dots)\), \(\mathbb S^l(\omega_i)\equiv(s_{i_{l+1}},s_{i_{l+2}},\dots)\), a "shifting-one-step-ahead" mapping; since \(\omega_i\) has infinite length, this is a (deterministic) mapping from \(\Omega\) to itself.
• \(X:\Omega\to S\) is a measurement function whose image is the first element in the sample point vector (a state): \(X(\omega_i)=s_{i_1}\), \(X(\mathbb S(\omega_i))=s_{i_2}\), …, \(X(\mathbb S^l(\omega_i))=s_{i_{l+1}}\). Then we define a sequence \(\{X_t(\omega_i)\}_{t=0}^\infty\).
• \(\mathbf X_{l+1}(\omega_i)\) is the \((l+1)\times1\) vector of the first \(l+1\) elements in \(\{X_t\}_{t=0}^\infty\). Since \(\mathbb S\) is deterministic, given \(\omega_i\) both \(\{X_t(\omega_i)\}\) and \(\mathbf X_{l+1}(\omega_i)\) are deterministic and not random at all; \(\mathbf X_{l+1}(\omega_i)\) can be interpreted as a vector of realized states in the first \(l+1\) steps of that Markov chain.
• Definition of \(\mathbb P\): the probability assigned to \(\mathbf X_{l+1}(\omega_i)=(s_{i_1},s_{i_2},\dots,s_{i_{l+1}})'\) can be computed two ways:
• using \((\mathbb S,\mathbb P)\): the probability assigned to \(\mathbf X_{l+1}(\omega_i)\) by \((\mathbb S,\mathbb P)\), \(\mathbb P(\{\omega_i:X(\mathbb S^j(\omega_i))=s_{i_{j+1}},\forall j=0,1,\dots,l\})\), adding up all probabilities of sample points whose first \(l+1\) elements coincide with \(\mathbf X_{l+1}(\omega_i)\).
• using the transition matrix \(\mathbf T\) (\(p_{s_a s_b}\) the probability of transferring from \(s_a\) to \(s_b\)): given an initial distribution \(q_0=(q^0_{s_1},q^0_{s_2},\dots,q^0_{s_k})\), \(\mathbb P\) is defined as a measure such that $$\mathbb P(\{\omega_i:X(\mathbb S^j(\omega_i))=s_{i_{j+1}},\forall j=0,1,\dots,l\})=q^0_{s_{i_1}}\prod_{k=1}^l p_{s_{i_k}s_{i_{k+1}}}$$ holds for any sequence \(\{i_1,i_2,\dots,i_{l+1}\}_{l=0}^\infty\) (\(i_j\in\{1,2,\dots,k\}\)).

Exercise 12.2. How should we understand the relationship between ergodicity defined for \((\mathbb S,\mathbb P)\) and that for the transition matrix \(\mathbf T\)?

• Ergodicity for \((\mathbb S,\mathbb P)\): given an initial distribution \(q_0\) and transition matrix \(\mathbf T\) we can recover the measure \(\mathbb P\), and define \(\mathbb S\) as in Exercise 12.1; if \(\mathbb P\) assigns probability either 0 or 1 to all invariant events under \(\mathbb S\), then \((\mathbb S,\mathbb P)\) is ergodic.
• Ergodicity for \(\mathbf T\): starting from any state (any initial distribution), \(\forall s_j\in S\) has a strictly positive probability of being reached within finite steps.
• The relationship: ergodicity for \((\mathbb S,\mathbb P)\) does rely on the initial distribution, while ergodicity for \(\mathbf T\) doesn't (since constructing the equivalent measure \(\mathbb P\) depends on \(q_0\), whether it is ergodic does depend on \(q_0\)). Ergodicity for \((\mathbb S,\mathbb P)\) allows the state space \(S\) to have multiple invariant events (as subsets of \(S\)), while ergodicity for \(\mathbf T\) requires that the only invariant event is \(S\). So given \(\mathbf T\), if \((\mathbb S,\mathbb P)\) is ergodic for any arbitrary \(q_0\), then it is a necessary but not sufficient condition for \(\mathbf T\) to be ergodic — we also need the condition that there is no transient state. Otherwise it is possible that the only invariant event w.r.t. \(\mathbb S\) is the whole state space but there are transient states: then \((\mathbb S,\mathbb P)\) is still ergodic for any arbitrary \(q_0\), but \(\mathbf T\) is not ergodic.

We can extend our usual version of the Law of Large Numbers (requiring i.i.d. random vectors) to one that allows inter-temporal dependency.

In the probability space \((\Omega,\mathcal F,\mathbb P)\) with \((\mathbb S,\mathbb P)\) measure preserving and \(\{\Lambda_j\}_{j=1}^\infty\) a countable partition of \(\Omega\) (each \(\Lambda_j\) invariant), for any \(\hat\Lambda\subset\Lambda_j\) with \(\hat\Lambda\ne\varnothing\), \(\hat\Lambda\) is not an invariant event. Define the indicator function $$\mathbf 1_\Lambda(\omega)=\begin{cases}1&\text{if }\omega\in\Lambda\\0&\text{otherwise}\end{cases}$$

This gives $$\mathbb P(\Lambda)=\sum_{j=1}^\infty\mathbb P(\Lambda_j)Q_r^j(\Lambda)\tag{12.1}$$ where \(Q_r^j(\Lambda_j)\) is interpreted as an ergodic building block.

A stochastic process is said to be exchangeable if the joint distributions of members of the sequence \(\{\mathbf x_t:t=0,1,\dots\}\) do not depend on their order of appearance. This is formalized below.

In the analysis above, we started with the measure space \((\Omega,\mathcal F,\mathbb P)\) and studied properties of measure-preserving transformations \(\mathbb S\), partitioning \(\Omega\) into a countable collection of invariant events \(\mathcal I=\{\Lambda_1,\Lambda_2,\dots\}\) (\(\Lambda_i\cap\Lambda_k=\varnothing\), \(\bigcup_j\Lambda_j=\Omega\)). Then by (12.1), for any \(\Lambda\in\mathcal F\), $$\mathbb P(\Lambda)=\sum_{j=1}^\infty\mathbb P(\Lambda_j\cap\Lambda)=\sum_{j=1}^\infty\Big(\frac{\mathbb P(\Lambda_j\cap\Lambda)}{\mathbb P(\Lambda_j)}\Big)\mathbb P(\Lambda_j)=\sum_{j=1}^\infty Q_r^j(\Lambda)\mathbb P(\Lambda_j)$$ where \(Q_r^j(\Lambda_j)=1\) and \(Q_r^j(\Lambda_k)=0\) (\(k\ne j\)). So each ergodic building block \(Q_r^j\) assigns probability 1 to only one corresponding invariant event \(\Lambda_j\), and their choices of corresponding invariant event \(\Lambda_j\) do not overlap with each other. This is the ergodic decomposition of \(\mathcal I\).

Now we want to go in the opposite direction. We want to start with a transformation \(\mathbb S\) and then study a set of probability measures \(\tilde{\mathbb P}\) for which \((\mathbb S,\tilde{\mathbb P})\) is measure preserving. We're going to interpret the \(Q_r^j\)'s constructed above as the "ergodic building blocks" of such wanted set of probability measures \(\tilde{\mathbb P}\) for which \((\mathbb S,\tilde{\mathbb P})\) is measure preserving. To motivate this construction, start with a more general proposition, then use the countable analog of this more general result.

Replicating this idea in the countable set-up: with \(Q_r^j\)'s (\(j=1,2,\dots\)) defined as in Proposition 12.2, take arbitrary probabilities \(\pi_j\ge0\) (\(\sum_{j=1}^\infty\pi_j=1\)) and construct $$\tilde{\mathbb P}(\Lambda)=\sum_{j=0}^\infty Q_r^j(\Lambda)\pi_j$$ Then \(\tilde{\mathbb P}\) is measure preserving but typically not ergodic.

Stationary and ergodic case. Consider the model \(\mathbf X:\Omega\to\mathbb R^n\), with \((\mathbb S,\mathbb P)\) defined by $$\mathbf X_{t+1}=\mathbf A\mathbf X_t+\mathbf B\mathbf W_{t+1},\quad \mathbf W_{t+1}\sim N(0,\mathbf I)\ \text{i.i.d.}$$ where \(\mathbf A\) is stable (eigenvalues with absolute values $<1$). Our goal is to find the stationary distribution and verify it is ergodic. \(\mathbf X_0\) is assumed to be deterministic (or normal) and \(\mathbf W_{t+1}\)'s are i.i.d. normal for all \(t\), so the stationary distribution of \(\{\mathbf X_t\}_{t=0}^\infty\) will also be normal. Behavior of moments: 1. Mean. \(\mu_{t+1}=\mathbb E[\mathbf X_{t+1}]=\mathbb E[\mathbf A\mathbf X_t+\mathbf B\mathbf W_{t+1}]=\mathbf A\mu_t+\mathbf 0\). If stationary, \(\mu=\mathbf A\mu\Rightarrow\mathbf 0=(\mathbf A-\mathbf I)\mu\Rightarrow\mu=\mathbf 0\). 2. Variance. \(\Sigma_{t+1}=\mathbf A\Sigma_t\mathbf A'+\mathbf B\mathbf B'\); stationary \(\Sigma=\mathbf A\Sigma\mathbf A'+\mathbf B\mathbf B'\). We guess \(\Sigma=\sum_{j=0}^\infty\mathbf A^j\mathbf B\mathbf B'(\mathbf A')^j\).

So for \((\mathbb S,\mathbb P)\) to be stationary, we have \(\mathbf X_t\sim N(\mathbf 0,\Sigma)\). By the nature of joint normal distribution, any point in \(\mathbb R^n\) has strictly positive probability of being reached, so the unique invariant event is \(\mathbb R^n\). Since \((\mathbb S,\mathbb P)\) assigns probability 1 to \(\mathbb R^n\), it is ergodic.

Stationary but not ergodic case. Consider a different VAR with non-zero mean of the stationary distribution. Partition the Markov state as $$\mathbf X_t=\begin{bmatrix}\mathbf x_t^1\\x_t^2\end{bmatrix},\quad\mathbf A=\begin{bmatrix}\mathbf A_{11}&a_{12}\\\mathbf 0&1\end{bmatrix},\quad\mathbf B=\begin{bmatrix}\mathbf B_1\\0\end{bmatrix}$$ i.e. $$\begin{bmatrix}\mathbf x_{t+1}^1\\x_{t+1}^2\end{bmatrix}=\begin{bmatrix}\mathbf A_{11}&a_{12}\\\mathbf 0&1\end{bmatrix}\begin{bmatrix}\mathbf x_t^1\\x_t^2\end{bmatrix}+\begin{bmatrix}\mathbf B_1\\0\end{bmatrix}\mathbf W_{t+1}$$ where \(x_t^2\ne0\), \(a_{12}\) is a scalar, and \(\mathbf x_t^1\) is a vector. Because the second row of \(\mathbf A\) is the row vector \([\mathbf 0,1]\), this implies \(x_t^2\) is constant for all \(t\). Suppose \(\mathbf x_t^1\) is stationary; then $$\mathbb E[\mathbf x_{t+1}^1]=\mathbf A_{11}\mathbb E[\mathbf x_t^1]+a_{12}\mathbb E[x_t^2]\Rightarrow(\mathbf I-\mathbf A_{11})\mathbb E[\mathbf x_t^1]=a_{12}\mathbb E[x_t^2]\Rightarrow\mu_1\equiv\mathbb E[\mathbf x_t^1]=(\mathbf I-\mathbf A_{11})^{-1}a_{12}x_0^2$$ For the variance, let \(\Sigma_t=\begin{bmatrix}\Sigma_t^{11}&0\\0&0\end{bmatrix}\); then $$\Sigma_{t+1}^{11}=\operatorname{Var}(\mathbf x_{t+1}^1)=\operatorname{Var}(\mathbf A_{11}\mathbf x_t^1+a_{12}x_t^2+\mathbf B_1\mathbf W_{t+1})=\mathbf A_{11}\Sigma_t^{11}\mathbf A_{11}'+\mathbf B_1\mathbf B_1'$$ Stationary: \(\Sigma^{11}=\sum_{j=0}^\infty\mathbf A_{11}^j\mathbf B_1\mathbf B_1'(\mathbf A_{11}')^j\). So for \((\mathbb S,\mathbb P)\) to be stationary, \(\mathbf X_t\sim N(\mu,\Sigma)\) where $$\mu=\begin{bmatrix}(\mathbf I-\mathbf A_{11})^{-1}a_{12}x_0^2\\x_0^2\end{bmatrix},\quad\Sigma=\begin{bmatrix}\Sigma^{11}&0\\0&0\end{bmatrix}$$ But clearly not any point in \(\mathbb R^n\) has strictly positive probability of being reached, so \(\mathbb R^n\) is no longer a unique invariant event. For each fixed value of \(x_0^2\), we will have a corresponding invariant event on \(\mathbb R^{n-1}\). Unless we know for sure that \(\mathbb P\) assigns probability 1 to one particular \(x_0^2\) and probability 0 to all others, we cannot conclude \((\mathbb S,\mathbb P)\) is ergodic.

Consider a transition matrix \(\mathbb P_{k\times k}\), with elements \(p_{ij}\) representing the probability of moving from state \(i\) to state \(j\) (\(\sum_{j=1}^k p_{ij}=1\)). Denote state \(i\) as a coordinate vector \(u_i=[0,\dots,1,\dots,0]'\) (1 in position \(i\), 0 everywhere else). We can represent the process \(\{\mathbf X_t\}\) with \(\mathbf X_t\in\mathbb R^k\) as realizations of the collection (linear combination) of coordinate vectors in each period \(t\). Further, we can represent any linear function \(f:\mathbb R^k\to\mathbb R\) of \(\mathbf X_t\)'s coordinates as a vector \(f_{k\times1}\), for all \(u\in\mathbb R^k\): \(f(u)=u'f_{k\times1}\). More generally we can think of conditional expectations, since $$\mathbb E[f(\mathbf X_{t+1})\mid\mathbf X_t=u]=u'\mathbb P f_{k\times1}$$

In general, to define \(\mathbb E[f(\mathbf X_{t+1})\mid\mathbf X_t=u]\) as a function of \(\mathbf X_t\), we can write that function as $$\mathbb E[f(\mathbf X_{t+1})\mid\mathbf X_t]=\mathbb P f_{k\times1}$$

There are many situations, such as in macroeconomic time series, where it is more reasonable to think about a process with a stochastic growth component as opposed to a stationary process. So we will be looking at the stationary increment process in this section. And following certain steps, we can always decompose a stationary increment process into several components including a martingale component. We are interested in this martingale decomposition because every martingale increment is a permanent shock to the stationary increment process of interest.

Notations: - Let \(\{X_t\}_{t=0}^\infty\) be a stationary and ergodic process. - Define \(\{Y_t\}_{t=0}^\infty\) by \(Y_{t+1}-Y_t=X_{t+1}\); alternatively \(Y_t=Y_0+\sum_{j=1}^t X_j\). - Denote the set of information up to period \(t\) (i.e. \(Y_0,X_1,X_2,\dots,X_t\)) by \(\mathcal F_t\).

We can do a martingale decomposition to a process \(\{Y_t\}\) whose difference process \(\{Y_{t+1}-Y_t\}\) is generated by a general stationary process \(\{X_t\}\). The algorithm follows.

• Step 1: Write down the process of interest \(Y_{t+1}-Y_t=X_{t+1}\) (12.2), where \(\{X_t\}\) is stationary (\(X_t\) is a general representation of a stationary process, including the additive-functional \(\kappa(X_t,W_{t+1})\) special case of §12.4.3 and the VAR special case \(\mathbf D\mathbf X_{t-1}+\mathbf F\mathbf W_t+\nu\) of §12.4.4).
• Step 2: Calculate \(\nu=\mathbb E[X_t]\) for all \(t\).
• Step 3: Define $$\overline X_t=\sum_{j=0}^\infty\mathbb E[X_{t+j}-\nu\mid\mathcal F_t]\tag{12.3}$$ $$\tilde X_t=\sum_{j=1}^\infty\mathbb E[X_{t+j}-\nu\mid\mathcal F_t]\tag{12.4}$$ Note \(X_{t+j}-\nu\) has zero unconditional expectation, so \(\mathbb E[\overline X_t]=0\), \(\mathbb E[\tilde X_t]=0\); also \(\overline X_t-\tilde X_t=\mathbb E[X_t-\nu\mid\mathcal F_t]=X_t-\nu\), and \(\mathbb E[\overline X_{t+1}\mid\mathcal F_t]=\tilde X_t\) (LIE).
• Step 4: Define \(M_{t+1}\equiv\overline X_{t+1}-\tilde X_t\) (12.5).
• Step 5: Decompose \(X_{t+1}\): $$X_{t+1}=(\overline X_{t+1}-\tilde X_{t+1})+\nu=\underbrace{(\overline X_{t+1}-\tilde X_t)}_{\equiv M_{t+1}}+(\tilde X_t-\tilde X_{t+1}+\nu)$$ where \(\mathbb E[M_{t+1}\mid\mathcal F_t]=\mathbb E[\overline X_{t+1}\mid\mathcal F_t]-\tilde X_t=0\).
• Step 6: Rewrite \(Y_t\) as a martingale decomposition: $$Y_t=Y_0+\sum_{j=1}^t X_j=Y_0+\sum_{j=1}^t\big[M_j+(\tilde X_{j-1}-\tilde X_j+\nu)\big]=Y_0+t\nu+\sum_{j=1}^t M_j+\big(\tilde X_0-\tilde X_t\big)$$

where: \(Y_0\) and \(\tilde X_0\) are invariant numbers; \(t\nu\) is the time trend (linear in \(t\)); \(\{\tilde X_t\}\) is stationary; \(\big\{\sum_{j=1}^t M_j\big\}\) is a martingale, since $$\mathbb E\Big[\sum_{j=1}^{t+1}M_j\mid\mathcal F_t\Big]=\underbrace{\mathbb E[M_{t+1}\mid\mathcal F_t]}_{=0}+\sum_{j=1}^t M_j=\sum_{j=1}^t M_j\tag{12.6}$$ The \(M_j\)'s are called martingale increments, and \(\{M_t\}\) is stationary.

By (12.6), \(M_j\) is the permanent shock in period \(j\) to the stationary increment process: once realized it never decays. Considering the difference in the expected detrended distance between today \(Y_t\) and the infinite future \(Y_{t+\infty}\) based on \(\mathcal F_t\) vs \(\mathcal F_{t+1}\), the difference is exactly \(M_{t+1}\) — it comes into place only because of period \(t+1\)'s realization and is still there even at the infinite horizon, so \(M_{t+1}\) is the permanent shock in period \(t+1\).

Let \(\{X_t\}_{t=1}^\infty\) be a (multivariate) Markov process, and define \(\{Y_t\}_{t=1}^\infty\) by $$Y_{t+1}-Y_t=\phi(X_{t+1})\tag{12.8}$$ Define the operator \(\mathbb T\) by $$(\mathbb T\phi)(x)=\mathbb E[\phi(X_{t+1})\mid X_t=x]$$ Let \(\mathbb L^2\) be the space of finite-second-moment functions, and \(\mathbb Z\) its zero-mean subspace: $$\mathbb L^2=\{\phi:\mathbb E[\phi(X_t)^2]<\infty\},\qquad\mathbb Z=\{\phi\in\mathbb L^2:\mathbb E[\phi(X_t)]=0\}$$ For \(\phi\in\mathbb Z\), \(\phi(X_{t+1})=\mathbb E[\phi(X_{t+1})\mid X_t]+e_{t+1}\), with \(e_{t+1}\) the estimation error uncorrelated with \(\mathbb E[\phi(X_{t+1})\mid X_t]\). So $$\operatorname{Var}(\phi(X_{t+1}))=\operatorname{Var}(\mathbb E[\phi(X_{t+1})\mid X_t])+\operatorname{Var}(e_{t+1})\ge\operatorname{Var}((\mathbb T\phi)(X_t))$$ Also \(\phi\in\mathbb Z\Rightarrow\mathbb E[\phi(X_t)]=0\), and \(\mathbb E[(\mathbb T\phi)(X_t)]=\mathbb E[\mathbb E[\phi(X_{t+1})\mid X_t]]=\mathbb E[\phi(X_{t+1})]=0\), so \(\mathbb T\phi\in\mathbb Z\). From \(\operatorname{Var}(\phi(X_{t+1}))\ge\operatorname{Var}((\mathbb T\phi)(X_t))\), $$\mathbb E[(\phi(X_t))^2]\ge\mathbb E[((\mathbb T\phi)(X_t))^2]\tag{12.9}$$ so \(\mathbb T\phi\) has zero mean and finite second moment, \(\mathbb T\phi\in\mathbb Z\). For \(\phi\in\mathbb Z\), denote \(\|\phi\|=(\mathbb E[(\phi(X_t))^2])^{1/2}\); then (12.9) implies \(\|\phi\|\ge\|\mathbb T\phi\|\) — i.e. \(\mathbb T\) is a weak contraction. Suppose we can further impose that \(\mathbb T\) is a strong contraction on \(\mathbb Z\): \(\|\mathbb T\phi\|\le\lambda\|\phi\|\), \(\lambda\in(0,1)\). Then by induction, for \(j\ge1\), \(\mathbb E[\phi(X_{t+j})\mid X_t]=(\mathbb T^j\phi)(X_t)\).

Martingale Extraction Algorithm for the Markov difference process. Suppose \(\{Y_t\}\)'s difference process is generated by a Markov process \(\{X_t\}\) and i.i.d. \(\{W_{t+1}\}\): \(Y_{t+1}-Y_t=\kappa(X_t,W_{t+1})\) (corresponding to \(X_{t+1}\) in the general case), \(X_{t+1}=\psi(X_t,W_{t+1})\); denote current-period variables \(x\equiv X_t\), \(w\equiv W_t\), and next-period variables \(x^\star\equiv X_{t+1}\), \(w^\star\equiv W_{t+1}\). The algorithm extracts the martingale increment in two parts: 1. Extract martingale from the deviation of the additive functional from its conditional mean. Define the conditional mean \(\bar f(x)=\mathbb E[\kappa(x,w^\star)\mid x]\) and the deviation \(\kappa_2(x,w^\star)=\kappa(x,w^\star)-\bar f(x)\). Then \(\mathbb E[\kappa_2(x,w^\star)\mid x]=\bar f(x)-\bar f(x)=0\), so \(\kappa_2\) is part of the martingale increment. 2. Extract martingale from the deviation of the conditional mean from the unconditional mean. Define \(\nu=\mathbb E[\kappa(x,w^\star)]=\mathbb E[\mathbb E[\kappa(x,w^\star)\mid x]]\) and the deviation \(f(x)=\bar f(x)-\nu\); define \(g(x)=f(x)+\mathbb T g(x)\), so \(g(x)=(\mathbb I-\mathbb T)^{-1}f(x)=\sum_{j=0}^\infty(\mathbb T^j f)(x)=\sum_{j=0}^\infty\mathbb E[f(X_{t+j})\mid X_t=x]\); define $$\kappa_1(x,w^\star)\equiv\sum_{j=0}^\infty\mathbb E[f(X_{t+1+j})\mid X_{t+1}=\psi(x,w^\star)]-\sum_{j=0}^\infty\mathbb E[f(X_{t+1+j})\mid X_t=x]=g(\psi(x,w^\star))-g(x)+f(x)$$ one can show \(\mathbb E[\kappa_1(x,w^\star)\mid x]=0\), so \(\kappa_1\) is also part of the martingale increment. 3. Put the two parts together and rewrite \(\{Y_t\}\): $$\kappa(x,w^\star)=\nu+f(x)+\kappa_2(x,w^\star)=\nu+(\kappa_1(x,w^\star)-g(\psi(x,w^\star))+g(x))+\kappa_2(x,w^\star)=\nu+\underbrace{(\kappa_1+\kappa_2)}_{\text{mart. increment}}-g(\psi(x,w^\star))+g(x)$$ $$\Rightarrow Y_t=Y_0+t\nu+\sum_{j=1}^t\kappa_a(X_{j-1},W_j)-g(X_t)+g(X_0)$$ where \(\sum_{j=1}^t\kappa_a(X_{j-1},W_j)\) is the martingale component, \([-g(X_t)]\) is the stationary part, \(t\nu\) is the trend, and \(g(X_0)+Y_0\) is the invariant part. This algorithm is consistent with the general case and with the even more special VAR Markov case below.

Specializing to the VAR Markov process \(\{\mathbf X_t\}\) generating \(\{Y_t\}\)'s difference process: $$\mathbf X_{t+1}=\mathbf A\mathbf X_t+\mathbf B\mathbf W_{t+1},\quad\mathbf W_{t+1}\sim N(0,\mathbf I)\ \text{i.i.d.},\ \mathbf A\text{ stable}$$ Define $$\mathbf Y_{t+1}-\mathbf Y_t=\mathbf D\mathbf X_t+\mathbf F\mathbf W_{t+1}+\nu$$ Let \(\mathbf H_{t+1}=\mathbf D\mathbf X_t+\mathbf F\mathbf W_{t+1}\); then \(\mathbf Y_{t+1}-\mathbf Y_t=\mathbf H_{t+1}+\nu\).

Define \(\overline{\mathbf H}_t=\mathbb E[\sum_{j=0}^\infty\mathbf H_{t+j}\mid\mathcal F_t]=\mathbf D\mathbf X_{t-1}+\mathbf F\mathbf W_t+\mathbf D\mathbf X_t+\mathbf D\mathbf A\mathbf X_t+\dots=\mathbf D\mathbf X_{t-1}+\mathbf F\mathbf W_t+\mathbf D(\mathbf I-\mathbf A)^{-1}\mathbf X_t\). Then the martingale increment is $$\overline{\mathbf H}_{t+1}-\mathbb E[\overline{\mathbf H}_{t+1}\mid\mathcal F_t]=(\mathbf F+\mathbf D(\mathbf I-\mathbf A)^{-1}\mathbf B)\mathbf W_{t+1}$$

By the martingale extraction of §12.4.3, putting everything together gives the VAR martingale decomposition $$\mathbf Y_t=\mathbf Y_0+t\nu+\Big[\sum_{j=1}^t(\mathbf F+\mathbf D(\mathbf I-\mathbf A)^{-1}\mathbf B)\mathbf W_j\Big]+\mathbf D(\mathbf I-\mathbf A)^{-1}(\mathbf X_0-\mathbf X_t)\tag{12.10}$$ Comparing with the general decomposition (12.7): \((\mathbf F+\mathbf D(\mathbf I-\mathbf A)^{-1}\mathbf B)\mathbf W_j\) maps into \(M_j\) and \(\mathbf D(\mathbf I-\mathbf A)^{-1}\mathbf X_t\) maps into \(\tilde X_t\). So \((\mathbf F+\mathbf D(\mathbf I-\mathbf A)^{-1}\mathbf B)\mathbf W_j\) is the martingale increment, i.e. the permanent shock.

Permanent shock. Consider the impulse response of \(\mathbf Y_t\) to \(\mathbf W_1\): suppose \(\mathbf X_0=\mathbf 0\) and \(\mathbf W_t=0\) for all \(t\ge2\); by (12.10), $$\mathbf Y_t=\mathbf Y_0+t\nu+(\mathbf F+\mathbf D(\mathbf I-\mathbf A)^{-1}\mathbf B)\mathbf W_1-\mathbf D(\mathbf I-\mathbf A)^{-1}\mathbf X_t\tag{12.13}$$ Even as \(t\to\infty\), the response of \(\mathbf Y_t\) to \(\mathbf W_1\) is always \((\mathbf F+\mathbf D(\mathbf I-\mathbf A)^{-1}\mathbf B)\mathbf W_1\) (never decays), so \(\mathbf W_1\) is a permanent shock with permanent response \((\mathbf F+\mathbf D(\mathbf I-\mathbf A)^{-1}\mathbf B)\mathbf W_1\). The above is the definition of permanent shock to the vector \(\mathbf Y_t\); we can also consider it element-wise: let \(Y_t^j,\nu^j,F^j,D^j\) be the \(j\)-th rows of \(\mathbf Y_t,\nu,\mathbf F,\mathbf D\) (\(Y_t^j,\nu^j\) scalars, \(F^j,D^j\) row vectors); by (12.10), $$Y_t^j=Y_0^j+t\nu^j+\Big[\sum_{j=1}^t(F^j+D^j(\mathbf I-\mathbf A)^{-1}\mathbf B)\mathbf W_j\Big]+D^j(\mathbf I-\mathbf A)^{-1}(\mathbf X_0-\mathbf X_t)$$ The permanent shock \(\mathbf W^j_t\) to the \(j\)-th element of \(\mathbf Y_t\) is parallel to \((F^j+D^j(\mathbf I-\mathbf A)^{-1}\mathbf B)'\); the permanent response is the norm of the vector \((F^j+D^j(\mathbf I-\mathbf A)^{-1}\mathbf B)\).

Identification. \((\mathbf F+\mathbf D(\mathbf I-\mathbf A)^{-1}\mathbf B)\) determines the permanent response; matrices \(\mathbf A,\mathbf D\) can be given by theory or estimated, but \(\mathbf F,\mathbf B\) cannot be identified without further assumptions — there is arbitrariness that we cannot get rid of. We can stack rows in \(\mathbf B\) to reduce the number of elements to estimate: suppose \(\mathbf X_t\) is \(n\times1\), \(\mathbf A\) is \(n\times n\), \(\mathbf W_t\) is \(m\times1\), \(\mathbf B\) is \(n\times m\) (\(n>m\)), \(\operatorname{rank}(\mathbf B)=m\); then a \(n\times n\) matrix \(\mathbf R\) stacks rows in \(\mathbf B\) into \(\mathbf{RB}=\begin{bmatrix}\mathbf B_1\\\mathbf 0_{(n-m)\times m}\end{bmatrix}\) (\(\mathbf B_1\) being \(m\times m\)). Then $$\mathbf{RX}_{t+1}=\mathbf{RAX}_t+\mathbf{RBW}_{t+1}\Rightarrow\mathbf Z_{t+1}=\mathbf C\mathbf Z_t+\begin{bmatrix}\mathbf B_1\\\mathbf 0\end{bmatrix}\mathbf W_{t+1}$$ (\(\mathbf Z_{t+1}=\mathbf{RX}_{t+1}\), \(\mathbf C=\mathbf{RAR}^{-1}\).) The error term \(\mathbf Z_{t+1}-\mathbf C\mathbf Z_t\) is observable, with estimable variance-covariance matrix \(\begin{bmatrix}\mathbf B_1\\\mathbf 0\end{bmatrix}\mathbf I\begin{bmatrix}\mathbf B_1'&\mathbf 0\end{bmatrix}=\begin{bmatrix}\mathbf B_1\mathbf B_1'&\mathbf 0\\\mathbf 0&\mathbf 0\end{bmatrix}\). But we can only estimate \(\mathbf B_1\mathbf B_1'\), not \(\mathbf B_1\) itself — since any \(m\times m\) orthogonal \(\mathbf Q\) (\(\mathbf{QQ}'=\mathbf I\)) gives \(\hat{\mathbf B}_1=\mathbf B_1\mathbf Q\) with \(\hat{\mathbf B}_1\hat{\mathbf B}_1'=\mathbf B_1\mathbf B_1'\). As \(\mathbf Q\) is arbitrary, we cannot pin down \(\hat{\mathbf B}_1\). But \(\mathbf B\) (or \(\mathbf B_1\)) is a crucial component of \((\mathbf F+\mathbf D(\mathbf I-\mathbf A)^{-1}\mathbf B)\), determining the permanent response. So we must use the techniques of §10.3 (Cholesky, long-run identification, sign restrictions) to restrict the scope of candidates for \(\mathbf B\).

Our goal is to identify permanent shocks. Consider a simple permanent income model $$K_{t+1}-K_t+C_t=[\exp(\rho)-1]K_t+Y_t$$ where \(Y_t\) is time-\(t\) income (\(\{\log Y_t\}_{t=0}^\infty\) a stationary increment process), \(K_t\) is time-\(t\) capital, \(\rho\) is a productivity parameter (\([\exp(\rho)-1]K_t\approx\rho K_t\) is production from capital \(K_t\)), and \(C_t\) is time-\(t\) consumption (from the macro model one can show \(\{\log C_t\}_{t=0}^\infty\) is a martingale). The objective is \(\sum_{t=0}^\infty\beta^t\log C_t\). Define \(\hat K_t=K_t/Y_t\), \(\hat C_t=\log C_t-\log Y_t\), and rewrite by dividing through by \(Y_t\): $$\hat K_{t+1}\frac{Y_{t+1}}{Y_t}-\hat K_t+\exp(\hat C_t)=[\exp(\rho)-1]\hat K_t+1$$ Specialize the state process \(\{X_t\}\) to a stationary Markov process (scalar as a special case of vector), using the VAR results $$X_{t+1}=AX_t+\mathbf{BW}_{t+1},\qquad\log(Y_{t+1})-\log(Y_t)=\nu+DX_t+\mathbf{FW}_{t+1}$$ with \(A\) stable ($<1$), \(\mathbf W_{t+1}\sim N(0,\mathbf I)\) i.i.d., \(\mathbf B,\mathbf F\) being \(k\times1\), and \(\mathbf W_{t+1}\) a \(k\times1\) individual-shock vector. Assume \(X_0=0\), \(\mathbf W_1\ne\mathbf 0\), \(\mathbf W_t=\mathbf 0\) (\(t\ne1\)), and consider the impulse response to \(\mathbf W_1\): $$\log(Y_t)=\log(Y_0)+t\nu+(\mathbf F+D(\mathbf I-A)^{-1}\mathbf B)\mathbf W_1-D(\mathbf I-A)^{-1}A^{t-1}X_t\xrightarrow{t\to\infty}\log(Y_0)+t\nu+(\mathbf F+D(\mathbf I-A)^{-1}\mathbf B)\mathbf W_1$$ (In a macro setting one would also linearize the system and solve for policy functions for consumption and saving; this step is skipped here.) What's important is the differential responses to transitory vs permanent shocks: if the shock \(\mathbf W_1\) is orthogonal to \((\mathbf F+D(\mathbf I-A)^{-1}\mathbf B)\), it has no effect on long-term income (transitory); if parallel, the effect on long-term income is persistent (permanent). Intuitively, transitory responses (e.g. \(-D(\mathbf I-A)^{-1}A^{t-1}\mathbf W_1\) of income \(Y_t\) to \(\mathbf W_1\)) are offset with consumption and savings decisions and won't permanently change income; the permanent responses, once \(\mathbf W_1\) is realized, permanently change the level of income.

Figures (scatter/path plots, paraphrased): under a permanent income shock log income gradually rises to a new level and log consumption immediately jumps to a new level and stays there (the response is permanent); under a transitory income shock log income spikes and then decays back to the original level while log consumption barely moves (the response is transitory).

In the prior section we produced a decomposition with (i) a time trend and (ii) a martingale component. Now we exploit that decomposition and a variant of the Central Limit Theorem (CLT) to characterize the limiting behavior of a stationary increment process. The next theorem extends the CLT from the i.i.d. case to martingale differences. Recall the set-up of a stationary increment process: \(Y_{t+1}-Y_t=X_{t+1}\) (\(\{X_t\}\) stationary, ergodic, \(\mathbb E[X_t]=\nu\)), \(\overline X_t=\sum_{j=0}^\infty\mathbb E[X_{t+j}-\nu\mid\mathcal F_t]\), \(M_t=\overline X_t-\tilde X_{t-1}=\overline X_t-\mathbb E[\overline X_t\mid\mathcal F_{t-1}]\), \(Y_t-Y_0=\sum_{j=1}^t M_j-\tilde X_t+\tilde X_0+t\nu\), \(\mathbb E[M_{t+1}\mid\mathcal F_t]=0\).

This characterizes the limiting distribution of \(Y_t\). By the martingale decomposition, $$Y_t-Y_0-t\nu=\sum_{j=1}^t M_j+(\tilde X_0-\tilde X_t)\Rightarrow\frac1{\sqrt t}(Y_t-Y_0-t\nu)=\frac1{\sqrt t}\Big(\sum_{j=1}^t M_j\Big)-\frac1{\sqrt t}\tilde X_t+\frac1{\sqrt t}\tilde X_0$$ The first term on the RHS converges to a normal by the CLT, and the other two converge to zero in probability, so $$\frac1{\sqrt t}(Y_t-Y_0-t\nu)\xrightarrow{d}N\big(0,\mathbb E[(M_j)^2]\big)$$

Observations: 1. \(\operatorname{Var}(M_t)=\mathbb E[(M_t)^2]\ne\operatorname{Var}(X_t)\). 2. \(\operatorname{Var}\big(\frac1{\sqrt t}\sum_{j=1}^t(X_j-\nu)\big)=\operatorname{Var}\big(\frac1{\sqrt t}(Y_t-Y_0-t\nu)\big)\to\operatorname{Var}(M_t)=\mathbb E[(M_j)^2]\). - \(\operatorname{Var}(M_t)\) is the spectral density of \(\{X_t-\nu\}_{t=-\infty}^\infty\) at frequency zero: by the definition of spectral density \(s_x(\omega)=\frac1{2\pi}\sum_{j=-\infty}^\infty\gamma_j e^{-i\omega j}\), at \(\omega=0\), \(s_x(0)=\frac1{2\pi}\sum_{j=-\infty}^\infty\gamma_j\).

Consider the process $$Y_t=r_1 Y_t^1+r_2 Y_t^2$$ where $$Y_t^1=Y_0^1+t\nu_1+\sum_{j=1}^t M_j^1+(\tilde X_0^1-\tilde X_t^1),\qquad Y_t^2=Y_0^2+t\nu_2+\sum_{j=1}^t M_j^2+(\tilde X_0^2-\tilde X_t^2)$$ We can write a similar decomposition for \(Y_t\): $$Y_t=(r_1 Y_0^1+r_2 Y_0^2)+t(r_1\nu_1+r_2\nu_2)+\sum_{j=1}^t(r_1 M_j^1+r_2 M_j^2)+\big((r_1\tilde X_0^1+r_2\tilde X_0^2)-(r_1\tilde X_t^1+r_2\tilde X_t^2)\big)$$ We say \(Y^1\) and \(Y^2\) are cointegrated if \(\exists r_1,r_2\ne0\) s.t. $$r_1\nu_1+r_2\nu_2=0\quad\text{and}\quad r_1 M_t^1+r_2 M_t^2=0,\ \forall t$$ \((r_1,r_2)\) is the cointegrating vector. Plugging in, $$Y_t=(r_1 Y_0^1+r_2 Y_0^2)+\big((r_1\tilde X_0^1+r_2\tilde X_0^2)-(r_1\tilde X_t^1+r_2\tilde X_t^2)\big)$$ all of which are stationary (the trend and permanent-shock terms are eliminated by the cointegrating vector). So when \(Y_t^1\) and \(Y_t^2\) are cointegrated, \(Y_{t+1}-Y_t\) contains only stationary components.

Let \(\{\mathbf W_{t+1}\}\) be a process of shocks, \(\mathbb E[\mathbf W_{t+1}\mid\mathcal F_t]=0\). Define the state process \(\{\mathbf X_t\}\) as a stationary Markov process $$\mathbf X_{t+1}=\phi(\mathbf X_t,\mathbf W_{t+1})$$ with \(\mathbf W_{t+1}\sim N(0,\mathbf I)\) i.i.d. and \(\mathbf X_t\) the state at time \(t\). Construct the observation process \(\{\mathbf Y_t\}\) as a function of state and shock: $$\mathbf Y_{t+1}-\mathbf Y_t=\kappa(\mathbf X_t,\mathbf W_{t+1})$$ Assumptions: (1) \(\mathbf X_0\) is observed; (2) there exists a known function \(\chi\) s.t. \(\mathbf W_{t+1}=\chi(\mathbf X_t,\mathbf Y_{t+1}-\mathbf Y_t)\) (the inversion assumption: we can back out \(\mathbf W_{t+1}\) from \(\mathbf X_t\) and \(\mathbf Y_{t+1}-\mathbf Y_t\)); (3) apply \(\phi\) to back out \(\mathbf X_{t+1}\) from \(\mathbf X_t\) and \(\mathbf W_{t+1}\); since \(\mathbf X_0\) is observed, we can use induction to back out all terms in \(\{\mathbf X_t\}\) — i.e. we have revealed states. Assume \(\mathbf Y_{t+1}-\mathbf Y_t\) has a density \(\psi(\cdot\mid\mathbf x)\) w.r.t. measure \(\tau\) conditional on \(\mathbf X_t=\mathbf x\).

This constructs the (conditional) likelihood process \(\{L_t\}\): $$L_t=\prod_{j=1}^t\psi(\mathbf Y_j-\mathbf Y_{j-1}\mid\mathbf X_{j-1})$$ with log-likelihood $$\log L_t=\sum_{j=1}^t\log\psi(\mathbf Y_j-\mathbf Y_{j-1}\mid\mathbf X_{j-1})$$ \(\log L_t\) has stationary increments (the increment \(\log\psi(\mathbf Y_j-\mathbf Y_{j-1}\mid\mathbf X_{j-1})\) is drawn from a stationary distribution), so we can do a martingale decomposition on \(\log L_t\).

Let \(\theta,\theta_0\) be two candidates for a parameter in the model (true value \(\theta_0\)). The density has two versions \(\psi(\cdot\mid\mathbf x,\theta)\), \(\psi(\cdot\mid\mathbf x,\theta_0)\), with two likelihood processes $$L_t(\theta)=\prod_{j=1}^t\psi(\mathbf Y_j-\mathbf Y_{j-1}\mid\mathbf X_{j-1},\theta),\qquad L_t(\theta_0)=\prod_{j=1}^t\psi(\mathbf Y_j-\mathbf Y_{j-1}\mid\mathbf X_{j-1},\theta_0)$$ Then $$\frac{L_{t+1}(\theta)}{L_{t+1}(\theta_0)}=\frac{L_t(\theta)}{L_t(\theta_0)}\cdot\frac{\psi(\mathbf Y_{t+1}-\mathbf Y_t\mid\mathbf X_t,\theta)}{\psi(\mathbf Y_{t+1}-\mathbf Y_t\mid\mathbf X_t,\theta_0)}\tag{12.15}$$ Take \(\mathbf y^\star\in\mathcal Y\) as the value of \(\mathbf Y_{t+1}-\mathbf Y_t\), and consider the conditional expectation of the last term on the RHS of (12.15): $$\mathbb E_{\theta_0}\Big[\frac{\psi(\mathbf Y_{t+1}-\mathbf Y_t\mid\mathbf X_t,\theta)}{\psi(\mathbf Y_{t+1}-\mathbf Y_t\mid\mathbf X_t,\theta_0)}\mid\mathcal F_t\Big]=\int_{\mathcal Y}\frac{\psi(\mathbf y^\star\mid\mathbf x,\theta)}{\psi(\mathbf y^\star\mid\mathbf x,\theta_0)}\psi(\mathbf y^\star\mid\mathbf x,\theta_0)\tau(d\mathbf y^\star)=\int_{\mathcal Y}\psi(\mathbf y^\star\mid\mathbf x,\theta)\tau(d\mathbf y^\star)=1$$ Plugging back into (12.15), $$\mathbb E_{\theta_0}\Big[\frac{L_{t+1}(\theta)}{L_{t+1}(\theta_0)}\mid\mathcal F_t\Big]=\frac{L_t(\theta)}{L_t(\theta_0)}\tag{12.16}$$ i.e. the process \(\big\{\frac{L_t(\theta)}{L_t(\theta_0)}\big\}_{t=1}^\infty\) is a positive martingale.

Since \(\log\) is strictly concave, by Jensen's inequality \(\mathbb E[\log(a)]\le\log(\mathbb E[a])\), so $$\mathbb E_{\theta_0}\Big[\log\Big(\frac{L_{t+1}(\theta)}{L_{t+1}(\theta_0)}\Big)\mid\mathcal F_t\Big]\le\log\Big(\mathbb E_{\theta_0}\Big[\frac{L_{t+1}(\theta)}{L_{t+1}(\theta_0)}\mid\mathcal F_t\Big]\Big)$$ Taking \(\log\) of both sides of (12.16), $$\mathbb E_{\theta_0}\Big[\log\Big(\frac{L_{t+1}(\theta)}{L_{t+1}(\theta_0)}\Big)\mid\mathcal F_t\Big]\le\log\Big(\frac{L_t(\theta)}{L_t(\theta_0)}\Big)\tag{12.17}$$ By strict concavity of \(\log\), (12.17) holds with equality only when \(\theta_0=\theta\). So the process \(\big\{\log\big(\frac{L_t(\theta)}{L_t(\theta_0)}\big)\big\}_{t=1}^\infty\) is a super-martingale.

Define $$v(\theta)=\mathbb E_{\theta_0}\big[\log(\psi(\mathbf Y_{t+1}-\mathbf Y_t\mid\mathbf X_t,\theta))-\log(\psi(\mathbf Y_{t+1}-\mathbf Y_t\mid\mathbf X_t,\theta_0))\big]$$ which is an unconditional expectation (not conditional on \(\mathcal F_t\)).

Note \(v(\theta)\le0\) is strict unless \(\theta_0=\theta\) or \(\psi(\cdot\mid\mathbf X_t,\theta)=\psi(\cdot\mid\mathbf X_t,\theta_0)\) (not true in general). By stationarity, applying the Birkhoff LLN to \(\log(\psi(\mathbf Y_j-\mathbf Y_{j-1}\mid\mathbf X_{j-1},\theta))\) and \(\log(\psi(\cdots\theta_0))\): $$\frac1t(\log(L_t(\theta))-\log(L_t(\theta_0)))=\frac1t\sum_{j=1}^t[\log(\psi(\cdots\theta))-\log(\psi(\cdots\theta_0))]\to\mathbb E_{\theta_0}[\log(\psi(\cdots\theta))-\log(\psi(\cdots\theta_0))]=v(\theta)$$ i.e. \(\frac1t(\log(L_t(\theta))-\log(L_t(\theta_0)))\to v(\theta)\). As long as \(\theta_0\ne\theta\), \(v(\theta)<0\), so $$\frac1t(\log(L_t(\theta))-\log(L_t(\theta_0)))<0\Rightarrow\log(L_t(\theta))-\log(L_t(\theta_0))\to-\infty\Rightarrow\frac{L_t(\theta)}{L_t(\theta_0)}\to0$$

By §12.6.4, \(v(\theta)=\mathbb E_{\theta_0}[\log(\psi(\mathbf Y_{t+1}-\mathbf Y_t\mid\mathbf X_t,\theta))]\) (dropping terms unrelated to \(\theta\)) is maximized at \(\theta=\theta_0\), so the first-order condition is $$\int_{\mathcal Y}\frac{\partial\log(\psi(\mathbf y^\star\mid\mathbf x,\theta))}{\partial\theta}\Big|_{\theta=\theta_0}\psi(\mathbf y^\star\mid\mathbf x,\theta_0)\tau(d\mathbf y^\star)=0\tag{12.19}$$

Since the increments of \(\{S_t\}\) are drawn from a stationary distribution (by stationarity of \(\mathbf X_t\) and i.i.d. of \(\mathbf W_{t+1}\)), we can apply the Birkhoff LLN and Billingsley CLT to the stationary-increment martingale process \(\{S_t\}\): $$\frac1t S_t\to\mathbb E_{\theta_0}[S_{t+1}-S_t\mid\mathcal F_t]=0\tag{12.20}$$ $$\frac1{\sqrt t}(S_t)\xrightarrow{d}N\big(0,\mathbb E[(S_{t+1}-S_t)(S_{t+1}-S_t)']\big)\tag{12.21}$$ where \(\mathbb E[(S_{t+1}-S_t)(S_{t+1}-S_t)']\) is exactly the variance-covariance matrix of \(\frac{\partial\log(\psi(\mathbf Y_{t+1}-\mathbf Y_t\mid\mathbf X_t,\theta))}{\partial\theta}\big|_{\theta=\theta_0}\), also called Fisher information. We also have $$\sqrt t\big(\theta_t^{\text{MLE}}-\theta_0\big)\xrightarrow{d}N\Big(0,\mathbb E[(S_{t+1}-S_t)(S_{t+1}-S_t)']^{-1}\Big)\tag{12.22}$$ where \(\theta_t^{\text{MLE}}\) is the maximum likelihood estimator based on data up to period \(t\).

Suppose there are multiple unknown parameters in the model, and we are only interested in one of them. Let \(\theta_0\) (a scalar) denote the parameter of interest, and stack all other unknown parameters into a vector \(\tilde\theta_0\). Since we are not interested in the parameters in \(\tilde\theta_0\), they are called nuisance parameters. We will illustrate that the more nuisance parameters we have, the less information about the parameter of interest.

Define the score process \(\{S_t\}_{t=0}^\infty\) for \(\theta_0\) (a process of random scalars) and the score process \(\{\tilde S_t\}_{t=0}^\infty\) for \(\tilde\theta_0\) (a process of random vectors), and stack \(S_t\) and \(\tilde S_t\) into a single vector: $$V=\mathbb E_{\theta_0,\tilde\theta_0}\Big[\begin{pmatrix}S_{t+1}-S_t\\\tilde S_{t+1}-\tilde S_t\end{pmatrix}\begin{pmatrix}S_{t+1}-S_t\\\tilde S_{t+1}-\tilde S_t\end{pmatrix}'\Big]\tag{12.23}$$ Then $$\sqrt t\Big(\begin{pmatrix}\theta_0^{\text{MLE}}\\\tilde\theta_0^{\text{MLE}}\end{pmatrix}-\begin{pmatrix}\theta_0\\\tilde\theta_0\end{pmatrix}\Big)\xrightarrow{d}N(0,V^{-1})$$ where the $(1,1)$ entry of \(V^{-1}\) is the variance of \(\theta_0^{\text{MLE}}\). To compute \(V^{-1}\), run a population least-square regression $$S_{t+1}-S_t=\beta'\big(\tilde S_{t+1}-\tilde S_t\big)+u_{t+1}\tag{12.24}$$ where \(u_{t+1}\) is a scalar residual uncorrelated with \(\tilde S_{t+1}-\tilde S_t\). Rewriting (12.24) and (12.23), and using the block-matrix inverses $$\begin{bmatrix}1&\beta'\\\mathbf 0&\mathbf I\end{bmatrix}^{-1}=\begin{bmatrix}1&-\beta'\\\mathbf 0&\mathbf I\end{bmatrix},\qquad\begin{bmatrix}1&\mathbf 0\\\beta&\mathbf I\end{bmatrix}^{-1}=\begin{bmatrix}1&\mathbf 0\\-\beta&\mathbf I\end{bmatrix}$$ we get $$V^{-1}=\begin{bmatrix}1&\mathbf 0\\-\beta&\mathbf I\end{bmatrix}\begin{bmatrix}\frac1{\mathbb E_{\theta_0,\tilde\theta_0}[u_{t+1}^2]}&\mathbf 0\\\mathbf 0&\mathbb E_{\theta_0,\tilde\theta_0}[(\tilde S_{t+1}-\tilde S_t)(\tilde S_{t+1}-\tilde S_t)']^{-1}\end{bmatrix}\begin{bmatrix}1&-\beta'\\\mathbf 0&\mathbf I\end{bmatrix}$$ so the $(1,1)$ entry of \(V^{-1}\) is \(\frac1{\mathbb E_{\theta_0,\tilde\theta_0}[u_{t+1}^2]}\) — exactly the Fisher information for estimating \(\theta_0\) with \(\tilde\theta_0\) unknown. The more nuisance parameters, the more \((\tilde S_{t+1}-\tilde S_t)\) explains and the smaller the residual variance \(\mathbb E[u_{t+1}^2]\), so the higher the Fisher information \(\frac1{\mathbb E[u_{t+1}^2]}\) and the higher the variance of \(\theta_0^{\text{MLE}}\).

If instead we know all the nuisance parameters \(\tilde\theta_0\), the Fisher information for \(\theta_0\) (a scalar) is \(\mathbb E_{\theta_0}[(S_{t+1}-S_t)^2]\) — the variance of the score-process increment. The important observation: $$\mathbb E_{\theta_0}[(S_{t+1}-S_t)^2]=\beta'\mathbb E_{\theta_0}\big[(\tilde S_{t+1}-\tilde S_t)(\tilde S_{t+1}-\tilde S_t)'\big]\beta+\mathbb E_{\theta_0,\tilde\theta_0}[u_{t+1}^2]\ge\mathbb E_{\theta_0,\tilde\theta_0}[u_{t+1}^2]$$ $$\Rightarrow\frac1{\mathbb E_{\theta_0}[(S_{t+1}-S_t)^2]}\le\frac1{\mathbb E_{\theta_0,\tilde\theta_0}[u_{t+1}^2]}$$ i.e. the Fisher information when nuisance parameters are known is greater than when they are unknown, so the variance of \(\theta_0^{\text{MLE}}\) is smaller.

§12.6 assumed the initial state \(\mathbf X_0\) was known and made the inversion assumption (a function \(\chi\) s.t. \(\mathbf W_{t+1}=\chi(\mathbf X_t,\mathbf Y_{t+1}-\mathbf Y_t)\), \(\mathbf X_{t+1}=\phi(\mathbf X_t,\mathbf W_{t+1})\)), so we could iteratively back out the entire sequence of shocks \(\{\mathbf W_{t+1}\}_{t=0}^\infty\): $$\mathbf W_1=\chi(\mathbf X_0,\mathbf Y_1-\mathbf Y_0),\quad\mathbf X_1=\phi(\mathbf X_0,\mathbf W_1),\quad\mathbf W_2=\chi(\mathbf X_1,\mathbf Y_2-\mathbf Y_1),\quad\dots$$ Now consider a more general setting: we do not observe the initial state \(\mathbf X_0\).

12.7.1 Regime Switching Model

• \(\{\mathbf X_t\}\) is an \(n\)-state Markov process, \(\mathbf X_t\) modeling the state at time \(t\); the realized state is an \(n\times1\) coordinate vector (if at time \(t\) the system is in state \(j\), then \(\mathbf X_t\) has zeros everywhere except a one at position \(j\)).
• \(\{\mathbf X_t\}\) evolves according to an \(n\times n\) transition matrix \(\mathbf P\), $$\mathbf P=\begin{bmatrix}\pi_{11}&\pi_{12}&\cdots&\pi_{1n}\\\pi_{21}&\pi_{22}&\cdots&\pi_{2n}\\\vdots&\vdots&\ddots&\vdots\\\pi_{n1}&\pi_{n2}&\cdots&\pi_{nn}\end{bmatrix},\quad\mathbf P\mathbf 1_n=\mathbf 1_n$$ the \((i,j)\) element \(\pi_{ij}\) being the probability of transferring to state \(j\) from state \(i\).
• Densities \(\{\psi_1,\dots,\psi_n\}\), where \(\mathbf Y_{t+1}-\mathbf Y_t\) has density \(\psi_j\) if state \(j\) is realized (these state-contingent densities are time-invariant).
• \(\mathbf Y^t=(\mathbf Y_1-\mathbf Y_0,\mathbf Y_2-\mathbf Y_1,\dots,\mathbf Y_t-\mathbf Y_{t-1})'\) is the vector of observations.
• \(\mathbf Q_0\) is a probability vector for \(\mathbf X_0\) (a "best guess" of the initial state).
• \(\mathbf Q_t\) is a conditional probability, $$\mathbf Q_t=\begin{bmatrix}q_t^1\\q_t^2\\\vdots\\q_t^n\end{bmatrix}=\mathbb P(\mathbf X_t\mid\mathbf Y^t,\mathbf Q_0)$$ where \(q_t^i\) gives the probability that the current state is the \(i\)-th state. Using Bayes' rule, write \(\mathbf Q_{t+1}\) as $$\begin{aligned}\mathbf Q_{t+1}&=\mathbb P(\mathbf X_{t+1}\mid\mathbf Y^{t+1},\mathbf Q_0)=\mathbb P(\mathbf X_{t+1}\mid\mathbf Y_{t+1}-\mathbf Y_t,\mathbf Y^t,\mathbf Q_0)\\&=\frac{\mathbb P(\mathbf X_{t+1},\mathbf Y_{t+1}-\mathbf Y_t\mid\mathbf Y^t,\mathbf Q_0)}{\mathbb P(\mathbf Y_{t+1}-\mathbf Y_t\mid\mathbf Y^t,\mathbf Q_0)}=\frac{\mathbb P(\mathbf X_{t+1},\mathbf Y_{t+1}-\mathbf Y_t\mid\mathbf Q_t)}{\mathbb P(\mathbf Y_{t+1}-\mathbf Y_t\mid\mathbf Q_t)}\end{aligned}\tag{12.25}$$ Finally, assume \(\mathbf X_{t+1}\) and \(\mathbf Y_{t+1}-\mathbf Y_t\) are independent conditional on \(\mathbf X_t\) for all \(t\). We characterize \(\mathbf Q_{t+1}\) explicitly in four steps.

Step 1: joint distribution of \((\mathbf X_{t+1},\mathbf Y_{t+1}-\mathbf Y_t)\) conditional on \(\mathbf X_t\). By conditional independence, $$\begin{aligned}&\underbrace{\mathbb P(\mathbf X_{t+1}\mid\mathbf X_t=\mathbf x_i)}_{\mathbf X_{t+1}\text{ density}}\underbrace{\mathbb P(\mathbf Y_{t+1}-\mathbf Y_t\mid\mathbf X_t=\mathbf x_i)}_{\mathbf Y_{t+1}-\mathbf Y_t\text{ density}}=\underbrace{\mathbf P'\mathbf x_i}_{=\mathbf x_i}\times\underbrace{\mathbf x_i'\operatorname{vec}\{\psi_i(\mathbf y^\star)\}}_{=\mathbf x_i'}\\&=\begin{bmatrix}\pi_{i1}\\\pi_{i2}\\\vdots\\\pi_{in}\end{bmatrix}\times\psi_i(\mathbf y^\star)=\begin{bmatrix}\pi_{i1}\psi_i(\mathbf y^\star)\\\pi_{i2}\psi_i(\mathbf y^\star)\\\vdots\\\pi_{in}\psi_i(\mathbf y^\star)\end{bmatrix}\end{aligned}$$ where \(\mathbf y^\star\equiv\mathbf Y_{t+1}-\mathbf Y_t\) for all \(t\).

Step 2: joint distribution of \((\mathbf X_{t+1},\mathbf Y_{t+1}-\mathbf Y_t)\) conditional on \(\mathbf Q_t\): $$\mathbb P(\mathbf X_{t+1},\mathbf Y_{t+1}-\mathbf Y_t\mid\mathbf Q_t)=\begin{bmatrix}q_t^1\pi_{11}\psi_1(\mathbf y^\star)+\dots+q_t^n\pi_{n1}\psi_n(\mathbf y^\star)\\q_t^1\pi_{12}\psi_1(\mathbf y^\star)+\dots+q_t^n\pi_{n2}\psi_n(\mathbf y^\star)\\\vdots\\q_t^1\pi_{1n}\psi_1(\mathbf y^\star)+\dots+q_t^n\pi_{nn}\psi_n(\mathbf y^\star)\end{bmatrix}\tag{12.26}$$ This can also be rewritten: since \(\mathbf Q_t\) is the vector of conditional probabilities for the current state, \(\mathbb E[\mathbf X_t\mathbf X_t'\mid\mathbf Q_t]=q_t^1\operatorname{diag}[\mathbf e_1]+\dots+q_t^n\operatorname{diag}[\mathbf e_n]=\operatorname{diag}[\mathbf Q_t]\) (\(\operatorname{diag}[\mathbf e_i]\) has 1 only in position \((i,i)\)), so $$\mathbb P(\mathbf X_{t+1},\mathbf Y_{t+1}-\mathbf Y_t\mid\mathbf Q_t)=\mathbb E[\mathbf P'\mathbf X_t\mathbf X_t'\operatorname{vec}\{\psi_i(\mathbf y^\star)\}\mid\mathbf Q_t]=\mathbf P'\underbrace{\operatorname{diag}\{\mathbf Q_t\}}_{\mathbb E[\mathbf X_t\mathbf X_t'\mid\mathbf Q_t]}\operatorname{vec}\{\psi_i(\mathbf y^\star)\}\tag{12.27}$$ consistent with (12.26).

Step 3: distribution of \(\mathbf Y_{t+1}-\mathbf Y_t\) conditional on \(\mathbf Q_t\). By the law of total probability, $$\mathbb P(\mathbf Y_{t+1}-\mathbf Y_t\mid\mathbf Q_t)=\sum_{i=1}^n\mathbb P(\mathbf X_{t+1}=\mathbf x_i,\mathbf Y_{t+1}-\mathbf Y_t\mid\mathbf Q_t)=\mathbf 1_n'\mathbf P'\operatorname{diag}\{\mathbf Q_t\}\operatorname{vec}\{\psi_i(\mathbf y^\star)\}=\mathbf Q_t'\operatorname{vec}\{\psi_i(\mathbf y^\star)\}$$ (using \(\mathbf 1_n'\mathbf P'=\mathbf 1_n'\), then \(\mathbf 1_n'\operatorname{diag}\{\mathbf Q_t\}=\mathbf Q_t'\).)

Step 4: construct \(\mathbf Q_{t+1}\). Using the terms from Steps 1–3 and the Bayes' rule of (12.25), $$\mathbf Q_{t+1}=\frac1{\underbrace{\mathbf Q_t'\operatorname{vec}\{\psi_i(\mathbf Y_{t+1}-\mathbf Y_t)\}}_{\mathbb P(\mathbf Y_{t+1}-\mathbf Y_t\mid\mathbf Q_t)}}\underbrace{\mathbf P'\operatorname{diag}\{\mathbf Q_t\}\operatorname{vec}\{\psi_i(\mathbf Y_{t+1}-\mathbf Y_t)\}}_{\mathbb P(\mathbf X_{t+1},\mathbf Y_{t+1}-\mathbf Y_t\mid\mathbf Q_t)}$$ We can then iteratively use this method to obtain a sequence of conditional probabilities \(\{\mathbf Q_{t+1}\}\) of the states in each period — this is exactly recursive learning (Bayesian filtering) under hidden states.