16. Learning under Uncertainty

Homogeneous agents (same beliefs, treated as one agent). A single stock with price \(P_t\) pays \(D_t\), with \(\delta_t\equiv\ln D_t\) satisfying (16.1):

\(\{Z_t\}\) standard BM, \(\sigma_\delta\) known, \(\mu_\delta\) unknown, taking one fixed value from \(\{\mu_1,\dots,\mu_n\}\) that never shifts. \(\{\mathcal F_t\}\) is the filtration. The subjective prior (period 0) belief \(\Pi_0(\mu_s)=\mathbf P\{\mu=\mu_s\mid\mathcal F_0\}\), \(\sum_{s=1}^n\Pi_0(\mu_s)=1\), \(\Pi_0(\mu_s)\geq0\); the posterior (period \(t\)) \(\Pi_t(\mu_s)=\mathbf P\{\mu=\mu_s\mid\mathcal F_t\}\).

16.1.2 Characterize the Belief Updating

By Bayes' rule (16.2):

By (16.1) the likelihood is normal (16.3): \(\mathbf P\{d\delta_t\mid\mu=\mu_s;\mathcal F_t\}=\frac{1}{\sqrt{2\pi}}e^{-\frac12\frac{(d\delta_t-\mu_s dt)^2}{\sigma_\delta^2 dt}}\). Substituting into (16.2), cancelling common factors gives (16.4):

The second-order Taylor expansion \(e^x\approx1+x+\frac12 x^2\) (with \(d\delta_t^2=\sigma_\delta^2 dt\), dropping \(\circ(dt)\)) gives (16.5): \(e^{\frac{\mu_s d\delta_t}{\sigma_\delta^2}-\frac12\frac{(\mu_s)^2}{\sigma_\delta^2}dt}\approx1+\frac{\mu_s d\delta_t}{\sigma_\delta^2}\). Substituting into (16.4) and simplifying (with \(\mathbb E_t[\mu_\delta]\equiv\sum_j\mu_j\Pi_t(\mu_j)\)) gives the belief dynamics (16.6) and innovation (16.7):

Observations. ① \(\mathbb E_t[d\hat Z_t]=0\), \(\mathrm{Var}(d\hat Z_t)=\mathrm{Var}(dZ_t)=dt\), so \(d\hat Z_t\sim\mathcal N(0,dt)\) and \(\{\hat Z_t\}\) is also standard BM. ② When \(\Pi_t(\mu_s)=1\) or \(0\), \(d\Pi_t(\mu_s)=0\): if the agent is fully (probability one) certain of being in/out of a state, the subjective probability stops moving. ③ One verifies \(\sum_s\Pi_t(\mu_s)=1\), \(\Pi_t(\mu_s)\geq0\) for all \(t\) (the dynamics stop the moment a belief hits zero, so it cannot go negative; and \(\sum_s d\Pi_t(\mu_s)=0\)).

16.1.3 Special Case: Homogeneity of Agents

Further assume agents are also homogeneous in risk tolerance \(\rho_i\) and initial wealth share \(\omega_i\). \(\{\Pi_t(\mu_s)\}\) still satisfies (16.6), (16.7).

By (16.1), \(\delta_{t+\tau}=\delta_t+\mu_\delta\tau+\sigma_\delta\sqrt\tau x_t\) (16.8), \(x_t\sim\mathcal N(0,1)\) (same as (15.89); \(\mu_\delta\) is still a constant, just unknown). The SDF \(\pi_t=\frac1\xi e^{-\phi t-\gamma\delta_t}\) (homogeneous CRRA, from (15.93)). The expectation is taken over both \(x_t\) and \(\mu_\delta\) (both unknown). By (15.94) the price (16.9) and price-dividend ratio (16.10):

Risk premium. By (1.53), under CRRA \(\mathbb E[r_t]-r_{f,t}=\gamma\sigma_P\sigma_C\rho_{PC}\) (16.12), equivalently (16.13): \((\mathbb E[r_t]-r_{f,t})\,dt=\gamma(\frac{dP_t}{P_t})(\frac{dC_t}{C_t})\). With \(D_t=C_t\), \(\frac{dC_t}{C_t}=d\delta_t=\mu_\delta\,dt+\sigma_\delta\,dZ_t\), \(\sigma_C=\sigma_\delta\), \(dZ_t^C=dZ_t\).

Without learning: \(\frac{P_t}{D_t}\) is a constant \(h\), \(\frac{dP_t}{P_t}=\frac{dD_t}{D_t}=d\delta_t\), \(\sigma_P=\sigma_\delta\); substituting into (16.13) gives (16.14):

With learning: \(\sigma_C=\sigma_\delta\), \(dZ_t^C=dZ_t\) unchanged, but by (16.10) \(\frac{P_t}{D_t}\) is no longer constant, so we differentiate (16.9). The result (16.15): \(\frac{dP_t}{P_t}=[\sigma_\delta+(\tilde{\mathbb E}[\mu_s]-\mathbb E_t[\mu_\delta])\frac1{\sigma_\delta}]\,dZ_t+\circ(dt)\), where \(\tilde{\mathbb E}[\mu_s]\) is the expectation under the weighted measure \(\tilde\Pi_t(\mu_s)=\frac{\Pi_t(\mu_s)k(\mu_s)}{\sum_s\Pi_t(\mu_s)k(\mu_s)}\). Substituting into (16.13) gives the with-learning risk premium (16.16):

Comparison. The with-learning premium $<$ without-learning premium iff \(\tilde{\mathbb E}[\mu_s]-\mathbb E_t[\mu_\delta]<0\), i.e. the weighted measure \(\tilde\Pi\) tilts toward smaller \(\mu_s\) than \(\Pi\), which happens when \(k'(\mu_s)<0\) (i.e. \(\gamma>1\), IES \(\frac1\gamma<1\)). Intuition: with IES $<1$ consumption is less responsive to prices, so \(\rho_{PC}\) shrinks and learning lowers the risk premium.

16.1.4 Heterogeneity of Agents

Restore heterogeneous risk tolerance \(\rho_i\) and initial share \(\omega_i\), but keep beliefs homogeneous. \(\{\Pi_t(\mu_s)\}\) still satisfies (16.6), (16.7). By (15.92), given \(\mu_\delta\) (16.17): \(P_t\mid\mu_\delta=e^{g(\delta_t)}\frac{1-e^{-\phi(T-t)}}\phi\mathbb E_t^{x_t,\tau}[e^{\delta_{t+\tau}-g(\delta_{t+\tau})}\mid\mu_\delta]\). Belief-weighting over the unknown \(\mu_\delta\) gives the price (16.18), (16.19):

where \(k(\delta_t,\mu_s)\equiv e^{g(\delta_t)-\delta_t}\frac{1-e^{-\phi(T-t)}}\phi\mathbb E_t^{x_t,\tau}[e^{\delta_{t+\tau}-g(\delta_{t+\tau})}\mid\mu_s]\) depends on both \(\delta_t\) (observable) and \(\mu_s\) (unobservable). \(g(\delta_t)\) is pinned by the functional fixed point (15.76) but hard to solve; its derivatives can be characterized as in §15.3.3 (\(g'(\delta_t)=1/\mathbb E_{\mathrm{CS}}^\star[\rho_i\mid\delta_t]>0\), \(g''(\delta_t)<0\)). Numerically simulable (16.20): draw \(\mu_\delta\) from \(\{\Pi_t(\mu_s)\}\), \(\tau\) from the truncated exponential, \(\delta_{t+\tau}\) from the normal, Monte-Carlo average to estimate \(\mathbb E_t^{x_t,\tau,\mu_\delta}[e^{\delta_{t+\tau}-g(\delta_{t+\tau})}]\), then compute \(P_t\).

16.1.5 Regime Shifting Model

Relax "\(\mu_\delta\) constant": let \(\mu_{\delta,t}\) shift back and forth within \(\{\mu_1,\dots,\mu_n\}\) (regime shifting). Now \(\{\Pi_t(\mu_s)\}\) no longer satisfies (16.6), (16.7). Agents have homogeneous beliefs over the same discrete state set; keep §16.1.1 assumptions but allow \(\mu_{\delta,t}\) to vary in time. Define transition rates \(\Lambda=\{\lambda_{ij}\}\): \(\lambda_{ij}=\mathbf P\{\mu_{\delta,t+dt}=\mu_j\mid\mu_{\delta,t}=\mu_i\}\), \(\sum_j\lambda_{ij}=1\). The belief dynamics split into two parts:

with \(d\hat Z_t=\frac{d\delta_t-\mathbb E_t[\mu_\delta]\,dt}{\sigma_\delta}\). Part A is Bayesian learning conditional on the regime not changing (same as (16.6)); Part B is the probability inflow/outflow from regime switches.

Same fundamental idea as §8 in He (2019a).

16.2.1 Setup

Observation equation (16.21):

with \(\mathbf Y_t,\mathbf F_0,\mathbf F_1,d\mathbf Z_t\) all \(n\times1\), \(\boldsymbol\omega\) being \(n\times n\), \(\beta_t\) being \(1\times1\), \(d\mathbf Z_t\sim\mathcal N(\mathbf 0,\mathbf I\,dt)\). State equation (16.22):

\(\boldsymbol\eta\) is \(n\times1\), \(\alpha_0,\alpha_1\) are \(1\times1\), and \(d\mathbf Z_t\) in (16.21), (16.22) is the same. Assumptions: the observation vector \(\mathbf Y_t\) is observed each period, the state \(\beta_t\) is not; the prior (at \(t\), before \(d\mathbf Y_t\)) distribution \(\beta_t\sim\mathcal N(\hat\beta_t,v_t)\).

16.2.2 Multivariate Normal Distribution

By the multivariate-normal definition (He 2019a §21), \(\mathbf Y_{t+dt}\) and \(\beta_{t+dt}\) are jointly normal at \(t\) (before \(d\mathbf Z_t\)) (16.23):

Each term follows from conditional expectations/variances of (16.21), (16.22): means (16.24), (16.25): \(\hat{\mathbf Y}_{(t+dt)\mid t}=\mathbf Y_t+(\mathbf F_0(\mathbf Y_t)+\mathbf F_1(\mathbf Y_t)\hat\beta_t)\,dt\), \(\hat\beta_{(t+dt)\mid t}=\hat\beta_t+(\alpha_0+\alpha_1\hat\beta_t)\,dt\); covariances (16.28), (16.31), (16.32), (16.33):

(The derivation repeatedly uses \((dt)^2=0\), \(d\mathbf Z_t\,dt=\mathbf 0\), \(d\mathbf Z_t(d\mathbf Z_t)'=\mathbf I_{n\times n}\,dt\).)

16.2.3 Updating

Lemma 16.1 (conditional normal). If \(\begin{pmatrix}\mathbf X\\\mathbf Y\end{pmatrix}\sim\mathcal N\!\left(\begin{pmatrix}\mathbf 0\\\mathbf 0\end{pmatrix},\begin{bmatrix}\mathbf S_{\mathbf X\mathbf X'}&\mathbf S_{\mathbf X\mathbf Y'}\\\mathbf S_{\mathbf Y\mathbf X'}&\mathbf S_{\mathbf Y\mathbf Y'}\end{bmatrix}\right)\), then \(\mathbf X\mid\mathbf Y\sim\mathcal N(\mathbf A\mathbf Y,\mathbf S_{\mathbf X\mathbf X'\mid\mathbf Y})\) with \(\mathbf A=\mathbf S_{\mathbf X\mathbf Y'}\mathbf S_{\mathbf Y\mathbf Y'}^{-1}\), \(\mathbf S_{\mathbf X\mathbf X'\mid\mathbf Y}=\mathbf S_{\mathbf X\mathbf X'}-\mathbf A\mathbf S_{\mathbf Y\mathbf Y'}\mathbf A'\).

Rewriting (16.23) in demeaned form (16.34) and applying Lemma 16.1 to \((\mathbf Y_{t+dt}-\hat{\mathbf Y}_{(t+dt)\mid t},\ \beta_{t+dt}-\hat\beta_{(t+dt)\mid t})\) gives the updated \(\beta_{t+dt}\mid\mathbf Y_{t+dt}\sim\mathcal N(\hat\beta_{t+dt},v_{t+dt})\), with \(\mathbf A=\boldsymbol\Sigma_{\beta\mathbf Y'\mid t}\boldsymbol\Sigma_{\mathbf Y\mathbf Y'\mid t}^{-1}\) (16.35), \(v_{t+dt}=\Sigma_{\beta\beta\mid t}-\boldsymbol\Sigma_{\beta\mathbf Y'\mid t}\boldsymbol\Sigma_{\mathbf Y\mathbf Y'\mid t}^{-1}\boldsymbol\Sigma_{\mathbf Y\beta\mid t}\) (16.36). Substituting (16.28), (16.32) gives \(\mathbf A=(\mathbf F_1(\mathbf Y_t)v_t+\boldsymbol\omega\boldsymbol\eta)'(\boldsymbol\omega\boldsymbol\omega')^{-1}\), hence the latent state's mean dynamics (16.37):

where \(d\tilde{\mathbf Z}_t=(\boldsymbol\omega\boldsymbol\omega')^{-\frac12}[d\mathbf Y_t-(\mathbf F_0(\mathbf Y_t)+\mathbf F_1(\mathbf Y_t)\hat\beta_t)\,dt]\) satisfies \(d\tilde{\mathbf Z}_t\sim\mathcal N(\mathbf 0,\mathbf I\,dt)\) (since (16.21) gives the mean and variance \(\boldsymbol\omega\boldsymbol\omega'\) of \(d\mathbf Y_t\)). Substituting (16.33), (16.31) gives the variance dynamics (16.39):

• (16.39) is a locally deterministic motion of \(v_t\) (\(\frac{dv_t}{dt}\) has no randomness at \(t\)).
• If \(\mathbf F_1(\mathbf Y_t)=\mathbf F_1\) is always constant, (16.39) shows \(v_t\) converges to the value where \(\frac{dv_t}{dt}=0\) (explicitly solvable from (16.39), a Riccati fixed point).
• If \(\mathbf F_1(\mathbf Y_t)\) depends on \(\mathbf Y_t\), convergence generally fails.