11. Decision Theory

Before laying out the set-up, we introduce some basic ideas about statistical models. Statistics can be used by agents both inside and outside the model — this is the dual role of statistics in economic analysis. Agents inside the model may use statistics to understand crucial features of the model (e.g. key parameters of interest), and the researcher outside the model uses statistics too.

There are three important concepts of uncertainty about a statistical model:

• Risk: uncertainty within a model. Uncertain outcomes with presumably known probabilities; these known probabilities can evolve over time. Suppose we have picked the right model, then this part of uncertainty tells us that we are still uncertain about the outcomes of that correct model.
• Ambiguity: uncertainty over models. Unknown weights for alternative possible models; we have several candidate models on hand but don't know which one is more accurate. Suppose we have correctly set up all the candidate models, then this part of uncertainty tells us that we are still uncertain about which candidate describes the reality in our case.
• Misspecification: uncertainty about models. Models we use are deliberate abstractions; by necessity there will always be model misspecification. How to cope with unknown flaws of models that are only meant to be approximations is the key link between misspecification and economic dynamics — i.e. where to impose structure and where to leave things non-parametric. This part of uncertainty tells us that we may be very wrong at the very beginning when we set up our candidate models.

• \(\Theta\): space of the unknown parameter \(\theta\); its measure is \(\lambda\).
• \(Y\): the data, a random vector; its realization is denoted \(y\), with space \(\mathcal Y\) (i.e. \(y\in\mathcal Y\)) and measure \(\tau\).
• \(\psi(y\mid\theta)\): the likelihood function — the probability density for \(Y\) given \(\theta\).
• \(D:\mathcal Y\to\mathcal D\): a decision rule mapping realized data \(y\in\mathcal Y\) into a decision \(d\in\mathcal D\). \(\mathcal D\) is the set of all possible decisions \(d\). The decision \(d\) is based on the observed data \(y\) and affects the level of utility. In a parameter-estimation problem, \(d\) may or may not directly be the choice of \(\theta\), but it will definitely determine the choice of \(\theta\) through the functional form of \(U(D(y),y,\theta)\).
• \(U(D(y),y,\theta)\): the utility function, our objective to be maximized by the choice of \(\theta\). In a parameter-estimation problem, given a predetermined \(D\) and observed \(y\), we pick the \(\theta\) that maximizes \(U\). The loss is defined as \(-U(D(y),y,\theta)\).

Example 11.1 (Model selection). Suppose we have two candidate models (actually one model with two possible choices of parameter), \(\Theta=\{\theta_1,\theta_2\}\). We want to use data \(y\) to decide which to use. Let the utility function be $$U(D(y),y,\theta)=\begin{cases}D(y)&\text{if }\theta=\theta_1\\1-D(y)&\text{if }\theta=\theta_2\end{cases}$$ Partition \(\mathcal Y\) into two disjoint subspaces \(\mathcal Y_1,\mathcal Y_2\) (\(\mathcal Y=\mathcal Y_1\cup\mathcal Y_2\), \(\mathcal Y_1\cap\mathcal Y_2=\varnothing\)), with the threshold decision rule \(D(y)\) being $$D(y)=\begin{cases}1&\text{if }y\in\mathcal Y_1\\0&\text{if }y\in\mathcal Y_2\end{cases}$$ Clearly we choose \(\theta=\theta_1\) when \(y\in\mathcal Y_1\) and \(\theta=\theta_2\) when \(y\in\mathcal Y_2\). So choosing a decision rule \(D\) is actually choosing a way of partitioning \(\mathcal Y\), which pins down the choice of \(\theta\) given any \(y\).

The two risk functions are $$\overline U(D\mid\theta_1)=\int_{\mathcal Y}D(y)\psi(y\mid\theta_1)\tau(dy)=\int_{\mathcal Y_1}\psi(y\mid\theta_1)\tau(dy)=\mathbb P(y\in\mathcal Y_1\mid\theta_1)$$ $$\overline U(D\mid\theta_2)=\int_{\mathcal Y}(1-D(y))\psi(y\mid\theta_2)\tau(dy)=\int_{\mathcal Y_2}\psi(y\mid\theta_2)\tau(dy)=\mathbb P(y\in\mathcal Y_2\mid\theta_2)$$ To maximize utility, it is reasonable to choose a partition (i.e. choose \(D\)) giving high expected utility in either \(\theta=\theta_1\) or \(\theta=\theta_2\). In the \((\overline U(D\mid\theta_1),\overline U(D\mid\theta_2))\) plane, all feasible \((\mathcal Y_1,\mathcal Y_2)\) combinations trace out a convex boundary; points on and inside the boundary exhaust all partitions, and we want to push toward the upper-right boundary.

From the previous example, the decision rule \(D\) determines the choice of \(\theta\) in the model-selection problem. But how can we obtain an optimal decision rule \(D^\star\)? What is the definition of an optimal rule? The following answers this from the Bayesian perspective.

Using the Bayesian perspective, introduce a subjective prior distribution of \(\theta\) (density \(\pi(\theta)\) w.r.t. measure \(\lambda\)), and define the posterior distribution density \(\overline\pi(\theta\mid y)\): $$\overline\pi(\theta\mid y)\equiv\frac{\psi(y\mid\theta)\pi(\theta)}{\int_\Theta\psi(y\mid\theta)\pi(\theta)\lambda(d\theta)}$$ For convenience, write \(\pi(d\theta)\) for \(\pi(\theta)\lambda(d\theta)\) and \(\overline\pi(d\theta\mid y)\) for \(\overline\pi(\theta\mid y)\lambda(d\theta)\), i.e. $$\overline\pi(d\theta\mid y)\equiv\frac{\psi(y\mid\theta)\pi(d\theta)}{\int_\Theta\psi(y\mid\theta)\pi(d\theta)}$$

The following problem is called the ex-ante problem: $$\max_D\int_\Theta\overline U(D\mid\theta)\pi(d\theta)$$ whose solution is denoted \(D^\star\). That is, before observing the data, average the risk function over \(\theta\) using the prior \(\pi\), then choose the optimal decision rule.

Rewrite the ex-ante problem: $$\begin{aligned}&\max_D\int_\Theta\overline U(D\mid\theta)\pi(d\theta)\\\Leftrightarrow&\max_D\int_\Theta\int_{\mathcal Y}U(D(y),y,\theta)\psi(y\mid\theta)\tau(dy)\pi(d\theta)\\\overset{\text{interchange}}{\Leftrightarrow}&\max_D\int_{\mathcal Y}\int_\Theta U(D(y),y,\theta)\frac{\pi(d\theta)\psi(y\mid\theta)}{\int_\Theta\psi(y\mid\theta)\pi(d\theta)}\Big(\int_\Theta\psi(y\mid\theta)\pi(d\theta)\Big)\tau(dy)\\\Leftrightarrow&\max_D\int_{\mathcal Y}\int_\Theta U(D(y),y,\theta)\overline\pi(d\theta\mid y)\Big(\int_\Theta\psi(y\mid\theta)\pi(d\theta)\Big)\tau(dy)\end{aligned}$$ Since \(\big(\int_\Theta\psi(y\mid\theta)\pi(d\theta)\big)\) depends only on \(y\) and is unrelated to \(D\), we can instead solve the ex-post problem: $$\max_{D(y)}\int_\Theta U(D(y),y,\theta)\overline\pi(d\theta\mid y)$$ whose solution is denoted \(d^\star(y)\). Solving for \(d^\star(y)\) for each \(y\) gives the decision rule \(\overline D^\star\).

Example 11.2 (Revisit the model selection example). Back to Example 11.1, with the posterior $$\overline\pi(\theta_1\mid y)=\frac{\psi(y\mid\theta_1)\pi(\theta_1)}{\psi(y\mid\theta_1)\pi(\theta_1)+\psi(y\mid\theta_2)\pi(\theta_2)},\quad \overline\pi(\theta_2\mid y)=\frac{\psi(y\mid\theta_2)\pi(\theta_2)}{\psi(y\mid\theta_1)\pi(\theta_1)+\psi(y\mid\theta_2)\pi(\theta_2)}$$ the Bayesian decision rule solves $$\begin{aligned}&\max_{D(y)}U(D(y),y,\theta_1)\overline\pi(\theta_1\mid y)+U(D(y),y,\theta_2)\overline\pi(\theta_2\mid y)\\\Leftrightarrow&\max_{D(y)}D(y)\overline\pi(\theta_1\mid y)+(1-D(y))\overline\pi(\theta_2\mid y)\\\Leftrightarrow&\max_{D(y)}D(y)(\overline\pi(\theta_1\mid y)-\overline\pi(\theta_2\mid y))+\overline\pi(\theta_2\mid y)\end{aligned}$$ with the constraint \(D(y)=1\) if \(y\in\mathcal Y_1\), \(0\) if \(y\in\mathcal Y_2\). The solution is $$\mathcal Y_1=\{y:\overline\pi(\theta_1\mid y)-\overline\pi(\theta_2\mid y)>0\},\quad \mathcal Y_2=\{y:\overline\pi(\theta_1\mid y)-\overline\pi(\theta_2\mid y)\le0\}$$ i.e. the Bayesian decision rule $$D^\star(y)=\begin{cases}1&\text{if }\overline\pi(\theta_1\mid y)>\overline\pi(\theta_2\mid y)\\0&\text{if }\overline\pi(\theta_1\mid y)\le\overline\pi(\theta_2\mid y)\end{cases}$$

Consider two decision rules \(D_1:\mathcal Y\to\mathcal D\), \(D_2:\mathcal Y\to\mathcal D\). If \(\overline U(D_2\mid\theta)\ge\overline U(D_1\mid\theta)\) for all \(\theta\in\Theta\), then for any prior \(\pi\) the ex-ante objective of \(D_2\) is no lower than that of \(D_1\): $$\overline U(D_2\mid\theta)\ge\overline U(D_1\mid\theta)\ \forall\theta\Rightarrow\int_\Theta\overline U(D_2\mid\theta)\pi(d\theta)\ge\int_\Theta\overline U(D_1\mid\theta)\pi(d\theta)\ \forall\pi$$ We then say \(D_2\) dominates \(D_1\), written \(D_2\succsim D_1\) (this is a partial order, since we only compare the pair \(D_1,D_2\), not all decision rules).

The Bayesian decision rule depends on our arbitrary choice of prior \(\pi\). To be more specific about the decision rule (i.e. eliminate the arbitrary component in selecting \(D\)), consider decision rules of the form $$\max_D\min_\pi\Big\{\int_\Theta\overline U(D\mid\theta)\pi(d\theta)+C(\pi)\Big\}$$ where \(D\) is a choice of decision rules, \(\pi\) is a prior distribution of \(\theta\), and \(C(\pi)\) is some convex function of \(\pi\) — an explicit way to confront prior uncertainty. Fundamentally this is a theory of caution: we pick a utility-maximizing decision rule for the worst case (given restrictions on \(\pi\) specified by \(C(\pi)\) of \(\theta\)'s distribution).

1. Benchmark example. Take \(C(\pi)\) as $$C=\begin{cases}0&\pi\in\Pi\\\infty&\pi\notin\Pi\end{cases}$$ Then the problem collapses to simply looking in the set \(\Pi\) for a solution: $$\max_D\min_\pi\Big\{\int_\Theta\overline U(D\mid\theta)\pi(d\theta)+C(\pi)\Big\}=\max_D\min_{\pi\in\Pi}\Big\{\int_\Theta\overline U(D\mid\theta)\pi(d\theta)\Big\}$$

2. The "relative entropy" penalty. For notational convenience (not strictly necessary), restrict \(\Theta\) and the possible prior \(\pi\) to finite (discrete) elements: \(\Theta=\{\theta_1,\dots,\theta_n\}\), \(\pi=(\pi_1,\dots,\pi_n)\). Define the penalty $$C(\pi)=\xi\sum_{j=1}^n\pi_j\big[\log\pi_j-\log\pi_j^0\big],\quad \xi>0$$ where \(\pi^0=(\pi_1^0,\dots,\pi_n^0)\) is the reference prior. This can be rewritten as $$C(\pi)=\xi\sum_{j=1}^n\Big(\frac{\pi_j}{\pi_j^0}\Big)\Big(\log\Big(\frac{\pi_j}{\pi_j^0}\Big)\Big)\pi_j^0$$

Suppose \(\theta\) has \(n\) possible values, \(\Theta=\{\theta_1,\dots,\theta_n\}\). The model-selection problem is $$\begin{aligned}&\max_D\min_\pi\Big\{\sum_{j=1}^n\pi_j\overline U(D\mid\theta_j)+C(\pi)\Big\}\\\Rightarrow&\max_D\min_\pi\Big\{\sum_{j=1}^n\pi_j\overline U(D\mid\theta_j)+\xi\sum_{j=1}^n\pi_j\big(\log\pi_j-\log\pi_j^0\big)\Big\}\\\Rightarrow&\max_D\min_\pi\Big\{\sum_{j=1}^n\pi_j\big[\overline U(D\mid\theta_j)+\xi\big(\log\pi_j-\log\pi_j^0\big)\big]\Big\}\end{aligned}$$ The Lagrangian of the inner (over \(\pi\)) problem is $$\mathcal L=\sum_{j=1}^n\pi_j\big[\overline U(D\mid\theta_j)+\xi(\log\pi_j-\log\pi_j^0)\big]+\lambda\Big(1-\sum_{j=1}^n\pi_j\Big)$$ (the constraint \(\pi_j\ge0\) holds automatically, since \(\log\to-\infty\) at \(0\).) The first-order condition for each \(\pi_j\): $$\forall j:\quad \overline U(D\mid\theta_j)+\xi(\log\pi_j-\log\pi_j^0)+\xi-\log s=0,\quad \log s=\lambda$$ Rearranging, $$\pi_j^\star=\frac{\pi_j^0\exp\big(-\tfrac1\xi\overline U(D\mid\theta_j)\big)}{\sum_{j=1}^n\pi_j^0\exp\big(-\tfrac1\xi\overline U(D\mid\theta_j)\big)}$$ This is the celebrated exponential tilting: it assigns higher weight (probability) to the \(\theta\)'s with which we have lower values of the utility \(\overline U\), and lower weight to the \(\theta\)'s with higher utility — confirming that \(\pi^\star\) is a very conservative prior.

Plugging \(\pi_j^\star\) back into the objective and taking logs (dividing both sides by \(\pi_j^0\) and taking \(\log\) gives \(\log\pi_j^\star-\log\pi_j^0=-\tfrac1\xi\overline U(D\mid\theta_j)-\log\sum_j\pi_j^0\exp[-\tfrac1\xi\overline U(D\mid\theta_j)]\)), the minimized objective is $$\begin{aligned}&\sum_{j=1}^n\pi_j^\star\Big[\overline U(D\mid\theta_j)+\xi\Big(-\tfrac1\xi\overline U(D\mid\theta_j)-\log\sum_{j=1}^n\pi_j^0\exp\big[-\tfrac1\xi\overline U(D\mid\theta_j)\big]\Big)\Big]\\=&\sum_{j=1}^n\pi_j^\star\Big[-\xi\log\Big(\sum_{j=1}^n\pi_j^0\exp\big(-\tfrac1\xi\overline U(D\mid\theta_j)\big)\Big)\Big]\\=&-\xi\log\Big(\sum_{j=1}^n\pi_j^0\exp\big(-\tfrac1\xi\overline U(D\mid\theta_j)\big)\Big)\end{aligned}\tag{11.1}$$

Reading the minimized objective as a certainty equivalence. Recall the expected utility from a random variable \(x\), \(U(x)=\sum_{j=1}^n\pi_j u(x_j)\), with certainty equivalence \(CE(x)=u^{-1}\big(\sum_{j=1}^n\pi_j u(x_j)\big)\). Notice (11.1) is similar to a certainty equivalent: take the utility function $$u(x_j)=-\exp\Big(-\tfrac1\xi x_j\Big)$$ Then its certainty equivalence $$\begin{aligned}CE(x)&=u^{-1}\Big(\sum_{j=1}^n\pi_j u(x_j)\Big)=-\xi\log\Big(-\sum_{j=1}^n\pi_j(-1)\exp\big(-\tfrac1\xi x_j\big)\Big)\end{aligned}$$ is exactly the form obtained in (11.1), i.e. with \(u(\overline U(D\mid\theta_j))=-\exp(-\tfrac1\xi\overline U(D\mid\theta_j))\), $$CE(\overline U(D\mid\theta_j))=-\xi\log\Big(\sum_{j=1}^n\pi_j^0\exp\big(-\tfrac1\xi\overline U(D\mid\theta_j)\big)\Big)$$ where the inside of the summation in (11.1) is strictly less than one (\(\sum_j\pi_j^0=1\) and \(\exp(-\tfrac1\xi\overline U)<1\)), so the entire term is positive. Thus the minimized objective is the certainty equivalence of \(\overline U(D\mid\theta_j)\) under the reference prior \(\pi_0\). By the max-min decision rule with the "relative entropy" penalty, we are selecting the decision rule that maximizes the certainty equivalence of \(\overline U(D\mid\theta_j)\) under a subjectively-picked reference prior \(\pi_0\).

Graphical argument for the two-candidate-parameter case (Example 11.1). Set \(C(\pi)=0\) (no restriction on \(\pi\), no anchor point \(\pi^0\)). The max-min problem is $$\max_D\min_\pi\overline U(D\mid\theta_1)\pi(\theta_1)+\overline U(D\mid\theta_2)\pi(\theta_2)$$ The inner minimization solves $$\min_{\pi(\theta_1)}\{\overline U(D\mid\theta_1)\pi(\theta_1)+\overline U(D\mid\theta_2)(1-\pi(\theta_1))\}=\min_{\pi(\theta_1)}\{[\overline U(D\mid\theta_1)-\overline U(D\mid\theta_2)]\pi(\theta_1)+\overline U(D\mid\theta_2)\}$$ \(\Rightarrow\pi^\star(\theta_2)=0\), \(\pi^\star(\theta_1)=1\) (if \(\overline U(D\mid\theta_1)<\overline U(D\mid\theta_2)\)), putting all weight on the parameter with lower utility. Then the maximization becomes \(\max_D\{\overline U(D\mid\theta_1)\}\); its result is obtained by moving \(\overline U(D\mid\theta_1)\) up until \(\overline U(D\mid\theta_1)=\overline U(D\mid\theta_2)\) (on the 45-degree line). By symmetry, min-max gives the same, corresponding to the unique intersection of the 45-degree line and the convex boundary. In a word, adding a minimization (whether before or after the Bayesian maximization) simply moves that point toward the 45-degree line while keeping it on the convex boundary — max-min and min-max give the same resulting optimal decision rule, the unique intersection of the 45-degree line and the boundary; hence there is only one resulting optimal decision rule.

Let \(y^\star\) be tomorrow's state and \(y\) today's state. We want to forecast \(y^\star\) with today's \(y\) based on some parameter \(\theta\) and decision \(D(y)\), with the conditional transition density \(\psi(y^\star\mid y,D,\theta)\). The parameter \(\theta\) is important because it affects our perspective about the future; but the issue is we don't know \(\theta\) and need to make inferences — we use the posterior density \(\overline\pi(d\theta\mid y)\) as the weight of different \(\theta\)'s (it is the solution to some minimization problem).

The utility function is defined on the conditional transition density \(\psi(y^\star\mid y,D,\theta)\), so the ex-post problem has objective function $$\phi(y^\star\mid y,D)\equiv\int_\Theta\psi(y^\star\mid y,D,\theta)\overline\pi(d\theta\mid y)\tag{11.5}$$ which defines a conditional transition density \(\phi(y^\star\mid y,D)\) without the parameter \(\theta\).

The RHS of (11.5) enables us to split the uncertainty in a dynamic problem (i.e. uncertainty in the transition function) into two parts:

• Part 1 (risk) comes from \(\psi(y^\star\mid y,D,\theta)\) — it determines the risk function \(\int_{\mathcal Y}\psi(y^\star\mid y,D,\theta)f(y\mid\theta)\tau(dy)\), where \(f(y\mid\theta)\) is the conditional density of \(y\) given \(\theta\). Here the risk function is interpreted as the unconditional (on today's state \(y\)) probability of having \(y^\star\) in the next period given parameter \(\theta\).
• Part 2 (ambiguity) comes from \(\overline\pi(d\theta\mid y)\) — it is chosen as a solution to certain problems with some incentives (e.g. the minimization problem as in the max-min decision rule).

The RHS of (11.5) can be regarded as a two-stage lottery: in the first stage we observe \(y\) and then draw a \(\theta\) according to \(\overline\pi(d\theta\mid y)\); in the second stage, based on \(y,D,\theta\), we draw a \(y^\star\) according to \(\psi(y^\star\mid y,D,\theta)\); repeating these two stages many times, we can estimate the distribution \(\phi(y^\star\mid y,D)\). The LHS of (11.5) collapses the two-stage lottery into a one-stage lottery — from the LHS itself, we cannot separate the uncertainty in \(\phi(y^\star\mid y,D)\) in any way.

Two-stage vs one-stage lottery. The two-stage lottery has more latitude for model calibration than the one-stage lottery: some empirical evidence may be hard to meet by model calibration with reasonable parameter values; splitting the uncertainty in the transition function into a risk part and an ambiguity part gives more room for calibration. So we don't simply reduce the two-stage lottery to a one-stage lottery.

Unavoidable subjectivity. We can never completely avoid subjective input in a model. In the two-stage lottery, the selection of \(\overline\pi(d\theta\mid y)\) is dependent on our subjective input. Take the max-min decision rule with the "relative entropy" penalty for example: the choice of \(\overline\pi(d\theta\mid y)\) depends on \(\overline\pi(d\theta)\), which depends on our subjectively-selected reference prior \(\pi^0\).