12. Stochastic Dynamic Programming: Commodity Model
本章主题：随机动态规划的商品模型。 §12.1 设定：需求平稳 \(P=D(q)\)（向下倾斜、严正、连续）；供给——代表性卖家每期获随机禀赋 \(\omega_t\in(m,M)\)（i.i.d.、密度 \(\mu(\omega)\)、无生产成本）；社会剩余 \(u(q)=\int_0^q D(x)dx\)（严增、严凹、连续可微）；存储成本 \(\phi(y)\)（严增、严凸、\(\phi(0)=0\)）。§12.2 贝尔曼方程：两状态 \(V(x,\omega)=\max_{y\in[0,x+\omega]}\{u(x+\omega-y)-\phi(y)+\beta\int_m^M V(y,\omega')\mu(\omega')d\omega'\}\)；单状态 \(s=x+\omega\)（\(y\)=结转存货）\(V(s)=\max_{y\in[0,s]}\{u(s-y)-\phi(y)+\beta\int_m^M V(y+\omega')\mu(\omega')d\omega'\}\)；f.o.c. \(u'(s-y)+\phi'(y)\ge\beta\int_m^M V'(y+\omega')\mu(\omega')d\omega'\)（内点取等，LHS 增 RHS 减 ⟹ 唯一 \(y^\star=g(s)\) 且随 \(s\) 递增）。
Chapter theme: the commodity model of stochastic dynamic programming. §12.1 Set-up: stationary demand \(P=D(q)\) (downward sloping, strictly positive, continuous); supply — the representative seller gets a random endowment \(\omega_t\in(m,M)\) each period (i.i.d., density \(\mu(\omega)\), no production cost); social surplus \(u(q)=\int_0^q D(x)dx\) (strictly increasing, strictly concave, continuously differentiable); cost of storage \(\phi(y)\) (strictly increasing, strictly convex, \(\phi(0)=0\)). §12.2 Bellman equation: two states \(V(x,\omega)=\max_{y\in[0,x+\omega]}\{u(x+\omega-y)-\phi(y)+\beta\int_m^M V(y,\omega')\mu(\omega')d\omega'\}\); one state \(s=x+\omega\) (\(y\) = inventory carried over) \(V(s)=\max_{y\in[0,s]}\{u(s-y)-\phi(y)+\beta\int_m^M V(y+\omega')\mu(\omega')d\omega'\}\); f.o.c. \(u'(s-y)+\phi'(y)\ge\beta\int_m^M V'(y+\omega')\mu(\omega')d\omega'\) (equality at interior; LHS increasing, RHS decreasing ⟹ unique \(y^\star=g(s)\) increasing in \(s\)).
12.1 Set-up
12.1.1 需求
设需求平稳（即不随 \(t\) 变化）：\(P=D(q)\)，其中 \(q\) 是需求且被消费的量。假设函数 \(D\) 向下倾斜、严格为正、连续。
12.1.2 供给
每期，代表性卖家获得禀赋 \(\omega_t\in(m,M)\)，\(0 12.1.3 社会剩余 定义社会剩余为
$$u(q)=\int_0^q D(x)dx$$
由于 \(D\) 向下倾斜、严正、连续，\(u\) 关于 \(q\) 严格递增、严格凹、连续可微。 12.1.4 存储成本 定义把 \(y\) 单位商品存储一期的成本为 \(\phi(y)\)，其中 \(\phi(y)\) 连续、关于 \(y\) 严格递增、严格凸、连续可微、且 \(\phi(0)=0\)。
12.1.1 Demand
Suppose the demand is stationary (i.e. doesn't change along with \(t\)) as \(P=D(q)\) where \(q\) is the amount demanded and consumed. Assume that the function \(D\) is downward sloping, strictly positive, and continuous.
12.1.2 Supply
In each period, the representative seller gets an endowment \(\omega_t\in(m,M)\) with \(0 12.1.3 Social surplus Define the social surplus as
$$u(q)=\int_0^q D(x)dx$$
Since we supposed that \(D\) is downward sloping, strictly positive and continuous, \(u\) is strictly increasing in \(q\), strictly concave in \(q\), and continuously differentiable. 12.1.4 Cost of storage Define the cost of storing \(y\) units of commodity for one period as \(\phi(y)\) where \(\phi(y)\) is continuous, strictly increasing in \(y\), strictly convex in \(y\), continuously differentiable, and \(\phi(0)=0\).
12.2 The Bellman Equation
12.2.1 两状态变量的贝尔曼方程
考虑状态变量 \(S=(x,\omega)\)，其中内生状态 \(x\) 是当前期从上期结转的初始存货、\(\omega\) 是外生随机冲击。社会计划者的贝尔曼方程为 $$V(x,\omega)=\max_{y\in\Gamma(x,\omega)}\left\{u(x+\omega-y)-\phi(y)+\beta\int_m^M V(y,\omega')\mu(\omega')d\omega'\right\}$$ 其中 \(\Gamma(x,\omega)=[0,x+\omega]\)。
12.2.2 仅一个状态变量的贝尔曼方程
考虑状态变量 \(s=x+\omega\)，\(x\)、\(\omega\) 同前。\(s\) 表示当前期的总供给。注意 \(y\) 不是下一期的总供给，而是结转到下一期的存货。社会计划者的贝尔曼方程为 $$V(s)=\max_{y\in\Gamma(s)}\left\{u(s-y)-\phi(y)+\beta\int_m^M V(y+\omega')\mu(\omega')d\omega'\right\}$$ 其中 \(\Gamma(s)=[0,s]\)。
12.2.3 一阶条件
仅一个状态变量的贝尔曼方程 $$V(s)=\max_{y\in\Gamma(s)}\left\{u(s-y)-\phi(y)+\beta\int_m^M V(y+\omega')\mu(\omega')d\omega'\right\}$$ 有一阶条件 $$u'(s-y)+\phi'(y)\ge\beta\int_m^M V'(y+\omega')\mu(\omega')d\omega'$$ 在 \(y^\star>0\)（内点解）时取等号。注意可对 \(u\) 假设 Inada 条件以排除角点解 \(y^\star=s\)。由于 LHS 关于 \(y\) 递增、RHS 关于 \(y\) 递减，故有唯一解 \(y^\star=g(s)\)。此外，更大的 \(s\) 把 LHS 下移、使最优策略更大，即 \(\hat s>s\Rightarrow g(\hat s)>g(s)\)，故 \(g\) 关于 \(s\) 递增。
12.2.1 Bellman equation with two state variables
Consider the state variable \(S=(x,\omega)\) where the endogenous state \(x\) is the current period's initial inventory carried over from last period and \(\omega\) is the exogenous stochastic shock. The Bellman equation for the social planner is as follows. $$V(x,\omega)=\max_{y\in\Gamma(x,\omega)}\left\{u(x+\omega-y)-\phi(y)+\beta\int_m^M V(y,\omega')\mu(\omega')d\omega'\right\}$$ with \(\Gamma(x,\omega)=[0,x+\omega]\).
12.2.2 Bellman equation with only one state variable
Consider the state variable \(s=x+\omega\) where \(x\) and \(\omega\) are defined as before. Here \(s\) means the total supply of the current period. And \(y\) doesn't mean the total supply of next period; instead, \(y\) means the inventory carried over to next period. The Bellman equation for the social planner is as follows. $$V(s)=\max_{y\in\Gamma(s)}\left\{u(s-y)-\phi(y)+\beta\int_m^M V(y+\omega')\mu(\omega')d\omega'\right\}$$ with \(\Gamma(s)=[0,s]\).
12.2.3 The first-order condition
The Bellman equation with only one state variable $$V(s)=\max_{y\in\Gamma(s)}\left\{u(s-y)-\phi(y)+\beta\int_m^M V(y+\omega')\mu(\omega')d\omega'\right\}$$ has the f.o.c. $$u'(s-y)+\phi'(y)\ge\beta\int_m^M V'(y+\omega')\mu(\omega')d\omega'$$ with equality if \(y^\star>0\) (interior solution). Note that we can assume an Inada condition on \(u\) to rule out the corner solution that \(y^\star=s\). Since the LHS is increasing in \(y\) and the RHS is decreasing in \(y\), there is a unique solution for \(y^\star=g(s)\). Moreover, a larger \(s\) will shift down the LHS so that the optimal policy is larger, i.e. for \(\hat s>s\), \(g(\hat s)>g(s)\). Therefore, \(g\) is increasing in \(s\).