Value Function

Definition

Value function $V_v^{\pi }:\mathcal{H}\to [0,1]$ with:

• A Policy $\pi$
• In Environment $v$
• A Discount Factor $y$
• A history $a_1{o}_1{r}_1\dots a_{t-1}{o}_{t-1}{r}_{t-1}=h_{<t}\in \mathcal{H}$ Is defined as: $$V_{v}$\pi(h_{<t}) := \mathbb{E}{v}^{\pi} [ \sum{k=1}^{\infty}y^{k-t}r_{k} | h_{<t}]

- The optimal value is defined as $V_{v}^{*}(h_{<t}) := sup_{\pi}V_{v}^{\pi}(h_{<t})$