Mathematical Approaches to AI Safety

A Metauni talk about AI Safety by Elliot Catt, a worker at deepmind.

Notes

• AI systems are highly capable, they might lead to x-risk fates.
• Ideally, we want to run experiments on AGI, though AGI has not been developed yet
• Next best thing is to use Universal Artificial Intelligence, which is a mathematical model that tells us how optimal ai systems will act
• Notations will be:
  • $\mathcal{A},\mathcal{O},\mathcal{R}$ are the finite sets of actions, observations and rewards repectively
  • $ϵ:=\mathcal{O}\times \mathcal{R}$ to denote set of percepts (additionally $e_t=o_t{r}_t$)
  • $\mathcal{H}:=(A\times ϵ{)}^{\ast }$ denotes set of finite interaction histories
  • $x_{1:t}=x_1{x}_2\dots x_t$
  • $x_{<t}:=x_1{x}_2\dots x_{t-1}$
  • We use $v$ to denote some environment
• The Value Function