Best Approximations are Characterized by Orthogonality

Theorem

With $W$ as a subspace of an inner product space $V$. Let $\beta \in V$. Then, Best approximations are characterized by an orthogonality relation

Explanation

The vector $\alpha$ is a best approximation to $\beta \in W⟺\beta -\alpha$ is Orthogonal to every vector in $W$

Proof

1. We first prove an equality that is useful. Let $\gamma \in V$, then $\beta -\gamma =(\beta -\alpha )+(\alpha -\gamma )$ and $||\beta -\gamma |{|}^2=||\beta -\alpha |{|}^2+2Re(⟨\beta -\alpha |\alpha -\gamma ⟩)+||\alpha -\gamma |{|}^2$ ($\ast$)

Proving $⟸$

Suppose $\beta -\alpha$ is orthogonal to every vector

Proving $⟹$

• Suppose that $||\beta -\alpha |{|}^2\le ||\beta -\gamma |{|}^2$ for every $\gamma \in W$
• It follows that $||\beta -\gamma |{|}^2-||\beta -\alpha |{|}^2\ge 0⟹2Re(⟨\beta -\alpha |\alpha -\gamma ⟩+||\alpha -\gamma |{|}^2>0$, from previous equality $(\ast )$
• Now, $\forall \gamma \in W$, as $\alpha \in W$, and this is true for all $\gamma \in W$, we see that every $r\in W$, $2Re(⟨\beta -\alpha |r⟩)+||r|{|}^2\ge 0$
• If $\gamma \in W,\gamma \ne \alpha$, we may take $r=-\frac{⟨\beta -\alpha |\alpha -\gamma ⟩}{||\alpha -\gamma ||}\cdot ⟨\alpha -\gamma ⟩$. We are subtracting the projection onto $\alpha -\gamma$
• Then, the inequality reduces to the statement $-2\cdot \frac{|⟨\beta -\alpha |\alpha -\gamma ⟩{|}^2}{⟨\alpha -\gamma |\alpha -\gamma ⟩}(\alpha -\gamma )+\frac{|⟨\beta -\alpha |\alpha -\gamma ⟩{|}^2}{⟨\alpha -\gamma |\alpha -\gamma ⟩}\ge 0$
• Then, this is equivalent to $-\frac{|⟨\beta -\alpha |\alpha -\gamma ⟩{|}^2}{||\alpha -\gamma |{|}^2}\ge 0$. Now, this only holds if $⟨\beta -\alpha |\alpha -\beta ⟩=0$, so its perpendicular to every vector in $W$