Least Squares Method

Least Square Solution Definition

• If $A$ is a $m\times n$ matrix,
• $b\in {\mathbb{R}}^m$
• a least square solution of $Ax=b$ is a vector $\hat{x}\in {\mathbb{R}}^n$ such that:
• $||b-A\hat{x}||\le ||b-Ax||,\forall x\in {\mathbb{R}}^n$ (Best Approximation) You can find $x$ with Least Squares Theorem

Intuition

• With an error vector $e$
• We reduce the error by finding the smallest $e$ (least square) for $||e|{|}^2=⟨e|e⟩$
• With $c=\left[\begin{array}{c}{y}_1\\ y_2\\ ⋮\\ y_n\end{array}\right]$
• With $A=\left[\begin{array}{cc}1& x_1\\ 1& x_2\\ ⋮& \\ 1& x_n\end{array}\right]$
• The vector $e=c-Ax$