With some [[../02 Areas/Math/Vector|Vector]] $\vec b$ that is not in some [[../02 Areas/Math/Linear Subspace|Linear Subspace]] $U$, what is the closest vector to $\vec b$, that is inside $U$ ($\vec b_U$). Let $\vec b_\perp$ be a the vector from $\vec b_U$ to $\vec b$ that is [[../02 Areas/Math/Orthogonal|Orthogonal]] to $U$. $\huge \begin{align} \vec b &= \vec b_{U} + \vec b_{\perp}\\ \vec b_{U} &\in U \\ \vec b_{\perp} &\in U^\perp \\ \end{align} $ Suppose that $U$ has a basis $\set{\vec u_{1}, \vec u_{2}, \dots}$. $ U = \op{Col}\underbrace{\mat{\vec u_{1} & \dots \vec u_{k}}}_{{A}}$ $\begin{align} U^\perp &= \op{Nul}\left( A^\intercal \right) \\ A^\intercal \vec b &= A^\intercal(\vec b_{U} + \vec b_{\perp}) \\ &= A^\intercal \vec b_{U} + A^\intercal \vec b_{\perp \ll a}\\ &= A^\intercal \vec b_{\perp} \end{align} $ $\begin{align} \vec b_{U} &\in \op{Col}{(A)}\\ \let \vec x \in \R^k&: \vec b_{U} = A\vec x \\ A^\intercal \vec b &= A^\intercal \vec b_{U} \\ &= \left( A^\intercal A\right)\vec x\\ {\pa{A^\intercal A}^{-1}} A^\intercal \vec b &= {\pa{A^\intercal A}^{-1}} \left( A^\intercal A\right)\vec x\\ \vec x&= \left( A^\intercal A \right)^{-1}A^\intercal \vec b \\ \vec b_{U} &= A\pa{A^\intercal A}^{-1} A^{\intercal} \vec b \end{align}$