An [[Overdetermined Linear System|Overdetermined System of Linear Equations]] is a system of linear equations that does not have enough degrees of freedom. >[!example] >$\large \begin{align} >2x+6y&=3\\ >4x+ y&=1\\ >-2x + y &= -2\\ >8x+ 2y&=4\\ >\end{align}$ > This system is over determined, there is no $x, y$ that satifies all of these equations. In these cases, it is most likely not possible for an exact solution, so you can get an approximation. What is meant by a 'aproximation' is for some [[System of Linear Equations]] represented by a [[Matrix]] $A\vec x = \vec b$, find a [[Vector]] $\vec c$ such that the error between $\vec c$ and $\vec b$ is minimized. Error in this context is denoted the least-square difference: $\large (b_{1}-c_{1})^2+(b_{2}-c_{2})^2 + \cdots + (b_{k} - c_{k})^2 \\ $ Which is the same as the [[Distance]] between $\vec b$ and $\vec u$ squared. $ \large \lvert \lvert \vec b - \vec c \rvert \rvert $ If $A\vec x=\vec b$ is [[Consistent]], then the closest [[Approximation]] is $\vec x$, which is the same thing as $\vec x \in U$ where $U=\op{Col}(A)$. If the equation is [[Inconsistent]], then the approximation would be the [[Orthogonal Projection]] of $\vec b$ unto $A$. $\large \begin{align} A \vec x &= \vec b\\ A^\intercal A \vec x &= A^\intercal \vec b \\ \end{align} $ #### Scatter plots Given some list of numbers $\vec x, \vec y$. | $x$ | $y$ | | -------- | -------- | | $x_1$ | $y_{1}$ | | $x_{2}$ | $y_{2}$ | | $\vdots$ | $\vdots$ | In order to connection each pair of $(x,y)$ with a trend line constitutes a [[Overdetermined Linear System|Overdetermined System]]. The error would be the least-squares formula: $\large \sum_{i=1}^n (x_i-f(x_{i}))^2 $ $\large \begin{align} ax_{1}+b &= y_{1} \\ ax_{2} + b &= y_{2} \\ &\vdots\\ ax_{n} + b &= y_{n} \\ \end{align} $ $\large \begin{align} \underbrace{\mat{ x_{1}&1\\ x_{2}&1 \\ \vdots&\vdots \\ x_{n} & 1 }}_{\huge X} \underbrace{ \mat{a\\b}}_{\huge {\vec a}} &= \underbrace{ \mat{y_{1}\\y_{2}\\\vdots\\y_{n}} }_{{\vec y}} \end{align}$ $\large \begin{align} X\vec a &= \vec y\\ X^\intercal X \vec a &= X^\intercal \vec b \\ \end{align}$ >[!example] >![[../../06 Mind Dump/202503260930|202503260930]]