1. 投影矩阵

投影矩阵
$$P=A(A^{\top }A{)}^{-1}{A}^{\top }$$

$p=Pb$

• 当$b\perp C(A)$,$p=A(A^{\top }A{)}^{-1}{A}^{\top }b=0;p=0$
• 当$b\in C(A),AX=b,p=A(A^{\top }A{)}^{-1}{A}^{\top }AX=A\{(A^{\top }A{)}^{-1}{A}^{\top }A\}X=AX=b$

• 其他情况
  $b=p+e$
  $p=Pb$
  $e=(I-P)b$

证明：当$A$的各列线性无关时，$A^{\top }A$可逆。

$$\begin{array}{rl}& 假设A^{\top }AX=0，我们只需要证明X=0。\\ & A^{\top }AX=0⟺X^{\top }{A}^{\top }AX=X^{\top }0=0\\ & (AX{)}^{\top }AX=0⟺y^{\top }y=0\\ & y^{\top }y=0\Rightarrow y=0⟺AX=0\\ & 因为A各列线性无关，所以X=0\end{array}$$

2. 最小二乘法

给出三个点$(1,1),(2,2),(3,2)$，让拟合成直线$y=C+Dt$的形式。

拟合直线实际上就是
$$\min ||AX-b|{|}^2=||e|{|}^2=e_1^2+e_2^2+e_3^2$$

先根据点和直线方程写出对应的矩阵
$$\begin{gathered} C+D=1 \\ C+2D=2 \\ C+3D=2 \end{gathered}$$
$$\begin{gathered} AX=b \\ A=\left[\begin{array}{cc}1& 1\\ 1& 2\\ 1& 3\end{array}\right] \\ X=\left[\begin{array}{c}C\\ D\end{array}\right] \\ b=\left[\begin{array}{c}1\\ 2\\ 2\end{array}\right] \end{gathered}$$

根据投影公式，我们先来求投影在列空间上的
$$\hat{X}=\left[\begin{array}{c}\hat{C}\\ \hat{D}\end{array}\right]$$
$$\begin{gathered} A\hat{X}=p \\ e=b-p=b-A\hat{X} \\ e\perp A \\ {A}^{\top }(b-A\hat{X})=0 \\ {A}^{\top }A\hat{X}=A^{\top }b \end{gathered}$$

$$A^{\top }A=\left[\begin{array}{ccc}1& 1& 1\\ 1& 2& 3\end{array}\right]\left[\begin{array}{cc}1& 1\\ 1& 2\\ 1& 3\end{array}\right]=\left[\begin{array}{cc}3& 6\\ 6& 14\end{array}\right]$$
$A^{\top }A可逆$

$$A^{\top }b=\left[\begin{array}{ccc}1& 1& 1\\ 1& 2& 3\end{array}\right]\left[\begin{array}{c}1\\ 2\\ 2\end{array}\right]=\left[\begin{array}{c}5\\ 11\end{array}\right]$$

近似方程
$$\begin{gathered} \left[\begin{array}{cc}3& 6\\ 6& 14\end{array}\right]\left[\begin{array}{c}\hat{C}\\ \hat{D}\end{array}\right]=\left[\begin{array}{c}5\\ 11\end{array}\right]⟺\{\begin{array}{ll}& 3\hat{C}+6\hat{D}=5\\ & 2\hat{C}+14\hat{D}=11\end{array} \\ \hat{C}=\frac{2}{3} \\ \hat{D}=\frac{1}{2} \\ y=1/2t+2/3 \end{gathered}$$

• 微积分求法

$$\begin{gathered} \begin{array}{rl}& E(C,D)=e_1^2+e_2^2+e_3^2=(C+D-1{)}^2+(C+2D-2{)}^2+(C+3D-2{)}^2\\ & \frac{\partial E(C,D)}{\partial C}=6C+12D-10\\ & \frac{\partial E(C,D)}{\partial D}=12C+28D-22\end{array} \\ \begin{array}{r}6C+12D-10=0\\ 12C+28D-22=0\end{array} \end{gathered}$$
偏导为0时，取得最小值；也可以计算出其最小值。
$$\begin{gathered} \hat{C}=\frac{2}{3} \\ \hat{D}=\frac{1}{2} \\ y=1/2t+2/3 \end{gathered}$$

可以求出替代点$p_1=(1,\frac{7}{6}),p_2=(2,\frac{5}{3}),p_3=(3,\frac{13}{6})$

这样可以得到误差
$$\begin{gathered} e=\left[\begin{array}{c}{e}_1\\ e_2\\ e_3\end{array}\right]=\left[\begin{array}{c}-\frac{1}{6}\\ \frac{1}{3}\\ -\frac{1}{6}\end{array}\right] \\ p=\left[\begin{array}{c}{p}_1\\ p_2\\ p_3\end{array}\right]=\left[\begin{array}{c}\frac{7}{6}\\ \frac{5}{3}\\ -\frac{13}{6}\end{array}\right] \end{gathered}$$
$$\begin{gathered} e+p=b \\ e\perp p \end{gathered}$$