线性回归-最小二乘法-概率视角

$$
\begin{aligned}
&\varepsilon \sim N\left(0, \sigma^{2}\right)\\
&y=f(w)+\varepsilon\\
&f(w)=w^{\top} x_i\\
&\begin{array}{l}
{y_i=w^{\top} x_i+\varepsilon} \\
{y_i | x_i; w \sim N\left(w^{\top} x_i, \sigma^{2}\right)}
\end{array}
\end{aligned}
$$

$$\mathbb{L}(w)=\log P\left (Y|X ; w \right)=\log \prod_{i=1}^{N} p\left(y_{i} | x_{i} ; w\right)=\sum_{i=1}^{N} \log p\left(y_{i} | x_{i}\right)$$

\begin{aligned}
&=\sum_{i=1}^{N} \log \frac{1}{\sqrt{2\pi}\sigma}+\log \operatorname{exp}\left\lbrace-\frac{\left(y_{i}-w^{\top} x_{i}\right)^{2}}{2\sigma^{2}}\right\rbrace \\
&=\sum_{i=1}^{N} \log \frac{1}{\sqrt{2 \pi} \sigma}-\frac{1}{2\sigma^{2}}\left(y_{i}-w^{\top} x_{i}\right)^{2}
\end{aligned}

\begin{aligned}
\hat{w} &=\arg\max_{w} \mathbb{L}(w) \\
&=\arg\max_{w}\sum_{i=1}^{N}-\frac{1}{2\sigma^2} \left(y_i-w^{\top} x_{i}\right)^2 \\
&=\arg\min_{w}\sum_{i=1}^{N}\left(y_i-w^{\top} x_{i}\right)^2
\end{aligned}

带有均值为$0$,方差为$\sigma^2$的高斯噪声的最大似然估计等价于最小二乘法。

非常感谢各位老板投喂!