数据

$$
\begin{aligned}
&\lbrace\left(x_{i}, y_{i}\right)\rbrace_{i=1}^{N}\\
&x_{i} \in \mathbb{R}^{p}, \quad y_{i} \in \lbrace 0,1\rbrace
\end{aligned}
$$

Sigmoid Function

$$
\sigma(z)=\frac{1}{1+e^{-z}}
$$

$$
\begin{cases}
{p_{1}=p(y=1 | x)=\sigma\left(w^{\top} x\right)=\frac{1}{1+e^{-w^{T} x}}, \quad y=1} \\
{p_{0}=p(y=0 | x)=1-p(y=1| x)=\frac{e^{-w^{T} x}}{1+e^{-w^{T} x}}, \quad y=0}
\end{cases}
$$

$$P(y | x)=p_1^{y}p_0^{1-y}$$

最大似然估计MLE:
$$
\begin{aligned}
\hat{w}&=\arg\max {w} \log P(Y | X) \\
&=\arg\max_w \log \prod{i=1}^{N} p\left(y_{i} | x_{i}\right) \\
&=\arg\max_w \sum_{i=1}^{N} \log p\left(y_{i} | x_{i}\right) \\
&=\arg\max_w \sum_{i=1}^{N}\left(y_{i} \log p_{1}+\left(1-y_{i}\right) \log p_{0}\right) \\
&=\arg\max_w \sum_{i=1}^{N} y_{i} \log \psi\left(x_{i}, w\right)+\left(1-y_{i}\right) \log \left(1-\psi\left(x_{i} ; w\right)\right)
\end{aligned}
$$