吳恩達機器學習筆記之邏輯迴歸

邏輯迴歸

原來的線性迴歸函數:
$h_{\theta}(x) = {\theta}^T * x$

${\theta}^T * x == \theta \cdot x$
表示兩個向量的內積, 即兩個向量做Dot-Product

在此基礎上增加Sigmoid函數,改成如下的邏輯迴歸函數:
$h_{\theta}(x) = \frac{1} {(1 + e^{-{\theta}^T * x})}$
其中:
$h_{\theta}(x) = g({\theta}^T * x)$
$g(z) = \frac{1} {(1 + e^{-z})}$

決策界限

傳統的線性迴歸函數 ${\theta}^T x$ 假如表示成如下:

${\theta}_0 + {\theta}_1x_1 + {\theta}_2x_2$

令${\theta}=\left[\begin{matrix}-3 \\1\\1\end{matrix}\right]$,可知：
$x_1+x_2=3$就是這個決策界限函數.

邏輯迴歸代價函數的簡單寫法

$Cost( h_{\theta}(x), y) = -y*log(h_{\theta}(x)) - (1-y)*log(1 - h_{\theta}(x))$

注意:這裏的log()函數相當於ln(),即以e爲底的對數.

最終代價函數如下:
$J(\theta) = 1/m * \sum_{i=1}^m Cost(h_{\theta}(x^{(i)}), y^{(i)})$
$J(\theta) = -1/m * \sum_{i=1}^m [y^{(i)} log(h_{\theta}(x^{(i)})) + (1-y^{(i)})log(1 - h_{\theta}(x^{(i)})]$

最終對上式求得偏導數如下:
$\frac{\partial}{\partial \theta_j}J(\theta)=1/m*\sum_{i=1}^m (h_{\theta}(x^{(i)})-y^{(i)})x_j^{(i)}$
其中求導過程如下:
這裏對複合函數求導：
令:
$f_1(\theta_j)=y^{(i)} log(h_{\theta}(x^{(i)}))$
$u(\theta_j)=h_{\theta}(x^{(i)}) ;g(u)=y^{(i)}log(u);$
$\therefore f_1'(\theta_j)=g'(u)*u'(\theta_j)$
$\therefore f_1'(\theta_j)=\frac{y^{(i)}}{u(\theta_j)}*u'(\theta_j)=\frac{y^{(i)}}{h_{\theta}(x^{(i)})}*u'(\theta_j)$

因爲:$log'(x)=1/x$
$(e^x)'=e^x$
$g'(z)=g(z)*(1-g(z))$

$u'(\theta_j)= \frac{e^{-{\theta}^T * x}} {(1 + e^{-{\theta}^T * x})^2}*x_j^{(i)}=h_{\theta}(x^{(i)})*(1-h_{\theta}(x^{(i)}))*x_j^{(i)}$
$\therefore f_1'(\theta_j)=y^{(i)}*(1-h_{\theta}(x^{(i)}))*x_j^{(i)}$

又有:
$f_2(\theta_j)= (1-y^{(i)})log(1 - h_{\theta}(x^{(i)})$
最終:
$f_2'(\theta_j)=(1-y^{(i)})* \frac{-h_{\theta}(x^{(i)})*(1-h_{\theta}(x^{(i)}))*x_j^{(i)}}{1 - h_{\theta}(x^{(i)})}$
消去分子分母后得到:
$f_2'(\theta_j)=-(1-y^{(i)})* h_{\theta}(x^{(i)})*x_j^{(i)}$

所以:
$\frac{\partial}{\partial \theta_j}J(\theta)=1/m*\sum_{i=1}^m (f_1'(\theta_j)+f_2'(\theta_j))$
$\frac{\partial}{\partial \theta_j}J(\theta)=-1/m*\sum_{i=1}^m (y^{(i)}-h_{\theta}(x^{(i)}))x_j^{(i)}$
然後提出一個負號,最終求得偏導數如下:
$\frac{\partial}{\partial \theta_j}J(\theta)=1/m*\sum_{i=1}^m (h_{\theta}(x^{(i)})-y^{(i)})x_j^{(i)}$