徹底理解BP算法

原創

leo_fighting

2019-08-23 01:21

本文內容

1.最簡單的神經網絡

如下圖所示一個簡單神經網絡,包括2個輸入,2個隱層神經元,2個輸出,輸入和隱層,隱層和輸出之間通過權重W*連接,不考慮使用偏置.

假設輸入 l1 = 0.2, l2 = 0.3; O1目標輸出是0, O2目標輸出是1;

權重 w1 = 0.1, w2 = 0.2, w3 = 0.2, w4 = 0.1;

w5 = 0.3, w6 = 0.4, w7 = 0.4, w8 = 0.3;

2. 前向傳播

$H_{1}^{in} = I_{1}*w_{1} + I_{2}*w_{3} = 0.2*0.1+0.3*0.2=0.08$

$H_{2}^{in} = I_{1}*w_{2} + I_{2}*w_{4} = 0.2*0.2+0.3*0.1=0.07$

$H_{1}^{out} =\frac{1}{1+e^{-H_{1}^{in}}} = \frac{1}{1+e^{-0.08}} = 0.51998934$

$H_{2}^{out} =\frac{1}{1+e^{-H_{2}^{in}}} = \frac{1}{1+e^{-0.07}} = 0.517492858$

$O_{1}^{in} = H_{1}^{out}*w_{5} + H_{2}^{out}*w_{7} = 0.51998934*0.3+0.517492858*0.4=0.362993945$

$O_{2}^{in} = H_{1}^{out}*w_{6} + H_{2}^{out}*w_{8} = 0.51998934*0.4+0.517492858*0.3=0.363243593$

$O_{1}^{out} =\frac{1}{1+e^{-O_{1}^{in}}} = \frac{1}{1+e^{-0.362993945}} = 0.58976499$

$O_{2}^{out} =\frac{1}{1+e^{-O_{2}^{in}}} = \frac{1}{1+e^{-0.363243593}} = 0.589825389$

3.計算損失

$loss_{O1} = \frac{1}{2}*(y_{o1}-O_{1}^{out})^2 = 0.5*(0-0.58976499)^{2} = 0.173911372$

$loss_{O2} = \frac{1}{2}*(y_{o2}-O_{2}^{out})^2 = 0.5*(1-0.589825389)^{2} = 0.084121606$

$loss_{All}= loss_{O1} +loss_{O2} =0.173911372+0.084121606= 0.258032978$

4. 反向傳播

以權重w5爲例, 求總損失關於w5的偏導數

根據鏈式求導法則:

$\frac{\partial loss_{All}}{\partial w_5} = \frac{\partial loss_{All}}{\partial O_{1}^{out}}*\frac{\partial O_{1}^{out}}{\partial w_5} = \frac{\partial loss_{All}}{\partial O_{1}^{out}}*\frac{\partial O_{1}^{out}}{\partial O_{1}^{in}}*\frac{\partial O_{1}^{in}}{\partial w_5}$

$=-(y_{o1}-O_{1}^{out})*O_{1}^{out}*(1-O_{1}^{out})*H_{1}^{out}$

求出偏導後就可以更新w5的權重了,其中 $\eta$ 是學習率

$w^\star_{5} = w_5 - \eta *\frac{\partial loss_{All}}{\partial w_5}$

再以權重w1爲例, 求總損失關於w1的偏導數

根據鏈式求導法則:

$\frac{\partial loss_{All}}{\partial w_1} = \frac{\partial loss_{All}}{\partial H^{in}_1}*\frac{\partial H^{in}_1}{\partial w_1} = \frac{\partial loss_{All}}{\partial H^{out}_1}*\frac{\partial H^{out}_1}{\partial H^{in}_1}*\frac{\partial H^{in}_1}{\partial w_1}$

$=\left ( \frac{\partial loss_{O1}}{\partial O^{in}_1} *\frac{\partial O^{in}_{1}}{\partial H^{out}_1}+ \frac{\partial loss_{O2}}{\partial O^{in}_2}*\frac{\partial O^{in}_{2}}{\partial H^{out}_1}\right )*\frac{\partial H^{out}_1}{\partial H^{in}_1}*\frac{\partial H^{in}_1}{\partial w_1}$

$=\left ( \frac{\partial loss_{O1}}{\partial O^{out}_1}* \frac{\partial O^{out}_1}{\partial O^{in}_1}*\frac{\partial O^{in}_{1}}{\partial H^{out}_1}+ \frac{\partial loss_{O2}}{\partial O^{out}_2}* \frac{\partial O^{out}_2}{\partial O^{in}_2}*\frac{\partial O^{in}_{2}}{\partial H^{out}_1}\right )*\frac{\partial H^{out}_1}{\partial H^{in}_1}*\frac{\partial H^{in}_1}{\partial w_1}$

$=\left ( -(y_{o1}-O_{1}^{out})*O_{1}^{out}*(1-O_{1}^{out})* w_5 -(y_{o2}-O_{2}^{out})*O_{2}^{out}*(1-O_{2}^{out})* w_6 \right )*H^{out}_1*(1-H^{out}_1)*I_{1}$

$=\left (-(0-0.58976499)*0.58976499*(1-0.58976499)*0.3-(1-0.589825389)*0.589825389*(1-0.589825389)*0.4\right )*0.51998934*(1-0.51998934)*0.2 = 0.000155405$

求出w1偏導後就可以更新w1的權重了,其中 $\eta$ 是學習率

$w^\star_{1} = w_1 - \eta *\frac{\partial loss_{All}}{\partial w_1}$

發表評論

所有評論

還沒有人評論，想成為第一個評論的人麼? 請在上方評論欄輸入並且點擊發布.

徹底理解BP算法

1.最簡單的神經網絡

2. 前向傳播

3.計算損失

4. 反向傳播

python gdal 安裝使用（Windows， python 3.6.8）

深入理解L1、L2正則化

SNIP：目標檢測論文解讀

C++ 模板類聲明和實現遇到的問題

MOBILE_FFMPEG_BUILD_DATE 在哪？

TridentNet：處理目標檢測中尺度變化新思路

Mac下配置sublime實現LaTeX

https://yachay.unat.edu.pe/blog/index.php?comment_area=format_blog&comment_component=blog&comment_co

linux以太網驅動總結