w4_2Multi-class Classification and Neural Networks編程解析

w4_2Multi-class Classification and Neural Networks編程解析

1. ex3.m

1.1 ex3.m框架分析

把數據可視化, 在plotData.m中添加:

input_layer_size  = 400;  % 20x20 Input Images of Digits
num_labels = 10;          % 10 labels, from 1 to 10
                          % (note that we have mapped "0" to label 10)

# 第1步加載數據並顯示,是裏面有各種手寫0-9的數字網格
load('ex3data1.mat'); % training data stored in arrays X, y
m = size(X, 1);
rand_indices = randperm(m);
sel = X(rand_indices(1:100), :);
displayData(sel);


#第2.1步 Vectorize Logistic Regression 
theta_t = [-2; -1; 1; 2];
X_t = [ones(5,1) reshape(1:15,5,3)/10];
y_t = ([1;0;1;0;1] >= 0.5);
lambda_t = 3;
[J grad] = lrCostFunction(theta_t, X_t, y_t, lambda_t);

# 第2.2 步 實現one-vs-All Training, 將手寫的0-9圖片區分出來
lambda = 0.1;
[all_theta] = oneVsAll(X, y, num_labels, lambda);

# 第3步進行預測
pred = predictOneVsAll(all_theta, X);

1.2 Regularized logistic regression 0 / 30

分兩步,先計算J(θ)$J(\theta )$ 再計算θ$\theta$

1.2.1 logistic regression Cost Function

帶正則化項的logistic迴歸的損失函數:
J(θ)=1m∑i=1m[−y(i)log(hθ(x(i)))−(1−y(i))log(1−hθ(x(i)))]+λ2m∑j=1nθ2j$J\left(\theta \right)=\frac{1}{m}\sum _{i=1}^m\left[-y^{(i)}\log \left(h_{\theta }(x^{(i)})\right)-\left(1-y^{(i)}\right)\log \left(1-h_{\theta }(x^{(i)})\right)\right]+\frac{\lambda }{2m}\sum _{j=1}^n{\theta }_j^2$

h_theta = sigmoid(X*theta);
left = -(y'*log(h_theta));
right = (1-y)'*log(1-h_theta);
theta_line_num = length(theta);
reg = sum(theta(2:theta_line_num,:) .^ 2);
J = 1/m*(left-right)+lambda/(2*m)*reg;

• 第1行: 先計算hθ$h_{\theta }$ , 注意這兒的sigmoid函數己經寫好了,在文件sigmoid.m中
• 第2行: 計算左半部分left=∑i=1m−y(i)log(hθ(x(i)))$\sum _{i=1}^m-y^{(i)}\log \left(h_{\theta }(x^{(i)})\right)$
• 第3行: 計算右半部分right=∑i=1m(1−y(i))log(1−hθ(x(i)))$\sum _{i=1}^m\left(1-y^{(i)}\right)\log \left(1-h_{\theta }(x^{(i)})\right)$
• 第4行: 計算列向量theta的長度: 因爲θ$\theta$ 是(28x1)的列向量,θ0${\theta }_0$ 不作爲penalty項,所以要剔除
• 第5行: 從列向量theta的第2行到最後一行取平方,再求和,得到正則化項
• 第6行: 將上述的left right reg,代入到公式,即可求得J(θ)$J(\theta )$

1.2.2 Gradient for regularized LR

下面計算grad:

h_theta = sigmoid(X*theta);  --> 這個上面己經有了
left_grad = X' * (h_theta-y);
grad = 1/m*left_grad + lambda/m*theta;
grad(1) = 1/m*(X(:,1))'*(h_theta-y);

θ$\theta$ 的更新公式：
∂J(θ)∂θj=(1m∑i=1m(hθ(x(i))−y(i))x(i)j)+λmθjfor j≥1$\frac{\mathrm{\partial }J\left(\theta \right)}{\mathrm{\partial }{\theta }_j}=\left(\frac{1}{m}\sum _{i=1}^m(h_{\theta }(x^{(i)})-y^{(i)})x_{{}_j}^{(i)}\right)+\frac{\lambda }{m}{\theta }_{j}forj\ge 1$
∂J(θ)∂θj=(1m∑i=1m(hθ(x(i))−y(i))x(i)j)for j=1$\frac{\mathrm{\partial }J\left(\theta \right)}{\mathrm{\partial }{\theta }_j}=\left(\frac{1}{m}\sum _{i=1}^m(h_{\theta }(x^{(i)})-y^{(i)})x_{{}_j}^{(i)}\right)forj=1$

• 第1行: 先計算hθ$h_{\theta }$ ,這個上面己經有了
• 第2,3行: 先計算左括號的部分,然後加上正則化項
• 第4行: 再單獨計算θ0${\theta }_0$ ,也就是代碼裏面的grad(1)

1.3 One-vs-all classifier training 0 / 20

實現 oneVsAll函數, 在oneVsAll.m中

initial_theta = zeros(n + 1, 1);
options = optimset('GradObj', 'on', 'MaxIter', 50);
for c=1:num_labels,
  theta = fmincg (@(t)(lrCostFunction(t, X, (y == c), lambda)), initial_theta, options);
  all_theta(c,:) = theta';
end;

其中fmincg函數的實現是在 fmincg.m 中, 上面函數直接抄的提示中的實現.

1.4 One-vs-all classifier prediction 0 / 20

在 predictOneVsAll.m中實現

all_preds = all_theta * X';
[max_vals, max_ndxs] = max(all_preds);
p = max_ndxs';

2. ex3_nn.m

2.1 ex3_nn.m框架分析

%% Setup the parameters you will use for this exercise


#第1步加載數據並顯示
load('ex3data1.mat');
m = size(X, 1);
sel = randperm(size(X, 1));
sel = sel(1:100);
displayData(X(sel, :));

input_layer_size  = 400;  % 20x20 Input Images of Digits
hidden_layer_size = 25;   % 25 hidden units
num_labels = 10;          % 10 labels, from 1 to 10   
                          % (note that we have mapped "0" to label 10)
# 第2步加載參數
load('ex3weights.mat');

# 第3步進行預測
pred = predict(Theta1, Theta2, X);

#第4步顯示預測結果
rp = randperm(m);
for i = 1:m
    displayData(X(rp(i), :));
    pred = predict(Theta1, Theta2, X(rp(i),:));
    # 按q退出
    s = input('Paused - press enter to continue, q to exit:','s');
    if s == 'q'
      break
    end
end

2.2 Neural network prediction function 0 / 30

3. 總結

3.1