基本概念
邏輯迴歸是一種概率型非線性迴歸模型。雖然名字裏面有迴歸,但是他其實是一種分類的方法,通常是用來研究在某些條件下某個結果會不會發生,例如:已知病人身體裏的腫瘤的情況,然後判斷這個腫瘤是良性還是惡性。
邏輯迴歸與線性迴歸
邏輯迴歸同線性迴歸一樣都需要有一個假設函數
在線性迴歸中:
但是我們引入了一個
因此在邏輯迴歸中的假設函數就是
當
當
代價函數J(θ)
在求解參數
目標就是求這個函數的值最大的時候的參數
lnL(θ)=Σmi=1[y(i)ln(hθ(x(i)))+(1−y(i))ln(1−hθ(x(i)))]
lnL(θ)=Σmi=1(y(i)ln(11+e−x(i))+(1−y(i))ln(1−11+e−x(i)))
lnL(θ)=Σmi=1(y(i)ln(ex(i)ex(i)+1)+(1−y(i))ln(1ex(i)+1))
lnL(θ)=Σmi=1(y(i)∗lnex(i)−y(i)∗ln(1+ex(i))−(1−y(i))∗ln(1+ex(i)))
lnL(θ)=Σmi=1(x(i)y(i)−ln(1+ex(i)))
我們要求得
爲了求得
對
因此完整的梯度下降應該爲:
Matlab code
function [J, grad] = costFunction(theta, X, y)
%COSTFUNCTION Compute cost and gradient for logistic regression
% J = COSTFUNCTION(theta, X, y) computes the cost of using theta as the
% parameter for logistic regression and the gradient of the cost
% w.r.t. to the parameters.
% Initialize some useful values
m = length(y); % number of training examples
% You need to return the following variables correctly
J = 0;
grad = zeros(size(theta));
% ====================== YOUR CODE HERE ======================
% Instructions: Compute the cost of a particular choice of theta.
% You should set J to the cost.
% Compute the partial derivatives and set grad to the partial
% derivatives of the cost w.r.t. each parameter in theta
%
% Note: grad should have the same dimensions as theta
%
z = X*theta;
hx = 1 ./ (1 + exp(-z));
J = -1/m *sum([y'*log(hx) + (1-y)'*log(1-hx)]);
%J = 1/m * sum([-y' * log(hx) - (1 - y)' * log(1 - hx)]);
for j = 1:length(theta)
grad(j)=1/m*sum((hx-y)'*X(:,j));
end;
% =============================================================
end
function g = sigmoid(z)
%SIGMOID Compute sigmoid functoon
% J = SIGMOID(z) computes the sigmoid of z.
% You need to return the following variables correctly
g = zeros(size(z));
% ====================== YOUR CODE HERE ======================
% Instructions: Compute the sigmoid of each value of z (z can be a matrix,
% vector or scalar).
sz = size(z);
for i = 1:sz(1),
for j = 1:sz(2),
g(i,j) = 1./(1+exp(-z(i,j)));
end;
end;
% =============================================================
end
function p = predict(theta, X)
%PREDICT Predict whether the label is 0 or 1 using learned logistic
%regression parameters theta
% p = PREDICT(theta, X) computes the predictions for X using a
% threshold at 0.5 (i.e., if sigmoid(theta'*x) >= 0.5, predict 1)
m = size(X, 1); % Number of training examples
% You need to return the following variables correctly
p = zeros(m, 1);
% ====================== YOUR CODE HERE ======================
% Instructions: Complete the following code to make predictions using
% your learned logistic regression parameters.
% You should set p to a vector of 0's and 1's
%
pp = sigmoid(X*theta);
pos = find(pp>=0.5);
p(pos,1)=1;
% =========================================================================
end