Week9_3Program_Anomaly Detection and Recommender Systems編程解析

Week9_3Program_Anomaly Detection and Recommender Systems編程解析

1. Anomaly detection分析

1.1 Estimate gaussian parameters 0 / 15

計算公式:
μi=1m∑j=1mx(j)i${\mu }_i=\frac{1}{m}\sum _{j=1}^m{x}_i^{(j)}$
σ2i=1m∑j=1m(x(j)i−μi)2${\sigma }_i^2=\frac{1}{m}\sum _{j=1}^m(x_i^{(j)}-{\mu }_i{)}^2$
X=(300x2) K=3 centroids=(3x2) idx=(300x1)
在 estimateGaussian.m 中添加

mu = mean(X);
sigma2 = var(X,opt=1);

octave中mean函數的作用:
mean (x) = SUM_i x(i) / N
octave中的var函數的作用:
var (x) = 1/(N-1) SUM_i (x(i) - mean(x))^2

1.2 Select threshold 0 / 15

計算公式:
p(x;μ,σ2)=12πσ2√e−(x−μ)22σ2$p(x;\mu ,{\sigma }^2)=\frac{1}{\sqrt{2\pi {\sigma }^2}}{e}^{-\frac{(x-\mu {)}^2}{2{\sigma }^2}}$
上述求p的過程己經在 multivariateGaussian.m 中完成了, 直接用就行,程序中是pval
下面是 selectThreshold.m中用到的公式:
precison=tp+fptp$precison=\frac{tp+fp}{tp}$
recall=tp+fntp$recall=\frac{tp+fn}{tp}$
F1=prec+rec2∗prec∗rec$F_1=\frac{prec+rec}{2\ast prec\ast rec}$

上圖是lecture 11中的課件截圖
* 將距離u1最近的找出來,取個平均值作爲新的u1 *

  predictions = (pval < epsilon);

  truePositives  = sum((predictions == 1) & (yval == 1));
  falsePositives = sum((predictions == 1) & (yval == 0));
  falseNegatives = sum((predictions == 0) & (yval == 1));

  precision = truePositives / (truePositives + falsePositives);
  recall = truePositives / (truePositives + falseNegatives);

  F1 = (2 * precision * recall) / (precision + recall);

2. Recommender Systems

2.1 Collaborative filtering cost 0 / 20

計算公式:
J(x(1),...,x(nm),θ(1),...,θ(nu))=12∑(i,j):r(i,j)=1((θ(j))Tx(i)−y(i,j))2$J(x^{(1)},...,x^{(n_m)},{\theta }^{(1)},...,{\theta }^{(n_u)})=\frac{1}{2}\sum _{(i,j):r(i,j)=1}{\left(({\theta }^{(j)}{)}^T{x}^{(i)}-y^{(i,j)}\right)}^2$

X(5x3) θ$\theta$ (4x3) Y(5x4)
在 cofiCostFunc.m 中添加, 實現沒有正則化項的costFunction

error = (X*Theta'-Y) .* R;
J = (1/2)*sum(sum(error .^ 2));

2.2 Collaborative filtering gradient 0 / 30

計算公式:
∂J∂x(i)k=∑j:r(i,j)=1((θ(j))Tx(i)−y(i,j))θ(j)k$\frac{\mathrm{\partial }J}{\mathrm{\partial }{x}_k^{(i)}}=\sum _{j:r(i,j)=1}\left(({\theta }^{(j)}{)}^T{x}^{(i)}-y^{(i,j)}\right){\theta }_k^{(j)}$
∂J∂θ(j)k=∑i:r(i,j)=1((θ(j))Tx(i)−y(i,j))x(i)k$\frac{\mathrm{\partial }J}{\mathrm{\partial }{\theta }_k^{(j)}}=\sum _{i:r(i,j)=1}\left(({\theta }^{(j)}{)}^T{x}^{(i)}-y^{(i,j)}\right)x_k^{(i)}$

# 下面兩行是2.1中添加的,計算costFunction   
error = (X*Theta'-Y) .* R;
J = (1/2)*sum(sum(error .^ 2)); 
# 下面兩行是2.2中的,梯度下降計算  
X_grad = error * Theta ;
Theta_grad = error' * X ;

2.3 Regularized cost 0 / 10

計算公式:
JnoReg(x(1),...,x(nm),θ(1),...,θ(nu))=12∑(i,j):r(i,j)=1((θ(j))Tx(i)−y(i,j))2$JnoReg(x^{(1)},...,x^{(n_m)},{\theta }^{(1)},...,{\theta }^{(n_u)})=\frac{1}{2}\sum _{(i,j):r(i,j)=1}{\left(({\theta }^{(j)}{)}^T{x}^{(i)}-y^{(i,j)}\right)}^2$
reg=λ2∑j=1nu∑k=1n(θ(j)k)2+λ2∑j=1nm∑k=1n(x(j)k)2$reg=\frac{\lambda }{2}\sum _{j=1}^{n_u}\sum _{k=1}^n({\theta }_k^{(j)}{)}^2+\frac{\lambda }{2}\sum _{j=1}^{n_m}\sum _{k=1}^n(x_k^{(j)}{)}^2$
J=JnoReg+reg$J=JnoReg+reg$

error = (X*Theta'-Y) .* R;
J_noReg = (1/2)*sum(sum(error .^ 2));
X_grad = error * Theta ;
Theta_grad = error' * X ;
# 下面實現正則化的costFunction
costRegLeft = lambda/2 * sum(sum(Theta.^2));
costRegRight = lambda/2 * sum(sum(X.^2));
Reg = costRegLeft + costRegRight;
J = J_noReg + Reg;

2.4 Gradient with regularization 0 / 10

計算公式:
XgradnoReg=∂J∂x(i)k=∑j:r(i,j)=1((θ(j))Tx(i)−y(i,j))θ(j)k$XgradnoReg=\frac{\mathrm{\partial }J}{\mathrm{\partial }{x}_k^{(i)}}=\sum _{j:r(i,j)=1}\left(({\theta }^{(j)}{)}^T{x}^{(i)}-y^{(i,j)}\right){\theta }_k^{(j)}$
ThetagradnoReg=∂J∂θ(j)k=∑i:r(i,j)=1((θ(j))Tx(i)−y(i,j))x(i)k$ThetagradnoReg=\frac{\mathrm{\partial }J}{\mathrm{\partial }{\theta }_k^{(j)}}=\sum _{i:r(i,j)=1}\left(({\theta }^{(j)}{)}^T{x}^{(i)}-y^{(i,j)}\right)x_k^{(i)}$
XReg=λx(i)k$XReg=\lambda x_k^{(i)}$
ThetaReg=λθ(i)k$ThetaReg=\lambda {\theta }_k^{(i)}$
ThetaGrad=ThetagradnoReg+XReg$ThetaGrad=Thet{a}_{g}ra{d}_{n}oReg+XReg$
ThetaGrad=ThetagradnoReg+ThetaReg$ThetaGrad=Thet{a}_{g}ra{d}_{n}oReg+ThetaReg$

# 下面是計算costFuncton,分兩步先計算不帶cost的J,再計算reg項
error = (X*Theta'-Y) .* R;
J_noReg = (1/2)*sum(sum(error .^ 2));
costRegLeft = lambda/2 * sum(sum(Theta.^2));
costRegRight = lambda/2 * sum(sum(X.^2));
Reg = costRegLeft + costRegRight;
J = J_noReg + Reg;

# 下面是計算grad,分兩步先計算不帶reg的grad,再計算reg
X_grad_noReg = error * Theta ;
Theta_grad_noReg = error' * X ;

X_grad = X_grad_noReg + lambda * X;
Theta_grad = Theta_grad_noReg + lambda * Theta;

3. 總結

1 Estimate gaussian parameters 0 / 15
2 Select threshold 0 / 15
3 Collaborative filtering cost 0 / 20
4 Collaborative filtering gradient 0 / 30
5 Regularized cost 0 / 10
6 Gradient with regularization 0 / 10