20180713DLday1課程筆記

Introduction

---

0

• Jian Tang
• [email protected]

1

• History 1950-1970 logic rules; 1980-1990 knowledge acquisition; 2010

-. machine learning

• DeepLearning⊂MachineLearning⊂ArtificialIntelligence$DeepLearning\subset MachineLearning\subset ArtificialIntelligence$
• machine learning
  
  • use statistical techniques, “learn” with data
  • extract features automatically, instead of by domain experts
  • learn automatically, instead of explicit programming
• Big Data-Big Computation-Big Model : Why deep learning now
• usage
  
  • …

2Probability

• Bayes’ Theorem

  • p(Y|X)=p(X|Y)p(Y)p(X),p(X)=∑Yp(X|Y)p(Y)$p(Y|X)=\frac{p(X|Y)p(Y)}{p(X)},p(X)=\sum _{Y}p(X|Y)p(Y)$
  • posterior ∝$\propto$ likelihood * prior

• variables

  • E[f] := the average value of f(X) under the distribution p(x)
  • E[f]=∑xp(x)f(x)$E[f]=\sum _{x}p(x)f(x)$
  • V[f], cov[x, y]

• distributions

  • binomial distribution
  • Bin(m|N,μ)=(Nm)μm(1−μ)N−m$Bin(m|N,\mu )=(\genfrac{}{}{0ex}{}{N}{m}){\mu }^m(1-\mu {)}^{N-m}$
  • E[m]=Nμ,var[m]=Nμ(1−μ)$E[m]=N\mu ,var[m]=N\mu (1-\mu )$

• multinomial variables

  • x可以取k種值，x=(0,0,1,0,0,0)T$x=(0,0,1,0,0,0{)}^T$ 表示x取了六種中的第三種

  μ=(μ1,μ2,...,μk)T$\mu =({\mu }_1,{\mu }_2,...,{\mu }_k{)}^T$ ，對應x向量每個位置上爲1的概率

  從而某個特定的x出現的概率 p(x|μ)=∏k=1Kμxkk$p(x|\mu )=\prod _{k=1}^K{\mu }_k^{x_k}$ (也就是μk${\mu }_k$ )

  • E[x|mu]=∑xp(x|μ)x=(μ1,μ2,...,μk)T=μ$E[x|mu]=\sum _{x}p(x|\mu )x=({\mu }_1,{\mu }_2,...,{\mu }_k{)}^T=\mu$

  • maximum likelihood estimation

  μk=mkN,mk=∑Nxnk觀察值的矩陣的每列和${\mu }_k=\frac{m_k}{N},m_k=\sum _N{x}_{nk}觀察值的矩陣的每列和$

• gaussian univariate distribution正態分佈

  • multivariate gaussian distribution
  • maximum likelihood estimation
  • mixture of gaussians-可以模擬其他各種分佈

• gradient descent梯度下降

  • a way to minimize an object function J(θ)$J(\theta )$
  • η$\eta$ : learning rate, which determines the size of the steps we take to reach a local minimum
  • update equation: θ=θ−η∗∇θJ(θ)$\theta =\theta -\eta \ast {\mathrm{\nabla }}_{\theta }J(\theta )$