1. Sequential
require ‘nn’;
mlp = nn.Sequential()
mlp:add(nn.Linear(10, 25)) -- Linear module (10 inputs, 25 hidden units)
mlp:add(nn.Tanh()) -- apply hyperbolic tangent transfer function on each hidden units
mlp:add(nn.Linear(25, 1)) -- Linear module (25 inputs, 1 output)
> mlp
nn.Sequential {
[input -> (1) -> (2) -> (3) -> output]
(1): nn.Linear(10 -> 25)
(2): nn.Tanh
(3): nn.Linear(25 -> 1)
}
> print(mlp:forward(torch.randn(10)))
-0.1815
[torch.Tensor of dimension 1]
插入網絡層
model = nn.Sequential()
model:add(nn.Linear(10, 20))
model:add(nn.Linear(20, 30))
model:insert(nn.Linear(20, 20), 2)
> model
nn.Sequential {
[input -> (1) -> (2) -> (3) -> output]
(1): nn.Linear(10 -> 20)
(2): nn.Linear(20 -> 20) -- The inserted layer
(3): nn.Linear(20 -> 30)
}刪除網絡層
model = nn.Sequential()
model:add(nn.Linear(10, 20))
model:add(nn.Linear(20, 20))
model:add(nn.Linear(20, 30))
model:remove(2)
> model
nn.Sequential {
[input -> (1) -> (2) -> output]
(1): nn.Linear(10 -> 20)
(2): nn.Linear(20 -> 30)
}2. Parallel
module = Parallel( inputDimension, outputDimension )
mlp = nn.Parallel(2,1); -- Parallel container will associate a module to each slice of dimension 2
-- (column space), and concatenate the outputs over the 1st dimension.
mlp:add(nn.Linear(10,3)); -- Linear module (input 10, output 3), applied on 1st slice of dimension 2
mlp:add(nn.Linear(10,2)) -- Linear module (input 10, output 2), applied on 2nd slice of dimension 2
-- After going through the Linear module the outputs are
-- concatenated along the unique dimension, to form 1D Tensor
> mlp:forward(torch.randn(10,2)) -- of size 5.
-0.5300
-1.1015
0.7764
0.2819
-0.6026
[torch.Tensor of dimension 5]
3. Concat
module = nn.Concat( dim )
mlp = nn.Concat(1);
mlp:add(nn.Linear(5,3))
mlp:add(nn.Linear(5,7))
> print(mlp:forward(torch.randn(5)))
0.7486
0.1349
0.7924
-0.0371
-0.4794
0.3044
-0.0835
-0.7928
0.7856
-0.1815
[torch.Tensor of dimension 10]
4. Convolutional Neural Network
net = nn.Sequential()
net:add(nn.SpatialConvolution(1, 6, 5, 5)) -- 1 input image channel, 6 output channels, 5x5 convolution kernel
net:add(nn.ReLU()) -- non-linearity
net:add(nn.SpatialMaxPooling(2,2,2,2)) -- A max-pooling operation that looks at 2x2 windows and finds the max.
net:add(nn.SpatialConvolution(6, 16, 5, 5))
net:add(nn.ReLU()) -- non-linearity
net:add(nn.SpatialMaxPooling(2,2,2,2))
net:add(nn.View(16*5*5)) -- reshapes from a 3D tensor of 16x5x5 into 1D tensor of 16*5*5
net:add(nn.Linear(16*5*5, 120)) -- fully connected layer (matrix multiplication between input and weights)
net:add(nn.ReLU()) -- non-linearity
net:add(nn.Linear(120, 84))
net:add(nn.ReLU()) -- non-linearity
net:add(nn.Linear(84, 10)) -- 10 is the number of outputs of the network (in this case, 10 digits)
net:add(nn.LogSoftMax()) -- converts the output to a log-probability. Useful for classification problems
print('Lenet5\n' .. net:__tostring());
5. Training
Loss Function（Criterion）
- 對於迴歸或二分類問題，可採用Mean Squared Error loss（MSE） ：
criterion = nn.MSECriterion()- 對於多分類問題或輸出層爲 Log-Softmax 的網絡，可採用Negative Log-likelihood ：
criterion = nn.ClassNLLCriterion() Stochastic Gradient Descent
trainer = nn.StochasticGradient(mlp, criterion)
trainer.learningRate = 0.001
trainer.maxIteration = 5
trainer:train(trainset)數據集有兩點要求：首先dataset[index]返回第Index個數據；其次就是dataset:size()函數返回整個數據集的example的個數。
以上爲一種簡單的訓練方式，更多的模型優化方式可參考Optimization package
參考資料
