【whr的深度學習總結1】使用Matconvnet訓練imbalance全連接網絡

matconvnet只提供了卷積函數，並沒有提供全連接函數，那麼如何在卷積函數上訓練全連接呢？

首先，我們要清楚一件事：卷積核爲1*1同時步長是1的網絡就是全連接。

那麼配置網絡的時候就只需執行卷積函數，同時配置卷積核的大小就可以。

這是我的配置文件：

function net = cnn_crp_init(varargin)
% CNN_MNIST_LENET Initialize a CNN similar for MNIST
opts.batchNormalization = true ;
opts.networkType = 'simplenn' ;
opts = vl_argparse(opts, varargin) ;

rng('default');
rng(0) ;

f=1/100 ;
net.layers = {} ;
net.layers{end+1} = struct('type', 'conv', ...
                           'weights', {{f*randn(1,1,5000,200, 'single'), zeros(1, 200, 'single')}}, ...
                           'stride', 1, ...
                           'pad', 0) ;
net.layers{end+1} = struct('type', 'relu') ;
net.layers{end+1} = struct('type', 'conv', ...
                           'weights', {{f*randn(1,1,200,200, 'single'), zeros(1,200,'single')}}, ...
                           'stride', 1, ...
                           'pad', 0) ;
net.layers{end+1} = struct('type', 'relu') ;                       
net.layers{end+1} = struct('type', 'conv', ...
                           'weights', {{f*randn(1,1,200,2, 'single'), zeros(1,2,'single')}}, ...
                           'stride', 1, ...
                           'pad', 0) ;                       
net.layers{end+1} = struct('type', 'softmaxloss') ;

% optionally switch to batch normalization
if opts.batchNormalization
  net = insertBnorm(net, 1) ;
  net = insertBnorm(net, 3) ;
  net = insertBnorm(net, 5) ;
end

% Meta parameters
net.meta.inputSize = [1 1 5000] ;
net.meta.trainOpts.learningRate = 0.001 ;
net.meta.trainOpts.numEpochs = 20 ;
net.meta.trainOpts.batchSize = 100 ;

% Fill in defaul values
net = vl_simplenn_tidy(net) ;

% Switch to DagNN if requested
switch lower(opts.networkType)
  case 'simplenn'
    % done
  case 'dagnn'
    net = dagnn.DagNN.fromSimpleNN(net, 'canonicalNames', true) ;
    net.addLayer('error', dagnn.Loss('loss', 'classerror'), ...
             {'prediction','label'}, 'error') ;
  otherwise
    assert(false) ;
end

% --------------------------------------------------------------------
function net = insertBnorm(net, l)
% --------------------------------------------------------------------
assert(isfield(net.layers{l}, 'weights'));
ndim = size(net.layers{l}.weights{1}, 4);
layer = struct('type', 'bnorm', ...
               'weights', {{ones(ndim, 1, 'single'), zeros(ndim, 1, 'single')}}, ...
               'learningRate', [1 1 0.05], ...
               'weightDecay', [0 0]) ;
net.layers{l}.biases = [] ;
net.layers = horzcat(net.layers(1:l), layer, net.layers(l+1:end)) ;

然後需要單獨配置網絡輸入的大小，由於matcovnet傳承了Caffe的配置，所以輸入數據是四維的：長寬通道數以及索引。我們的數據是5000*9963，所以我將輸入配置改成：

data = single(reshape(F',1,1,5000,[]));
set = [ones(1,size(res.train{c},1)) 3*ones(1,size(res.test{c},1))];

將5000作爲通道數就可以了，即1*1*5000

這裏的數據是不平衡的，所以按照之前訓練出來的結果是偏向負樣本的（負樣本多）。所以我就使用了一個策略：在minibatch裏保證正負樣本一樣多。

這裏填上我所寫的代碼:

訓練時：

for t=1:opts.batchSize/2:size(subset_n,4)
  %t1=t1+opts.batchSize;
  batchSize= min(floor(opts.batchSize/2), size(subset_n,4) - t + 1) ;
  batchSize_p=batchSize;
  batchSize_n=batchSize-batchSize_p;

  fprintf('%s: epoch %02d: %3d/%3d: ', mode, epoch, ...
          fix(t/opts.batchSize)+1, ceil(size(subset_n,4)/opts.batchSize)) ;

  numDone = 0 ;
  error = [] ;
  for s=1:opts.numSubBatches
    % get this image batch and prefetch the next
    batchStart = t + (labindex-1) + (s-1) * numlabs ;
    batchEnd = min(t+opts.batchSize/2-1, size(subset_n,4)) ;
    batch = batchStart : opts.numSubBatches * numlabs : batchEnd ;
    [im, labels] = getBatch(imdb, batch, batchSize_p) ;
   
    if opts.prefetch
      if s==opts.numSubBatches
        batchStart = t + (labindex-1) + opts.batchSize ;
        batchEnd = min(t+2*opts.batchSize-1, size(subset_n,4)) ;
      else
        batchStart = batchStart + numlabs ;
      end
      nextBatch = subset(batchStart : opts.numSubBatches * numlabs : batchEnd) ;
      getBatch(imdb, nextBatch) ;
    end

提取數據時：

function [images, labels] = getSimpleNNBatch(imdb, batch, batchSize_p)
% --------------------------------------------------------------------
allImageN = imdb.images.data(:,:,:,imdb.images.labels.train{imdb.images.class}(:,2)==-1);
images_n = allImageN(:,:,:,batch);
allImageP = imdb.images.data(:,:,:,imdb.images.labels.train{imdb.images.class}(:,2)==1); 
idx = randperm(size(allImageP,4));
allImageP = allImageP(:,:,:,idx);
images_p = allImageP(:,:,:,1:batchSize_p);
images = cat(4,images_p,images_n);
im_num = size(images,4);
labels = [ones(size(images_p,4),1);-1.*ones(size(images_n,4),1)];
im_indx = randperm(im_num);
labels = labels(im_indx,:)';
images = images(:,:,:,im_indx);

這樣就保證了數據訓練不會太偏斜，使得訓練效果並不好。

附件：在我的github裏提供了完整的代碼。歡迎大家賞光。