tensorflow代價敏感因子、增加正則化項、學習率衰減

1.代價敏感：

 

    outputs, end_points = vgg.all_cnn(Xinputs,
                                          num_classes=num_classes,
                                          is_training=True,
                                          dropout_keep_prob=0.5,
                                          spatial_squeeze=True,
                                          scope='all_cnn'

    cross_entrys=tf.nn.softmax_cross_entropy_with_logits(logits=outputs, labels=Yinputs)
    # w_temp = tf.matmul(Yinputs, w_ls) #代價敏感因子w_ls=tf.Variable(np.array(w,dtype='float32')，name="w_ls",trainable=False)，w是權重項鍊表
    # loss=tf.reduce_mean(tf.multiply(cross_entrys,w_temp))  #代價敏感下的交叉熵損失

2. 正則化項:

    weights_norm=tf.reduce_sum(input_tensor=weight_dacay*tf.stack([tf.nn.l2_loss(i) for i in tf.get_collection('weights')]),name='weights_norm' )
    loss=tf.add(cross_entrys,weights_norm) #包含正則化項損失，對應於caffe裏面的weight-decay因子λ，因爲在梯度反向傳遞時'l2-正則化:1/2*λ*||W||^2'對應的更新值就是權重衰減因子，W-△w=w-(△w_分類損失部分+λ*w)=-△w_分類損失部分+(1-λ)*w。通常λ=0.001~0.0005

3. 學習率衰減:

global_step = tf.Variable(0, trainable=False)
add_g=global_step.assign_add(1)
starter_learning_rate = 0.001
decay_steps = 10

#tf.train.下面有多個衰減函數可用
learning_rate = tf.train.exponential_decay(starter_learning_rate, global_step, decay_steps, decay_rate=0.01)

#train_op = tf.train.MomentumOptimizer(learning_rate,0.9).minimize(loss) #用於優化損失
#decayed_learning_rate = learning_rate *  decay_rate ^ (global_step / decay_steps)
init = tf.initialize_all_variables()
# 啓動圖 (graph),查看衰減狀態
with tf.Session() as sess:
    sess.run(init)
    for i in range(15):
        _,r=sess.run([add_g, learning_rate])
        print(_,"=",r)