Keras：自建數據集圖像分類的模型訓練、保存與恢復

數據擴增

在數據集中的數據不多的情況下,可以使用圖片生成器ImageDataGenerator用來生成一個batch的圖像數據，進行數據擴增.

示例:

#!/usr/bin/python
# coding:utf8

from keras.preprocessing.image import ImageDataGenerator, array_to_img, img_to_array, load_img
datagen = ImageDataGenerator(rotation_range=40,
                             width_shift_range=0.2,
                             height_shift_range=0.2,
                             shear_range=0.2,
                             zoom_range=0.2,
                             horizontal_flip=True,
                             fill_mode='nearest')
img = load_img('/image/panda.jpg')
x = img_to_array(img)
x = x.reshape((1,)+x.shape)

i = 0
for batch in datagen.flow(x, batch_size=1,
                          save_to_dir='/train/1/', save_prefix='pandas', save_format='jpg'):
    i += 1
    if i>60:
        break

輸出:

模型的訓練、保存與恢復

將擴增之後的圖片數據集按類分爲訓練集驗證集.
圖片數據放置形式:

train\
      0\
        dog_0_1.jpg
        dog_0_2.jpg
        dog_0_3.jpg
        ...
      1\
        panda_0_1.jpg
        panda_0_2.jpg
        panda_0_3.jpg
        ...
validation\
      0\
        dog_0_1.jpg
        dog_0_2.jpg
        dog_0_3.jpg
        ...
      1\
        panda_0_1.jpg
        panda_0_2.jpg
        panda_0_3.jpg
        ... 

訓練模型並保存模型(分別保存整個模型、保存模型權重、保存模型結構).
示例:

#!/usr/bin/python
# coding:utf8

from keras.models import Sequential
from keras.layers import Conv2D, MaxPooling2D
from keras.layers import Dropout, Flatten, Dense
from keras.preprocessing.image import ImageDataGenerator
# 建立模型
model = Sequential()
model.add(Conv2D(32, 3, activation='relu', input_shape=(150,150,3)))
model.add(MaxPooling2D(pool_size=(2, 2)))
model.add(Conv2D(32, 3, activation='relu'))
model.add(MaxPooling2D(pool_size=(2, 2)))
model.add(Conv2D(64, 3, activation='relu'))
model.add(MaxPooling2D(pool_size=(2, 2)))
model.add(Flatten())
model.add(Dense(64, activation='relu'))
model.add(Dropout(0.5))
model.add(Dense(1, activation='sigmoid'))
# 編譯
model.compile(loss='binary_crossentropy',
              optimizer='rmsprop',
              metrics=['accuracy'])

train_datagen = ImageDataGenerator(rescale=1./255,
                                   shear_range=0.2,
                                   zoom_range=0.2,
                                   horizontal_flip=True)
test_datagen = ImageDataGenerator(rescale=1./255)
# 從圖片中直接產生數據和標籤
train_generator = train_datagen.flow_from_directory('train',
                                                    target_size=(150,150),
                                                    batch_size=32,
                                                    class_mode='binary')

validation_generator = test_datagen.flow_from_directory('validation',
                                                        target_size=(150,150),
                                                        batch_size=32,
                                                        class_mode='binary')
model.fit_generator(train_generator,
                    steps_per_epoch=1000,
                    epochs=60,
                    validation_data=validation_generator,
                    validation_steps=200)
# 保存整個模型
model.save('model.hdf5')

# 保存模型的權重
model.save_weights('model_weights.h5')

# 保存模型的結構
json_string = model.to_json()
open('model_to_json.json','w').write(json_string)
yaml_string = model.to_yaml()
open('model_to_yaml.yaml','w').write(json_string)

保存完整模型

通過model.save()將模型和權重保存在一個.hdf5文件中.

model.save('model/model.hdf5')

完整模型的恢復

通過keras.models.load_model()可以將模型重新實例化.

示例:

#!/usr/bin/python
# coding:utf8

from keras.models import load_model
import numpy as np
from keras.preprocessing import image
from keras.preprocessing.image import load_img

# 加載權重
model = load_model('model.hdf5')

# 加載圖像
img = load_img('pandas_0_96.jpg',target_size=(150, 150))
img = image.img_to_array(img) / 255.0
img = np.expand_dims(img, axis=0)

predictions = model.predict(img)
print (predictions)

輸出:

[[0.9967361]]

保存模型權重

通過model.save_weights()可以將模型的權重保存到指定路徑下的.h5文件中.

model.save_weights('model/model_weights.h5')

模型權重的恢復

首先建立對應模型,然後通過model.load_weights()從.hdf5文件中加載到當前模型.
如果想將權重載入不同的模型（有些層相同）中，則設置by_name=True，只有名字匹配的層纔會載入權重.

示例:

#!/usr/bin/python
# coding:utf8

from keras.preprocessing.image import load_img
from keras.models import Sequential
from keras.layers import Conv2D, MaxPooling2D
from keras.layers import Dropout, Flatten, Dense
import numpy as np
from keras.preprocessing import image

# 建立模型
model = Sequential()
model.add(Conv2D(32, 3, activation='relu', input_shape=(150,150,3)))
model.add(MaxPooling2D(pool_size=(2, 2)))
model.add(Conv2D(32, 3, activation='relu'))
model.add(MaxPooling2D(pool_size=(2, 2)))
model.add(Conv2D(64, 3, activation='relu'))
model.add(MaxPooling2D(pool_size=(2, 2)))
model.add(Flatten())
model.add(Dense(64, activation='relu'))
model.add(Dropout(0.5))
model.add(Dense(1, activation='sigmoid'))
model.compile(loss='binary_crossentropy', optimizer='rmsprop', metrics=['accuracy'])

# 加載權重
classifier = model.load_weights('model_weights.h5')

# 加載圖像
img = load_img('pandas_0_96.jpg',target_size=(150, 150))
img = image.img_to_array(img) / 255.0
img = np.expand_dims(img, axis=0)
predictions = model.predict(img)
print (predictions)

輸出:

[[0.9967361]]

保存模型結構

通過model.to_json()或model.to_yaml()，返回代表模型的JSON字符串，僅包含網絡結構，不包含權值.這樣可以把模型序列化爲json或yaml文件，然後保存到本地.
保存爲json文件:

json_string = model.to_json()
open('model_to_json.json','w').write(json_string)

保存爲yaml文件:

yaml_string = model.to_yaml()
open('model_to_yaml.yaml','w').write(json_string)

模型結構的恢復

通過model.to_json或model.to_yaml可以從JSON字符串或YAML字符串中重構原模型.
JSON字符串或YAML字符串可以通過open(filepath).read()從對應的json文件或yaml文件中讀取.

示例:

from keras.models import model_from_yaml
yaml_string = open('model_to_yaml.yaml').read()
model = model_from_yaml(yaml_string)
# 打印出模型
model.summary()

打印出模型概況:

_________________________________________________________________
Layer (type)                 Output Shape              Param #   
=================================================================
conv2d_1 (Conv2D)            (None, 148, 148, 32)      896       
_________________________________________________________________
max_pooling2d_1 (MaxPooling2 (None, 74, 74, 32)        0         
_________________________________________________________________
conv2d_2 (Conv2D)            (None, 72, 72, 32)        9248      
_________________________________________________________________
max_pooling2d_2 (MaxPooling2 (None, 36, 36, 32)        0         
_________________________________________________________________
conv2d_3 (Conv2D)            (None, 34, 34, 64)        18496     
_________________________________________________________________
max_pooling2d_3 (MaxPooling2 (None, 17, 17, 64)        0         
_________________________________________________________________
flatten_1 (Flatten)          (None, 18496)             0         
_________________________________________________________________
dense_1 (Dense)              (None, 64)                1183808   
_________________________________________________________________
dropout_1 (Dropout)          (None, 64)                0         
_________________________________________________________________
dense_2 (Dense)              (None, 1)                 65        
=================================================================
Total params: 1,212,513
Trainable params: 1,212,513
Non-trainable params: 0
_________________________________________________________________

dog:

panda:

---

Keras版本:2.1.5

參考:

Keras中文文檔::圖片生成器ImageDataGenerator
Keras中文文檔::關於Keras模型
The Keras Blog::Building a simple Keras + deep learning REST API