好像還挺好玩的GAN4——Keras搭建ACGAN利用卷積給生成結果貼上標籤
學習前言
請各位發糞圖強!
什麼是ACGAN
ACGAN一種帶條件約束的DCGAN,在生成模型(D)和判別模型(G)的建模中均引入條件變量y(conditional variable y)。
ACGAN相當於是DCGAN和CGAN的結合,將深度卷積網絡和標籤帶入到GAN當中。
使用額外信息y對模型增加條件,可以指導數據生成過程。這些條件變量y可以基於多種信息,例如類別標籤,用於圖像修復的部分數據,來自不同模態(modality)的數據。
在存在類別標籤的情況下,將深度卷積網絡帶入到GAN當中,提高圖片的生成質量。
這個簡單直接的改進被證明非常有效。
簡單來講,普通的GAN輸入的是一個N維的正態分佈隨機數,而ACGAN會爲這個隨機數添上標籤,其利用Embedding層將正整數(索引值)轉換爲固定尺寸的稠密向量,並將這個稠密向量與N維的正態分佈隨機數相乘,從而獲得一個有標籤的隨機數。
與此同時,ACGAN將深度卷積網絡帶入到存在標籤的GAN中,可以生成更加高質量的圖片。
神經網絡構建
1、Generator
生成網絡的輸入是一個帶標籤的隨機數,具體操作方式是生成一個N維的正態分佈隨機數,再利用Embedding層將正整數(索引值)轉換爲N維的稠密向量,並將這個稠密向量與N維的正態分佈隨機數相乘。
輸入的數進行reshape後利用上採樣與卷積生成圖像。
def build_generator(self):
model = Sequential()
# 先全連接到64*7*7的維度上
model.add(Dense(32 * 7 * 7, activation="relu", input_dim=self.latent_dim))
# reshape成特徵層的樣式
model.add(Reshape((7, 7, 32)))
# 7, 7, 64
model.add(Conv2D(64, kernel_size=3, padding="same"))
model.add(BatchNormalization(momentum=0.8))
model.add(Activation("relu"))
# 上採樣
# 7, 7, 64 -> 14, 14, 64
model.add(UpSampling2D())
model.add(Conv2D(128, kernel_size=3, padding="same"))
model.add(BatchNormalization(momentum=0.8))
model.add(Activation("relu"))
# 上採樣
# 14, 14, 128 -> 28, 28, 64
model.add(UpSampling2D())
model.add(Conv2D(64, kernel_size=3, padding="same"))
model.add(BatchNormalization(momentum=0.8))
model.add(Activation("relu"))
# 上採樣
# 28, 28, 64 -> 28, 28, 1
model.add(Conv2D(self.channels, kernel_size=3, padding="same"))
model.add(Activation("tanh"))
model.summary()
noise = Input(shape=(self.latent_dim,))
label = Input(shape=(1,), dtype='int32')
label_embedding = Flatten()(Embedding(self.num_classes, self.latent_dim)(label))
model_input = multiply([noise, label_embedding])
img = model(model_input)
return Model([noise, label], img)
2、Discriminator
普通GAN的判別模型的目的是根據輸入的圖片判斷出真僞。
在ACGAN中,其不僅要判斷出真僞,還要判斷出種類,主幹網絡利用卷積構成。。
因此它的輸入一個28,28,1維的圖片,輸出有兩個:
一個是0到1之間的數,1代表判斷這個圖片是真的,0代表判斷這個圖片是假的。與普通GAN不同的是,它使用的是卷積神經網絡。
另一個是一個向量,用於判斷這張圖片屬於什麼類。
def build_discriminator(self):
model = Sequential()
# 28,28,1 -> 14,14,16
model.add(Conv2D(16, kernel_size=3, strides=2, input_shape=self.img_shape, padding="same"))
model.add(LeakyReLU(alpha=0.2))
model.add(Dropout(0.25))
# 14,14,16 -> 8,8,32
model.add(Conv2D(32, kernel_size=3, strides=2, padding="same"))
model.add(LeakyReLU(alpha=0.2))
model.add(Dropout(0.25))
model.add(BatchNormalization(momentum=0.8))
# 8,8,32 -> 4,4,64
model.add(ZeroPadding2D(padding=((0,1),(0,1))))
model.add(Conv2D(64, kernel_size=3, strides=2, padding="same"))
model.add(LeakyReLU(alpha=0.2))
model.add(Dropout(0.25))
model.add(BatchNormalization(momentum=0.8))
# 4,4,64 -> 4,4,128
model.add(Conv2D(128, kernel_size=3, strides=1, padding="same"))
model.add(LeakyReLU(alpha=0.2))
model.add(Dropout(0.25))
model.add(GlobalAveragePooling2D())
img = Input(shape=self.img_shape)
features = model(img)
validity = Dense(1, activation="sigmoid")(features)
label = Dense(self.num_classes, activation="softmax")(features)
return Model(img, [validity, label])
訓練思路
ACGAN的訓練思路分爲如下幾個步驟:
1、隨機選取batch_size個真實的圖片和它的標籤。
2、隨機生成batch_size個N維向量和其對應的標籤label,利用Embedding層進行組合,傳入到Generator中生成batch_size個虛假圖片。
3、Discriminator的loss函數由兩部分組成,一部分是真僞的判斷結果與真實情況的對比,一部分是圖片所屬標籤的判斷結果與真實情況的對比。
4、Generator的loss函數也由兩部分組成,一部分是生成的圖片是否被Discriminator判斷爲1,另一部分是生成的圖片是否被分成了正確的類。
實現全部代碼
from __future__ import print_function, division
import tensorflow as tf
from keras.datasets import mnist
from keras.backend.tensorflow_backend import set_session
from keras.layers import Input, Dense, Reshape, Flatten, Dropout, multiply
from keras.layers import BatchNormalization, Activation, Embedding, ZeroPadding2D, GlobalAveragePooling2D
from keras.layers.advanced_activations import LeakyReLU
from keras.layers.convolutional import UpSampling2D, Conv2D
from keras.models import Sequential, Model
from keras.optimizers import Adam
import matplotlib.pyplot as plt
import os
import numpy as np
class ACGAN():
def __init__(self):
# 輸入shape
self.img_rows = 28
self.img_cols = 28
self.channels = 1
self.img_shape = (self.img_rows, self.img_cols, self.channels)
# 分十類
self.num_classes = 10
self.latent_dim = 100
# adam優化器
optimizer = Adam(0.0002, 0.5)
# 判別模型
losses = ['binary_crossentropy', 'sparse_categorical_crossentropy']
self.discriminator = self.build_discriminator()
self.discriminator.compile(loss=losses,
optimizer=optimizer,
metrics=['accuracy'])
# 生成模型
self.generator = self.build_generator()
# conbine是生成模型和判別模型的結合
# 判別模型的trainable爲False
# 用於訓練生成模型
noise = Input(shape=(self.latent_dim,))
label = Input(shape=(1,))
img = self.generator([noise, label])
self.discriminator.trainable = False
valid, target_label = self.discriminator(img)
self.combined = Model([noise, label], [valid, target_label])
self.combined.compile(loss=losses,
optimizer=optimizer)
def build_generator(self):
model = Sequential()
# 先全連接到64*7*7的維度上
model.add(Dense(32 * 7 * 7, activation="relu", input_dim=self.latent_dim))
# reshape成特徵層的樣式
model.add(Reshape((7, 7, 32)))
# 7, 7, 64
model.add(Conv2D(64, kernel_size=3, padding="same"))
model.add(BatchNormalization(momentum=0.8))
model.add(Activation("relu"))
# 上採樣
# 7, 7, 64 -> 14, 14, 64
model.add(UpSampling2D())
model.add(Conv2D(128, kernel_size=3, padding="same"))
model.add(BatchNormalization(momentum=0.8))
model.add(Activation("relu"))
# 上採樣
# 14, 14, 128 -> 28, 28, 64
model.add(UpSampling2D())
model.add(Conv2D(64, kernel_size=3, padding="same"))
model.add(BatchNormalization(momentum=0.8))
model.add(Activation("relu"))
# 上採樣
# 28, 28, 64 -> 28, 28, 1
model.add(Conv2D(self.channels, kernel_size=3, padding="same"))
model.add(Activation("tanh"))
model.summary()
noise = Input(shape=(self.latent_dim,))
label = Input(shape=(1,), dtype='int32')
label_embedding = Flatten()(Embedding(self.num_classes, self.latent_dim)(label))
model_input = multiply([noise, label_embedding])
img = model(model_input)
return Model([noise, label], img)
def build_discriminator(self):
model = Sequential()
# 28,28,1 -> 14,14,16
model.add(Conv2D(16, kernel_size=3, strides=2, input_shape=self.img_shape, padding="same"))
model.add(LeakyReLU(alpha=0.2))
model.add(Dropout(0.25))
# 14,14,16 -> 8,8,32
model.add(Conv2D(32, kernel_size=3, strides=2, padding="same"))
model.add(LeakyReLU(alpha=0.2))
model.add(Dropout(0.25))
model.add(BatchNormalization(momentum=0.8))
# 8,8,32 -> 4,4,64
model.add(ZeroPadding2D(padding=((0,1),(0,1))))
model.add(Conv2D(64, kernel_size=3, strides=2, padding="same"))
model.add(LeakyReLU(alpha=0.2))
model.add(Dropout(0.25))
model.add(BatchNormalization(momentum=0.8))
# 4,4,64 -> 4,4,128
model.add(Conv2D(128, kernel_size=3, strides=1, padding="same"))
model.add(LeakyReLU(alpha=0.2))
model.add(Dropout(0.25))
model.add(GlobalAveragePooling2D())
img = Input(shape=self.img_shape)
features = model(img)
validity = Dense(1, activation="sigmoid")(features)
label = Dense(self.num_classes, activation="softmax")(features)
return Model(img, [validity, label])
def train(self, epochs, batch_size=128, sample_interval=50):
# 載入數據庫
(X_train, y_train), (_, _) = mnist.load_data()
# 歸一化
X_train = (X_train.astype(np.float32) - 127.5) / 127.5
X_train = np.expand_dims(X_train, axis=3)
y_train = y_train.reshape(-1, 1)
valid = np.ones((batch_size, 1))
fake = np.zeros((batch_size, 1))
for epoch in range(epochs):
# --------------------- #
# 訓練鑑別模型
# --------------------- #
idx = np.random.randint(0, X_train.shape[0], batch_size)
imgs, labels = X_train[idx], y_train[idx]
# ---------------------- #
# 生成正態分佈的輸入
# ---------------------- #
noise = np.random.normal(0, 1, (batch_size, self.latent_dim))
sampled_labels = np.random.randint(0, 10, (batch_size, 1))
gen_imgs = self.generator.predict([noise, sampled_labels])
d_loss_real = self.discriminator.train_on_batch(imgs, [valid, labels])
d_loss_fake = self.discriminator.train_on_batch(gen_imgs, [fake, sampled_labels])
d_loss = 0.5 * np.add(d_loss_real, d_loss_fake)
# --------------------- #
# 訓練生成模型
# --------------------- #
g_loss = self.combined.train_on_batch([noise, sampled_labels], [valid, sampled_labels])
print ("%d [D loss: %f, acc.: %.2f%%, op_acc: %.2f%%] [G loss: %f]" % (epoch, d_loss[0], 100*d_loss[3], 100*d_loss[4], g_loss[0]))
if epoch % sample_interval == 0:
self.sample_images(epoch)
def sample_images(self, epoch):
r, c = 2, 5
noise = np.random.normal(0, 1, (r * c, 100))
sampled_labels = np.arange(0, 10).reshape(-1, 1)
gen_imgs = self.generator.predict([noise, sampled_labels])
gen_imgs = 0.5 * gen_imgs + 0.5
fig, axs = plt.subplots(r, c)
cnt = 0
for i in range(r):
for j in range(c):
axs[i,j].imshow(gen_imgs[cnt,:,:,0], cmap='gray')
axs[i,j].set_title("Digit: %d" % sampled_labels[cnt])
axs[i,j].axis('off')
cnt += 1
fig.savefig("images/%d.png" % epoch)
plt.close()
def save_model(self):
def save(model, model_name):
model_path = "saved_model/%s.json" % model_name
weights_path = "saved_model/%s_weights.hdf5" % model_name
options = {"file_arch": model_path,
"file_weight": weights_path}
json_string = model.to_json()
open(options['file_arch'], 'w').write(json_string)
model.save_weights(options['file_weight'])
save(self.generator, "generator")
save(self.discriminator, "discriminator")
if __name__ == '__main__':
if not os.path.exists("./images"):
os.makedirs("./images")
acgan = ACGAN()
acgan.train(epochs=20000, batch_size=256, sample_interval=200)
實現效果爲: