PaddleHub之《青春有你2》作業：五人識別
一、任務簡介
圖像分類是計算機視覺的重要領域，它的目標是將圖像分類到預定義的標籤。近期，許多研究者提出很多不同種類的神經網絡，並且極大的提升了分類算法的性能。本文以自己創建的數據集：青春有你2中選手識別爲例子，介紹如何使用PaddleHub進行圖像分類任務。
#CPU環境啓動請務必執行該指令
%set_env CPU_NUM=1
env: CPU_NUM=1
#安裝paddlehub
!pip install paddlehub==1.6.0 -i https://pypi.tuna.tsinghua.edu.cn/simple
Looking in indexes: https://pypi.tuna.tsinghua.edu.cn/simple
Collecting paddlehub==1.6.0
[?25l Downloading https://pypi.tuna.tsinghua.edu.cn/packages/7f/9f/6617c2b8e9c5d847803ae89924b58bccd1b8fb2c98aa00e16531540591f2/paddlehub-1.6.0-py3-none-any.whl (206kB)
[K |████████████████████████████████| 215kB 10.0MB/s eta 0:00:01
[?25hRequirement already satisfied: protobuf>=3.6.0 in /opt/conda/envs/python35-paddle120-env/lib/python3.7/site-packages (from paddlehub==1.6.0) (3.10.0)
Requirement already satisfied: pandas; python_version >= "3" in /opt/conda/envs/python35-paddle120-env/lib/python3.7/site-packages (from paddlehub==1.6.0) (0.23.4)
Requirement already satisfied: requests in /opt/conda/envs/python35-paddle120-env/lib/python3.7/site-packages (from paddlehub==1.6.0) (2.22.0)
Requirement already satisfied: sentencepiece in /opt/conda/envs/python35-paddle120-env/lib/python3.7/site-packages (from paddlehub==1.6.0) (0.1.85)
Requirement already satisfied: chardet==3.0.4 in /opt/conda/envs/python35-paddle120-env/lib/python3.7/site-packages (from paddlehub==1.6.0) (3.0.4)
Requirement already satisfied: gunicorn>=19.10.0; sys_platform != "win32" in /opt/conda/envs/python35-paddle120-env/lib/python3.7/site-packages (from paddlehub==1.6.0) (20.0.4)
Requirement already satisfied: six>=1.10.0 in /opt/conda/envs/python35-paddle120-env/lib/python3.7/site-packages (from paddlehub==1.6.0) (1.12.0)
Requirement already satisfied: numpy; python_version >= "3" in /opt/conda/envs/python35-paddle120-env/lib/python3.7/site-packages (from paddlehub==1.6.0) (1.16.4)
Requirement already satisfied: yapf==0.26.0 in /opt/conda/envs/python35-paddle120-env/lib/python3.7/site-packages (from paddlehub==1.6.0) (0.26.0)
Requirement already satisfied: tensorboard>=1.15 in /opt/conda/envs/python35-paddle120-env/lib/python3.7/site-packages (from paddlehub==1.6.0) (2.1.0)
Requirement already satisfied: flask>=1.1.0 in /opt/conda/envs/python35-paddle120-env/lib/python3.7/site-packages (from paddlehub==1.6.0) (1.1.1)
Requirement already satisfied: flake8 in /opt/conda/envs/python35-paddle120-env/lib/python3.7/site-packages (from paddlehub==1.6.0) (3.7.9)
Requirement already satisfied: tb-paddle in /opt/conda/envs/python35-paddle120-env/lib/python3.7/site-packages (from paddlehub==1.6.0) (0.3.6)
Requirement already satisfied: colorlog in /opt/conda/envs/python35-paddle120-env/lib/python3.7/site-packages (from paddlehub==1.6.0) (4.1.0)
Requirement already satisfied: Pillow in /opt/conda/envs/python35-paddle120-env/lib/python3.7/site-packages (from paddlehub==1.6.0) (6.2.0)
Requirement already satisfied: nltk in /opt/conda/envs/python35-paddle120-env/lib/python3.7/site-packages (from paddlehub==1.6.0) (3.4.5)
Requirement already satisfied: cma==2.7.0 in /opt/conda/envs/python35-paddle120-env/lib/python3.7/site-packages (from paddlehub==1.6.0) (2.7.0)
Requirement already satisfied: opencv-python in /opt/conda/envs/python35-paddle120-env/lib/python3.7/site-packages (from paddlehub==1.6.0) (4.1.1.26)
Requirement already satisfied: pre-commit in /opt/conda/envs/python35-paddle120-env/lib/python3.7/site-packages (from paddlehub==1.6.0) (1.21.0)
Requirement already satisfied: pyyaml in /opt/conda/envs/python35-paddle120-env/lib/python3.7/site-packages (from paddlehub==1.6.0) (5.1.2)
Requirement already satisfied: setuptools in /opt/conda/envs/python35-paddle120-env/lib/python3.7/site-packages (from protobuf>=3.6.0->paddlehub==1.6.0) (41.4.0)
Requirement already satisfied: pytz>=2011k in /opt/conda/envs/python35-paddle120-env/lib/python3.7/site-packages (from pandas; python_version >= "3"->paddlehub==1.6.0) (2019.3)
Requirement already satisfied: python-dateutil>=2.5.0 in /opt/conda/envs/python35-paddle120-env/lib/python3.7/site-packages (from pandas; python_version >= "3"->paddlehub==1.6.0) (2.8.0)
Requirement already satisfied: certifi>=2017.4.17 in /opt/conda/envs/python35-paddle120-env/lib/python3.7/site-packages (from requests->paddlehub==1.6.0) (2019.9.11)
Requirement already satisfied: urllib3!=1.25.0,!=1.25.1,<1.26,>=1.21.1 in /opt/conda/envs/python35-paddle120-env/lib/python3.7/site-packages (from requests->paddlehub==1.6.0) (1.25.6)
Requirement already satisfied: idna<2.9,>=2.5 in /opt/conda/envs/python35-paddle120-env/lib/python3.7/site-packages (from requests->paddlehub==1.6.0) (2.8)
Requirement already satisfied: grpcio>=1.24.3 in /opt/conda/envs/python35-paddle120-env/lib/python3.7/site-packages (from tensorboard>=1.15->paddlehub==1.6.0) (1.26.0)
Requirement already satisfied: google-auth<2,>=1.6.3 in /opt/conda/envs/python35-paddle120-env/lib/python3.7/site-packages (from tensorboard>=1.15->paddlehub==1.6.0) (1.10.0)
Requirement already satisfied: wheel>=0.26; python_version >= "3" in /opt/conda/envs/python35-paddle120-env/lib/python3.7/site-packages (from tensorboard>=1.15->paddlehub==1.6.0) (0.33.6)
Requirement already satisfied: google-auth-oauthlib<0.5,>=0.4.1 in /opt/conda/envs/python35-paddle120-env/lib/python3.7/site-packages (from tensorboard>=1.15->paddlehub==1.6.0) (0.4.1)
Requirement already satisfied: werkzeug>=0.11.15 in /opt/conda/envs/python35-paddle120-env/lib/python3.7/site-packages (from tensorboard>=1.15->paddlehub==1.6.0) (0.16.0)
Requirement already satisfied: markdown>=2.6.8 in /opt/conda/envs/python35-paddle120-env/lib/python3.7/site-packages (from tensorboard>=1.15->paddlehub==1.6.0) (3.1.1)
Requirement already satisfied: absl-py>=0.4 in /opt/conda/envs/python35-paddle120-env/lib/python3.7/site-packages (from tensorboard>=1.15->paddlehub==1.6.0) (0.8.1)
Requirement already satisfied: Jinja2>=2.10.1 in /opt/conda/envs/python35-paddle120-env/lib/python3.7/site-packages (from flask>=1.1.0->paddlehub==1.6.0) (2.10.1)
Requirement already satisfied: click>=5.1 in /opt/conda/envs/python35-paddle120-env/lib/python3.7/site-packages (from flask>=1.1.0->paddlehub==1.6.0) (7.0)
Requirement already satisfied: itsdangerous>=0.24 in /opt/conda/envs/python35-paddle120-env/lib/python3.7/site-packages (from flask>=1.1.0->paddlehub==1.6.0) (1.1.0)
Requirement already satisfied: pycodestyle<2.6.0,>=2.5.0 in /opt/conda/envs/python35-paddle120-env/lib/python3.7/site-packages (from flake8->paddlehub==1.6.0) (2.5.0)
Requirement already satisfied: mccabe<0.7.0,>=0.6.0 in /opt/conda/envs/python35-paddle120-env/lib/python3.7/site-packages (from flake8->paddlehub==1.6.0) (0.6.1)
Requirement already satisfied: pyflakes<2.2.0,>=2.1.0 in /opt/conda/envs/python35-paddle120-env/lib/python3.7/site-packages (from flake8->paddlehub==1.6.0) (2.1.1)
Requirement already satisfied: entrypoints<0.4.0,>=0.3.0 in /opt/conda/envs/python35-paddle120-env/lib/python3.7/site-packages (from flake8->paddlehub==1.6.0) (0.3)
Requirement already satisfied: moviepy in /opt/conda/envs/python35-paddle120-env/lib/python3.7/site-packages (from tb-paddle->paddlehub==1.6.0) (1.0.1)
Requirement already satisfied: identify>=1.0.0 in /opt/conda/envs/python35-paddle120-env/lib/python3.7/site-packages (from pre-commit->paddlehub==1.6.0) (1.4.10)
Requirement already satisfied: importlib-metadata; python_version < "3.8" in /opt/conda/envs/python35-paddle120-env/lib/python3.7/site-packages (from pre-commit->paddlehub==1.6.0) (0.23)
Requirement already satisfied: toml in /opt/conda/envs/python35-paddle120-env/lib/python3.7/site-packages (from pre-commit->paddlehub==1.6.0) (0.10.0)
Requirement already satisfied: aspy.yaml in /opt/conda/envs/python35-paddle120-env/lib/python3.7/site-packages (from pre-commit->paddlehub==1.6.0) (1.3.0)
Requirement already satisfied: cfgv>=2.0.0 in /opt/conda/envs/python35-paddle120-env/lib/python3.7/site-packages (from pre-commit->paddlehub==1.6.0) (2.0.1)
Requirement already satisfied: nodeenv>=0.11.1 in /opt/conda/envs/python35-paddle120-env/lib/python3.7/site-packages (from pre-commit->paddlehub==1.6.0) (1.3.4)
Requirement already satisfied: virtualenv>=15.2 in /opt/conda/envs/python35-paddle120-env/lib/python3.7/site-packages (from pre-commit->paddlehub==1.6.0) (16.7.9)
Requirement already satisfied: pyasn1-modules>=0.2.1 in /opt/conda/envs/python35-paddle120-env/lib/python3.7/site-packages (from google-auth<2,>=1.6.3->tensorboard>=1.15->paddlehub==1.6.0) (0.2.7)
Requirement already satisfied: rsa<4.1,>=3.1.4 in /opt/conda/envs/python35-paddle120-env/lib/python3.7/site-packages (from google-auth<2,>=1.6.3->tensorboard>=1.15->paddlehub==1.6.0) (4.0)
Requirement already satisfied: cachetools<5.0,>=2.0.0 in /opt/conda/envs/python35-paddle120-env/lib/python3.7/site-packages (from google-auth<2,>=1.6.3->tensorboard>=1.15->paddlehub==1.6.0) (4.0.0)
Requirement already satisfied: requests-oauthlib>=0.7.0 in /opt/conda/envs/python35-paddle120-env/lib/python3.7/site-packages (from google-auth-oauthlib<0.5,>=0.4.1->tensorboard>=1.15->paddlehub==1.6.0) (1.3.0)
Requirement already satisfied: MarkupSafe>=0.23 in /opt/conda/envs/python35-paddle120-env/lib/python3.7/site-packages (from Jinja2>=2.10.1->flask>=1.1.0->paddlehub==1.6.0) (1.1.1)
Requirement already satisfied: imageio-ffmpeg>=0.2.0; python_version >= "3.4" in /opt/conda/envs/python35-paddle120-env/lib/python3.7/site-packages (from moviepy->tb-paddle->paddlehub==1.6.0) (0.3.0)
Requirement already satisfied: proglog<=1.0.0 in /opt/conda/envs/python35-paddle120-env/lib/python3.7/site-packages (from moviepy->tb-paddle->paddlehub==1.6.0) (0.1.9)
Requirement already satisfied: decorator<5.0,>=4.0.2 in /opt/conda/envs/python35-paddle120-env/lib/python3.7/site-packages (from moviepy->tb-paddle->paddlehub==1.6.0) (4.4.0)
Requirement already satisfied: imageio<3.0,>=2.5; python_version >= "3.4" in /opt/conda/envs/python35-paddle120-env/lib/python3.7/site-packages (from moviepy->tb-paddle->paddlehub==1.6.0) (2.6.1)
Requirement already satisfied: tqdm<5.0,>=4.11.2 in /opt/conda/envs/python35-paddle120-env/lib/python3.7/site-packages (from moviepy->tb-paddle->paddlehub==1.6.0) (4.36.1)
Requirement already satisfied: zipp>=0.5 in /opt/conda/envs/python35-paddle120-env/lib/python3.7/site-packages (from importlib-metadata; python_version < "3.8"->pre-commit->paddlehub==1.6.0) (0.6.0)
Requirement already satisfied: pyasn1<0.5.0,>=0.4.6 in /opt/conda/envs/python35-paddle120-env/lib/python3.7/site-packages (from pyasn1-modules>=0.2.1->google-auth<2,>=1.6.3->tensorboard>=1.15->paddlehub==1.6.0) (0.4.8)
Requirement already satisfied: oauthlib>=3.0.0 in /opt/conda/envs/python35-paddle120-env/lib/python3.7/site-packages (from requests-oauthlib>=0.7.0->google-auth-oauthlib<0.5,>=0.4.1->tensorboard>=1.15->paddlehub==1.6.0) (3.1.0)
Requirement already satisfied: more-itertools in /opt/conda/envs/python35-paddle120-env/lib/python3.7/site-packages (from zipp>=0.5->importlib-metadata; python_version < "3.8"->pre-commit->paddlehub==1.6.0) (7.2.0)
Installing collected packages: paddlehub
Found existing installation: paddlehub 1.5.0
Uninstalling paddlehub-1.5.0:
Successfully uninstalled paddlehub-1.5.0
Successfully installed paddlehub-1.6.0
二、任務實踐
Step1、基礎工作
加載數據文件
導入python包
!unzip -o file.zip -d ./dataset/
unzip: cannot find or open file.zip, file.zip.zip or file.zip.ZIP.
!unzip ./data/train.zip -d ./dataset/train
!ls
Archive: ./data/train.zip
replace ./dataset/train/anqi/anqi0.jpg? [y]es, [n]o, [A]ll, [N]one, [r]ename:
import paddlehub as hub
Step2、加載預訓練模型
接下來我們要在PaddleHub中選擇合適的預訓練模型來Finetune，由於是圖像分類任務，因此我們使用經典的ResNet-50作爲預訓練模型。PaddleHub提供了豐富的圖像分類預訓練模型，包括了最新的神經網絡架構搜索類的PNASNet，我們推薦您嘗試不同的預訓練模型來獲得更好的性能。
import os
def generate_train_tlist():
# 待搜索的目錄路徑
result=[]
path = "dataset/train"
# 待搜索的名稱
stars = {'yushuxin': 0, 'xujiaqi': 1, 'zhaoxiaotang': 2, 'anqi': 3, r'wangchengxuan': 4}
for root, dirs, files in os.walk(path):
for f in files:
ff = os.path.join(root, f)
# print(ff)
fff='t'+ff.strip('dataset/')
# print(fff)
# print(f)
name=f[:-5]
# print(name)
# print(' %s %d'% ( fff, stars[name]))
result.append('%s %d'% ( fff, stars[name]))
return result
train_list=generate_train_tlist()
with open("./dataset/train_list.txt", "w") as f:
for line in train_list:
print(line)
f.writelines(line)
f.writelines("\n")
train/anqi/anqi4.jpg 3
train/anqi/anqi8.jpg 3
train/anqi/anqi7.jpg 3
train/anqi/anqi3.jpg 3
train/anqi/anqi0.jpg 3
train/anqi/anqi5.jpg 3
train/anqi/anqi2.jpg 3
train/anqi/anqi6.jpg 3
train/anqi/anqi9.jpg 3
train/anqi/anqi1.jpg 3
train/wangchengxuan/wangchengxuan5.jpg 4
train/wangchengxuan/wangchengxuan7.jpg 4
train/wangchengxuan/wangchengxuan4.jpg 4
train/wangchengxuan/wangchengxuan2.jpg 4
train/wangchengxuan/wangchengxuan9.jpg 4
train/wangchengxuan/wangchengxuan8.jpg 4
train/wangchengxuan/wangchengxuan3.jpg 4
train/wangchengxuan/wangchengxuan0.jpg 4
train/wangchengxuan/wangchengxuan6.jpg 4
train/wangchengxuan/wangchengxuan1.jpg 4
train/yushuxin/yushuxin0.jpg 0
train/yushuxin/yushuxin9.jpg 0
train/yushuxin/yushuxin5.jpg 0
train/yushuxin/yushuxin4.jpg 0
train/yushuxin/yushuxin3.jpg 0
train/yushuxin/yushuxin2.jpg 0
train/yushuxin/yushuxin7.jpg 0
train/yushuxin/yushuxin6.jpg 0
train/yushuxin/yushuxin8.jpg 0
train/yushuxin/yushuxin1.jpg 0
train/xujiaqi/xujiaqi4.jpg 1
train/xujiaqi/xujiaqi2.jpg 1
train/xujiaqi/xujiaqi6.jpg 1
train/xujiaqi/xujiaqi1.jpg 1
train/xujiaqi/xujiaqi9.jpg 1
train/xujiaqi/xujiaqi8.jpg 1
train/xujiaqi/xujiaqi5.jpg 1
train/xujiaqi/xujiaqi3.jpg 1
train/xujiaqi/xujiaqi0.jpg 1
train/xujiaqi/xujiaqi7.jpg 1
train/zhaoxiaotang/zhaoxiaotang7.jpg 2
train/zhaoxiaotang/zhaoxiaotang6.jpg 2
train/zhaoxiaotang/zhaoxiaotang1.jpg 2
train/zhaoxiaotang/zhaoxiaotang9.jpg 2
train/zhaoxiaotang/zhaoxiaotang2.jpg 2
train/zhaoxiaotang/zhaoxiaotang0.jpg 2
train/zhaoxiaotang/zhaoxiaotang4.jpg 2
train/zhaoxiaotang/zhaoxiaotang5.jpg 2
train/zhaoxiaotang/zhaoxiaotang8.jpg 2
train/zhaoxiaotang/zhaoxiaotang3.jpg 2
!hub install ernie
Module ernie already installed in /home/aistudio/.paddlehub/modules/ernie
module = hub.Module(name="resnet_v2_50_imagenet")
[32m[2020-04-26 13:03:09,478] [ INFO] - Installing resnet_v2_50_imagenet module[0m
[32m[2020-04-26 13:03:09,498] [ INFO] - Module resnet_v2_50_imagenet already installed in /home/aistudio/.paddlehub/modules/resnet_v2_50_imagenet[0m
Step3、數據準備
接着需要加載圖片數據集。我們使用自定義的數據進行體驗，請查看適配自定義數據
from paddlehub.dataset.base_cv_dataset import BaseCVDataset
class DemoDataset(BaseCVDataset):
def __init__(self):
# 數據集存放位置
self.dataset_dir = "dataset"
super(DemoDataset, self).__init__(
base_path=self.dataset_dir,
train_list_file="train_list.txt",
# validate_list_file="validate_list.txt",
test_list_file="test_list.txt",
label_list_file="label_list.txt",
)
dataset = DemoDataset()
Step4、生成數據讀取器
接着生成一個圖像分類的reader，reader負責將dataset的數據進行預處理，接着以特定格式組織並輸入給模型進行訓練。
當我們生成一個圖像分類的reader時，需要指定輸入圖片的大小
data_reader = hub.reader.ImageClassificationReader(
image_width=module.get_expected_image_width(),
image_height=module.get_expected_image_height(),
images_mean=module.get_pretrained_images_mean(),
images_std=module.get_pretrained_images_std(),
dataset=dataset)
[32m[2020-04-26 13:07:58,826] [ INFO] - Dataset label map = {'虞書欣': 0, '許佳琪': 1, '趙小棠': 2, '安崎': 3, '王承渲': 4}[0m
Step5、配置策略
在進行Finetune前，我們可以設置一些運行時的配置，例如如下代碼中的配置，表示：
-
use_cuda：設置爲False表示使用CPU進行訓練。如果您本機支持GPU，且安裝的是GPU版本的PaddlePaddle，我們建議您將這個選項設置爲True； -
epoch：迭代輪數； -
batch_size：每次訓練的時候，給模型輸入的每批數據大小爲32，模型訓練時能夠並行處理批數據，因此batch_size越大，訓練的效率越高，但是同時帶來了內存的負荷，過大的batch_size可能導致內存不足而無法訓練，因此選擇一個合適的batch_size是很重要的一步； -
log_interval：每隔10 step打印一次訓練日誌； -
eval_interval：每隔50 step在驗證集上進行一次性能評估； -
checkpoint_dir：將訓練的參數和數據保存到cv_finetune_turtorial_demo目錄中； -
strategy：使用DefaultFinetuneStrategy策略進行finetune；
更多運行配置，請查看RunConfig
同時PaddleHub提供了許多優化策略，如AdamWeightDecayStrategy、ULMFiTStrategy、DefaultFinetuneStrategy等，詳細信息參見策略
config = hub.RunConfig(
use_cuda=True, #是否使用GPU訓練，默認爲False；
num_epoch=3, #Fine-tune的輪數；
checkpoint_dir="cv_finetune_turtorial_demo" ,#模型checkpoint保存路徑, 若用戶沒有指定，程序會自動生成；
batch_size=3, #訓練的批大小，如果使用GPU，請根據實際情況調整batch_size；
# eval_interval=3, #模型評估的間隔，默認每100個step評估一次驗證集；
log_interval=10,
strategy=hub.finetune.strategy.DefaultFinetuneStrategy()) #Fine-tune優化策略；
[32m[2020-04-26 13:06:01,681] [ INFO] - Checkpoint dir: cv_finetune_turtorial_demo[0m
Step6、組建Finetune Task
有了合適的預訓練模型和準備要遷移的數據集後，我們開始組建一個Task。
由於該數據設置是一個二分類的任務，而我們下載的分類module是在ImageNet數據集上訓練的千分類模型，所以我們需要對模型進行簡單的微調，把模型改造爲一個二分類模型：
- 獲取module的上下文環境，包括輸入和輸出的變量，以及Paddle Program；
- 從輸出變量中找到特徵圖提取層feature_map；
- 在feature_map後面接入一個全連接層，生成Task；
input_dict, output_dict, program = module.context(trainable=True)
img = input_dict["image"]
feature_map = output_dict["feature_map"]
feed_list = [img.name]
task = hub.ImageClassifierTask(
data_reader=data_reader,
feed_list=feed_list,
feature=feature_map,
num_classes=dataset.num_labels,
config=config)
[32m[2020-04-26 13:06:04,337] [ INFO] - 267 pretrained paramaters loaded by PaddleHub[0m
Step5、開始Finetune
我們選擇finetune_and_eval接口來進行模型訓練，這個接口在finetune的過程中，會週期性的進行模型效果的評估，以便我們瞭解整個訓練過程的性能變化。
run_states = task.finetune_and_eval()
[32m[2020-04-26 13:06:10,841] [ INFO] - Strategy with slanted triangle learning rate, L2 regularization, [0m
[32m[2020-04-26 13:06:10,873] [ INFO] - Try loading checkpoint from cv_finetune_turtorial_demo/ckpt.meta[0m
[32m[2020-04-26 13:06:10,874] [ INFO] - PaddleHub model checkpoint not found, start from scratch...[0m
[32m[2020-04-26 13:06:10,909] [ INFO] - PaddleHub finetune start[0m
[36m[2020-04-26 13:06:12,540] [ TRAIN] - step 10 / 50: loss=0.89901 acc=0.73333 [step/sec: 7.16][0m
[36m[2020-04-26 13:06:13,915] [ TRAIN] - step 20 / 50: loss=0.38457 acc=1.00000 [step/sec: 10.22][0m
[36m[2020-04-26 13:06:15,447] [ TRAIN] - step 30 / 50: loss=0.11394 acc=1.00000 [step/sec: 6.95][0m
[36m[2020-04-26 13:06:16,902] [ TRAIN] - step 40 / 50: loss=0.06314 acc=1.00000 [step/sec: 7.48][0m
[36m[2020-04-26 13:06:18,353] [ TRAIN] - step 50 / 50: loss=0.04763 acc=1.00000 [step/sec: 7.62][0m
[32m[2020-04-26 13:06:18,432] [ INFO] - Load the best model from cv_finetune_turtorial_demo/best_model[0m
[32m[2020-04-26 13:06:18,433] [ INFO] - Evaluation on test dataset start[0m
share_vars_from is set, scope is ignored.
[34m[2020-04-26 13:06:19,083] [ EVAL] - [test dataset evaluation result] loss=0.00011 acc=1.00000 [step/sec: 19.32][0m
[32m[2020-04-26 13:06:19,085] [ INFO] - Saving model checkpoint to cv_finetune_turtorial_demo/step_51[0m
[32m[2020-04-26 13:06:20,090] [ INFO] - PaddleHub finetune finished.[0m
Step6、預測
當Finetune完成後，我們使用模型來進行預測，先通過以下命令來獲取測試的圖片
import numpy as np
import matplotlib.pyplot as plt
import matplotlib.image as mpimg
with open("dataset/test_list.txt","r") as f:
filepath = f.readlines()
# print(filepath)
data = [filepath[0].split(" ")[0],filepath[1].split(" ")[0],filepath[2].split(" ")[0],filepath[3].split(" ")[0],filepath[4].split(" ")[0]]
print(data)
label_map = dataset.label_dict()
index = 0
run_states = task.predict(data=data)
results = [run_state.run_results for run_state in run_states]
print(results)
print(50*'*')
for batch_result in results:
print(batch_result)
print(50*'*')
batch_result = np.argmax(batch_result, axis=2)[0]
print(batch_result)
for result in batch_result:
index += 1
result = label_map[result]
print("input %i is %s, and the predict result is %s" %
(index, data[index - 1], result))
[32m[2020-04-26 13:08:42,487] [ INFO] - PaddleHub predict start[0m
[32m[2020-04-26 13:08:42,487] [ INFO] - Load the best model from cv_finetune_turtorial_demo/best_model[0m
['dataset/test/yushuxin.jpg', 'dataset/test/xujiaqi.jpg', 'dataset/test/zhaoxiaotang.jpg', 'dataset/test/anqi.jpg', 'dataset/test/wangchengxuan.jpg']
[[array([[9.99820173e-01, 7.76551133e-06, 3.23944623e-05, 1.07691012e-04,
3.19731771e-05],
[2.23369602e-06, 9.99973655e-01, 9.16190515e-07, 2.26883185e-05,
5.21339530e-07],
[2.94848701e-06, 1.81872983e-05, 9.99861002e-01, 8.90642877e-06,
1.08885535e-04]], dtype=float32)], [array([[2.0881200e-06, 4.9753922e-05, 5.9350541e-06, 9.9994111e-01,
1.0736142e-06],
[3.4572211e-05, 3.4797737e-05, 3.2656546e-05, 3.2816235e-05,
9.9986517e-01]], dtype=float32)]]
**************************************************
[array([[9.99820173e-01, 7.76551133e-06, 3.23944623e-05, 1.07691012e-04,
3.19731771e-05],
[2.23369602e-06, 9.99973655e-01, 9.16190515e-07, 2.26883185e-05,
5.21339530e-07],
[2.94848701e-06, 1.81872983e-05, 9.99861002e-01, 8.90642877e-06,
1.08885535e-04]], dtype=float32)]
**************************************************
[0 1 2]
input 1 is dataset/test/yushuxin.jpg, and the predict result is 虞書欣
input 2 is dataset/test/xujiaqi.jpg, and the predict result is 許佳琪
input 3 is dataset/test/zhaoxiaotang.jpg, and the predict result is 趙小棠
[array([[2.0881200e-06, 4.9753922e-05, 5.9350541e-06, 9.9994111e-01,
1.0736142e-06],
[3.4572211e-05, 3.4797737e-05, 3.2656546e-05, 3.2816235e-05,
9.9986517e-01]], dtype=float32)]
**************************************************
[3 4]
input 4 is dataset/test/anqi.jpg, and the predict result is 安崎
input 5 is dataset/test/wangchengxuan.jpg, and the predict result is 王承渲
share_vars_from is set, scope is ignored.
[32m[2020-04-26 13:08:42,793] [ INFO] - PaddleHub predict finished.[0m