計算YOLOv3在COCO數據集上的mAP值

安裝darknet

  1. 詳細步驟和解釋請參考YOLOv3的官網,這裏給出相應的命令行和需要特別注意的地方

  2. 從github上下載項目源碼
    git clone https://github.com/pjreddie/darknet
    cd darknet
  3. 對源碼進行編譯(編譯默認運行YOLOv3時是不使用GPU的,我們假設跑YOLOv3時需要使用GPU,所以需要修改編譯選項)
    1. 只需要將當前文件夾下(darknet/)的Makefile文件中第一行的GPU=0改爲GPU=1,改完後的Makefile:
      GPU=1
      CUDNN=0
      OPENCV=0
      OPENMP=0
      DEBUG=0
      
      ...
      ...
      ...
    2. 編譯

      make
  4. 將權重文件(yolov3.weights)下載到當前目錄下
    wget https://pjreddie.com/media/files/yolov3.weights
  5. 測試
    #命令行中的‘-i 0’代表使用第0號顯卡,假如不想使用GPU可以用‘-nogpu’替代‘-i 0’
    ./darknet -i 0 detector test cfg/coco.data cfg/yolov3.cfg yolov3.weights data/dog.jpg 

COCO數據集的下載和配置

  1. 通過腳本文件(scripts/get_coco_dataset.sh)獲取COCO數據集(數據集下載速度會很慢,建議從Function的數據集下載站下載,然後逐行運行scripts/get_coco_dataset.sh中的每一行命令)
    cp scripts/get_coco_dataset.sh data
    cd data
    bash get_coco_dataset.sh
  2. 修改COCO的配置文件(cfg/coco.data)
    classes= 80
    train  = <path-to-coco>/trainvalno5k.txt # 注意這裏的<path-to-coco>需要填入COCO數據集的文件夾路徑
    valid  = <path-to-coco>/5k.txt # 注意這裏的<path-to-coco>需要填入COCO數據集的文件夾路徑
    names = data/coco.names
    backup = backup
    eval = coco #注意這行是爲了生成json格式的檢測結果,官網裏並沒有說,如果去掉這一行生成的文件是classes(80)類個txt文件
  3. 修改YOLOv3模型的配置文件(cfg/yolov3.cfg)

    [net]
    # Testing
    # batch=1
    # subdivisions=1
    # Training
    batch=64 #注意這裏的batch有可能使得顯存爆掉,這時需要修改,GeForce RTX 2080Ti需要修改爲8
    subdivisions=8
    ....

對COCO驗證集進行檢測

  1. 在darknet目錄下運行下面的命令行

    ./darknet -i 0 detector valid cfg/coco.data cfg/yolov3.cfg yolov3.weights # '-i 0'代表使用0號GPU

    運行完上述命令後會生成results/coco_results.json文件,該文件保存了檢測結果

計算mAP

方法源於:https://blog.csdn.net/xidaoliang/article/details/88397280

  1. 安裝pycocotools
    pip install pycocotools
  2. 在darknet目錄下編寫一個python腳本(compute_coco_mAP.py)用於計算mAP(注意根據自己的實際情況修改兩個文件路徑)

    #-*- coding:utf-8 -*-
    import matplotlib.pyplot as plt 
    from pycocotools.coco import COCO 
    from pycocotools.cocoeval import COCOeval 
    import numpy as np 
    import skimage.io as io 
    import pylab,json 
    pylab.rcParams['figure.figsize'] = (10.0, 8.0) 
    def get_img_id(file_name): 
        ls = [] 
        myset = [] 
        annos = json.load(open(file_name, 'r')) 
        for anno in annos: 
          ls.append(anno['image_id']) 
        myset = {}.fromkeys(ls).keys() 
        return myset 
    if __name__ == '__main__': 
        annType = ['segm', 'bbox', 'keypoints']#set iouType to 'segm', 'bbox' or 'keypoints'
        annType = annType[1] # specify type here
        cocoGt_file = 'data/coco/annotations/instances_val2014.json' #需要根據自己的實際情況配置該路徑
        cocoGt = COCO(cocoGt_file)#取得標註集中coco json對象
        cocoDt_file = 'results/coco_results.json' #需要根據自己的實際情況配置該路徑
        imgIds = get_img_id(cocoDt_file) 
        print(len(imgIds))
        cocoDt = cocoGt.loadRes(cocoDt_file)#取得結果集中image json對象
        imgIds = sorted(imgIds)#按順序排列coco標註集image_id
        imgIds = imgIds[0:5000]#標註集中的image數據
        cocoEval = COCOeval(cocoGt, cocoDt, annType) 
        cocoEval.params.imgIds = imgIds#參數設置
        cocoEval.evaluate()#評價
        cocoEval.accumulate()#積累
        cocoEval.summarize()#總結
  3. 運行compute_coco_mAP.py腳本

    python compute_coco_mAP.py
  4. 有可能在運行compute_coco_mAP.py腳本的過程中會報錯,可能的解決方法:

    1. 升級scikit-image

      pip install -U scikit-image
    2. 修改Numpy版本爲1.16
      pip install numpy==1.16

結果展示

  1. img_size爲608*608(img_size的設置可以通過修改cfg/yolov3.cfg文件中的width和height來實現)時:
     Average Precision  (AP) @[ IoU=0.50:0.95 | area=   all | maxDets=100 ] = 0.334
     Average Precision  (AP) @[ IoU=0.50      | area=   all | maxDets=100 ] = 0.585
     Average Precision  (AP) @[ IoU=0.75      | area=   all | maxDets=100 ] = 0.345
     Average Precision  (AP) @[ IoU=0.50:0.95 | area= small | maxDets=100 ] = 0.194
     Average Precision  (AP) @[ IoU=0.50:0.95 | area=medium | maxDets=100 ] = 0.365
     Average Precision  (AP) @[ IoU=0.50:0.95 | area= large | maxDets=100 ] = 0.439
     Average Recall     (AR) @[ IoU=0.50:0.95 | area=   all | maxDets=  1 ] = 0.291
     Average Recall     (AR) @[ IoU=0.50:0.95 | area=   all | maxDets= 10 ] = 0.446
     Average Recall     (AR) @[ IoU=0.50:0.95 | area=   all | maxDets=100 ] = 0.470
     Average Recall     (AR) @[ IoU=0.50:0.95 | area= small | maxDets=100 ] = 0.304
     Average Recall     (AR) @[ IoU=0.50:0.95 | area=medium | maxDets=100 ] = 0.502
     Average Recall     (AR) @[ IoU=0.50:0.95 | area= large | maxDets=100 ] = 0.593
    
  2. img_size爲416*416時:
     Average Precision  (AP) @[ IoU=0.50:0.95 | area=   all | maxDets=100 ] = 0.314
     Average Precision  (AP) @[ IoU=0.50      | area=   all | maxDets=100 ] = 0.559
     Average Precision  (AP) @[ IoU=0.75      | area=   all | maxDets=100 ] = 0.318
     Average Precision  (AP) @[ IoU=0.50:0.95 | area= small | maxDets=100 ] = 0.142
     Average Precision  (AP) @[ IoU=0.50:0.95 | area=medium | maxDets=100 ] = 0.341
     Average Precision  (AP) @[ IoU=0.50:0.95 | area= large | maxDets=100 ] = 0.464
     Average Recall     (AR) @[ IoU=0.50:0.95 | area=   all | maxDets=  1 ] = 0.278
     Average Recall     (AR) @[ IoU=0.50:0.95 | area=   all | maxDets= 10 ] = 0.419
     Average Recall     (AR) @[ IoU=0.50:0.95 | area=   all | maxDets=100 ] = 0.442
     Average Recall     (AR) @[ IoU=0.50:0.95 | area= small | maxDets=100 ] = 0.239
     Average Recall     (AR) @[ IoU=0.50:0.95 | area=medium | maxDets=100 ] = 0.482
     Average Recall     (AR) @[ IoU=0.50:0.95 | area= large | maxDets=100 ] = 0.611
    
  3. img_size爲320*320時:
     Average Precision  (AP) @[ IoU=0.50:0.95 | area=   all | maxDets=100 ] = 0.286
     Average Precision  (AP) @[ IoU=0.50      | area=   all | maxDets=100 ] = 0.521
     Average Precision  (AP) @[ IoU=0.75      | area=   all | maxDets=100 ] = 0.284
     Average Precision  (AP) @[ IoU=0.50:0.95 | area= small | maxDets=100 ] = 0.103
     Average Precision  (AP) @[ IoU=0.50:0.95 | area=medium | maxDets=100 ] = 0.315
     Average Precision  (AP) @[ IoU=0.50:0.95 | area= large | maxDets=100 ] = 0.449
     Average Recall     (AR) @[ IoU=0.50:0.95 | area=   all | maxDets=  1 ] = 0.260
     Average Recall     (AR) @[ IoU=0.50:0.95 | area=   all | maxDets= 10 ] = 0.389
     Average Recall     (AR) @[ IoU=0.50:0.95 | area=   all | maxDets=100 ] = 0.408
     Average Recall     (AR) @[ IoU=0.50:0.95 | area= small | maxDets=100 ] = 0.189
     Average Recall     (AR) @[ IoU=0.50:0.95 | area=medium | maxDets=100 ] = 0.454
     Average Recall     (AR) @[ IoU=0.50:0.95 | area= large | maxDets=100 ] = 0.601
    

     

關於結果的解釋請參考:https://blog.csdn.net/u014734886/article/details/78831884

發表評論
所有評論
還沒有人評論,想成為第一個評論的人麼? 請在上方評論欄輸入並且點擊發布.
相關文章