NVIDIA NGC鏡像使用筆記

NGC鏡像列表官方web地址:

https://ngc.nvidia.com/catalog/all

NGC命令官方使用說明地址:

https://docs.nvidia.com/ngc/ngc-catalog-cli-user-guide/index.html

 

查看NGC鏡像列表

# ngc registry image list
+--------------------------------------+--------------------------------------+---------------------------------+------------+--------------+------------+
| Name                                 | Repository                           | Latest Tag                      | Image Size | Updated Date | Permission |
+--------------------------------------+--------------------------------------+---------------------------------+------------+--------------+------------+
| CUDA                                 | nvidia/cuda                          | latest                          | 1.76 GB    | Jun 19, 2020 | unlocked   |
| Caffe2                               | nvidia/caffe2                        | 18.08-py3                       | 1.3 GB     | Oct 18, 2019 | unlocked   |
| NVCaffe                              | nvidia/caffe                         | 20.03-py3                       | 2.08 GB    | Mar 26, 2020 | unlocked   |
| Microsoft Cognitive Toolkit          | nvidia/cntk                          | 18.08-py3                       | 2.4 GB     | Oct 18, 2019 | unlocked   |
| MXNet                                | nvidia/mxnet                         | 20.03-py3                       | 2.56 GB    | Mar 26, 2020 | unlocked   |
| PyTorch                              | nvidia/pytorch                       | 20.03-py3                       | 3.38 GB    | Mar 26, 2020 | unlocked   |
| TensorFlow                           | nvidia/tensorflow                    | 20.03-tf2-py3                   | 2.78 GB    | Jun 19, 2020 | unlocked   |
| Theano                               | nvidia/theano                        | 18.08                           | 1.49 GB    | Oct 18, 2019 | unlocked   |
| Torch                                | nvidia/torch                         | 18.08-py2                       | 1.24 GB    | Oct 18, 2019 | unlocked   |
...

輸入#號後命令後,返回的是NGC鏡像列表。列表解釋:

Title Describe
Name 鏡像名稱
Repository 鏡像路徑/地址
Lasted Tag 默認鏡像的鏡像版本
Image Size 默認鏡像的大小
Updated Date 默認鏡像的更新時間
Permission 未知

查看某個鏡像中所有鏡像版本/tag

ngc registry image info [鏡像路徑/地址]

以鏡像列表的caffe2爲例 ,需要注意的是命令用到的是caffe2的鏡像地址repository,非鏡像名稱name。

# ngc registry image info nvidia/caffe     
--------------------------------------------------
  Image Repository Information
    Name: NVCaffe
    Short Description: NVIDIA Caffe, also known as NVCaffe, is an NVIDIA-maintained fork of Berkeley Vision and Learning Center (BVLC) Caffe tuned for NVIDIA GPUs, particularly in multi-GPU configurations.
    Built By: NVIDIA
    Publisher: NVIDIA
    Logo: https://static.nvidiagrid.net/ngc/containers/nvcaffe.png
    Labels: Deep Learning, Training
    Public: Yes
    Last Updated: Mar 26, 2020
    Latest Image Size: 2.08 GB
    Latest Tag: 20.03-py3
    Tags: 
        20.03-py3
        20.02-py3
        20.01-py3
        19.12-py3
        19.11-py3
        19.10-py2
        19.09-py2
        19.08-py2
        19.07-py2
        19.06-py2
  ...

查看某一鏡像tag的詳細信息

我們是在使用NGC時不免會想得到一個鏡像的詳細信息,如鏡像中使用到了什麼版本的cuda,cudnn等,這時我們可以通過json格式輸出鏡像的詳細信息,以此來選擇合適的鏡像。

ngc registry image --format_type json info [鏡像路徑]:[鏡像tag]

 以鏡像caffe2中tag 19.11-py3爲例 ,需要注意的是命令用到的是caffe2的鏡像地址repository,非鏡像倉庫名稱name。

# ngc registry image --format_type json info nvidia/caffe:19.11-py3    
{
    "architecture": "amd64",
    "fsLayers": [
        {
            "blobSum": "sha256:b591970b7b882ee9d190d3f8ea11a075782eb47aaee74df8553f84413281ee1a"
        },
...
...
    ],
    "history": [
        {
            "v1Compatibility": "{\"architecture\":\"amd64\",\"config\":{\"Hostname\":\"\",\"Domainname\":\"\",\"User\":\"\",\"AttachStdin\":false,\"AttachStdout\":false,\"AttachStderr\":false,\"Tty\":false,\"OpenStdin\":false,\"StdinOnce\":false,\"Env\":[\"PATH=/usr/local/mpi/bin:/usr/local/nvidia/bin:/usr/local/cuda/bin:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin\",\"CUDA_VERSION=10.2.89\",\"CUDA_DRIVER_VERSION=440.30.00\",\"CUDA_CACHE_DISABLE=1\",\"_CUDA_COMPAT_PATH=/usr/local/cuda/compat\",\"ENV=/etc/shinit_v2\",\"BASH_ENV=/etc/bash.bashrc\",\"NVIDIA_REQUIRE_CUDA=cuda\\u003e=9.0\",\"NCCL_VERSION=2.5.6\",\"CUBLAS_VERSION=10.2.2.89\",\"CUDNN_VERSION=7.6.5.32\",\"TRT_VERSION=6.0.1.8\",\"NSIGHT_SYSTEMS_VERSION=2019.5.2\",\"DALI_VERSION=0.15.0\",\"DALI_BUILD=947079\",\"LD_LIBRARY_PATH=/usr/local/cuda/compat/lib:/usr/local/nvidia/lib:/usr/local/nvidia/lib64\",\"NVIDIA_VISIBLE_DEVICES=all\",\"NVIDIA_DRIVER_CAPABILITIES=compute,utility,video\",\"MOFED_VERSION=4.4-1.0.0\",\"IBV_DRIVERS=/usr/lib/libibverbs/libmlx5\",\"OPENMPI_VERSION=3.1.4\",\"OMPI_MCA_btl_vader_single_copy_mechanism=none\",\"LIBRARY_PATH=/usr/local/cuda/lib64/stubs:\",\"CAFFE_VERSION=0.17.3\",\"NVIDIA_CAFFE_VERSION=19.11\",\"PYTHONPATH=:/usr/local/python\",\"NVIDIA_BUILD_ID=8776850\"],\"Cmd\":null,\"ArgsEscaped\":true,\"Image\":\"sha256:0a69bd6b813ec5dc5dd2b73b1db842d4e1050ff86dd3b478387c33ea4df844c2\",\"Volumes\":null,\"WorkingDir\":\"/workspace\",\"Entrypoint\":[\"/usr/local/bin/nvidia_entrypoint.sh\"],\"OnBuild\":null,\"Labels\":{\"com.nvidia.build.id\":\"8776850\",\"com.nvidia.build.ref\":\"6b1ea48d7c7e81888171e242f1e205e985ae73bb\",\"com.nvidia.caffe.version\":\"0.17.3\",\"com.nvidia.cublas.version\":\"10.2.2.89\",\"com.nvidia.cuda.version\":\"9.0\",\"com.nvidia.cudnn.version\":\"7.6.5.32\",\"com.nvidia.nccl.version\":\"2.5.6\",\"com.nvidia.nsightsystems.version\":\"2019.5.2\",\"com.nvidia.tensorrt.version\":\"6.0.1.8\",\"com.nvidia.volumes.needed\":\"nvidia_driver\"}},\"container\":\"4825a54e4a1fd7a0d15dee754baef65db7e6f8fba489efd75d40bc80dc74c116\",\"container_config\":{\"Hostname\":\"\",\"Domainname\":\"\",\"User\":\"\",\"AttachStdin\":false,\"AttachStdout\":false,\"AttachStderr\":false,\"Tty\":false,\"OpenStdin\":false,\"StdinOnce\":false,\"Env\":[\"PATH=/usr/local/mpi/bin:/usr/local/nvidia/bin:/usr/local/cuda/bin:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin\",\"CUDA_VERSION=10.2.89\",\"CUDA_DRIVER_VERSION=440.30.00\",\"CUDA_CACHE_DISABLE=1\",\"_CUDA_COMPAT_PATH=/usr/local/cuda/compat\",\"ENV=/etc/shinit_v2\",\"BASH_ENV=/etc/bash.bashrc\",\"NVIDIA_REQUIRE_CUDA=cuda\\u003e=9.0\",\"NCCL_VERSION=2.5.6\",\"CUBLAS_VERSION=10.2.2.89\",\"CUDNN_VERSION=7.6.5.32\",\"TRT_VERSION=6.0.1.8\",\"NSIGHT_SYSTEMS_VERSION=2019.5.2\",\"DALI_VERSION=0.15.0\",\"DALI_BUILD=947079\",\"LD_LIBRARY_PATH=/usr/local/cuda/compat/lib:/usr/local/nvidia/lib:/usr/local/nvidia/lib64\",\"NVIDIA_VISIBLE_DEVICES=all\",\"NVIDIA_DRIVER_CAPABILITIES=compute,utility,video\",\"MOFED_VERSION=4.4-1.0.0\",\"IBV_DRIVERS=/usr/lib/libibverbs/libmlx5\",\"OPENMPI_VERSION=3.1.4\",\"OMPI_MCA_btl_vader_single_copy_mechanism=none\",\"LIBRARY_PATH=/usr/local/cuda/lib64/stubs:\",\"CAFFE_VERSION=0.17.3\",\"NVIDIA_CAFFE_VERSION=19.11\",\"PYTHONPATH=:/usr/local/python\",\"NVIDIA_BUILD_ID=8776850\"],\"Cmd\":[\"|2\",\"NVIDIA_BUILD_REF=6b1ea48d7c7e81888171e242f1e205e985ae73bb\",\"PYVER=3.6\",\"/bin/sh\",\"-c\",\"rm -rf /opt/mellanox/DEBS/4.7-1.0.0/DEBS\"],\"ArgsEscaped\":true,\"Image\":\"sha256:0a69bd6b813ec5dc5dd2b73b1db842d4e1050ff86dd3b478387c33ea4df844c2\",\"Volumes\":null,\"WorkingDir\":\"/workspace\",\"Entrypoint\":null,\"OnBuild\":null,\"Labels\":{\"com.nvidia.build.id\":\"8776850\",\"com.nvidia.build.ref\":\"6b1ea48d7c7e81888171e242f1e205e985ae73bb\",\"com.nvidia.caffe.version\":\"0.17.3\",\"com.nvidia.cublas.version\":\"10.2.2.89\",\"com.nvidia.cuda.version\":\"9.0\",\"com.nvidia.cudnn.version\":\"7.6.5.32\",\"com.nvidia.nccl.version\":\"2.5.6\",\"com.nvidia.nsightsystems.version\":\"2019.5.2\",\"com.nvidia.tensorrt.version\":\"6.0.1.8\",\"com.nvidia.volumes.needed\":\"nvidia_driver\"}},\"created\":\"2019-11-13T01:25:47.958284932Z\",\"docker_version\":\"18.06.0-ce\",\"id\":\"5c4e301aec4260b05edb24eda582a5c59b498e9a2f796c33bce7ca9a4f56fffe\",\"os\":\"linux\",\"parent\":\"fda4f4fd76f910dc8f69f6079556cef18cd4bf470daf55a98cf2440f8fbe71b9\"}"
        },
...
...

每個鏡像的詳細信息不同,上例中我們可以從history中查看到鏡像中所使用的環境信息。

下載NGC鏡像

在鏡像中找到合適的鏡像tag後,通過此命令將鏡像下載到本地

ngc registry image pull [鏡像路徑]:[鏡像tag]

以鏡像caffe2中tag 19.11-py3爲例 ,需要注意的是命令用到的是caffe2的鏡像地址repository,非鏡像倉庫名稱name。

# ngc registry image pull nvidia/caffe:19.11-py3
...
5c5837a082b9: Waiting
6f25f220bc2b: Waiting
b591970b7b88: Waiting
7ddbc47eeb70: Downloading [====================>                              ]  10.86MB/26.69MB
c1bbdc448b72: Download complete
8c3b70e39044: Download complete
45d437916d57: Download complete
08675124ee65: Downloading [==============================================>    ]  8.354MB/9.054MB
258aa4bcaca7: Download complete
f2e8775ec207: Downloading [>                                                  ]  10.25MB/715.2MB
...

 

發表評論
所有評論
還沒有人評論,想成為第一個評論的人麼? 請在上方評論欄輸入並且點擊發布.
相關文章