Pytorch——Build PyTorch CNN - Object Oriented Neural Networks

Building neural networks with PyTorch

When say model, we mean our network. The words model and network mean the same thing. What we want our network to ultimately do is model or approximate a function that maps image inputs to the correct output class.
說到模型，我們指的是我們的網絡。模型和網絡是一個意思。我們希望我們的網絡最終做的是建模或近似一個將圖像輸入映射到正確輸出類別的函數。

Let’s build a simple lizard class to demonstrate how classes encapsulate data and code:

class Lizard: #class declaration
    def __init__(self, name): #class constructor (code)
        self.name = name #attribute (data)
    
    def set_name(self, name): #method declaration (code)
        self.name = name #method implementation (code)

The first line declares the class and specifies the class name, which in this case is Lizard.
The second line defines a special method called the class constructor. Class constructors are called when a new instance of the class is created. As parameters, we have self and name.

The self parameter gives us the ability to create attribute values that are stored or encapsulated within the object. 參數self使我們能夠創建存儲或封裝在對象中的屬性值。
When we call this constructor or any of the other methods, we don’t pass the self parameter. Python does this for us automatically.

Argument values for any other parameter are arbitrarily passed by the caller, and these passed values that come in to the method can be used in a calculation or saved and accessed later using self.任何其他參數的參數值都是由調用者任意傳遞的，這些傳入方法的傳遞值可以在計算中使用，或者稍後使用self保存並訪問。

After we’re done with the constructor, we can create any number of specialized methods like this one here that allows a caller to change the name value that was stored in self. All we have to do here is call the method and pass a new value for the name. Let’s see this in action.

> lizard = Lizard('deep')
> print(lizard.name)
deep

> lizard.set_name('lizard')
> print(lizard.name)
lizard

PyTorch’s torch.nn package

To build neural networks in PyTorch, we use the torch.nn package, which is PyTorch’s neural network (nn) library. We typically import the package like so:

import torch.nn as nn

PyTorch’s nn.Module class

As we know, deep neural networks are built using multiple layers. This is what makes the network deep. Each layer in a neural network has two primary components:

1. A transformation (code)
2. A collection of weights (data)

Within the nn package, there is a class called Module, and it is the base class for all of neural network modules which includes layers.

This means that all of the layers in PyTorch extend the nn.Module class and inherit all of PyTorch’s built-in functionality within the nn.Module class. In OOP this concept is known as inheritance.

Neural networks and layers in PyTorch extend the nn.Module class. This means that we must extend the nn.Module class when building a new layer or neural network in PyTorch.

PyTorch nn.Modules have a forward() method

The tensor input is passed forward though the network.
The goal of the overall transformation is to transform or map the input to the correct prediction output class, and during the training process, the layer weights (data) are updated in such a way that cause the mapping to adjust to make the output closer to the correct prediction.整體轉換的目標是將輸入轉換或映射到正確的預測輸出類，並且在訓練過程中，以這種方式更新層權重(數據)，從而調整映射，使輸出更接近正確的預測。

When we are building layers and networks, we must provide an implementation of the forward() method. The forward method is the actual transformation.

PyTorch’s nn.functional package

The nn.functional package contains methods that subclasses of nn.Module use for implementing their forward() functions.

Building a neural network in PyTorch

The steps are as follows:

1. Create a neural network class that extends the nn.Module base class.
2. In the class constructor, define the network’s layers as class attributes using pre-built layers from torch.nn.
3. Use the network’s layer attributes as well as operations from the nn.functional API to define the network’s forward pass.

class Network(nn.Module): # line 1
    def __init__(self):
        super().__init__() # line 3
        self.layer = None

    def forward(self, t):
        t = self.layer(t)
        return t

1. Specify the nn.Module class in parentheses on line 1.
2. Insert a call to the super class constructor on line 3 inside the constructor.

Define the network’s layers as class attributes

We’re building a CNN, so the two types of layers we’ll use are linear layers and convolutional layers.

class Network(nn.Module):
    def __init__(self):
        super().__init__()
        self.conv1 = nn.Conv2d(in_channels=1, out_channels=6, kernel_size=5)
        self.conv2 = nn.Conv2d(in_channels=6, out_channels=12, kernel_size=5)
        
        self.fc1 = nn.Linear(in_features=12 * 4 * 4, out_features=120)
        self.fc2 = nn.Linear(in_features=120, out_features=60)
        self.out = nn.Linear(in_features=60, out_features=10)
        
    def forward(self, t):
        # implement the forward pass
        return t

We used the abbreviation fc in fc1 and fc2 because linear layers are also called fully connected layers. They also have a third name that we may hear sometimes called dense.
So linear, dense, and fully connected are all ways to refer to the same type of layer. PyTorch uses the word linear, hence the nn.Linear class name.

We used the name out for the last linear layer because the last layer in the network is the output layer.

CNN Layer Parameters

Parameter vs Argument

Parameters are used in function definitions as place-holders while arguments are the actual values that are passed to the function. The parameters can be thought of as local variables that live inside a function.
In our network’s case, the names are the parameters and the values that we have specified are the arguments.在我們的網絡中，名稱是parameter，我們指定的值是argument。

Two types of parameters

1. Hyperparameters
2. Data dependent hyperparameters

When we construct a layer, we pass values for each parameter to the layer’s constructor. With our convolutional layers have three parameters and the linear layers have two parameters.

• Convolutional layers： in_channels / out_channels / kernel_size
• Linear layers：in_features / out_features

Hyperparameters

In general, hyperparameters are parameters whose values are chosen manually and arbitrarily.
We choose hyperparameter values mainly based on trial and error and increasingly by utilizing values that have proven to work well in the past. For building our CNN layers, these are the parameters we choose manually.我們選擇超參數的值主要是基於嘗試和錯誤，並越來越多地使用過去已經證明工作良好的值。對於構建CNN層，這些是我們手動選擇的參數。

On pattern that shows up quite often is that we increase our out_channels as we add additional conv layers, and after we switch to linear layers we shrink our out_features as we filter down to our number of output classes.
一般來講，我們增加了輸出通道，因爲我們添加了額外的conv層，當我們切換到線性層後，我們縮小了輸出特性，因爲我們過濾了一部分輸出類的數量。

Data dependent hyperparameters

Data dependent hyperparameters are parameters whose values are dependent on data. The first two data dependent hyperparameters that stick out are the in_channels of the first convolutional layer, and the out_features of the output layer.

The in_channels of the first convolutional layer depend on the number of color channels present inside the images that make up the training set. Since we are dealing with grayscale images, we know that this value should be a 1.輸入通道的大小取決於圖像的顏色通道個數。

The out_features for the output layer depend on the number of classes that are present inside our training set. Since we have 10 classes of clothing inside the Fashion-MNIST dataset, we know that we need 10 output features.輸出特徵的大小取決於分類的個數。

In general, the input to one layer is the output from the previous layer, and so all of the in_channels in the conv layers and in_features in the linear layers depend on the data coming from the previous layer.

When we switch from a conv layer to a linear layer, we have to flatten our tensor.
當我們從一個conv層轉換到一個線性層時，我們必須使我們的張量變平。

Summary of layer parameters

self.conv1 = nn.Conv2d(in_channels=1, out_channels=6, kernel_size=5)
self.conv2 = nn.Conv2d(in_channels=6, out_channels=12, kernel_size=5)

self.fc1 = nn.Linear(in_features=12 * 4 * 4, out_features=120)
self.fc2 = nn.Linear(in_features=120, out_features=60)
self.out = nn.Linear(in_features=60, out_features=10)

c