Pytorch——ArgMax and Reduction Tensor Ops

Tensor reduction operations

A reduction operation on a tensor is an operation that reduces the number of elements contained within the tensor.

1. Reshaping operations gave us the ability to position our elements along particular axes.
2. Element-wise operations allow us to perform operations on elements between two tensors.
3. Reduction operations allow us to perform operations on elements within a single tensor.

Reduction operation example

Suppose we the following 3 x 3 rank-2 tensor:

> t = torch.tensor([
    [0,1,0],
    [2,0,2],
    [0,3,0]
], dtype=torch.float32)

# Let’s look at our first reduction operation, a summation:

> t.sum()
tensor(8.)

> t.numel()
9

> t.sum().numel()
1

# We can see that

> t.sum().numel() < t.numel()
True

Since the number of elements have been reduced by the operation, we can conclude that the sum() method is a reduction operation.
As you may expect, here are some other common reduction functions:

> t.sum()
tensor(8.)

> t.prod()
tensor(0.)

> t.mean()
tensor(.8889)

> t.std()
tensor(1.1667)

Reducing tensors by axes

> t = torch.tensor([
    [1,1,1,1],
    [2,2,2,2],
    [3,3,3,3]
], dtype=torch.float32)

This is a 3 x 4 rank-2 tensor. Having different lengths for the two axes will help us understand these reduce operations
.Let’s consider the sum() method again. Only, this time, we will specify a dimension to reduce. We have two axes so we’ll do both. Check it out.

> t.sum(dim=0)  # 在維度0上進行加和，也就是數組相加，故1+2+3=6
tensor([6., 6., 6., 6.])

> t.sum(dim=1)  # 在維度1上進行加和，也就是每個數組內的數相加，1+1+1+1=4，2+2+2+2=8等
tensor([ 4.,  8., 12.])

When we sum across the first axis（第一個軸就是第零個維度）, we are taking the summation of all the elements of the first axis.

​> t[0]
tensor([1., 1., 1., 1.])

> t[1]
tensor([2., 2., 2., 2.])

> t[2]
tensor([3., 3., 3., 3.])

> t[0] + t[1] + t[2]
tensor([6., 6., 6., 6.])

The second axis in this tensor contains numbers that come in groups of four. Since we have three groups of four numbers, we get three sums.

> t[0].sum()
tensor(4.)

> t[1].sum()
tensor(8.)

> t[2].sum()
tensor(12.)

> t.sum(dim=1)
tensor([ 4.,  8., 12.])

Argmax tensor reduction operation

Argmax returns the index location of the maximum value inside a tensor.
When we call the argmax() method on a tensor, the tensor is reduced to a new tensor that contains an index value indicating where the max value is inside the tensor.

t = torch.tensor([
    [1,0,0,2],
    [0,3,3,0],
    [4,0,0,5]
], dtype=torch.float32)

> t.max()  # 最大值爲5
tensor(5.)

> t.argmax()
tensor(11)

> t.flatten()
tensor([1., 0., 0., 2., 0., 3., 3., 0., 4., 0., 0., 5.])

We’ll have a look at the flattened output for this tensor. If we don’t specific an axis to the argmax() method, it returns the index location of the max value from the flattened tensor, which in this case is indeed 11.
Let’s see how we can work with specific axes now.

> t.max(dim=0)  # 數組間進行比較
(tensor([4., 3., 3., 5.]), tensor([2, 1, 1, 2]))

> t.argmax(dim=0)
tensor([2, 1, 1, 2])

> t.max(dim=1)  # 數組內進行比較
(tensor([2., 3., 5.]), tensor([3, 1, 3]))

> t.argmax(dim=1)
tensor([3, 1, 3])

Notice how the call to the max() method returns two tensors. The first tensor contains the max values and the second tensor contains the index locations for the max values.

Accessing elements inside tensors

> t = torch.tensor([
    [1,2,3],
    [4,5,6],
    [7,8,9]
], dtype=torch.float32)

> t.mean()
tensor(5.)

> t.mean().item()
5.0

When we call mean on this 3 x 3 tensor, the reduced output is a scalar valued tensor. If we want to actually get the value as a number, we use the item() tensor method. This works for scalar valued tensors.
Have a look at how we do it with multiple values:

> t.mean(dim=0).tolist()
[4.0, 5.0, 6.0]

> t.mean(dim=0).numpy()
array([4., 5., 6.], dtype=float32)

When we compute the mean across the first axis, multiple values are returned, and we can access the numeric values by transforming the output tensor into a Python list or a NumPy array.