Working with Tensors in PyTorch

Introduction

Tensors are the fundamental data structure in PyTorch, similar to NumPy arrays, but with additional capabilities for GPU acceleration and automatic differentiation. Understanding how to work with tensors is essential for any deep learning project.

Creating Tensors

import torch

# From list
a = torch.tensor([1, 2, 3])

# 2D Tensor (Matrix)
b = torch.tensor([[1, 2], [3, 4]])

# With random values
c = torch.rand(2, 3)

# Zeros and ones
d = torch.zeros(2, 2)
e = torch.ones(2, 2)

Checking Shape and Type

print("Shape of b:", b.shape)
print("Data type of c:", c.dtype)

Basic Tensor Operations

x = torch.tensor([10.0, 20.0])
y = torch.tensor([1.0, 2.0])

# Element-wise addition
print(x + y)

# Multiplication
print(x * y)

# Dot product
print(torch.dot(x, y))

Reshaping Tensors

You can reshape tensors using view() or reshape():

f = torch.arange(6)  # [0, 1, 2, 3, 4, 5]
f_reshaped = f.view(2, 3)
print(f_reshaped)

Tensor Indexing and Slicing

m = torch.tensor([[5, 6, 7], [8, 9, 10]])

print(m[0])     # First row
print(m[:, 1])  # Second column
print(m[1, 2])  # Element at row 1, col 2

Moving Tensors to GPU

if torch.cuda.is_available():
    a = a.to('cuda')
    print("Tensor on GPU:", a)
else:
    print("CUDA not available")

Advanced Tensor Initialization

import torch

# Identity matrix
i = torch.eye(3)

# Range with steps
r = torch.arange(0, 10, step=2)

# Like existing tensor
base = torch.ones(2, 3)
same_shape = torch.empty_like(base)

print("Identity matrix:", i)
print("Range with step:", r)
print("Empty like base:", same_shape)

• torch.eye creates an identity matrix.
• torch.arange creates evenly spaced values within a range.
• torch.empty_like creates an uninitialized tensor with the same shape as another.

Tensor Type Conversion

x = torch.tensor([1.5, 2.5])

# Convert to int
x_int = x.int()

# Convert to float
x_float = x_int.float()

print(x_int)
print(x_float)

• We can convert tensor data types using methods like .int(), .float(), etc.
• This is useful when matching tensor types for operations (e.g., during loss calculations).

Tensor Broadcasting

a = torch.tensor([[1, 2], [3, 4]])
b = torch.tensor([10, 20])

# Broadcasting adds b to each row of a
result = a + b

print(result)

• Broadcasting allows operations between tensors of different shapes by implicitly expanding the smaller tensor.
• Here, vector b is added to each row of matrix a.

Tensor Cloning and Detaching

t = torch.tensor([[1.0, 2.0]], requires_grad=True)
t_clone = t.clone().detach()

print("Original tensor:", t)
print("Detached clone:", t_clone)

• .clone() creates a copy of the tensor.
• .detach() removes the tensor from the computation graph, useful in gradient-sensitive operations.

Use Case: Batch Tensor Reshaping

x = torch.arange(12).view(3, 4)  # 3 samples, 4 features
x_reshaped = x.view(-1, 2, 2)     # Reshape into 3 samples of 2x2

print(x)
print(x_reshaped)

• This example shows how to reshape a 2D tensor into a 3D tensor while preserving batch structure.
• -1 tells PyTorch to infer the correct dimension size automatically.

Tensor Comparison and Boolean Masking

x = torch.tensor([1, 3, 5, 7])
mask = x > 3

print("Mask:", mask)
print("Filtered values:", x[mask])

• mask is a Boolean tensor showing where conditions are met.
• Used frequently in data filtering and conditional logic.

Use Case: Normalizing Tensors

x = torch.tensor([1.0, 2.0, 3.0])
x_norm = (x - x.mean()) / x.std()

print(x_norm)

• Tensor normalization is common in machine learning for stabilizing training.
• We subtract the mean and divide by the standard deviation to bring values to a standard scale.

Conclusion

Tensors are the core data structure in PyTorch, supporting flexible shape transformations, mathematical operations, and easy movement between CPU and GPU. Mastery of tensors will enable you to build and debug deep learning models effectively.