Getting Started with PyTorch: A Beginner’s Guide to Deep Learning

PyTorch, created by Meta’s AI Research lab, has become one of the most popular deep learning frameworks in both academia and industry. Its flexibility and user-friendly design makes it a top choice for researchers and professionals alike. High-profile applications of PyTorch, such as Tesla’s self-driving car AI and key defense projects, highlight it’s robustness and versatility in real-world, high-stakes environments.

What is PyTorch?

PyTorch is a user-friendly and robust framework for developing deep learning models. Think of it like a set of building blocks that help us create artificial intelligence systems, such as image recognition or natural language processing models. PyTorch is popular because it’s flexible, works well with Python, and can run fast on both our computer’s CPU and powerful GPUs for larger tasks.

Its core components include:

• Tensors Multi-dimensional arrays with GPU acceleration, supporting efficient operations and high versatility in data manipulation. They enable complex mathematical operations essential for deep learning and scientific computing.
• Autograd System Enables automatic differentiation by tracking tensor operations, simplifying gradient computation for model optimization. This allows for flexible experimentation and rapid prototyping of custom models.
• Neural Network Module (torch.nn) Provides layers, loss functions, and model-building tools, seamlessly integrating with autograd for flexible deep learning. Its modularity allows for the ease of building complex architectures.
• Optimizers (torch.optim) Contains algorithms like SGD and Adam to update model parameters, enabling fine-tuning during training. These optimizers are customizable, offering control over training dynamics and performance.

Key advantages of PyTorch are:

• Dynamic computation graph: PyTorch allows flexible model building by enabling real-time adjustments to the computation graph during runtime, making experimentation easier.
• Pythonic syntax: PyTorch uses an intuitive, Python-friendly syntax. This simplifies coding, especially for developers familiar with Python.
• Community support: PyTorch has a large, active community that provides extensive tutorials, resources, and troubleshooting support. This helps users learn and resolve issues quickly.

PyTorch vs TensorFlow

Both PyTorch and TensorFlow are critical tools for machine learning. Let’s explore the difference between both:

PyTorch Basics

PyTorch enables efficient model building for a wide range of applications, including neural networks, computer vision, and natural language processing. But first, let’s learn how we can install PyTorch and about its most important concept called Tensors.

PyTorch Installation

Installing PyTorch is straightforward and varies slightly based on the operating system. PyTorch’s website offers an interactive tool to help generate the appropriate installation command for CPU support. Follow these simple steps to install PyTorch on your device:

Open your command shell and run the command:

pip3 install torch torchvision torchaudio 

For GPU support, add the appropriate torch version for CUDA:

pip3 install torch torchvision torchaudio --extra-index-url <https://download.pytorch.org/whl/cu113> 

Set up a virtual environment to manage dependencies:

python -m venv myenv 

# Unix/macOS systems
source myenv/bin/activate  

# On Windows
myenv\Scripts\activate 

# To deactivate, use `deactivate` 

PyTorch Tensors

Tensors form the backbone of PyTorch. They are like arrays in NumPy but with enhanced ability. Unlike NumPy arrays, tensors can leverage GPU acceleration for faster computation, making them for deep learning applications.

A tensor extends beyond 2D structures, supporting various dimensions, which is crucial for handling the complexities of deep learning models. Next, we’ll see the different ways to manipulate tensors.

Creating and Manipulating Tensors in PyTorch

We can perform multiple mathematical and logical operations on PyTorch tensors. We can also manipulate the shape of tensors and change their dimensions. Let’s walk through how to create, manipulate, and perform operations on tensors in PyTorch.

Creating tensors

• 1D Tensor Creation

A 1D tensor is a list of numbers. Here’s how to create a basic 1D tensor:

# Creating a 1D tensor 
tensor_one_d = torch.tensor([1, 2, 3, 4]) 
print(tensor_one_d)  # Output: tensor([1, 2, 3, 4]) 

• 2D Tensor Creation

A 2D tensor extends this idea by arranging numbers in rows and columns, much like a matrix.

# Creating a 2D tensor 
tensor_two_d = torch.tensor([[1, 2], [3, 4]]) 
print(tensor_two_d)  # Output: tensor([[1, 2], [3, 4]]) 

This is useful for more complex data structures, like images or tabular data, where multiple dimensions are required.

Indexing, Slicing, and Reshaping

• Indexing

Just like lists or arrays, we can access individual elements in a tensor by using their index. Let’s see how we can retrieve the first element from the 1D tensor we created earlier:

# Indexing to access an element 
element = tensor_one_d [0] 
print(element)  # Output: tensor(1) 

Indexing works the same way in higher-dimensional tensors. For instance, tensor_2d[0,1] would give you the value 2.

• Slicing

Slicing allows us to extract specific sections of a tensor. Below is an example of slicing a range from the 1D tensor:

# Slicing a tensor 
slice_one_d = tensor_one_d[1:3] 
print(slice_one_d)  # Output: tensor([2, 3]) 

This slices out elements at indices 1 and 2, resulting in a smaller tensor.

• Reshaping

Reshaping is important in deep learning when we need to change the shape of the tensor without altering its data.

The view() function allows us to specify a new shape for a tensor, where each dimension in the new shape must be compatible with the number of elements in the original tensor. It changes the shape of the tensor but keeps the total number of elements the same.

Here’s how to reshape a 2D tensor into a 1D tensor:

# Reshaping a tensor 
reshaped_tensor = tensor_two_d.view(4)  # Reshapes the 2D tensor to a 1D tensor 
print(reshaped_tensor)  # Output: tensor([1, 2, 3, 4]) 

Performing Mathematical Operations

PyTorch allows easy execution of element-wise mathematical operations on tensors.

• Adding tensors:

Now, let’s perform a basic operation of adding two tensors together. We’re adding tensor_one and tensor_two, which are two tensors with the same shape. The addition happens element-wise: tensor_one[i] + tensor_two[i] for each element:

Tensor_one = torch.tensor([1.0, 2.0, 3.0]) 
Tensor_two = torch.tensor([4.0, 5.0, 6.0]) 
addition = x + y 
print(addition) 

The output of the code will be:

tensor([5., 7., 9.]) 

• Matrix multiplication:

We can perform a matrix multiplication between two 2D tensors using the torch.matmul() function:

a = torch.tensor([[1, 2], [3, 4]]) 
b = torch.tensor([[5, 6], [7, 8]]) 
multiplication = torch.matmul(a, b)  # or use a @ b 
print(multiplication) 

The output of the code will be:

tensor([[19, 22], 
        [43, 50]]) 

• Adding Scalars to Tensors

We can add a scalar value to each element in the tensor like so:

# Adding a scalar to a tensor 
added_tensors = tensor_one_d + 1 
print(added_tensors)  # Output: tensor([2, 3, 4, 5]) 

• Multiplying Scalars with Tensors

Similarly, scalar multiplication multiplies every element in the tensor by the given value:

# Multiplying tensor elements by a scalar 
multiplied_tensors = tensor_one_d * 2 
print(multiplied_tensors)  # Output: tensor([2, 4, 6, 8]) 

Advanced Tensor Operations

Let’s dive into more advanced operations like stacking, concatenating, splitting tensors, and moving them across devices.

• Stacking Tensors

Stacking creates a new dimension and stacks tensors along that dimension. The torch.stack() function stacks a sequence of tensors along a new dimension, creating an additional axis in the output tensor. This operation is particularly useful for organizing data in batches.

Here’s an example of torch.stack():

# Stacking tensors along a new dimension 
stacked_tensors = torch.stack((tensor_one_d, tensor_one_d)) 
print(stacked_tensors)  # Output: tensor([[1, 2, 3, 4], [1, 2, 3, 4]]) 

This is useful for organizing batches of data for neural network processing.

• Concatenating Tensors

We can also concatenate tensors along an existing dimension. The torch.cat() function concatenates tensors along an existing dimension, which is useful when merging data, such as combining different parts of a dataset for training:

# Concatenating tensors along an existing dimension 
concatenated_tensors = torch.cat((tensor_one_d, tensor_one_d), dim=0) 
print(concatenated_tensors)  # Output: tensor([1, 2, 3, 4, 1, 2, 3, 4]) 

Concatenation is particularly handy when we want to combine data for training purposes.

• Splitting Tensors

To break tensors into smaller chunks, use torch.chunk(). It divides a tensor into smaller chunks along a specified dimension, allowing you to break data into manageable pieces:

# Splitting tensors into chunks 
split_tensors = torch.chunk(tensor_one_d, 2) 
print(split_tensors)  # Output: (tensor([1, 2]), tensor([3, 4])) 

This splits the tensor into two equal chunks.

Moving Tensors Across Devices

One of the great features of PyTorch is the ability to seamlessly move tensors between devices, such as a CPU and a GPU.

• Changing Tensor Data Type

First, let’s change the data type of the tensor from integer to floating point:

# Changing tensor data type 
float_tensor = tensor_one_d.float() 
print(float_tensor)  # Output: tensor([1., 2., 3., 4.]) 

• Moving Tensors to GPU

If a GPU is available, we can move the tensor to it for faster computation. We do this like so:

# Moving tensor to GPU (if available) 
if torch.cuda.is_available(): 
  tensor_gpu = tensor_one_d.to('cuda') 
  print(tensor_gpu)  # Output will be a CUDA tensor representation 

This allows our deep learning models to take advantage of GPU acceleration, which is important for scaling models to larger datasets.

Understanding tensor operations is crucial for working effectively with neural networks. This foundational knowledge will enable the construction and optimization of complex neural network architectures.

Wrapping Up

Excellent! Let’s revise what we learned throughout the tutorial:

• PyTorch is a flexible and user-friendly deep learning framework developed by Facebook’s AI Research lab, ideal for both research and industry applications.
• Key components include tensors for multi-dimensional arrays and an autograd system for automatic differentiation, enabling efficient model training.
• The framework offers advantages like a dynamic computation graph, intuitive Pythonic syntax, and strong community support, making it accessible for newcomers.
• PyTorch is compared to TensorFlow, which has strengths in ease of use and prototyping. TensorFlow is often preferred for large-scale, production-ready systems.

You can take this free course Intro to PyTorch and Neural Networks to learn more about PyTorch and its basics.