⬡ Hub
Skip to content

PyTorch: From Novice to Professional

PyTorch is an open-source machine learning library primarily developed by Facebook's AI Research lab (FAIR). It's widely used for applications such as computer vision and natural language processing. It is known for its flexibility, dynamic computation graph, and Pythonic interface.

Key Features:

  • Tensor Computation: Similar to NumPy, but with strong GPU acceleration.
  • Automatic Differentiation (Autograd): Automatically computes gradients required for backpropagation.
  • Neural Networks: A rich library of modules and loss functions for building and training deep neural networks.
  • Dynamic Computation Graph: Allows for more flexible network architectures than static graphs.

Getting Started: Installation

You can install PyTorch using pip or conda. It's highly recommended to install it with GPU support if you have a compatible NVIDIA GPU.

Using pip:

pip install torch torchvision torchaudio

Using conda (with CUDA 11.8 support as an example):

conda install pytorch torchvision torchaudio pytorch-cuda=11.8 -c pytorch -c nvidia

(Adjust CUDA version as per your system)

Basic Concepts: Tensors

Tensors are the fundamental data structure in PyTorch. They are similar to NumPy arrays but can run on GPUs.

Creating Tensors

import torch

# Uninitialized tensor
x = torch.empty(5, 3)
print(x)

# Randomly initialized tensor
x = torch.rand(5, 3)
print(x)

# Tensor of zeros with dtype long
x = torch.zeros(5, 3, dtype=torch.long)
print(x)

# Tensor directly from data
x = torch.tensor([[1, 2], [3, 4]])
print(x)

# From a NumPy array
import numpy as np
a = np.array([1, 2, 3])
y = torch.from_numpy(a)
print(y)

Tensor Operations

Tensors support various operations like addition, subtraction, multiplication, etc.

import torch

x = torch.tensor([1, 2, 3])
y = torch.tensor([4, 5, 6])

# Addition
print(x + y)
print(torch.add(x, y))

# In-place addition
y.add_(x)
print(y)

# Slicing
x = torch.rand(4, 4)
print(x[:, 1]) # all rows, second column

# Reshaping
x = torch.randn(4, 4)
y = x.view(16)
z = x.view(-1, 8)  # -1 infers the other dimension
print(x.size(), y.size(), z.size())

Further Topics:

  • Automatic Differentiation (autograd)
  • Building Neural Networks (nn.Module)
  • Optimizers and Loss Functions
  • Training Loops
  • Loading Data (torch.utils.data)
  • GPU Acceleration

This document serves as an introductory guide. Dive into subsequent files for detailed explanations, advanced topics, and practical examples.