AI INFRASTRUCTURE

PyTorch Internals

Unraveling the C++ backend, JIT compiler, and CUDA kernels.

3
MODULES
92%
GPU UTIL
HARD

Autograd Internals

How automatic differentiation works under the hood.

Start Learning
EXPERT

Dispatcher & Operators

Kernel dispatch mechanism and ATen library.

Start Learning
HARD

Distributed Data Parallel

Multi-GPU training architecture and communication.

Start Learning
+
Add New Note