Skip to content

To-do 24

What should you prepare before the session?

Handling data and opimizers in PyTorch

Use the wiki → PyTorch. The list below refers to this wiki.

In the wiki, in section "2. Distributed PyTorch", go over the tutorial "Distributed Data Parallel in PyTorch - Video Tutorials". Complete the following parts of the tutorial:

  1. Introduction
  2. What is DDP
  3. Single-Node Multi-GPU Training
  4. Multi-Node Training