PyTorch

For multiple GPUs, use distributed data parallel

Related libraries

• skorch
• PyTorch BigGraph

Tutorials

• https://pytorch.org/tutorials/index.html
• Deep Learning With PyTorch - Full Course

Tips

• The shortest guide for pytorch training on GPUs
• How 🤗 Accelerate runs very large models thanks to PyTorch - How Hugging Face run very large models.