GPU

Prominent GPUs

• NVIDIA A100
• NVIDIA H100

Cloud computing options

• recommended from fast.ai course
  • https://www.crestle.com/ - launch Jupyter notebook with GPU instantly. pay hourly.
  • https://www.paperspace.com/ - slightly more hassle than crestle, but much cheaper.

Parallelization

• How to train a model on 10k H100 GPUs?