Transformer model
- http://jalammar.github.io/illustrated-transformer/
- https://www.youtube.com/watch?v=-QH8fRhqFHM : GPT is more about decoders (generation), BERT is more about encoders (translation and representation).
Google’s T5 paper provides a unified framework to understand and train transformer models.
https://huggingface.co/blog/how-to-train shows how to train a transformer model from scratch.