Generalization v.s. Memorization: Tracing Language Models' Capabilities Back to Pretraining Data

• https://www.arxiv.org/abs/2407.14985
• Xinyi Wang, Antonis Antoniades, Yanai Elazar, Alfonso Amayuelas, Alon Albalak, Kexun Zhang, William Yang Wang

Training data leakage and memorization in language models