Can Long-Context Language Models Subsume Retrieval, RAG, SQL, and More?

RAG, In context learning,

we introduce LOFT, a benchmark of real-world tasks requiring context up to millions of tokens designed to evaluate LCLMs’ performance on in-context retrieval and reasoning. … our findings reveal LCLMs’ surprising ability to rival state-of-the-art retrieval and RAG systems.