In-context learning

Long contexts

The LLMs‘s length of context determine how much input you can pack. So, there has been efforts to increase the context length, while retaining the ability to use all the information within the longer context.

The “needle in the haystack” benchmark is used to see whether an LLM can answer the question properly by using some random information injected in the context somewhere. It has been reported that Claude 3 suspected it was being tested.

Liu2023lost investigates how LLMs use long contexts.