LLMs' capacities
What are the demonstrated capacities of LLMs?
Apparently, you can keep scaling up the model and dataset size and keep improving the model as far as we know: Kaplan2020scaling. Pre-trained language models are currently dominating the field because fine-tuning approach on a small dataset can benefit from a better, bigger pre-trained model. In other words, even with a limited dataset, by using a better pre-trained model, we can improve performance on the limited dataset. Here is a review of such pre-trained language models: https://github.com/thunlp/PLMpapers
The power of the large language models is not limited to common NLP tasks. They have been exhibiting impressive performance on various tasks. For instance, zero-shot model achieved SoTA on the Tuebingen causal discovery benchmark (cause-effect pairs)[^1], showing that they have similar causal “intuition” about the world.
137 emergent abilities of large language models identify abilities that are only present in large models. See also Webb et al., Emergent Analogical Reasoning in Large Language Models.
It can be used for “non-language data” as well. For instance, we can think of Language models for health records or Language models for genetics.
Science in general
- Liang2023can: can LLMs give useful feedbacks?
- Huang2023benchmarking: can LLMs do research?
Math & CS
- It can do some Recursion: Recursive LLM prompts
Political science
- Wu2023large shows that LLMs can score ideologies of politicians