The Linear Representation Hypothesis and the Geometry of Large Language Models
- https://arxiv.org/abs/2311.03658
- Kiho Park, Yo Joong Choe, Victor Veitch
“Geometry of large language models” can be a nice title of a review paper.
the ‘linear representation hypothesis’ is the idea that high-level concepts are represented linearly as directions in some representation space.