Monotonic Representation of Numeric Properties in Language Models

• https://arxiv.org/abs/2403.10381
• Benjamin Heinzerling, Kentaro Inui

LLMs, Representation learning

https://www.threads.net/@yyahn/post/C-1I_UoNjXX?xmt=AQGz6GbYncnzE9ycGb6n7xnl1e98z65lNDYcFH5_FP_dAw

This paper reports that there is a low-dim subspaces that encode numeric properties monotonically, which can be used to “edit” the LLMs’ response by increasing or decreasing in the numeric dimension.