Language Models as Agent Models
When performing next word predic- tion given a textual context, an LM can infer and represent properties of an agent likely to have produced that context.
Claims:
In the course of performing next-word pre- diction in context, current LMs sometimes infer approximate, partial representations of the beliefs, desires and intentions possessed by the agent that produced the context, and other agents mentioned within it.
Once these representations are inferred, they are causally linked to LM prediction, and thus bear the same relation to generated text that an intentional agent’s state bears to its communciative actions.