Whose Opinions Do Language Models Reflect?

LLM Personas

If we ask general survey questions to LLMs and then measure the probability for each answer, we can measure its alignment with the opinions of existing subpopulations: where and are opinion distributions, is a set of questions.

A recent paper argued that this may be due to simple bias towards the first answer in the survey. see Dominguez-Olmedo2023questioning