Generative Agent Simulations of 1,000 People

• https://arxiv.org/abs/2411.10109
• Joon Sung Park, Carolyn Q. Zou, Aaron Shaw, Benjamin Mako Hill, Carrie Cai, Meredith Ringel Morris, Robb Willer, Percy Liang, Michael S. Bernstein

AI Agent, Personas

The generative agents replicate participants’ responses on the General Social Survey 85% as accurately as participants replicate their own answers two weeks later, and perform comparably in predicting personality traits and outcomes in experimental replications.

The study starts with in-depth interviews (2 hr audio) and the interview script is used as agent memory for LLM agents. Then the actual participant responses on General Social Survey, Big Five Personality Inventory, economic games, and behavioral experiments are compared withg simulated participant responses with agents. When the agents were queried, the entire interview script was fed into the prompt.

The sample of 1,000 participants were drawn using stratified sampling to create a representative sample.

The baselines are: demographic and persona-based agents. The demographic baseline used age, gender, race, and political ideology. The persona baseline used a brief paragraph written by each participant explaining their personal background, personality, and demographic details (see Park2022social).

These findings suggest that, when informing language models about human behavior, interviews are more effective and efficient than survey-based methods.

Access to an agent bank can help lay the foundations for replicable science using AI-based tools. Our agent bank of 1,000 generative agents offers a resource toward these goals. To balance scientific potential with privacy concerns, the authors at Stanford University provide a two-pronged access system for research: open access to aggregated responses on fixed tasks (e.g., GSS) and restricted access to individualized responses on open tasks.