Generative Agents: Interactive Simulacra of Human Behavior — Joon Sung Park

Joon Sung Park · 2023 · 45:15 · ▶ Watch

Summary

Central question. Can a computer simulate believable human behavior, convincingly enough to give “an illusion of life”? Researchers in human-computer interaction (HCI) and AI have wanted this for four decades, for usability testing, game characters, social robots and social-science experiments. Park argues that large language models (LLMs) are the missing ingredient, because they are trained on broad data that “encode a tremendous amount about us how we live talk and behave”.

Main thesis. An LLM alone gives plausible behavior in the moment. It does not give a coherent person over days. To get that, wrap the model in an agent architecture with three parts:

1. A memory stream, a log of everything the agent experiences, in plain language.
2. Reflection, which periodically turns raw observations into higher-level thoughts that are stored back in the same memory.
3. Planning, which fixes a day’s intentions top-down so behavior stays consistent over time.

Twenty-five such agents, each started from one paragraph of description, live in “Smallville”, a small sandbox town. From that seed alone they wake, work, gossip about a mayoral election, form new acquaintances, and (after one agent decides to hold a party) organise a Valentine’s Day party that five agents attend.

Evidence offered. Interviews with the agents, ranked by 100 human raters (TrueSkill, a generalisation of Elo). The full architecture beats every ablation, and beats answers written by human authors. He reports a large effect against the “prompt an LLM” baseline (Cohen’s d of 8.16). He also lists failures: missed memories, embellished memories, and agents that are too polite because of instruction tuning.

The second half: an application. He shows an earlier, simpler system, Social Simulacra (2022), that uses the same idea to prototype online communities. A designer writes a community’s goals, rules and moderation plan, and the tool populates it with synthetic members, posts and trolls. It has three features: generate, what if, and multiverse (many possible outcomes, not one prediction).

Stance. An engineer-researcher presenting a method, not a philosopher. He is candid about errors and non-determinism, and he treats the simulation as a way to explore possibilities, not to forecast one outcome. He does not make claims about replacing human participants in this talk. That claim arrived later (see Part 1).

Map — talking points

#	Point
1	The four-decade dream of simulating believable behavior, and why it stalled
2	The opportunity: LLMs encode how we live, talk and behave
3	Smallville: 25 agents, each seeded with one paragraph
4	Agents act, talk and affect the world; users can chat, intervene or embody an agent
5	A day in the life of the Lin family
6	Emergent group behavior: information spreads, relationships form, a party is coordinated
7	The architecture: memory stream, retrieval (recency, importance, relevance)
8	Reflection: trees of thoughts built from observations
9	Planning: top-down, recursive, re-planning on events
10	Evaluation by interviewing the agents; TrueSkill ranking by 100 raters
11	Errors: forgotten and embellished memories
12	End-to-end results: how far the news and the party invitation spread
13	Boundaries: instruction tuning makes agents overly polite and agreeable
14	Q&A: can you rewind and rerun? Non-determinism and butterfly effects
15	Application: Social Simulacra, prototyping online communities
16	Three features: generate, what if, multiverse
17	Designer study with 16 participants

Putting it together

The mental model in one paragraph. An LLM is a compressed record of how people write and talk, so it can produce the next plausible line for any character. What it cannot do is be a character over time, because it has no persistent self, and its context window is too small to hold a life. Park’s architecture supplies the missing parts, borrowed in spirit from cognitive science: a long-term store (memory stream), a way to find what matters now (retrieval by recency, importance and relevance), a way to turn experience into self-understanding (reflection), and a way to commit to intentions (plans). Put many such agents in a shared world and social phenomena appear without being scripted. The same recipe, with simpler agents, lets a designer see a community’s possible futures before launch. The talk’s honest limit: this shows the agents are believable to observers. It does not show they are accurate models of real people. Everything that followed in this line of work is an attempt to close that gap.

Three tensions to keep open

1. Believable versus accurate. A character a rater finds convincing is not a measurement of what a real person would do (idea 5).
2. Emergence versus the model’s defaults. The group dynamics are real outputs of the system, but they are also shaped by a tuned model that is polite, agreeable and homogeneous (idea 6).
3. Prototype versus prediction. Social Simulacra shows many possible worlds so the designer is not misled by one. Selling a single “answer” (a survey result, a forecast) is the opposite stance (idea 7).

Reading path

• Start: the paper, Park et al., Generative Agents: Interactive Simulacra of Human Behavior (UIST 2023), arXiv:2304.03442. It is readable and the architecture figure is the whole talk.
• Then: Social Simulacra (UIST 2022), arXiv:2208.04024. Then Argyle et al., “Out of One, Many” (2023), arXiv:2209.06899, the “silicon samples” paper the talk’s evaluation is positioned against.
• Deeper: Park et al., Generative Agent Simulations of 1,000 People (2024), arXiv:2411.10109, then the 2025 critiques listed in Part 1.

Part 1 — Before you read

Key terms

Term	Meaning here	Watch out
Generative agent	An LLM wrapped with memory, reflection and planning so that it acts as a persistent character	”Agent” in AI also means any tool-using program. Here it means a simulated person
Believability	Whether an observer finds the behavior consistent with a plausible person	Not the same as accuracy against a real person. This is the central measurement gap
Simulacrum / simulacra	A copy or likeness. In the title it is plain English: an imitation of behavior	In Baudrillard (1981), a copy with no original. Park does not use that sense; the philosophical echo is real, though (idea 5)
Memory stream	The agent’s log of experience in natural language, with timestamps	Not a neural memory. It is text in a database
Retrieval	Selecting which few memories to put in the prompt	Needed because the context window is too small to hold a life
Reflection	A higher-level summary of recent memories, stored back into memory	A kind of memory, not a separate module of thought
Ablation	Removing one component to see what breaks	The talk compares versions without observation, planning or reflection
TrueSkill / Elo	Ranking systems that turn many head-to-head comparisons into a score	Used here to rank conditions by human preference
Instruction tuning	Training that makes a model follow instructions and be helpful	Cause of the “overly polite” agents
Synthetic respondent / silicon sample / digital twin	An LLM-generated stand-in for a survey respondent	The later use of this line of work, which the 2023 talk only anticipates

People named (a reading list)

Person	Why they come up	Start with
Michael Bernstein, Percy Liang	Park’s advisors at Stanford	Bernstein’s HCI work on social computing; Liang’s work on foundation models
Meredith Ringel Morris, Carrie Cai	Collaborators and mentors on the project (captions give “Mary Morris”, “Kerry Kai”)	Their work on human-AI interaction
Joseph O’Brien	Student collaborator (captions: “Joey O’Brien”)	The two papers
Agents: Isabella Rodriguez, Klaus Mueller, John, Mei and Eddy Lin, Sam Moore, Tom, Maria, Latoya	Characters in the demonstration	Appendix of the paper

The speaker’s own frame

Park was a PhD student at Stanford, in computer science, when he gave this talk (he says he is a “rising fourth year”). He was a co-author of the two papers he presents. The framing throughout is HCI: build a system, evaluate it with people, and describe both strengths and failures. In 2024 he and colleagues published the 1,000-person study, and by 2026 he had co-founded a company (Simile) that sells simulated consumers. A later talk from him on that chapter, “Simulating Humanity: from Generative Agents to 8 Billion Digital Twins”, takes the idea from sandbox towns to commercial prediction. This page covers the first chapter.

The current debate (2023–2026)

• From characters to individuals. In November 2024 Park and colleagues built agents of 1,052 real people from two-hour interviews, and compared them with the General Social Survey. The agents reproduced participants’ answers at about 85% of the rate at which participants reproduced their own answers two weeks later, and showed smaller accuracy gaps across racial and ideological groups than agents given only demographics (arXiv:2411.10109; Stanford HAI summary). A revised version reports 83% to 86% depending on the inputs, against 74% for demographic-only agents. Note the benchmark: it is test-retest consistency, not truth.
• Commercialisation. Park’s company Simile reportedly raised a $100 million Series A in February 2026 and a $200 million Series B at a $2 billion valuation in July 2026, to build a “confidence model” that predicts the accuracy of each simulation (Dealroom; Let’s Data Science). The accuracy claims there are the company’s own.
• Critiques of synthetic samples. A 2025 study with GPT-4 and Llama 3.1 on US election survey data found that accuracy fails to hold across demographic groupings and that answers are homogenised, under-representing minority opinions (Li et al., arXiv:2507.02919). Another found the many design choices in building a synthetic sample shift accuracy so much that no configuration is reliably best (Cummins, arXiv:2509.13397). A review of consumer-research use found synthetic respondents beat chance on individual answers but overstated positive attitudes toward known brands and varied far less than real people, and suggests early-stage concept testing rather than high-stakes decisions (NIM). Conference reports of independent tests are more negative still but are secondhand.
• How it connects to this talk. The agents’ politeness and the reruns that drift are the 2023 seeds of two 2025 critiques: homogenisation and unreliable single runs.

The ideas

Park spoke for 45 minutes to a language-and-speech audience, so the talk is compressed and light on theory. Several older traditions sit behind it, including some he does not name. They meet roughly like this:

 COGNITIVE ARCHITECTURES ─────┐                 ┌──── BELIEVABLE CHARACTERS
 Newell · Soar · ACT-R        │                 │     Bates · Disney animation · The Sims
 memory science (Tulving,     ▼                 ▼
 Ebbinghaus)           ┌───────────────────────────────┐
                       │  PARK: LLM + memory +         │ ◄──── AGENT-BASED MODELLING
 LLMs AS COMPRESSED ──►│  reflection + plans           │       Schelling · Epstein & Axtell
 CULTURE               │  = believable agents          │       emergence from simple rules
 Argyle · Aher         └───────────────────────────────┘
                          ▲            ▲            ▲
        EVALUATION OF     │   ALIGNMENT│            │ HCI PROTOTYPING
        "HUMAN-LIKE"      │   Instruct-│GPT, RLHF   │ personas · Wizard of Oz · pre-mortems
        Turing · Elo      │   sycophancy              Nielsen · Cooper

Which idea covers which tradition:

• The case for LLMs as a social substrate and the old believable-character problem: idea 1.
• Cognitive architecture and memory science: idea 2 (and reflection, idea 3).
• Planning and cognitive science of goals: idea 4.
• Evaluation and the believability-versus-accuracy gap: idea 5.
• Agent-based modelling, emergence and alignment effects: idea 6.
• HCI prototyping, personas and pre-mortems: idea 7.

1. The opportunity: LLMs as a record of how we live

In short. For decades, simulating a person was blocked by the sheer size of the space of human behavior. Park’s claim is that LLMs, trained on traces of our lives, now hold enough of that space to be the “core ingredient” that had been missing. The rest of the talk is about turning that raw ingredient into something that holds together over time.

🎙 In the talk. He describes the dream: “simulate believable human behavior behavior that is so compelling and so human-like that they provide an illusion of life”. Uses range from “model human processors for usability testing to social robots NPCs” to “social simulations that would test social science and economics theories”. The blocker was that “the space of possibility the way we behave and communicate we found was much too advanced and too complex to navigate with existing methods”. The opportunity: models trained on “Broad data that reflect our lives like the traces on our social web Wikipedia and more”, so “these models encode a tremendous amount about us how we live talk and behave”. The right method can turn them into “the core ingredient that had been missing the past decades”.

📚 Deeper theory

• Why the old methods hit a wall. Classic simulations needed someone to write the rules of behavior by hand. People’s behavior depends on context, history, relationships and language, so hand-written rules were either shallow or enormous. An LLM trades rules for statistics: it has seen so many examples that it can continue almost any situation plausibly. That is also its limit. It continues text, not necessarily behavior (idea 5).
• A model of the writer, not the person. What the model compresses is how people write about themselves and others online. That is a skewed sample of humanity: more English, more educated, more online, and with the voice of public text. The “believable” simulated person inherits this skew. The later homogenisation critique (Part 1) is the same point at scale.
• Roles, not individuals. Early evidence for the idea came from showing that an LLM can play a type of person when prompted. Whether it can play a specific person is a harder claim, and it is the one the 2024 study tested.

📚 Roots: Believable characters — from animation to NPCs

The problem. Long before LLMs, game and animation makers needed characters that seem alive without being alive. The question was never “is it intelligent?” but “does the audience accept it?”

The point. Park adopts this exact bar. His measure is believability, which he says “has long been a central design and engineering goal in a line of prior literature”. That choice defines what the whole paper can and cannot claim: observers’ acceptance, not fidelity to a real human.

The chain of thinkers

1. Disney animators and “the illusion of life” (1930s–1981). Frank Thomas and Ollie Johnston’s The Illusion of Life (1981) codified principles (squash and stretch, anticipation, and so on) that make drawn characters feel alive. Park’s phrase “illusion of life” is the same words.
2. Joseph Bates, “The Role of Emotion in Believable Agents” (1994). Carnegie Mellon computer scientist who brought the animators’ term “believable” into AI. His point: a believable agent need not be realistic or smart; it must not break the audience’s belief in the character. His work was part of CMU’s Oz project.
3. Game AI and the NPC. Non-player characters followed scripts and state machines for decades. The Sims (2000) showed that simple needs-driven agents could produce lively household stories, but with hand-built behavior.†
4. Cognitive architectures as a route to human-like agents. Soar and ACT-R aimed to be general models of cognition. The “model human processor” Park mentions is Card, Moran and Newell’s 1983 HCI model. These were accurate in the lab and brittle in open worlds.
5. LLM-driven characters (2022–). Once a model could speak any role, the bottleneck moved from writing behaviors to keeping a character consistent. That is where Park starts.

Where others push back. Believability is audience-dependent and cheap to satisfy for short episodes. Researchers in cognitive science argue that a system can pass observers while having no model of the person behind the behavior. Critics of the “illusion of life” frame also worry about people anthropomorphising and over-trusting such agents.

What they leave out. Who the observers are. Raters in the study are crowd-workers judging text answers; players, patients and survey clients might apply different standards.

Check yourself

• What is the difference between believability and accuracy?
• Why was the “space of possibility” too large for older methods?
• What skew might an LLM bring from its training data?

---

2. The architecture: a memory stream and what to retrieve

In short. Each agent has one big log of everything it has experienced, in plain English. Because the log quickly outgrows the model’s context window, the system retrieves only the few records that matter for the situation now. Retrieval scores memories by how recent, important and relevant they are.

🎙 In the talk. The setting: Smallville has houses, a cafe, a bar, a school and shops, with “25 generated agents”, each initialised with “one paragraph of natural language description”. That paragraph is sealed into the agent’s memory at the start, “and that is it that is all the input that we ever give to these agents”. Actions are generated as sentences (“Isabella Rodriguez is drinking coffee”) and then translated into movements and emoji. The memory stream is “the primary database that maintains a comprehensive record of an agent’s experience in natural language”. The problem: “fitting the entire memory stream to a language model can distract the model”, and “not even a few hours worth of memory in Smallville can even fit into the limited context window”. So retrieval “takes the agent’s current situation as input and returns a subset of the memory stream”. It combines three scores: “we biased towards retrieving memories that are more recent more important and more relevant”. Recency is “an exponential decay function”. Importance asks the model to rate each event, “almost like just asking the agent this happened to you is this important for you”, so breakfast is low and a graduation or breakup is high. Relevance uses “cosine similarity” between embeddings of the query and the memory.

📚 Deeper theory

• Retrieval is the design. The agent’s “self” is whatever retrieval puts in the prompt. Change the weights and you change the person. This is the real tunable of the system and the least visible one.
• Natural language as the memory format makes the system inspectable (you can read what an agent “thinks”) but lossy and expensive. Later systems add structured and vector stores.
• The importance score is the model’s guess about human salience, which inherits the model’s biases about what matters.
• Why decay? Human memory forgets, and a prompt window must too. Recency-weighting solves an engineering problem and happens to match a psychological finding.
• Retrieval-augmented generation (RAG). This is a RAG pattern whose query is “the agent’s situation” and whose corpus is “the agent’s past”. The same pattern now powers most enterprise assistants.

📚 Roots: Memory — from forgetting curves to cognitive architectures

The problem. A system that acts over time needs a store of the past and a way to decide what to bring to mind. Psychology and cognitive science studied both long before LLMs.

The point. Park’s three retrieval factors line up with long-standing findings: recent things are easier to recall, important events are remembered, and cues that match the memory pull it up. The architecture borrows the shape, not the mechanisms.

The chain of thinkers

1. Hermann Ebbinghaus (1885). The first systematic study of memory; the forgetting curve, with retention falling steeply at first and then levelling off. (The exact shape of the curve is still debated.) Park’s recency decay is the engineering cousin.
2. Endel Tulving (1972). In the chapter “Episodic and Semantic Memory”, distinguished episodic memory (events) from semantic memory (facts). The memory stream is episodic in form; reflections (idea 3) play a semantic role.
3. Allen Newell and Herbert Simon; Soar (1980s). General problem solving with working memory and long-term memory. The limited context window plays the role of working memory.
4. John Anderson, ACT-R. Its “base-level activation” makes the chance of retrieving a memory depend on how recently and often it was used, and on cues in the current context. This is the closest ancestor of the recency-plus-relevance score.†
5. Roger Schank and Robert Abelson, scripts (1977). Everyday knowledge as stored routines (the restaurant script). A day plan (idea 4) is a script written on the fly.
6. Retrieval-augmented language models (2020–). Embedding search over a text store becomes standard; Park adds recency and importance to cosine relevance.

Where others push back. Human memory is reconstructive: we rebuild the past each time we recall. A log of verbatim records is not how people remember, and it makes the agent’s recall too perfect in some ways and too brittle in others (idea 6 shows the errors).

What they leave out. Emotion and forgetting-by-design. The system never represses or distorts a memory the way people do, except by accident.

Check yourself

• Why can’t the whole memory stream go into the prompt?
• Name the three retrieval factors and how each is computed.
• If you changed the importance prompt, what would change about the agent’s personality?

---

3. Reflection: turning observations into a self

In short. With observations alone the agents cannot generalise. So the system periodically asks itself questions about the recent past and writes the answers back as new memories. These stack into trees: facts at the bottom, character traits at the top.

🎙 In the talk. “With just the [raw] observational memory our agents struggle to generalize to make inferences” (the caption reads “wrong”; the context shows he said raw). The system “periodically” synthesizes clusters of records into higher-level thoughts called reflections. “Once they are synthesized these Reflections are just a type of memory so they are stored in the memory stream just like the observational memory”. The method: first “generate questions on what to reflect them by looking at the 100 most recent records”, then retrieve what is relevant to those questions and synthesize an answer. The result is a tree where “the leaf nodes are observations and the non-leaf nodes are thoughts that become higher level”. For Klaus, a student researcher, observations like reading about gentrification become “Klaus spends many hours reading”, then research activity, then dedication to research. The step from facts to summary matters: “that’s no longer a fact it’s a summary of the known facts”. Higher up the tree “you’re really starting to get into the territory of who Klaus is”. Similar trees can be formed about colleagues and places.

📚 Deeper theory

• Reflection is how the system gets a “personality”. The top of the tree is a character sheet that the agent wrote itself, and later behavior is conditioned on it. This is a feedback loop: act, summarise, act in line with the summary.
• It also fixes drift. Without reflection, behavior is a function of recent text. With reflection, durable traits persist beyond the retrieval window.
• Reflections can be wrong, and they compound. A mistaken generalisation is stored as a memory and then retrieved as if it were fact (idea 6, embellishment).
• Questions first, answers second. Choosing what to reflect on is itself generated by the model, which makes it another place where model priors steer the “person”.
• Related to later agent-memory designs. Summaries of summaries, “memory consolidation” and self-written notes are now standard in agent frameworks.†

📚 Roots: Self-narrative — how a person becomes a person to themselves

The problem. A person is not a pile of events. Events become a self when they are interpreted, summarised and linked to goals and traits.

The point. Reflection is a mechanical version of this move: from “he read three articles” to “he is dedicated to research”. The talk does not claim the agent has a self, but the tree has the shape that philosophers of identity describe.

The chain of thinkers

1. William James, Principles of Psychology (1890). The “stream of thought” and the idea of the self as something built from continuous experience. Park’s “memory stream” echoes the phrase.
2. Frederic Bartlett, Remembering (1932). Memory is reconstruction guided by schemas, not playback. Reflection plays the role of a schema.
3. Jerome Bruner (1990s), narrative identity. People make sense of themselves by telling a story; Dan McAdams develops this as the “narrative identity” of a person.
4. Daniel Dennett, “The Self as a Center of Narrative Gravity” (1992). The self is a useful fiction constructed by narration. Reflection makes the narrative explicit, in a way the model then follows.
5. Daniel Kahneman’s experiencing vs remembering self (2000s). What we remember of an episode (peak, end) differs from the experience. The importance score is a crude peak detector.†

Where others push back. A summary written by a language model is not understanding. Critics of LLM “theory of mind” claims argue that these tree nodes are plausible text, not beliefs held by an agent.

What they leave out. Contradiction. People hold inconsistent traits and revise them under pressure; a tree with tidy top-level nodes may be less human than the raw records.

Check yourself

• Where are reflections stored, and why does that matter?
• How does the system decide what to reflect on?
• Give a case where a reflection could make the agent worse.

---

4. Planning: consistency over a day

In short. The model optimises for what is believable right now. That can make behavior incoherent over hours. Plans fix intentions in advance, in coarse strokes first and then in detail, and the agent can revise them when something happens.

🎙 In the talk. “Optimizing for believability in the moment can this can sacrifice the believability over time”, so “what we needed was for the agents to plan over a longer time Horizon”. A plan is “a future sequence of actions”, kept in the same memory stream, and generated by prompting the model with a summary of the agent and its current status. To control granularity: “a top-down approach where we recursively generate more plants in detail” (the caption’s “plants” means plans). The day is split into roughly seven chunks and then decomposed “into 5 to 15 minute actions”. This can go the other way: plans for a month or a year, or minute by minute; five to fifteen minutes “is actually a reasonable interval” for the game. Plans are revised: if John sees his son taking a walk, “he might decide to start an impromptu conversation”, and the model decides whether to edit the plan.

📚 Deeper theory

• Why a plan is a kind of memory. Because plans sit in the memory stream, other processes (retrieval, reflection, conversation) can see them. That is how Isabella’s intention to hold a party becomes a fact that other agents learn about.
• Hierarchical decomposition is the oldest idea in planning. It trades flexibility for coherence, and the paper’s “5 to 15 minute” granularity is a tuning choice for a game, not a law of human behavior.
• Re-planning on interruption (a reaction check at every step) is where the simulation becomes costly: each tick may call the model several times. The cost explains why only 25 agents ran and why reruns were rare (idea 6).
• Human days are not so planned. Much real behavior is habit and improvisation. A model-written plan is a tidy account of the day, closer to how people describe their plans than how they live them.

📚 Roots: Planning — from means-ends analysis to hierarchical tasks

The problem. To act coherently over time, an agent has to choose actions that serve goals that are not yet achieved.

The point. Park’s top-down, recursive plan is classical hierarchical planning with a language model standing in for hand-written operators.

The chain of thinkers

1. Newell and Simon, General Problem Solver (1957–59). Means-ends analysis: reduce the difference between the current state and the goal.
2. STRIPS (Fikes and Nilsson, 1971). Planning as search over states with operators.
3. Hierarchical Task Networks (1970s–90s). Break tasks into sub-tasks recursively, as Park’s day-to-minutes decomposition does.†
4. Michael Bratman, Intention, Plans, and Practical Reason (1987). Philosophy of action. Plans are commitments that stabilise behavior and filter what we consider. The BDI (belief-desire-intention) agent architectures of the 1990s build on this.
5. Lucy Suchman, Plans and Situated Actions (1987). An HCI classic arguing that plans are resources for action, not blueprints that action follows. Real action is improvised in context. Park’s re-planning is a nod to this.
6. LLM planners (2022–). Chain-of-thought, ReAct and related methods use the model to propose steps in language.

Where others push back. Suchman’s critique still applies: simulated humans who follow a plan are following a story about human action. And the model writes plans that read as typical days, so diversity of lifestyles is likely narrower than reality.

What they leave out. Constraints that real people face: money, health, obligations, other people’s refusal.

Check yourself

• Why does optimising for the next moment hurt over the day?
• What does “top-down recursive” mean for plan granularity?
• How does Suchman’s view change how you read the simulation?

---

5. Evaluation: believability, interviews and the human-author result

In short. The team tested the agents by interviewing them in natural language and having 100 people rank the answers across versions of the system. Every component mattered, and the full system beat answers written by human authors. Park is careful to note failures. The deeper issue, which the talk only brushes, is that believable is not accurate.

🎙 In the talk. “The main dependent variable that we use is believability”. The method: “we’re leveraging methodological Opportunity by interviewing it in natural language”, with “five categories of questions five questions each” so that agents must “retrieve and synthesize information to stay in character”. Example: Klaus introduces himself correctly, and says he would “quickly turn off the stove” if breakfast were burning. Procedure: test the full architecture, ablated versions and human authors, then ask “100 human evaluators to rank the answers”, and compute TrueSkill ratings, “a generalization of the ELO rating system”. Result: “the components of our agent protection observation plan and reflection each contribute critically to the believability of the agent architecture”, and the full system “significantly outperforms every other conditions including the human author condition”. Against the baseline that represents prior work, prompting an LLM with no memory, plan or reflection, the full architecture has a standardised effect size of 8.16, which he describes as “a standard deviation difference”. But “this did not mean that our agents work without flaws”: Rajiv says he has been “following the election too closely” though he never learned of the candidacy, and Isabella confirmed Sam’s candidacy and then embellished it with a non-existent announcement.

📚 Deeper theory

• What the test does and doesn’t show. It shows that, in text answers about their own made-up lives, agents with memory, reflection and planning read as more consistent than agents without. It does not test whether the agents match real people’s behavior, because the characters are invented and there is no ground truth.
• Beating the human authors. The humans wrote answers as if they were the character, without access to the agent’s memory log. So the comparison measures consistency with the logged history, an advantage for the system by design. This is a fair result about memory, and an unfair headline about humanness.
• Effect size. A Cohen’s d of 8.16 is enormous because the baseline condition has almost no information about the character. It tells you the components matter, not that the system is eight standard deviations from reality.
• Interviewing as a probe. Asking the agent questions tests recall and character consistency, a narrow slice. It is not a test of group behavior, which the paper checks separately (idea 6).
• The step to validity. The 2024 study’s design changed the question: compare each agent with the real person it was built from, normalised by that person’s own consistency over two weeks (Part 1). That is the move from believability to something like accuracy.

📚 Roots: Evaluating “human-like” — Turing, Elo and validity

The problem. How do you measure whether a simulated person is good? Imitation, accuracy, usefulness and safety are all different answers.

The point. Park uses an imitation-style test with human raters, which suits characters in a game but leaves open validity for science and markets.

The chain of thinkers

1. Alan Turing, “Computing Machinery and Intelligence” (1950). The imitation game: judge by indistinguishability in conversation. Park describes his own test as “a steering test” in the transcript, which is the caption’s garble of Turing test (see Source notes).
2. Arpad Elo (1960s) and Microsoft’s TrueSkill (2006). Ranking from pairwise outcomes. TrueSkill extends Elo to multi-way comparisons and uncertainty.†
3. Construct validity (Cronbach and Meehl, 1955). A measure is valid if it captures the thing you claim. Believability raters measure a perception, not a population.†
4. Silicon sampling (Argyle et al., 2023). The claim that LLMs conditioned on demographic backstories reproduce group-level survey patterns (“algorithmic fidelity”). A different validity standard: match to real survey distributions.
5. Turing Experiments (Aher, Arriaga and Kalai, 2022). Replicate classic human-subject studies with LLMs.†
6. Individual-level checks (Park et al., 2024). Compare each simulated person with the real person. See Part 1 for the numbers and the critiques.

Where others push back. Critics say imitation tests reward fluent text over truth, and that raters cannot detect homogenised or sycophantic output. Measurement researchers add that agreement with a single survey does not establish transfer to new questions.

What they leave out. Calibration: knowing when the simulation is wrong. Simile’s reported “confidence model” (Part 1) is an attempt at exactly this.

Check yourself

• What was compared with what in the interview test?
• Why is a Cohen’s d of 8.16 not the same as “close to real people”?
• What does the 2024 study do differently?

---

6. Emergence, errors and the shape of the model

In short. Without scripting, the agents spread news, formed relationships and organised a party. But they also forgot, embellished and were too nice, and the results changed from run to run. Both the emergent behavior and the flaws come from the model underneath.

🎙 In the talk. The group phenomena: “emergent social dynamics where new relationships form information diffuses and coordination arises”. Information: Sam’s mayoral run spread, and “seven agents heard about” it and 12 about the party. Relationships: Latoya and Sam, strangers at the start, meet in a park and later remember each other. Coordination: from Isabella’s one intention to hold a party, invitations spread and five agents attended; Rajiv, who did not come, “explained that he was too busy” and “still did not show up”, which Park calls “extremely realistic human behavior in my experience”. Limits: “the instruction tuning of the [agent’s] model seems to guide the agent to be overly polite”, so agents stay agreeable even with family, and “Isabella never really refuse ideas” even when they did not fit her identity.

In the Q&A someone asks whether the world can be rewound and rerun. Park says the full 25-agent run was “too slow and expensive for now”, but smaller three- or five-agent runs show that “they still all follow the general believable pattern”, although models “are not entirely deterministic”. Because the agents form a community, “even the small perturbance at the beginning of this simulation could actually change the outcome”. His example: whether Klaus asks Maria to the party or she asks him, or neither, “sometimes changes”. That one shift changes who meets whom.

📚 Deeper theory

• What “emergent” means here. No rule says “spread the news” or “throw a party”. Memory, retrieval and conversation make those outcomes likely. That is emergence in the agent-based modelling sense, but the “rules” are a language model’s priors, which nobody wrote or can fully read.
• Politeness as a systematic bias. Instruction tuning trained the model to be helpful and agreeable. A population of agents built on it will under-produce conflict, refusal and rudeness, which are normal in real communities. This is a bias in the dynamics, not only in the words.
• Homogeneity. Twenty-five characters from one model share a “voice”. The 2025 critiques (Part 1) measure the same thing in survey answers.
• Sensitivity and what to report. Park’s rerun point is that a single run is one draw from many possible worlds. Honest reporting needs many runs, which the cost prevented in 2023.
• Errors to a purpose. Forgotten and embellished memories look like human flaws, but they are not modelled on human memory; they are model failures that happen to resemble it. Treating them as realism is a risk.

📚 Roots: Agent-based modelling and emergence

The problem. How do large-scale patterns arise from many interacting individuals? Social scientists answered by simulating simple agents.

The point. Generative agents are agent-based models whose agents have language, memory and open-ended behavior. That raises both the richness and the opacity.

The chain of thinkers

1. Thomas Schelling, Micromotives and Macrobehavior (1978). Mild individual preferences about neighbours produce strong segregation. The classic proof that macro patterns need not be intended.
2. Robert Axelrod, The Evolution of Cooperation (1984). Simple strategies in a repeated game produce cooperation; tournaments of programs.
3. Joshua Epstein and Robert Axtell, Growing Artificial Societies (1996). The Sugarscape model; “generative social science”: explain a phenomenon by growing it from rules.†
4. Mark Granovetter, “The Strength of Weak Ties” (1973). Why information crosses communities through acquaintances. Relevant to how news and invitations spread.
5. Complexity and chaos (Lorenz, 1963). Sensitive dependence on initial conditions: the butterfly effect Park describes in reruns.†
6. LLM alignment and sycophancy (2022–). Research on RLHF-trained models finds a tendency to agree with the user and avoid conflict. This explains the polite agents.†

Where others push back. Agent-based modellers insist on transparent rules, so that a mechanism can be tested. A language model’s rules are not transparent, which weakens explanation even when prediction works. Others note that the model may have memorised stories about parties and elections, so “emergence” can be recall.

What they leave out. Calibration to real data at the group level, and the cost of failure in domains where polite averages mislead.

Check yourself

• What makes a behavior “emergent” in this system, and what is not emergent?
• Why does instruction tuning bias group dynamics?
• Why are single runs a weak basis for conclusions?

---

7. Social Simulacra: prototyping a community before it exists

In short. The second half of the talk applies the idea to designers. Online communities fail in ways their designers did not foresee. A tool that fills a mock community with synthetic members can show the good, the bad and the ugly before launch, as a set of possibilities rather than a forecast.

🎙 In the talk. The 2022 paper, he says, was “I think first paper in the field that sort of tried to use a large language model to simulate human behavior”. The problem: “even today as more and more people populate these systems we continue to be surprised by the things that happen in them like unexpected trolling”. A prototype of a social system must capture “those were social dynamics that might arise when the system reaches critical massive users”, and “where are we going to get thousands or even tens of thousands of diverse test users”. Social Simulacra is a way “to populate a social Computing system with generative agents and behaviors for the purpose of prototyping the system design”, built as a tool for subreddits. Three features:

1. Generate. From goals, rules and member descriptions it produces a populated community page. Examples: a new user in a Los Angeles community asking where to go, and a troll replying to a personal-finance post with a joke at the poster’s expense.
2. What if. It regenerates a conversation from the middle with a different actor, for example a troll or an advertiser instead of a helpful professor, and lets the designer test a moderator’s reply.
3. Multiverse. It shows many outcomes from one start, because “no matter how likely something is to happen there’s no guarantee that it will happen”. The aim is to “explore the design space with inductive insights rather than over relying on the one prediction made by the model”.

Study: 16 designers. Without such a tool “the design practices for social Computing systems are reactive”, and one participant said “all the rules are said in reaction to some dumpster fire”. With it they found both unexpected good behavior (impromptu friendships) and bad (trolls changing the tone).

📚 Deeper theory

• A different epistemic stance. Social Simulacra is a design aid: success is whether designers see things they would not have thought of. It does not need the simulation to be statistically accurate. This is the opposite of selling predictions, and it is the safer use.
• Multiverse as honesty. Showing several outcomes tells the user the output is a distribution. Dropping that feature is the commonest way simulation tools become misleading.
• Villains need the model to be willing. The troll example works because the tool can role-play bad actors. A model tuned for politeness is poor at this (idea 6), which is a limit on what the tool shows.
• Link to survey simulation. Replace “designer” with “researcher” and “community” with “population”, and the same machinery makes synthetic respondents. The design-aid stance transfers only if outputs are treated as hypotheses to check with real people.

📚 Roots: HCI prototyping — personas, pre-mortems and Wizard of Oz

The problem. Designers must learn how people will use a system before building it. Prototyping tools exist for screens and flows, but not for crowds.

The point. Social Simulacra adds a missing kind of prototype: one that simulates the population using the design.

The chain of thinkers

1. Alan Cooper, The Inmates Are Running the Asylum (1999). Popularised personas: fictional users that stand in for real ones during design. Generative agents are personas that act.
2. Wizard of Oz studies (John Kelley, 1984). A human secretly plays the machine, so a design can be tested before the technology exists. Social Simulacra reverses it: a machine plays the humans.
3. Jakob Nielsen’s usability heuristics and small-sample testing (1990s). Five users find most problems in a screen flow, but not crowd dynamics.
4. Gary Klein’s pre-mortem (2007). Imagine the project failed and ask why. “What if” and “multiverse” are a pre-mortem engine.†
5. Social computing research (CSCW tradition). Findings on moderation, norms and trolling (Kraut and Resnick, Building Successful Online Communities, 2011) provide the design rules the tool takes as input.
6. Policy and design sandboxes. The same idea applied to simulating a regulation or product before launch, now a commercial category.

Where others push back. A model that has read Reddit will reproduce Reddit’s typical trolls, not the new failure modes a novel design could create. Designers may also over-trust a vivid simulated thread.

What they leave out. Whether simulated findings changed any real design outcome. The study measures designers’ perceived insight, not community health later.

Check yourself

• Why is “multiverse” more important than “generate”?
• How does this use differ from predicting a survey result?
• What kinds of failure would the tool be unlikely to show?

Q&A

(Add questions and answers here only if worth keeping.)

---

About the source and how this page was made

• Speaker: Joon Sung Park
• Event: Seminar at the Center for Language and Speech Processing (CLSP), Johns Hopkins University
• Date: 2023-08-25
• Uploaded: 2023-08-25
• Length: 45:15
• Channel: Center for Language & Speech Processing (CLSP), JHU
• Video: Generative Agents: Interactive Simulacra of Human Behavior - Joon Sung Park (Stanford)

Source notes

Speakers

Label	Who	Role
Park	Joon Sung Park	Presenter, then a rising fourth-year PhD student in computer science at Stanford; names his advisors and collaborators in the opening
Audience member	Not named	Asks one question at about about rewinding and rerunning the simulation

Identification and confidence. High. The channel (Johns Hopkins CLSP), the video title and the talk’s own opening agree: first-person account of the paper “with my advisors Michael and Percy”, and a title matching the 2023 paper. The questioner’s identity is unknown.

Dating. The upload date is 2023-08-25, and Park says “now that it’s summer” and that the social-simulacra paper “was published 2022 so it was last year”. The talk is dated to the upload date. The paper was published in 2023 (UIST), consistent with this.

Caption corrections used on this page. The captions are auto-generated and garble many names and terms:

• “ChatGPT” is garbled into an unrelated word.
• “Brazilian” is Percy (Liang); “Mary Morris” is Meredith Ringel Morris; “Kerry Kai” is Carrie Cai; “Joey O’Brien” is Joseph O’Brien.
• “hops Cafe” is the Hobbs Cafe; “Seymour” and “Moore” are Sam Moore; “Latoya” is Latoya Williams; “may” is Mei Lin.
• “true skill” is TrueSkill; “yellow rating” is Elo rating; “coincidence d” is Cohen’s d.
• “plants” is plans; “wrong observational memory” is raw observational memory (inferred from context).
• “a steering test” is, from context, a Turing test.
• “Asian” for “agent” appears repeatedly; “patrons”/“similar crap” for “Simulacra”.
• “four decades” is garbled to “4 over four decades” at the start.

Slips and claims to check.

• At Park calls the 2022 Social Simulacra paper “the first paper in the field” to use an LLM to simulate human behavior. This is his own priority claim. Related work existed in 2022 (for example Aher et al.’s “Turing Experiments” and Argyle et al.’s silicon-sampling work, both of which appeared in 2022), so treat “first” as his claim, not an established fact.
• The talk’s effect size (d = 8.16) compares against a baseline with no memory, planning or reflection, so it should not be read as distance from real human behavior.
• The talk says the study asked “100 human evaluators”; the paper’s design is the source for the description of the conditions.

Not in this talk. The 1,000-person study, the commercial company and the accuracy critiques are later developments added from outside sources in Part 1. They are not Park’s claims in 2023.

---

🎙 marks what the speaker says. 📚 marks added background and the wider debate, not the speaker’s claim. † marks background from general knowledge that was not checked against a source.