Anamnesis — Survey Simulation With Backstories, Behind a GUI

Background

Simulating survey respondents with an LLM has settled into a standard recipe: hand the model a short demographic sketch — 35, Hispanic, woman, moderate — and ask how such a person would answer. The recipe is convenient and it has a documented failure mode. Cheng et al. (2023) showed these prompts drift toward caricature, and Wang et al. (2025) found they flatten the internal variation of the groups they claim to represent.

Anthology (Moon et al., 2024) replaced the sketch with a life story. A base language model, prompted with interview questions from the American Voices Project, writes a multi-paragraph narrative covering upbringing, work, family, and values. That narrative conditions the model at survey time. Alterity (Kang et al., 2025) pushed on the same idea from the direction of group perception, asking whether a conditioned model reproduces a group’s own reasoning or an outsider’s picture of it.

Both methods work, and both shipped as research code. The gap Anamnesis targets sits between a published method and a social scientist who wants to use it. Commercial services fill that gap today — Synthetic Users, Expected Parrot, Artificial Societies — and the paper’s objection is specific. Two of the three keep their conditioning methodology private, which forecloses reproduction; the open one reduces persona conditioning to the short-attribute prompting that Anthology was built to replace.

Core Idea

The workflow has four stages, and each one corresponds to a design decision worth naming.

A user builds a survey graphically, with multiple-choice, multi-select, open-ended, ranking, and image or audio questions. They then pick a target population and a sample size, drawn against a pool of roughly 35,000 pre-sampled backstories. A target can be as loose as “women aged 18–24” or as specific as “voters aged 25–44 with a college degree, evenly split between Democrat and Republican.” The run executes, each backstory answering the whole instrument in sequence. Results land in a dashboard that can be re-sliced by demographic attribute without re-running anything.

The interesting complication is in stage two. In the original Anthology experiments each backstory was paired with a real respondent from a completed survey, so its demographics were observed labels and balancing was deterministic: given quotas and known attributes, greedy selection or Hungarian matching picks the right respondents. Anamnesis has no such pairing. It stores, for each backstory b and dimension d, an inferred distribution $p_{b,d}(c)$ over categories. Every selection decision now happens under uncertainty.

Method

Two ways to sample a population

Anamnesis offers two algorithms, and the choice encodes what the researcher wants.

Top-K probability ranking scores each backstory by joint compatibility with the filter, multiplying probabilities across dimensions and summing over selected categories, then takes the top S. This treats the filtered set as one group and ignores internal composition. It suits studies that want the personas most likely to belong to the target population.

Balanced demographic matching enforces representation across cells. Selected categories expand into a cross-product of demographic cells G; the sample size divides into slots $K_g$ per cell, uniformly or by user weights; and selection becomes an assignment problem — fill every slot, use each backstory at most once, maximize total compatibility.

Hungarian matching over the full pool would be impractical for an interactive tool, so the candidate space is pruned first. The top M backstories per cell, 50 by default, form a shared candidate pool, and the assignment runs over that reduced matrix. The paper reports this as $O(S^3)$ with $S \le 50$, fast enough to run client-side while the backend stays free for worker throughput.

Execution under rate limits

A survey run costs $O(S \times Q)$ LLM calls for S personas and Q questions, and $O(S \times Q \times N)$ when demographic inference uses repeated sampling. Three constraints follow: per-persona state has to survive across questions, concurrency has to stay inside API and vLLM limits, and a run has to be reproducible after the fact.

The engine is a dispatcher–queue–worker arrangement. Each run is snapshotted at launch with its filters, answering algorithm, model configuration, and concurrency bounds. Tasks decompose into persona–question units and go onto a message queue; workers consume them asynchronously while keeping each persona’s questions in order, accumulating context incrementally.

Extending the demographic space

Researchers regularly need attributes nobody annotated — marital status, occupation, political leaning. Anamnesis lets a user define a new categorical dimension in the interface and runs a demographic survey across the pool to estimate a distribution for every backstory.

Two modes produce that distribution. Logprobs mode takes a single constrained forward pass and reads the full categorical distribution off the token probabilities, which requires a self-hosted vLLM backend. N-sample mode asks the question N times and takes the empirical distribution, converging as N grows at higher inference cost. The second mode exists because most researchers do not run their own serving infrastructure, and the abstraction downstream is identical either way.

Answering algorithms

The default is Anthology’s sequential accumulation: the backstory prepends the first question, and each question–answer pair joins the context before the next question. A persona therefore sees its own prior answers, which is what holds beliefs consistent across an instrument. A zero-shot mode conditioning only on a short demographic description is available alongside, so the contribution of backstory conditioning can be measured inside the same run rather than argued for. Responses parse through structured output first — guided decoding on vLLM, JSON schema on OpenRouter — with a small parser model as fallback.

Experiments

Public opinion. The platform replicates three Pew American Trends Panel waves: Wave 34 on biomedical and food issues (2,537 respondents), Wave 92 on political typology (10,916), and Wave 99 on AI and human enhancement (10,260). Questions keep their original wording and options, including the randomized Likert ordering of the original fielding. Two metrics carry over from Moon et al.: average Wasserstein distance for how well the response distribution matches, and the Frobenius norm between response correlation matrices for whether a persona stays internally consistent across questions.

Backstory conditioning wins on distributional match in every wave, and the twenty questions and thousands of virtual respondents behind these numbers were configured through the graphical builder. That is the claim the paper most wants to land: a social scientist can stress-test an instrument against a balanced virtual population before recruiting anyone.

The human row deserves as much attention as the winning row. It is the distance between two halves of the real respondent pool, and at 0.057–0.091 WD it sits far below anything the platform reaches. The gap between 0.147 and 0.057 is the remaining distance between simulated and real opinion on Wave 34, and it is roughly the same size as the gap the paper just closed. All of Table 1 also runs on a single backbone, LLaMA-3.1-8B.

Multimodal preference. The New Yorker Caption Contest supplies a controlled test of whether backstories still help when the stimulus is an image. Across 49 contests with randomized caption order, Gemini 2.5 Flash at temperature 1.0 makes 20 choices per contest under each answering algorithm. Anthology conditioning reaches 59.2% majority-vote accuracy (95% CI 45.2–71.8%) against 51.0% for the zero-shot baseline (37.5–64.4%). The confidence intervals overlap heavily at 49 contests, so the paired within-item comparison carries more weight: the mean vote share going to the human-preferred caption rises from 52.0% to 59.8%, a paired gain of 7.8 points (95% CI 3.2–12.8, p = 0.0024).

Limitations

The authors put the binding constraint first. Simulation quality is bounded by the diversity of the backstory pool, and LLM-generated backstories inherit their training data’s demographic skew rather than a census. That produces the failure mode Alterity was built to name: shallow binding, where the model voices an outsider’s stereotype of a demographic instead of the group’s own reasoning. Anamnesis carries Alterity’s multi-turn transcripts to deepen the binding, and the authors still advise treating sensitive-topic results with suspicion.

Every backstory and every simulation runs in English, which compresses cultural and linguistic variation into English-language reasoning patterns. Multimodal conditioning inherits whatever perceptual blind spots the underlying model has. And personas are temporally static — nothing in a backstory updates when the world changes, so longitudinal tracking of shifting opinion is out of reach without regenerating the pool.

Two further cautions belong to the evaluation. The main table rests on one 8B model, so how the advantage scales with backbone quality is unmeasured. And the caption-contest study is 49 items, which is why its headline accuracy interval is 27 points wide.

References

• Original paper: Anamnesis: An Open-Source Platform for Large-Scale Backstory-Conditioned Survey Simulation
• Platform: simulate.group
• Anthology (backstory conditioning): Moon et al. (2024), EMNLP 2024, pp. 19864–19897
• Alterity (deep binding, partisan misperception): Kang et al. (2025), COLM 2025
• Demographic-prompting baselines: Santurkar et al. (2023), Whose Opinions Do Language Models Reflect?, ICML 2023
• Caricature and flattening in persona simulation: Cheng et al. (2023), EMNLP 2023; Wang et al. (2025), Nature Machine Intelligence 7(3)
• Interview-conditioned agents: Park et al. (2024), Generative Agent Simulations of 1,000 People — covered earlier in this series
• New Yorker Caption Contest benchmark: Hessel et al. (2022), arXiv:2209.06293; Jain et al. (2020)
• American Voices Project: Stanford Center on Poverty and Inequality (2021)