CAMEL — Role-Playing Communicative Agents for LLM Society

TL;DR

Why it matters. Most early multi-agent LLM pipelines required a human to steer every turn. CAMEL asks whether two chat agents can complete an open-ended task cooperatively, without human intervention in the loop.
What it proposes. Role-playing + inception prompting — a two-agent protocol where an AI User drives toward a goal and an AI Assistant executes, each given a role-anchoring system prompt that outlasts the conversational context.
Headline result. On the AI Society dataset (25,000 conversations), agents complete the assigned task in the majority of sessions, though roughly 15% exhibit “conversation collapse” under GPT-3.5.
Limit. All task-quality evaluation is LLM-self-graded; role flipping and degenerate loops remain unsolved failure modes within the paper’s own scope.
What to keep. The inception-prompting template has outlasted the datasets. Anchoring role, goal, and refusal phrases together in one system prompt is what holds a cooperative conversation together long enough to finish a task.

Background

RLHF-tuned chat models (InstructGPT, ChatGPT) showed that instruction following could be taught at scale, enabling single-agent task completion across diverse domains. The ceiling appeared at multi-step tasks requiring planning, execution, and verification in sequence. The raw capability was there; what was missing was a mechanism for one agent to hold a persistent role while another held a complementary one.

The most common workaround was human-in-the-loop orchestration: a human user prompts the AI step by step, essentially acting as both planner and verifier. This keeps the human as the indispensable scheduler. CAMEL’s framing is sharper: given that LLMs can both follow instructions and generate instructions, can two instances cooperate without any human in the middle? If yes, what is the minimal protocol that makes it work?

Core Idea

End-to-end CAMEL role-playing session: Human idea flows through Task Specifier to AI User (Stock Trader) and AI Assistant (Python Programmer) exchanging multi-turn instructions and code — **Fig. 1.**CAMEL role-playing framework end-to-end. A human idea enters the Task Specifier, which generates a concrete task; the AI User (Stock Trader) then issues step-by-step instructions; the AI Assistant (Python Programmer) executes and reports code solutions until CAMEL_TASK_DONE.Source: Li et al. 2023, misc/framework.png

The protocol assigns two roles and one goal, then leaves the agents to negotiate turn by turn. The AI User is responsible for issuing concrete, single-step instructions. The AI Assistant is responsible for executing them and reporting results. Neither is supposed to initiate the other’s job.

The challenge is that LLMs, without guidance, tend to hedge, break character, or assume the other agent’s responsibilities after a few turns. The authors call their solution inception prompting: a carefully designed system prompt that embeds the role so deeply that it survives the accumulation of in-context turns.

A schematic AI User inception prompt reads:

You are a {human_role}. You are interacting with a {ai_role} AI assistant.
You want the assistant to help you with the following task:

  Task: {task}

You must instruct the assistant one step at a time.
Do NOT perform the task yourself — only provide instructions.
When the task is complete, say: "CAMEL_TASK_DONE".

A parallel prompt anchors the assistant to execution, forbids unsolicited planning, and prohibits the phrases that would signal role abandonment (e.g., “As an AI language model”, “I cannot help with this”). The asymmetry — User plans, Assistant acts — rests entirely on text constraints, with no architectural separation behind it.

Method

Task specification

A third LLM call, the task specifier, takes the two roles and a generic task seed and produces a concrete, actionable version. For role pair (Python programmer, stock trader), the seed “develop trading software” might become “write a Python backtesting script for a mean-reversion strategy on SPY, with configurable window size and stop-loss parameters”. This specificity step matters: vague tasks produce vague instruction sequences that are hard to evaluate.

AI Society and Code datasets

The paper generates two datasets using the protocol above.

AI Society: 50 AI backbone roles (programmer, doctor, teacher, chef, …) × 50 human roles × 10 tasks per pair = 25,000 conversations. Each conversation is a multi-turn role-playing exchange until the CAMEL_TASK_DONE signal or a maximum turn limit.

Code: A subset oriented toward programming tasks, with role pairs like (Python programmer, domain expert) generating conversations that contain executable code.

Both datasets are released and have been used as instruction-following training data in subsequent work.

Conversation dynamics and failure modes

CAMEL role-playing framework diagram with description of failure modes: role flipping, repeated instructions, flake replies, infinite loops, premature termination — **Fig. 2.**CAMEL role-playing architecture and named failure modes. The three-node diagram represents the AI User, Task Specifier, and AI Assistant; the description enumerates the failure modes inception prompting is designed to suppress: role flipping, instruction repetition, flake replies, infinite message loops, and premature termination.Source: camel-ai.org documentation

The paper documents four failure modes from conversation analysis:

Role flipping: the assistant starts issuing instructions and the user starts executing — roles quietly invert mid-conversation.
Conversation deviation: the exchange drifts from the original task to meta-commentary, small talk, or tangential topics.
Premature termination: CAMEL_TASK_DONE is declared before the task is meaningfully complete.
Degenerate messages: repetitive acknowledgements (“Great! Let’s continue.”) that add no information and consume context budget without advancing the task.

Role flipping is the most theoretically interesting failure: it suggests the model’s default cooperative disposition overrides the role assignment when maintaining the role requires non-cooperative behavior (refusing to take over the partner’s job).

Experiments

Conversations were sampled from the AI Society dataset and analyzed along three dimensions: instruction-following consistency, role adherence across turns, and task completion.

The qualitative finding is that inception prompting substantially reduces all four failure modes relative to simpler system prompts that merely name the role. Role-locked conversations that reach CAMEL_TASK_DONE tend to produce on-topic, step-structured exchanges where the assistant’s outputs could plausibly be used to complete the task.

The paper’s analysis is largely qualitative and case-study driven — representative conversations are annotated and presented, but aggregate task-completion rates with statistical confidence are not the primary instrument. This is a design choice: the goal is to show that autonomous cooperation is possible and to map where it breaks, leaving leaderboard numbers aside.

Limitations

Three limits the authors acknowledge. First, the backbone model coverage is narrow: experiments run on GPT-3.5 and GPT-4; whether inception prompting transfers to smaller or differently aligned models is unexamined. Second, the evaluation is self-graded — a model checks whether the conversation completed the task, which conflates completion with quality and cannot detect confident-but-wrong execution. Third, the binary AI User / AI Assistant split does not generalize to tasks requiring more than two agents, dynamic role assignment, or task trees with branching rather than linear execution.

Interpretively, the failure-mode taxonomy (role flipping, deviation, premature termination, degenerate messages) stays descriptive. CAMEL names what breaks without explaining why it breaks, which is part of why the same failures keep surfacing in multi-agent frameworks built years later.

Sapiens Q Take

Three things we would take from this paper. First, inception prompting operates at the structural level: the system prompt has to fix both what the agent does and what it refuses to do, and both halves have to be present for the role to hold. Subsequent work (AutoGen, LangGraph, etc.) re-learned this lesson by designing hard-coded role separators or tool-access restrictions that do architecturally what inception prompting attempts textually. Second, CAMEL is the origin of the CAMEL-AI ecosystem, and the authors later built OASIS (million-agent social simulation) on the same role-playing abstraction. Reading CAMEL and OASIS together shows how a two-agent protocol scales to a platform through layers of environment, recommendation, and infrastructure added incrementally. Third, the failure mode of role flipping has a structural analogue in multi-agent coordination protocols generally: whenever maintaining a role requires refusing a cooperative action, cooperative fine-tuning creates pressure toward role dissolution. This is a design consideration for any system where agents must specialize while also being collectively trained to be helpful.

References

Original paper: CAMEL: Communicative Agents for “Mind” Exploration of Large Language Model Society
Code and ecosystem: camel-ai/camel
NeurIPS 2023 proceedings: NeurIPS poster #72905
Direct successor (scale-up): Yang et al. (2024) OASIS, arXiv:2411.11581
Contemporary multi-agent frameworks: AutoGen (Wu et al. 2023); AgentScope (Liu et al. 2024)