1.10 Agent Personas: Simulated Users

How to test an agent against users who behave differently from the tidy messages you would write by hand.

Why Personas

A hand-written test sends the same polite message every run. Real users are impatient, vague, change their minds, or try to steer the agent somewhere it should not go. A persona describes one such user once, and the Simulate user turn step has CertiQA write that user’s next message, in character, based on what the conversation looks like so far. You write the goal of the turn (“ask for a refund and push back on the first answer”); the persona decides how it is said.

Personas are on Team Pro Plus and above (and during the trial).

Create a Persona

Open Persona Profiles from the command palette or the agent tools in the project, then Add Persona:

Field	What to put in it
Name	A short label, for example “Frustrated customer” or “Technical user”
System prompt template	Instructions for playing this user: who they are, what they want, how they talk, what they will and will not accept
Default variables	Optional values the template can use

A good prompt is concrete and states limits: “You are an impatient customer who wants a refund for order 4471. You reply in one or two short sentences. You never give your card number. If the agent offers store credit, refuse once, then accept.” Avoid prompts that ask the persona to attack the agent: use the OWASP red-team sweep for that.

A persona belongs to one project. Anyone with read access to the project can see its personas, members who can author can create and change them, and deleting one needs project admin.

Use a Persona in a Case

1. Add a Simulate user turn step to an agent case.
2. Choose the persona and write the turn goal.
3. Follow it with the checks that matter: expected tool calls, the agent’s reply, a judge rubric, a budget.

When the step runs, CertiQA gives the persona the conversation so far and the turn goal, sends the message it writes to your agent, and records both in the transcript exactly like a typed message. You can use variables in the goal (for example the order number an earlier tool call returned). The conversation can run for as many turns as the case has.

What to Know Before You Rely On It

Simulated Users Are Not Deterministic

• The persona’s wording changes from run to run, as a person’s would. Write assertions about what the agent must do (call the refund tool, refuse to share data, stay in budget), not about exact phrases, and use repeats to see how often it holds.
• Each simulated turn is a model call made on your plan’s AI quota. A long persona conversation costs more than a scripted one.
• A persona only knows the conversation and its prompt. It does not see your agent’s tools, system prompt or the mocks you set.
• Review the transcript of a failing run before deciding the agent is wrong: sometimes the persona wandered off the goal. Tighten the prompt if so.