← All ideas
0

Agents: Run multi-turn simulations with an AI customer and replay real conversations

LaunchedAgentsIdea

Fin Simulations play a full conversation with an AI customer, mock connector outputs and judge the result; Batch Test replays real conversations (https://fin.ai/updates/simulations-for-procedures, https://fin.ai/updates). Voiceflow tests support AI-simulated users with success criteria and assertions on tool calls (https://www.voiceflow.com/docs/documentation/measure/tests.md). Our tests (lib/agents-tests.ts) are one question plus an expected behavior, which does not cover booking or qualification flows. First scope: a test can be a persona plus a goal; an LLM plays the customer for up to N turns in dry run and the judge checks the goal and which tools were called; plus 'Replay last 20 conversations' that reruns real visitor messages against the current instructions and highlights answers that changed.

Comments 0

No comments yet.