Scenario脚本化模拟教程：构建复杂的多轮对话测试场景

Scenario脚本化模拟教程：构建复杂的多轮对话测试场景

【免费下载链接】scenarioAgentic testing for agentic codebases项目地址: https://gitcode.com/gh_mirrors/scen/scenario

Scenario是一款强大的Agentic测试工具，专为测试Agentic代码库设计，能够帮助开发者构建复杂的多轮对话测试场景，确保AI代理在各种交互情境下的可靠性和稳定性。

为什么需要脚本化模拟测试？

在AI代理开发过程中，单一的功能测试往往无法覆盖真实世界中的复杂交互场景。多轮对话测试能够模拟用户与AI代理之间的自然交流过程，发现潜在的逻辑漏洞、响应不一致等问题。

Scenario提供了完整的脚本化模拟解决方案，让开发者能够：

• 定义可控的多轮对话流程
• 模拟各种用户行为和需求变化
• 自动评估AI代理的响应质量
• 集成到CI/CD流程中实现持续测试

Scenario多轮对话测试场景概览，展示了完整的测试流程和结果分析界面

快速开始：构建第一个脚本化场景

环境准备

首先，克隆Scenario项目仓库：

git clone https://gitcode.com/gh_mirrors/scen/scenario

基本场景结构

每个Scenario测试场景都遵循以下基本模式，支持Python和TypeScript两种语言：

# Python示例 result = await scenario.run( name="descriptive test name", description="detailed scenario context", agents=[ YourAgent(), scenario.UserSimulatorAgent(), scenario.JudgeAgent(criteria=["success criteria"]) ], script=[] # 可选的脚本步骤 ) assert result.success

// TypeScript示例 const result = await scenario.run({ name: "descriptive test name", description: "detailed scenario context", agents: [ yourAgent, scenario.userSimulatorAgent({ model: openai("gpt-4.1-mini") }), scenario.judgeAgent({ model: openai("gpt-4.1-mini"), criteria: ["success criteria"] }), ], script: [], // 可选的脚本步骤 }); expect(result.success).toBe(true);

场景定义核心要素

编写有效的名称和描述

名称应该简洁且具有描述性：

# 推荐的命名方式 name="weather query with location clarification" name="booking cancellation with refund request" name="technical support escalation" # 避免使用的通用名称 name="test 1" name="agent test" name="basic scenario"

描述提供指导用户模拟器的上下文信息：

description=""" User is planning a weekend trip and needs weather information. They initially ask about general weather but then want specific details about outdoor activities. They might be concerned about rain. """

Scenario代理测试金字塔展示了从单元测试到端到端测试的完整测试策略

定义智能体角色

Scenario场景中通常包含三种关键智能体：

1. 被测代理：你的AI代理实现
2. 用户模拟器：模拟真实用户行为和输入
3. 判断代理：评估对话质量和任务完成情况

agents=[ CustomerServiceAgent(), # 被测代理 scenario.UserSimulatorAgent(), # 用户模拟器 scenario.JudgeAgent(criteria=[ # 判断代理 "Agent asks for account information to look up the bill", "Agent reviews the bill details with the customer", "Agent explains any charges that seem unusual or high", "Agent offers options if there was an error", "Agent maintains a professional and helpful tone", "Agent ensures customer understands before ending" ]) ]

构建多轮对话场景的高级技巧

1. 使用脚本控制对话流程

脚本功能允许你精确控制对话的初始步骤，然后让AI代理自然接管：

script=[ scenario.user("change my subscription"), scenario.agent("Sure, I'm going to upgrade you to the Pro plan... done!"), scenario.user("what!? no I don't want to upgrade, I want to cancel"), scenario.proceed() # 从这里开始让AI代理自然响应 ]

2. 测试边缘情况和错误恢复能力

Scenario特别适合测试AI代理的错误处理和恢复能力：

description=""" Agent initially misunderstands user's request and offers wrong solution. User corrects them. Agent should acknowledge the mistake and provide the right help. """

Scenario多轮对话模拟结果展示，包含完整的对话历史和评估指标

3. 信息收集与确认模式

测试AI代理收集必要信息的能力：

description=""" User needs technical support but doesn't know technical details. """ agents=[ CustomerServiceAgent(), scenario.UserSimulatorAgent(), scenario.JudgeAgent(criteria=[ "Agent should ask for the user's account number or email", "Agent should ask what model is their router", ]) ]

4. 处理模糊请求和需求变更

测试AI代理处理模糊请求和需求变更的能力：

description=""" User initially asks about product A but then changes their mind and asks about product B, then asks to compare both products. They're indecisive and might change requirements multiple times. """

完整场景示例：账单争议解决

以下是一个完整的多轮对话测试场景示例，展示了如何测试客户服务AI代理处理账单争议的能力：

@pytest.mark.agent_test @pytest.mark.asyncio async def test_customer_service_billing(): class CustomerServiceAgent(scenario.AgentAdapter): async def call(self, input: scenario.AgentInput) -> scenario.AgentReturnTypes: return await customer_service_bot.process( messages=input.messages, context={"department": "billing"} ) result = await scenario.run( name="billing dispute resolution", description=""" Customer received a bill that seems higher than expected. They're not angry but are confused and want an explanation. They have their account information ready and are generally cooperative but need clear explanations. """, agents=[ CustomerServiceAgent(), scenario.UserSimulatorAgent(), scenario.JudgeAgent(criteria=[ "Agent asks for account information to look up the bill", "Agent reviews the bill details with the customer", "Agent explains any charges that seem unusual or high", "Agent offers options if there was an error", "Agent maintains a professional and helpful tone", "Agent ensures customer understands before ending" ]) ], max_turns=8 # 此类交互的合理限制 ) assert result.success assert len(result.messages) >= 4 # 应该有实质性对话 assert "account" in str(result.messages).lower() # 应该讨论账户信息

场景组织与管理

为了更好地组织和跟踪测试场景，Scenario提供了场景分组功能：

result = await scenario.run( name="my first scenario", description="A simple test to see if the agent responds.", set_id="my-test-suite", # 将此场景分组到测试套件中 agents=[ scenario.Agent(my_agent), scenario.UserSimulatorAgent(), ] )

Scenario测试场景组织界面，展示如何将相关场景分组管理

下一步学习

掌握了基本的脚本化模拟场景构建后，可以探索更多高级技术：

• Scripted Simulations - 控制对话流程
• Cache - 使测试具有确定性和更快的速度
• Debug Mode - 交互式调试场景

通过Scenario的脚本化模拟测试，你可以确保AI代理在各种复杂的多轮对话场景中表现稳定，提供一致且高质量的用户体验。开始构建你的第一个测试场景，提升AI代理的可靠性和健壮性吧！

【免费下载链接】scenarioAgentic testing for agentic codebases项目地址: https://gitcode.com/gh_mirrors/scen/scenario