AI agent testing terms that point to evidence, not theatre.
This glossary defines the language behind Agent Torture Lab reports: scenario families, transcript evidence, launch recommendations, retesting, and the failure modes that matter before launch.
Core vocabulary for AI agent launch testing.
AI agent testing
The process of checking whether an AI agent behaves safely, accurately, and usefully in the situations it is likely to face after launch.
Adversarial customer simulation
A test conversation that applies realistic pressure: confusion, policy challenges, unsafe requests, prompt probes, or urgent escalation needs.
Scenario family
A broad category of test cases, such as privacy, policy bending, prompt injection, safety, escalation, tone, or conversion.
Scenario pack
A curated group of test scenarios selected for a channel, industry, or launch goal. Public pages can describe families without publishing proprietary exact prompts.
Transcript evidence
The captured customer turn and agent reply used to support a finding. Strong reports show the exchange instead of only naming the failure.
Prompt injection
An attempt to make the agent ignore its intended instructions, reveal hidden policy, change role, or use tools in an unintended way.
Expected safer behavior
The behavior the agent should have used instead, such as refusing a risky request, asking a clarifying question, or escalating to a human.
Launch report
The practical output of a test run: score, findings, transcript evidence, severity, fix guidance, and retest recommendations.
Severity
The practical risk level of a finding. It should reflect customer harm, business exposure, trust damage, compliance risk, or broken conversion.