Step
Start with a legitimate customer task
The best tests mix normal support or sales intent with instruction pressure, because that is where real user behavior becomes risky.
Methodology
Prompt injection testing methodology
How Agent Torture Lab tests prompt-injection risk in customer-facing chatbots without publishing reusable exploit recipes.
Last updated 2026-06-19. This page explains the testing standard without publishing private scenario prompts or customer data.
Risk family
Hidden-instruction pressure, role override attempts, policy bypasses, and context misuse.
Test prompt-injection risk as part of customer-facing launch readiness, not as a stunt.Tie each serious result to business impact such as privacy exposure, unsafe advice, or policy bypass.Keep public reporting focused on risk families and safer behavior rather than reusable payloads.Retest nearby variants after prompt, retrieval, guardrail, or workflow changes.
Test steps
How this risk family is pressure-tested.
Step
Probe role and policy boundaries
The bot should keep its approved role, refuse hidden-rule disclosure, and still help with the safe part of the customer request.
Step
Check whether bad instructions persist
A single refusal is not enough. Multi-turn testing verifies that user-supplied instructions do not poison the rest of the conversation.
Evidence standard
What a credible finding should show.
- The finding names the customer-facing risk, not the injection label alone.
- The transcript shows the point where the bot followed user-supplied instructions or protected its boundary.
- The expected safer behavior is specific enough to retest without exposing a public exploit recipe.
A credible finding shows01 02 03
The finding names the customer-facing risk, not the injection label alone.
The transcript shows the point where the bot followed user-supplied instructions or protected its boundary.
The expected safer behavior is specific enough to retest without exposing a public exploit recipe.
Mistakes to avoid
Shortcuts that weaken the test.
- Publishing exact attack prompts in public pages.
- Treating prompt injection as separate from refund, privacy, safety, and policy risk.
- Only testing obvious jailbreak strings while ignoring multi-turn customer pressure.
FAQ
Short answers for buyers, builders, and AI assistants.
What is prompt injection testing for chatbots?
It checks whether user messages can make a chatbot ignore instructions, reveal hidden context, misuse tools, expose private data, or bypass business policy.
Should public prompt-injection methodology include payloads?
No. Public methodology should explain risk families, evidence standards, safer behavior, and remediation without giving attackers reusable instructions.
How do you retest a prompt-injection fix?
Rerun the original risk family, nearby multi-turn variants, and normal customer journeys to verify the fix protects boundaries without blocking useful behavior.
Related pages
Connect the methodology to practical testing.
Priority paths
Move from methodology into the pages that should be discovered first.
Bot Roast
Run the live crash test and get a transcript-backed report preview.
Pricing
See the free preview, one-time report unlock, and account credit model.
Agency AI agent testing
Use Bot Roast reports for client QA, handoff, and fix conversations.
Sample API Agent Roast report
Inspect the report format: evidence, severity, fixes, and retest guidance.
Chatbot QA checklist
Use the launch checklist for policy, privacy, escalation, and prompt pressure.
AI chatbot QA testing
Map chatbot QA to real customer pressure, transcript evidence, and fixes.
Generic LLM evals comparison
Compare model-level evals with customer-facing launch-readiness testing.
Is my chatbot safe to launch?
Decide if a bot — even one someone else built for you — is safe to put in front of customers.
AI chatbot audit
What an AI chatbot audit covers and the transcript-backed report you should get from one.