What this prompt does
This prompt builds a prompt-injection defense system and red-team test suite for an AI application that handles untrusted user input. It generates [attack_count] attack test cases spanning direct injection, indirect injection, prompt leaking, role-playing attacks, and delimiter attacks, then designs a detection prompt that classifies input as safe, suspicious, or malicious. On top of that it adds input sanitization, output validation to catch a compromised model, system-prompt hardening, and a defense-in-depth pipeline of input filter, hardened prompt, and output filter.
The variables target the defense to your application. [app_type] and [app_purpose] define what is being protected, such as a support chatbot that processes refunds, which shapes the relevant attack surface. [attack_count] sets how many injection test cases the red-team suite contains. [sensitive_data] names what must never leak — PII, order details, internal pricing — so the output validation and safe-handling rules guard the right things. [target_model] ensures the hardening techniques, like XML tags and instruction hierarchy, suit how that model parses instructions.
When to use it
- You are shipping an AI feature that touches user input and want a red-team pass before launch.
- The assistant could be tricked into revealing its system prompt or ignoring guardrails.
- Sensitive data (
[sensitive_data]) is in context and a leak would be a real incident. - You need layered defenses — input filter, hardened prompt, output filter — not a single fragile rule.
- Building a test suite of
[attack_count]injection attacks to probe the system systematically. - Planning monitoring and incident response for when an injection attempt slips through.
Example output
Expect a red-team suite of [attack_count] injection test cases across direct, indirect, prompt-leaking, role-playing, and delimiter attacks, plus a detection prompt that labels input safe, suspicious, or malicious. It includes input-sanitization rules, output-validation checks for a compromised model, system-prompt hardening for [target_model], a defense-in-depth layering, monitoring and alerting guidance, a red-team report template, safe-handling rules for [sensitive_data], and an incident-response plan for successful injections.
Pro tips
- Layer the defenses — input filter, hardened system prompt, and output validation — since defense-in-depth is what keeps an injection attempt from becoming an incident, not any single filter.
- Sanitize input without breaking legitimate use; over-aggressive filtering blocks real users, so tune the rules against the
[attack_count]test cases. - Validate output for compromise signals like off-topic replies or system-prompt leakage; catching a hijacked response is the last line before damage.
- Harden the system prompt with structure (XML tags, instruction hierarchy) suited to
[target_model], since hardening techniques are model-specific. - Treat
[sensitive_data]handling as non-negotiable — define exactly what can appear in context and what must never be echoed back. - Plan the incident-response path before launch; assume some injection eventually succeeds and decide how you detect and contain it.
