Red teaming is structured attempts to make a system fail. Done well, it is a repeating program. Done poorly, it is a brainstorm that never becomes evidence.
Set the scope
Name the system, the user, and the harms you care about: hallucination, unsafe advice, security leakage, policy bypass, or domain error. A boundless "try to break it" session produces stories, not coverage.
Design scenarios
- Ambiguous instructions that a real user might send.
- Adversarial prompts that ask the system to ignore rules.
- Domain edge cases that require professional judgment.
- Multi-step agent workflows where an early error compounds.
Staff the roles
Separate people who invent attacks, people who execute them, and people who classify results. Domain experts should review failures that look plausible. Security specialists should own tool and data-exfiltration cases.
Capture evidence
Store the prompt, the output, the classification, and the severity. Group findings into categories you can track over time. Then schedule the next round against the same categories after a model or policy change.