What should you test for before release, in addition to quality?
Why D is the answer
Prompt injection and jailbreak attempts
Testing for prompt injection and jailbreak attempts finds weaknesses before attackers do.
In detail
- Test for prompt injection and jailbreaks as well as quality.
- Repeat these tests after each change.
Use case
An illustrative scenario, not a real incident.
Before launch, a team tries to make its assistant reveal its rules, and it fixes what works.
See it in different ways
flowchart LR A["User input"] --> B["AWS WAF"] B --> C["Input guardrail"] C --> D["Foundation model"] D --> E["Output guardrail"] E --> F["Grounding check"] F --> G["Response"]
sequenceDiagram participant U as User participant I as IAM participant G as Guardrails participant M as Model participant L as CloudTrail U->>I: Is this caller allowed? I->>G: Filter the input G->>M: Safe prompt M-->>G: Output G-->>U: Filtered output I->>L: Records the call
flowchart LR S0["Write attacks"] --> S1["Run them"] --> S2["Fix weaknesses"] --> S3["Test again"]
Key point: Test against adversarial input.
Why the other options are wrong
- A. The size of the model's training data
- Training data size is not a safety test.
- B. The cost per token of the model
- Token cost is not a safety test.
- C. The size of the vector index
- Index size is not a safety test.