Your system prompt is 12 pages of rules, and the model follows them inconsistently. What should you do?
Why B is the answer
Trim it to what matters, and move detail into retrievable references
Long prompts dilute attention. Keep the core short and load detail on demand, then use evals to confirm that behavior improved.
In detail
- Long prompts dilute attention.
- Keep the core short, and load detail on demand.
- Use evals to confirm that behavior improved.
- Remove a rule and measure, so you know it mattered.
Use case
An illustrative scenario, not a real incident.
A 12-page system prompt is followed unevenly. The team keeps the core rules and moves the rest into references loaded when needed, and evals improve.
See it in different ways
flowchart LR
A["Task"] --> B["Read the tool index"]
B --> C{"Which capability?"}
C --> D["Load that definition"]
D --> E["Call it"]mindmap
root((Eval set))
Real traffic
Historical cases
Edge cases
Known failures
Human review
Labeled samples
Slices
Segment
Task type
Riskflowchart TD
Q{"How often does the rule apply?"}
Q -->|"Every request"| A0["Core prompt"]
Q -->|"Some requests"| A1["Retrievable reference"]Key point: Short prompts are followed better.
Why the other options are wrong
- A. Repeat the most important rules several times in different places
- Repetition adds length, which is the cause of the problem.
- C. Rewrite all of the rules in capital letters so that they carry more weight
- Capitals do not reduce the amount of text.
- D. Translate the whole prompt into structured JSON for the model to read
- JSON changes the format, not the volume.