The system prompt establishes the LLM's operating context for the entire conversation. It defines persona (who the model acts as), context (relevant background), rules (what it can/cannot do), and output format. System prompts are cached by most providers (reducing cost on repeated requests). Poor system prompts lead to off-topic responses, wrong tone, or ignored constraints. The system prompt is processed before user input — treat it as configuration, not conversation.
Structuring prompts to maximise cache hit rate.
"You are a helpful assistant" is a useless persona. "You are a senior TypeScript engineer who specialises in type safety and has written compilers" gives the model a specific knowledge domain and response style to emulate.
Cache threshold is ≥1024 tokens (OpenAI) or ≥2048 (Anthropic). A rich 2000-token system prompt with persona, rules, context, and examples costs 90% less after the first call with Anthropic caching.
Don't expect the model to infer constraints. "Never recommend deprecated APIs" "Always add error handling" "If asked about competitors, stay neutral" — explicit rules produce consistent behaviour across thousands of calls.
Sign in to share your feedback and join the discussion.