Zero-Shot Prompting: What LLMs Know Without Examples

What LLMs Know Without Being Told

Zero-shot prompting means giving the model a task with no examples. It works because large language models were trained on enormous corpora that contain implicit patterns for most common tasks. Understanding what zero-shot can and can't do — and the prompting techniques that improve it — is foundational before adding examples, tools, or reasoning chains.

How Zero-Shot Works

flowchart LR
    pretraining["Pretraining
Text prediction on web-scale data
Implicit patterns for many tasks"]
    instruct["Instruction tuning
Fine-tuned on instruction-response pairs
(RLHF, DPO)"]
    zeroShot["Zero-shot inference
Task description → completion
No examples needed"]

    pretraining --> instruct --> zeroShot

Instruction Clarity

// The most impactful zero-shot improvement: specificity

// ❌ Vague — model has too many valid interpretations
const vaguePrompt = "Summarize this article.";

// ✅ Specific — format, length, and audience are all constrained
const clearPrompt = `
Summarize the following article in exactly 3 bullet points.
Each bullet point should be 1 sentence, max 20 words.
Target audience: non-technical product managers.
Focus on business impact, not technical details.

Article:
${articleText}
`;

// What to specify:
// - Output format (bullet points, numbered list, JSON, prose)
// - Output length (3 bullets, 100 words, one paragraph)
// - Target audience (engineers, executives, customers)
// - Perspective/focus (business impact, technical approach, risks)
// - What to exclude ("no technical jargon", "do not include introductory phrases")

Role Prompting

Assigning a role adjusts the model's register (vocabulary, depth, assumptions about what you know) and can activate relevant knowledge clusters from pretraining. It's not magic — a role doesn't give the model capabilities it doesn't have — but it does shift output style meaningfully.

// Role prompting via system message
const systemPrompt = `You are a senior software engineer reviewing a code change for production readiness.

Your review focuses on:
- Security vulnerabilities (OWASP Top 10)
- Performance implications at scale
- Error handling and edge cases
- Backward compatibility

You do not comment on style or formatting unless it affects readability.
Your tone is direct and technical. You cite specific line numbers.`;

const response = await openai.chat.completions.create({
  model: 'gpt-4o',
  messages: [
    { role: 'system', content: systemPrompt },
    { role: 'user', content: `Review this code:

${codeChange}` }
  ]
});

// Role effects:
// - Shifts vocabulary and assumed expertise level
// - Activates relevant knowledge structures from pretraining
// - Sets tone (formal/informal, direct/diplomatic)
// - Constrains scope (what to include/exclude in the response)

// Roles that work well: "senior [role]", "expert in [domain]", "[role] specialising in [area]"
// Roles that don't help: "a helpful assistant", "an AI", vague personas

Zero-Shot Chain-of-Thought

Adding “Let's think step by step” significantly improves accuracy on reasoning tasks. This is the zero-shot version of chain-of-thought prompting — no examples needed, just the instruction to reason before answering.

// Zero-shot CoT for reasoning tasks
const withoutCoT = `
A customer's subscription started on March 15. They cancelled after 47 days.
Their plan costs $29/month. They paid monthly. How much should they be refunded?
`;

const withCoT = `
A customer's subscription started on March 15. They cancelled after 47 days.
Their plan costs $29/month. They paid monthly. How much should they be refunded?

Think through this step by step before giving your final answer.
`;

// withCoT is significantly more likely to get the right answer
// because it forces the model to surface its reasoning

// For production: extract just the final answer
const finalAnswerPrompt = `...(calculation prompt)...

Think step by step. Then give ONLY the final dollar amount refund on the last line,
formatted as: REFUND: $XX.XX`;

Format Specification

// Specify output format explicitly — don't let the model choose
const structuredPrompt = `
Analyse the sentiment of the following customer review.

Return your analysis as a JSON object with this exact structure:
{
  "sentiment": "positive" | "negative" | "neutral" | "mixed",
  "confidence": 0.0 to 1.0,
  "key_phrases": ["phrase1", "phrase2"],  // up to 3 phrases driving the sentiment
  "summary": "one sentence"
}

Return ONLY the JSON object. No explanation, no markdown code blocks.

Review: ${reviewText}
`;

// Or use structured output for guaranteed valid JSON:
const response = await openai.chat.completions.create({
  model: 'gpt-4o-mini',
  messages: [{ role: 'user', content: reviewText }],
  response_format: { type: 'json_object' },
});

When to Upgrade from Zero-Shot

• Consistent bad format: Switch to few-shot with format examples, or use structured output
• Task-specific style not matching: Provide 2-3 examples of your exact style
• Factual errors on specialised domain: Add context via RAG or switch to a domain-specific model
• Reasoning errors on multi-step problems: Add explicit chain-of-thought instruction or use a reasoning model (o1, o3)
• High volume, cost concerns: Fine-tune on your zero-shot prompts + ideal outputs

Pitfalls

Negation and exclusions

LLMs are worse at following negative instructions than positive ones. “Do not include technical jargon” is less reliable than “Use plain language suitable for a non-technical reader.” Rephrase exclusions as positive constraints wherever possible.

Role prompting on factual tasks

Telling the model it's “an expert in quantum physics” does not give it knowledge it doesn't have. On highly specialised, factual questions, roles increase confidence without increasing accuracy — which is worse than no role at all.