Designing Reliable Planning Loops

Problem Context

A single LLM call is good at one-shot reasoning; it is bad at multi-step tasks where intermediate state matters. Planning loops give the agent an explicit plan, execute the steps, observe results, and revise. The choice of planning loop — ReAct, Plan-and-Execute, ReWOO, Reflexion — is the single biggest lever on latency, cost, and reliability of an agent.

The Four Planning Loops You Need to Know

• ReAct(Thought → Action → Observation, repeat) — reactive, one step ahead. Cheap when steps are obvious, expensive when they aren't.
• Plan-and-Execute — make a full plan up front, execute step-by-step, replan only on failure. Far fewer LLM calls than ReAct for long tasks.
• ReWOO (Reason WithOut Observation) — generate the entire plan as a DAG of tool calls with placeholders, execute the DAG, stitch results in a single final reasoning call. Maximum parallelism, minimum LLM calls.
• Reflexion — run the task, evaluate the outcome, write a critique, retry with the critique in the prompt. Best for tasks with an automatic verifier (tests pass, schema validates).

flowchart LR
    subgraph ReAct
        R1[Thought] --> R2[Action] --> R3[Obs] --> R1
    end
    subgraph PlanExecute
        P1[Plan all steps] --> P2[Exec step i]
        P2 --> P3{All done?}
        P3 -- no --> P2
        P3 -- yes --> P4[Final answer]
    end
    subgraph ReWOO
        W1[Plan DAG with<br/>placeholders] --> W2[Exec tools<br/>in parallel] --> W3[Stitch + answer]
    end
      

Choosing a Loop: A Rubric

• Steps depend on prior observations? ReAct or Plan-and-Execute. Not ReWOO.
• Steps are independent (parallel tool calls)? ReWOO is 5–10× cheaper.
• Task has a verifier (tests, schema, eval LLM)? Wrap any loop in Reflexion.
• Latency matters more than cost? ReWOO (parallel) > Plan-and-Execute > ReAct (sequential).

Plan-and-Execute, Minimal Implementation

async def plan_and_execute(task: str):
    # 1. Plan
    plan = await llm.invoke(PLAN_PROMPT.format(task=task))
    steps = parse_plan(plan)  # ['search arxiv for X', 'summarise top 5', ...]

    results = []
    for i, step in enumerate(steps):
        # 2. Execute one step
        try:
            r = await executor.run(step, prior=results)
            results.append(r)
        except StepFailed as e:
            # 3. Replan from current state
            steps = await llm.invoke(REPLAN_PROMPT.format(
                task=task, done=results, failed=step, error=str(e)))
            steps = parse_plan(steps)
            continue

    return await llm.invoke(SYNTHESIZE_PROMPT.format(task=task, results=results))

The Cost Math Nobody Runs

For a 5-tool-call task on gpt-4o-mini at $0.15/M input + $0.60/M output, with 4 KB of context per call:

• ReAct: ~10 calls (each step is two — thought + observation digest) ≈ $0.012.
• Plan-and-Execute: 1 plan + 5 step + 1 synth = 7 calls ≈ $0.009.
• ReWOO: 1 plan + 1 synth = 2 LLM calls (tools run in parallel) ≈ $0.003.

At 1M tasks/month, that's $12K vs $9K vs $3K. The loop choice is the cost choice.

Failure Modes

• Ungrounded plans — Plan-and-Execute makes up tool names that don't exist. Inject the tool registry into the planner prompt.
• Replan storms — every failure triggers a full replan; tasks never finish. Cap replans to 2, then fail loud.
• ReWOO placeholder mismatch — placeholder #E2 never gets filled because step 2 returned an error. Validate the DAG before executing.
• Reflexion infinite loops — the critic keeps finding new flaws. Cap retries and require monotonically improving scores.

Production Checklist

• Pick the loop on cost/latency math, not vibes.
• Inject the available tool registry into every planner prompt — no hallucinated tools.
• Cap iterations (max steps, max replans, max reflections) and surface the cap as a typed error.
• Trace every loop iteration as a span; debugging a planning loop without traces is impossible.

Key Takeaways

• ReAct is the default; ReWOO is the optimisation; Plan-and-Execute is the middle ground; Reflexion is the wrapper.
• The cheapest loop is the one whose structure matches your task's dependency graph.
• Caps and tool-registry grounding are what turn a research loop into a production loop.