Tree-of-Thought: Search Over Reasoning PathsMixed sources2 sources, drawn from a mix of official docs, project repositories and research.This rates where the sources come from. It doesn't prove every claim.
Propose several next steps, judge the partial states, keep the best and expand them. How to build a Tree-of-Thought search, why the evaluator decides everything, and when a reasoning model makes it unnecessary.
By Amit UpadhyayPublished 10 minArticle 4 of 13 in Prompt Engineering
AI assistant · answers from this article
1 of 6
One path is enough
Nothing to add yet
When one line of reasoning reaches the answer, a tree only adds cost.
On this page
Tree-of-thought, in one sentence
Tree-of-thought has a model propose several next steps, scores each one and keeps the best, so one bad early choice does not ruin the answer.
You reach for it when a problem needs search: several options must be tried, and early choices can lead to dead ends. Not before: it costs many more model calls, and an evaluator that has to be right for the search to work.
The cheat sheet
What to use when
You do not need all of this on day one. Each row is one signal and the piece that answers it. Stop at the row your system has reached, or pick one to see it in the picture above.
Where you areThe signalWhat to useWhat it costs
None. A single chain of reasoning gets the task right.Nothing. One chain of reasoningNothing
The model picks an order early, meets a conflict at step seven, and talks itself into a workaround.Several candidate steps, scored, with the best keptWhere the idea comes from ↓Ten to fifty model calls for one answer
The model grades its own proposals kindly, and the tree just follows the first chain.An evaluator that is real code, or a separate judgeThe evaluator ↓A checker to write
Nobody worked out how many model calls one answer takes.A beam, with the calls per level multiplied out in advanceA breadth-first tree ↓A good branch can fall out of the beam
Four options come back, and they are one option phrased four ways.Ask for options that differ in a named way, and remove duplicatesPitfalls ↓A deduplication step
A reasoning model at higher effort solves the same puzzle in one call.Keep the tree for an audit trail, an external checker or a small modelWhen models already reason ↓Measuring the simpler setup first
Start With One Chain
If chain-of-thought, or a reasoning model at higher effort, gets the task right, stop there. A tree costs ten to fifty model calls for one answer. It earns that on a narrow class of problems, where a wrong early choice can't be recovered inside a single answer.
Chain-of-thought gives the model room to work, but it still commits to one path. If the second step is a bad choice, every step after it is spent defending that choice. You see this on planning tasks: ask a model to order eleven database migrations so that no foreign key breaks, and it picks an order early, notices the conflict at migration seven, and then talks itself into a workaround instead of going back.
Tree-of-Thought (ToT) is the fix for that specific failure. Instead of one chain, you generate several candidate next steps, score each one, keep the promising ones, and expand them further. It is ordinary search, the same breadth-first or depth-first search you wrote in a data structures course, with the model doing two jobs: proposing moves and judging positions.
Reach for Tree-of-Thought when
The task has intermediate states you can name: a partial plan, a half-filled schedule, a draft patch
A wrong early choice is expensive to recover from inside a single answer
You can judge a partial state, either with the model or, better, with real code such as tests or a validator
You can afford ten to fifty model calls for one answer, because the quality of that answer matters more than its cost
If none of these hold, stop at chain-of-thought or a reasoning model with higher effort. ToT is a tool for a narrow class of problems, and it is easy to spend a lot of tokens on it for nothing.
Pro deep dive10 min full read
Keep reading with Pro
This is one of our advanced production deep dives. Pro unlocks the rest of it, every other deep dive, and the See, Animate, Test and Build modes on every topic.
What it covers
Propose several next steps, judge the partial states, keep the best and expand them. How to build a Tree-of-Thought search, why the evaluator decides everything, and when a reasoning model makes it unnecessary.