Problem Context
You built a chatbot. It talks to an LLM API, streams responses, and users like it. But it can't do anything. It can't look up data, create tickets, check order status, or remember what you discussed yesterday. It's a conversation to nowhere.
The path from chatbot to agent is well-defined: add tools (so it can act), add memory (so it can remember), and add planning (so it can handle multi-step tasks). Each layer builds on the previous one. This article walks through each upgrade on a real codebase.
- You have a working chat interface but users keep asking it to do things it can't
- You want to add tool calling but aren't sure how to handle the back-and-forth between LLM and tools
- Your chatbot forgets everything between sessions and users are frustrated
- You need the chatbot to handle multi-step tasks like "book a room and send the confirmation"
This article evolves a basic chatbot into an agent, one capability at a time, with code at every step.
Concept Explanation
A chatbot answers questions. An agent takes actions. The difference is three capabilities:
flowchart TD
CB["Chatbot\n(stateless Q&A)"] --> T["+ Tools\n(can take actions)"]
T --> M["+ Memory\n(remembers context)"]
M --> P["+ Planning\n(multi-step tasks)"]
P --> A["Agent\n(autonomous assistant)"]
style CB fill:#6b7280,color:#fff,stroke:#4b5563
style T fill:#d97706,color:#fff,stroke:#b45309
style M fill:#059669,color:#fff,stroke:#047857
style P fill:#4f46e5,color:#fff,stroke:#4338ca
style A fill:#7c3aed,color:#fff,stroke:#6d28d9
Layer 1: Tools
The LLM can request function calls. Your code executes them and returns results. The LLM incorporates results into its response. This is the difference between "I can tell you how to check order status" and "Your order #4521 shipped yesterday."
Layer 2: Memory
Persistent storage of conversation history and user context across sessions. Without memory, every conversation starts from zero. With memory, the agent knows your preferences, previous requests, and ongoing tasks.
Layer 3: Planning
The ability to decompose a complex request into steps, execute them in order, handle failures mid-plan, and report progress. "Book a room and send the confirmation" becomes: (1) search available rooms, (2) select best match, (3) create booking, (4) send confirmation email.
Implementation
Step 1: The Starting Chatbot
// The basic chatbot — stateless, no tools
public class SimpleChatbot
{
private readonly ChatClient _client;
public async Task<string> ChatAsync(string userMessage)
{
var messages = new List<ChatMessage>
{
new SystemChatMessage("You are a helpful assistant."),
new UserChatMessage(userMessage)
};
var response = await _client.CompleteChatAsync(messages);
return response.Value.Content[0].Text;
}
}
Step 2: Add Tools (Layer 1)
public class ToolEnabledChatbot
{
private readonly ChatClient _client;
private readonly ToolRegistry _tools;
public async Task<string> ChatAsync(string userMessage)
{
var messages = new List<ChatMessage>
{
new SystemChatMessage("You are a helpful assistant with access to tools."),
new UserChatMessage(userMessage)
};
var options = new ChatCompletionOptions
{
ToolChoice = ChatToolChoice.CreateAutoChoice()
};
foreach (var tool in _tools.GetAll())
options.Tools.Add(tool);
// Tool-calling loop
while (true)
{
var response = await _client.CompleteChatAsync(messages, options);
if (response.Value.FinishReason == ChatFinishReason.ToolCalls)
{
// Add assistant message with tool calls
messages.Add(new AssistantChatMessage(response.Value));
// Execute each tool call
foreach (var toolCall in response.Value.ToolCalls)
{
var result = await _tools.ExecuteAsync(
toolCall.FunctionName,
toolCall.FunctionArguments.ToString());
messages.Add(new ToolChatMessage(toolCall.Id, result));
}
continue; // Let LLM process tool results
}
return response.Value.Content[0].Text;
}
}
}
Step 3: Add Memory (Layer 2)
public class MemoryEnabledAgent
{
private readonly ChatClient _client;
private readonly ToolRegistry _tools;
private readonly IChatStore _store;
public async Task<string> ChatAsync(string sessionId, string userMessage)
{
// Load conversation history
var history = await _store.LoadHistoryAsync(sessionId)
?? new List<ChatMessage>
{
new SystemChatMessage("""
You are a helpful assistant. You remember previous
conversations. Reference past context when relevant.
""")
};
history.Add(new UserChatMessage(userMessage));
var options = new ChatCompletionOptions
{
ToolChoice = ChatToolChoice.CreateAutoChoice()
};
foreach (var tool in _tools.GetAll())
options.Tools.Add(tool);
// Tool-calling loop (same as before)
var response = await ExecuteWithTools(history, options);
history.Add(new AssistantChatMessage(response));
// Trim history if too long (keep system + last N turns)
if (TokenCounter.Count(history) > 12_000)
history = SummarizeAndTrim(history);
// Persist
await _store.SaveHistoryAsync(sessionId, history);
return response;
}
private List<ChatMessage> SummarizeAndTrim(List<ChatMessage> history)
{
var system = history.First();
var recent = history.TakeLast(10).ToList();
// Keep system message + recent turns
return new List<ChatMessage> { system }
.Concat(recent).ToList();
}
}
Step 4: Add Planning (Layer 3)
public class PlanningAgent
{
public async Task<string> HandleComplexTask(
string sessionId, string userRequest)
{
// Step 1: Ask LLM to create a plan
var planPrompt = $"""
The user wants: {userRequest}
Create a step-by-step plan using your available tools.
Return JSON: {{"steps": [{{"action": "tool_name",
"params": {{}}, "purpose": "why"}}]}}
""";
var plan = await GetPlan(planPrompt);
// Step 2: Execute plan with progress tracking
var results = new List<StepResult>();
foreach (var step in plan.Steps)
{
try
{
var result = await _tools.ExecuteAsync(
step.Action, step.Params);
results.Add(new StepResult
{
Step = step,
Success = true,
Output = result
});
}
catch (Exception ex)
{
results.Add(new StepResult
{
Step = step,
Success = false,
Error = ex.Message
});
// Ask LLM to handle the failure
var recovery = await HandleFailure(step, ex, results);
if (recovery.ShouldAbort) break;
}
}
// Step 3: Summarize results for the user
return await SummarizeExecution(userRequest, results);
}
}
Step 5: Putting It All Together
// The complete agent — tools + memory + planning
public class FullAgent
{
public async Task<string> ProcessAsync(string sessionId, string message)
{
// Classify: simple chat or complex task?
var complexity = await ClassifyComplexity(message);
return complexity switch
{
TaskComplexity.Simple => await _memoryAgent.ChatAsync(sessionId, message),
TaskComplexity.MultiStep => await _planningAgent.HandleComplexTask(sessionId, message),
_ => await _memoryAgent.ChatAsync(sessionId, message)
};
}
private async Task<TaskComplexity> ClassifyComplexity(string message)
{
var response = await _client.CompleteChatAsync(new[]
{
new UserChatMessage($"""
Classify this request:
- "simple" if it's a question or single action
- "multi_step" if it requires multiple actions in sequence
Request: {message}
Return only: simple or multi_step
""")
});
return response.Value.Content[0].Text.Trim().Contains("multi_step")
? TaskComplexity.MultiStep
: TaskComplexity.Simple;
}
}
Pitfalls
1. Adding all layers at once
Each layer multiplies the complexity and debugging surface. Add tools first, validate they work, then add memory, validate persistence, then add planning. Debugging a planning failure is nearly impossible if you're also unsure about your tool execution.
2. Unbounded tool-calling loops
Without a loop limit, the LLM can call tools indefinitely — especially if tool results are ambiguous. Set a maximum iteration count (typically 5-10) and break out with a "I couldn't complete this task" message if exceeded.
3. Storing raw chat history forever
Chat history grows linearly. Tool call results are often verbose. After 20 turns, your context window is exhausted. Implement summarization — periodically compress old turns into a summary while keeping recent turns verbatim.
4. Planning without failure handling
Plans that assume every step succeeds are fragile. Real-world tool calls fail — APIs time out, permissions are denied, data doesn't exist. Every plan needs error handling: retry, skip, or re-plan from the point of failure.
Practical Takeaways
- Add layers incrementally. Chatbot → tools → memory → planning. Each layer should be stable before adding the next. This is evolutionary architecture, not big-bang design.
- The tool-calling loop is the core pattern. LLM requests a function call → your code executes it → result goes back to the LLM. Master this loop; everything else builds on it.
- Memory makes the agent useful. Users won't adopt a tool that forgets everything between sessions. Even simple session-based history dramatically improves the experience.
- Planning is optional for most use cases. Tools + memory handle 90% of agent scenarios. Add planning only when users genuinely need multi-step workflows — it adds significant complexity.
- Set limits everywhere. Max tool calls per turn, max history length, max plan steps. Without limits, agents loop, costs spiral, and users wait forever.