Function Calling and Tool Use: Structured Outputs from LLMs

Beyond Text Output

Function calling lets LLMs do something they can't do alone: take structured action. Instead of completing your prompt with text, the model returns a structured call to a function you define — with typed arguments it extracted from the conversation. You execute the function, pass the result back, and the model continues. This is how agents work.

Tool Schema Design

The model selects tools and generates arguments based entirely on your schema descriptions. Bad descriptions mean wrong tool selection. Treat tool descriptions like API documentation — write them for someone who can't see your code.

sequenceDiagram
    participant User
    participant LLM
    participant App
    participant DB

    User->>LLM: "Show me orders from last week over $500"
    LLM->>App: tool_call: searchOrders({minAmount:500, dateFrom:"2025-01-13"})
    App->>DB: SELECT * FROM orders WHERE ...
    DB->>App: [{id: 123, amount: 750, ...}]
    App->>LLM: tool_result: [{id:123, ...}, {id:456, ...}]
    LLM->>User: "Found 2 orders: order #123 for $750..."

import OpenAI from 'openai';

const openai = new OpenAI();

// Define tools with precise descriptions — the model reads these
const tools: OpenAI.Chat.Completions.ChatCompletionTool[] = [
  {
    type: 'function',
    function: {
      name: 'searchOrders',
      description: `Search customer orders by date range and amount.
        Use when the user asks about orders, purchases, or transactions.
        Returns an array of order objects with id, amount, status, and date.`,
      parameters: {
        type: 'object',
        properties: {
          minAmount: {
            type: 'number',
            description: 'Minimum order amount in USD. Omit if no minimum.',
          },
          maxAmount: {
            type: 'number',
            description: 'Maximum order amount in USD. Omit if no maximum.',
          },
          dateFrom: {
            type: 'string',
            description: 'Start date in ISO 8601 format (YYYY-MM-DD). Omit for all dates.',
          },
          dateTo: {
            type: 'string',
            description: 'End date in ISO 8601 format (YYYY-MM-DD). Omit for today.',
          },
          status: {
            type: 'string',
            enum: ['pending', 'processing', 'shipped', 'delivered', 'cancelled'],
            description: 'Filter by order status. Omit to return all statuses.',
          },
        },
        required: [],  // All params optional — model decides what to include
        additionalProperties: false,
      },
    },
  },
];

The Conversation Loop

type Message = OpenAI.Chat.Completions.ChatCompletionMessageParam;

// Tool implementations
const toolHandlers: Record<string, (args: unknown) => Promise<unknown>> = {
  searchOrders: async (args) => {
    // CRITICAL: validate args before executing
    const parsed = SearchOrdersSchema.parse(args); // Zod validation
    return db.orders.findMany({
      where: {
        amount: { gte: parsed.minAmount, lte: parsed.maxAmount },
        createdAt: { gte: parsed.dateFrom, lte: parsed.dateTo },
        status: parsed.status,
      },
    });
  },
};

async function runAgentLoop(userMessage: string): Promise<string> {
  const messages: Message[] = [
    { role: 'system', content: 'You are a helpful order management assistant.' },
    { role: 'user', content: userMessage },
  ];

  // Loop until the model stops calling tools
  while (true) {
    const response = await openai.chat.completions.create({
      model: 'gpt-4o',
      messages,
      tools,
      tool_choice: 'auto',  // model decides when to call tools
    });

    const message = response.choices[0].message;
    messages.push(message);

    // No tool calls → final text response
    if (!message.tool_calls) {
      return message.content ?? '';
    }

    // Execute each tool call
    for (const toolCall of message.tool_calls) {
      const args = JSON.parse(toolCall.function.arguments);
      const result = await toolHandlers[toolCall.function.name]?.(args) ?? 'Tool not found';

      // Append tool result to conversation
      messages.push({
        role: 'tool',
        tool_call_id: toolCall.id,
        content: JSON.stringify(result),
      });
    }
    // Loop continues — model sees tool results and decides next step
  }
}

Parallel Tool Calls

// When the model calls multiple tools in one turn, execute them in parallel
// This is a performance optimisation — the model already decided to call all of them

const response = await openai.chat.completions.create({
  model: 'gpt-4o',
  messages,
  tools,
});

const toolCalls = response.choices[0].message.tool_calls ?? [];

// Execute all tool calls in this turn concurrently
const toolResults = await Promise.all(
  toolCalls.map(async (toolCall) => {
    const args = JSON.parse(toolCall.function.arguments);
    const result = await toolHandlers[toolCall.function.name]?.(args);
    return {
      role: 'tool' as const,
      tool_call_id: toolCall.id,
      content: JSON.stringify(result),
    };
  })
);

messages.push(response.choices[0].message, ...toolResults);

Controlling Tool Use

// tool_choice controls whether and which tools the model uses

// auto — model decides (default)
tool_choice: 'auto'

// none — never use tools (text-only response)
tool_choice: 'none'

// required — must call at least one tool
tool_choice: 'required'

// specific function — must call this exact function
tool_choice: { type: 'function', function: { name: 'searchOrders' } }

// Use 'required' for structured data extraction tasks:
// "Extract the key details from this support ticket"
// + required tool: extractTicketDetails
// Guarantees structured output, never returns unstructured text

Pitfalls

Trusting model-generated arguments directly

Never execute a tool call with unvalidated model output. The model can hallucinate argument values, generate SQL fragments, produce out-of-range numbers, or construct paths that traverse directories. Always validate against a schema (Zod, Joi, AJV) before passing to your handler. Treat LLM-generated arguments like untrusted user input.

Overloading a single tool

A callDatabase(sql: string) tool that accepts raw SQL is a security and reliability disaster. Use narrow, typed tools with explicit parameters. The model should be selecting typed parameters — not constructing strings that your code interprets.

Missing tool call IDs in history

Every tool_calls entry in the assistant message must have a corresponding tool message with the matchingtool_call_id. Missing a tool result causes an API error on the next turn. If a tool fails, still append a tool message with an error description so the model can gracefully handle it.