Building a Production Tool Registry

Problem Context

The naive approach to giving an LLM tools is "dump every JSON schema into the system prompt and hope for the best." This breaks at ~20 tools (context bloat, latency, hallucinated tool calls) and again at ~100 (cost, security boundaries, multi-tenant scoping). A tool registry is the runtime indirection that turns "a bag of functions" into "a versioned, scoped, observable capability catalogue".

What a Registry Actually Owns

• Schema — JSON schema, version, deprecation status.
• Implementation — the function reference (local, HTTP, gRPC, MCP server).
• Scope — which agents/tenants/roles can call it.
• Cost & latency budget — for routing decisions and circuit breakers.
• Side-effect class — read, write, destructive — for permission prompts and audit.

flowchart LR
    A[Agent] --> RT[Tool router]
    RT --> R[(Registry)]
    R --> M[Match by name+version]
    M --> S{Scope check<br/>tenant + role + class}
    S -- ok --> EX[Execute<br/>idempotent + traced]
    S -- deny --> ER[Typed error]
    EX --> AU[Audit log]
      

Just-in-Time Tool Selection

Don't expose all 100 tools to the LLM. Run a small retrieval step first:

async function selectTools(query: string, agent: AgentCtx): Promise<ToolSchema[]> {
  // 1. Vector search over tool descriptions
  const candidates = await registry.search(query, { topK: 12 });
  // 2. Filter by agent's allowed scope
  const allowed = candidates.filter(t => registry.canCall(agent, t));
  // 3. Take top 6 to fit context budget
  return allowed.slice(0, 6).map(t => t.schema);
}

Six tools in the prompt are debuggable; sixty are not. Even with frontier models, accuracy on tool selection drops sharply past ~15 schemas.

Versioning Without Pain

• Names are name@semver: send_email@1.2.0.
• Agents pin a major version; minor/patch upgrades are transparent.
• Deprecation is a flag, not a deletion — old agents keep working until they upgrade.
• Schema changes that drop a required field are a major bump, full stop.

Side-Effect Classes

Tag every tool with one of:

• read — idempotent, cacheable, safe to retry.
• write — mutates state; must accept an idempotency_key.
• destructive — irreversible (delete, send_email, charge_card); requires explicit user confirmation or a privileged role.

The router enforces the policy: read tools execute freely, write tools require an idempotency key, destructive tools require both a confirmation token and an audit entry.

MCP Makes the Registry Portable

The Model Context Protocol (MCP) standardises tool servers across vendors. Your registry can mount external MCP servers (GitHub, Slack, Linear, your own internal services) as namespaced tool collections — github.create_issue, slack.post — without rewriting client code. Treat each MCP server as a remote tool provider behind the same scope and audit pipeline.

Failure Modes

• Hallucinated tools — the LLM invents get_user_by_email when only get_user_by_id exists. Validate every call against the registry; reject unknown names with a typed error the agent can recover from.
• Schema drift — tool implementation diverges from its schema. Generate schemas from the implementation (Zod, Pydantic) — never hand-write.
• Cross-tenant tool leak — agent for tenant A calls a tool with tenant B's data. Scope must be checked at the registry, not by the tool itself.
• Retry storms on non-idempotent writes — the agent retries a payment after a timeout. Idempotency keys at the protocol level.

Production Checklist

• Tools live in a registry, not in the system prompt.
• JIT selection: vector-search → scope filter → top N into prompt.
• Every tool has a version, side-effect class, and audit entry.
• Schemas generated from code; calls validated against the registry; unknown tools = typed error, not 500.

Key Takeaways

• The registry pattern turns "a bag of functions" into a governable capability catalogue.
• JIT tool selection beats prompt-stuffing once you have more than a dozen tools.
• Versioning, scoping, and side-effect classes are how tools survive contact with real users.