LLM Function Calling & Tool Use: Production Architecture

Claude Sonnet 4 has the strongest tool schema adherence in benchmarks — 95%+ schema compliance vs 88-92% for GPT-4o on complex nested schemas. It also supports parallel tool calls natively and has a 200K context window for injecting large tool results.

Alternatives: OpenAI GPT-4o, Google Gemini 2.5 Pro, Mistral Large 2

Define tool schemas in TypeScript/Zod and compile to JSON Schema at build time. This gives you: type safety in your tool executor (validate args before execution), auto-generated JSON Schema for the LLM API, and runtime validation that catches schema drift. Never write JSON Schema by hand — one typo makes the schema invalid and the LLM silently falls back to hallucinated args.

Alternatives: JSON Schema directly, OpenAI function definitions, Pydantic (Python)

When the LLM returns multiple tool_use blocks, execute them in parallel with Promise.all(). For 3 tools that each take 500ms sequentially (1500ms total), parallel execution takes 500ms — a 3x latency reduction. Use Promise.allSettled() instead of Promise.all() so a single tool failure doesn't abort all parallel calls.

Alternatives: Promise.allSettled() for error isolation, Sequential execution (fallback), Worker threads for CPU-intensive tools

Validate all LLM-supplied arguments before execution — models hallucinate invalid argument types or values ~5-10% of the time. `safeParse()` returns an error result (which gets returned as a tool_result error) instead of throwing and crashing your server. Validation cost: <0.1ms per call.

Alternatives: JSON Schema Validator (AJV), Manual argument checking

Anthropic's API requires a tool_result for every tool_use block. If you omit it, you get a 400 error. If a tool fails, return: `{type: 'tool_result', tool_use_id: '...', is_error: true, content: 'Error: API returned 404'}`. The model will then either try a different approach or explain to the user what went wrong — much better than a crash.

Alternatives: Retry the tool call once, Fallback to alternative tool

Without a loop guard, a model stuck in a tool-calling loop (e.g., search → not found → search → not found...) can run up $5-50 in a single user session. 10 tool calls covers 99% of legitimate use cases. At call 10, return whatever partial result you have with a note that the tool limit was reached.

Alternatives: LangGraph recursion_limit, Time-based session timeout, Token budget tracking

Tool results consume input tokens — at $3/M for Claude Sonnet 4, a 10K token tool result costs $0.03 per call, and with 10 tool calls per session that's $0.30 in tool-result tokens alone. Compress results: for web search, return {title, url, snippet} not full HTML. For API responses, extract only the fields specified in the tool's output_schema.

Alternatives: Raw HTML (for web search — avoid this), Full API response payload

Mitigation: The model repeatedly calls the same tool with slightly different arguments hoping for a different result. Implement a session-level counter (max 10 calls) and a deduplication check: if the same (tool_name, args_hash) pair appears twice in a session, skip execution and return 'duplicate call detected'. Break the loop by returning 'max tools reached' and ask the model to synthesize a best-effort answer from what it has.

Mitigation: Omitting a tool_result for any tool_use block causes Anthropic API 400 errors ('all tool_use blocks must have a corresponding tool_result'). Fix: use a try-catch wrapper around every tool executor that always returns a result — either success or error. Track outstanding tool_use IDs in a Map and assert all are fulfilled before sending the next message.

Mitigation: The LLM invents argument values not present in the user's input (e.g., making up a zip code for a weather lookup). Prevention: add explicit descriptions to schema properties explaining where the value should come from. If required arguments cannot be inferred from context, use `ask_user` tool pattern — define a tool that signals 'I need more information from the user' instead of hallucinating.

Mitigation: External tools (search, weather, CRM) have their own rate limits. When a tool returns a rate-limit error, the model often retries the same call — compounding the problem. Fix: implement exponential backoff in the tool executor (not visible to the model), cache identical tool calls within a session (same args → same result for 60 seconds), and return rate-limit errors as tool_result errors with a `retry_after` hint in the message.