The waiter-and-kitchen analogy
A waiter cannot cook. A kitchen cannot speak to guests. Between them lives a slip — a piece of paper with checkboxes for exactly what the kitchen can make ("medium, no onions, side of fries"). The waiter speaks human. The kitchen reads slips. Neither needs to learn the other's job.
Function calling is the slip. The LLM speaks human, your code speaks API. The schema is the slip — a contract describing each available function, its parameters, and its types. The model fills in the slip; your code reads it and acts.
How a function call flows
- You send the model a request plus a list of available tools, each with a JSON Schema for its parameters.
- The model decides whether to call a tool. If yes, it returns a structured
tool_useblock with the function name and typed arguments. - Your code executes the function and returns the result back to the model.
- The model uses the result to compose the final answer (or call another tool).
Every modern LLM API supports this — Anthropic, OpenAI, Google, etc. The wire format differs slightly; the pattern is identical.
Why structured calls beat parsing prose
Before function calling, you had to parse free text: "the model said 'send an email to alice@…'". Brittle, regex-laden, sometimes wrong. With function calling:
- Typed arguments — strings, numbers, enums, nested objects. Validated against your schema.
- Atomic decisions — the model either calls or doesn't; no partial nonsense.
- Easy auditing — every action is a serialised event, replayable.
- Multi-tool composition — chain tools without re-prompting prose.
Designing tools that work
- Crisp names.
get_user_by_emailbeatslookup. The name is a hint to the model. - Tight descriptions. One sentence per tool, one sentence per parameter. The model reads this every call.
- Constrain with enums wherever possible (
status: "open" | "closed"). Eliminates a class of nonsense args. - Required fields explicit. Optional fields should default cleanly server-side.
- One tool, one job. Avoid "do_thing(action: 'create' | 'delete' | …)" — split into separate tools. The model's intent comes through more clearly.
Common pitfalls
- Tool sprawl. 50 tools and the model picks wrong. Group, scope, or split agents.
- Vague descriptions. "Tool that does stuff with users" — model guesses, badly.
- Free-text parameters where enums fit. Eternal source of "close-enough" bugs.
- Side effects without confirmation. A tool that sends emails, deletes data, charges cards — wrap in confirmation gates, not raw execution.
- Long-running tools — model waits, you blow the request budget. Make heavy operations async; return a job ID; let the model poll.
Variants you'll see
- Single tool call — model picks one tool at a time, you execute, send result back, model continues.
- Parallel tool calls — model emits multiple tool calls in one response (faster when calls are independent).
- Streaming tool calls — arguments streamed as the model decides them; gives you early visibility but adds complexity.
- Forced tool use — make the model call a specific tool (or any tool); useful when you require structured output.
Function calling vs structured outputs
They overlap but aren't the same:
- Structured outputs / JSON mode — the model's final answer is a JSON object matching a schema.
- Function calling — the model decides to invoke a tool with structured arguments mid-conversation.
Often you use both: tools to fetch / act, structured output to format the final response.
In one line
Function calling is what turns an LLM from a "smart writer" into a "smart user of your software." The schema is the contract; the loop is the runtime.