Tool Use

Also known as: function calling, tool calling, actions

The model calls external functions or APIs through structured schemas, grounding it in real data and actions.

TrackAgentic systems FamilyCore loop Maturity Established ChangesTool/action

Problem

A language model only knows what was in its training data and its prompt. It can’t check today’s order status, query a database or send an email. Asked anyway, it either refuses or invents a plausible answer.

Real tasks need live data and side effects the model cannot produce on its own.
Free-text output is hard for code to parse reliably.
Every tool you expose widens what can go wrong, from wrong arguments to unwanted actions.
Too many tools in the prompt make the model pick the wrong one more often.

Solution

Describe each tool to the model as a name, a one-line purpose and a JSON schema for its arguments. The model replies either with an answer or with a structured tool call. The runtime, not the model, validates the arguments against the schema, checks permissions, executes the call and returns the result to the model as a new message. The model then decides whether it needs another call or can answer.

Diagram

The model only proposes calls. The runtime validates and executes them, which is where permissions and logging live.

When to use

Use it when

The answer depends on live or private data (orders, tickets, inventory)
The task needs an action in another system (create, update, send)
Exact computation matters, such as math, dates or currency
Downstream code needs structured output it can trust

Avoid it when

The model already has everything it needs in the prompt
The tool would take an irreversible action with no approval step
You'd be exposing dozens of tools at once; group them or route first

Trade-offs

Latency

One extra model turn per tool call

Cost

Tool schemas add prompt tokens on every call

Complexity

Schemas, validation and error handling

Reliability

Grounded answers instead of guesses

Gains

Answers grounded in real, current data
Structured outputs that downstream code can parse
A natural place to enforce permissions and audit every action

Costs

Tool descriptions need the same care as prompts
Each tool is a new failure surface (timeouts, bad arguments, rate limits)
Write actions need extra controls, such as approval steps and idempotency

Worked example

Scenario
An internal IT helpdesk agent with three tools, get_device(user), reset_mfa(user) and open_ticket(summary, priority).

Employee“My authenticator app stopped working after I got a new phone.”
ModelCalls get_device(user="a.shah") to check enrollment before acting.
RuntimeValidates the call against the schema, runs it, returns {mfa_device: "iPhone 12", enrolled: 2024-03-02}.
ModelSees the device mismatch and calls reset_mfa(user="a.shah"). The runtime’s policy marks reset_mfa as sensitive and asks the employee to confirm first.
ModelExplains how to re-enroll on the new phone. No ticket is needed.
Audit logRecords both calls with arguments and results. A review shows reset_mfa is the most-called write tool, which justifies its confirmation step.

Failure modes

Failure	Symptom	Mitigation
Hallucinated arguments	Model invents an ID or email that looks valid	Validate against the schema and look up IDs before writes
Wrong tool chosen	Similar tools get confused, e.g. search_orders vs get_order	Write distinct one-line descriptions; merge or route when tools overlap
Unhandled tool errors	Timeouts or 500s make the agent loop or give up silently	Return errors to the model as readable messages with a retry limit
Unsafe side effects	A write runs twice or without consent	Idempotency keys and confirmation for sensitive tools