Skip to content

Tool Augmented AI

Everything about function calling, aka tool calling, aka tool use, in one place. How it works, how to use it with every major provider, how to build training data, how to fine-tune a model for it, and how to evaluate it.

Try it live: toolcalling.ai

Don't just read about function calling. Run it.

Bring your JSON schema and a prompt, run a real function call, and see which tools the model picks and with which arguments. Then go to the Arena: two anonymous models answer the same tool-calling task, and you vote for the better one. Every blind vote feeds the community leaderboard of which models really get tool calling right.

Open the workspace Vote in the Arena Leaderboard

  • Workspace: build and debug function calls; catch vague descriptions, missing enums, and guessed defaults before your users do.
  • Arena: blind, pairwise comparison of models on real tool-calling tasks.
  • Leaderboard: models ranked by community votes. Free to start.

Start here

  • Learn

    What function calling is, the request/response loop, and how every provider differs.

    What is Function Calling · Using it · Providers · MCP

  • Build training data

    The end-to-end recipe: coverage matrix, every edge and advanced case, dataset mix, quality gates.

    The Recipe · Training Data

  • Fine-tune and serve

    Chat templates, loss masking, LoRA, DPO/RL, then serving your model with tool parsing.

    Fine-Tuning Guide

  • Evaluate and benchmark

    Metrics, offline benchmarks, and live head-to-head voting at toolcalling.ai.

    Evaluation · Benchmarks

What is function calling?

Function calling connects a language model to the outside world: live data, APIs, databases, and actions. The model reads the user's message, chooses a function, and fills in its arguments as JSON. It can also turn unstructured or noisy input (natural language, web pages, documents) into structured data. The model never runs the function itself: it proposes a call, your code runs it, and the model uses the result. See What is Function Calling.

Recipe at a glance

  1. Scenario: pick a domain and the user's role. Expose only the tools that role may use (Best Practices).
  2. Schema: write 3-8 tools, plus 2-5 similar distractor tools (Schema).
  3. System prompt: tone, policies, defaults, confirmation rules, and out-of-scope behavior (System Instruction).
  4. Seed conversations: one per combination of the taxonomies below (The Recipe).
  5. Variants: 4-5 per seed (Variants).
  6. Validate: run the quality gates, balance the mix, and deduplicate (Training Data).
  7. Fine-tune and serve (Fine-Tuning Guide).
  8. Evaluate per slice, on held-out unseen tools (Evaluation), and benchmark live at toolcalling.ai.

Dataset taxonomy

To train a model for tool calling, cover every value of each taxonomy below in your conversation examples. The Recipe turns this into a coverage matrix.

1. Turn

  • single turn: one user message, one final answer.
  • multiple turns: follow-up questions and back-and-forth (Turn).

2. Function Call

  • single call: exactly one function call.
  • multiple parallel calls: several independent calls at once.
  • multiple consecutive calls: calls in sequence; a later one depends on an earlier result.
  • no call: nothing to call, or no tool supports the request.
  • Advanced: conditional calls, loops/pagination, retry, fallback, and cross-turn reuse (Function Call).

3. Adversarial User Cases (details)

  • Missing information
  • Happy path vs Unhappy path
  • User cancels request
  • User changes params
  • User asks params
  • User rejects providing params
  • Recover from failure
  • System pivoting
  • Toxic user
  • Noisy input
  • Conflicting info, Multi-intent, Out-of-role request
  • Prompt injection from the user or from a tool output
  • Destructive actions and duplicate requests

4. Tool Results

  • Success, Empty, Partial failure, Error with hint, Timeout / rate limit, Huge output (Tool Output)

5. Major Domains

  • Data Extraction: noisy natural language to JSON, where the data is buried in unstructured text.
  • Web Scraping: HTML/XML to JSON
  • Business and Sales
  • Finance
  • Education
  • Healthcare: scheduling, records lookup, triage support.
  • Text Analysis: sentiment analysis, entity extraction, text classification
  • Developer tools: code search, CI, ticketing, databases
  • Personal assistant: calendar, email, travel, smart home