Voice Agent with Tool-Calling

What you build: a hosted-AI voice agent with a custom tool (track_order). When a caller asks about their order, the agent decides to call the tool, Voicebip POSTs a tool.invocation request to your endpoint, your server hits your orders API and returns JSON, and the agent speaks it — all within the call.

Caller: "Where's my order AC-4821?"
   │
   ▼
Agent decides → POST agent.webhook_url ──▶ your server
                { event_type:"tool.invocation",       │  hits your orders API
                  tool:{ name:"track_order",           │
                         arguments:{order_id:"AC-4821"}}}  ← { "status":"shipped", "eta":"tomorrow" }
   ┌──────────────────────────────────────────────────┘
   ▼
Agent speaks: "Your order shipped and arrives tomorrow."

Who it’s for: any developer with an existing backend API (orders, bookings, account balance) who wants the phone agent to act on live data, not just recite static docs. This is the biggest leap in capability — the difference between a demo and a product.

Prerequisites

• A hosted-AI agent (see Inbound support + KB for the base build).
• An HTTPS webhook_url on the agent to receive tool.invocation and return results. It must be a public address (Voicebip’s SSRF guard rejects private/loopback/link-local hosts) and respond within a few seconds — the caller is waiting.
• Your workspace signing secret (dashboard → Workspace → Signing Secret) to verify signatures.

Step 1 — Define the tool

tool_definitions is an OpenAI-compatible JSON array. Define one custom function:

1
[
2
  {
3
    "type": "function",
4
    "function": {
5
      "name": "track_order",
6
      "description": "Look up the status and ETA of a customer's order by its ID.",
7
      "parameters": {
8
        "type": "object",
9
        "properties": {
10
          "order_id": { "type": "string", "description": "The order reference, e.g. AC-4821" }
11
        },
12
        "required": ["order_id"]
13
      }
14
    }
15
  }
16
]

Voicebip validates the model’s arguments against this JSON-Schema before calling your webhook — if the model hallucinates a bad shape, your endpoint is never hit and the model is asked to self-correct.

Alongside your custom tool, the agent always has these built-in tools it can call with no webhook — use them in your prompt:

Step 2 — Create the agent with the tool + webhook

tool_definitions is a string (stringified JSON). Custom tools dispatch to webhook_url, so set both. Instruct the model when to use the tool in system_prompt:

1
{
2
  "display_name": "Acme Orders Agent",
3
  "ai_provider": "hosted",
4
  "ai_model": "<model_id>",
5
  "tts_voice": "<voice_id>",
6
  "greeting_text": "Hi, this is Acme. Ask me about your order any time.",
7
  "webhook_url": "https://you.example.com/tools",
8
  "system_prompt": "You help callers track orders. When a caller gives an order ID, call track_order. Read back the status and ETA in one sentence. If they have no ID or ask for a person, call transfer_call.",
9
  "tool_definitions": "[{\"type\":\"function\",\"function\":{\"name\":\"track_order\",\"description\":\"Look up the status and ETA of an order by its ID.\",\"parameters\":{\"type\":\"object\",\"properties\":{\"order_id\":{\"type\":\"string\"}},\"required\":[\"order_id\"]}}}]"
10
}

Step 3 — Handle tool.invocation on your server

When the model calls track_order, Voicebip POSTs to your webhook_url (HMAC-signed, same scheme as everything else). Verify the signature, run the tool, and return a JSON body — that body is fed straight back to the model, which turns it into speech.

1
// Express — tool dispatcher
2
import express from "express";
3
import crypto from "crypto";
4
5
const app = express();
6
app.use(express.raw({ type: "application/json" }));
7
8
app.post("/tools", async (req, res) => {
9
  // 1. verify signature (timestamp . raw body)
10
  const sig = req.header("X-Voicebip-Signature");
11
  const ts  = req.header("X-Voicebip-Timestamp");
12
  const mac = crypto.createHmac("sha256", process.env.VOICEBIP_SIGNING_SECRET)
13
    .update(`${ts}.${req.body}`).digest("hex");
14
  if (sig !== `sha256=${mac}`) return res.sendStatus(401);
15
16
  const evt = JSON.parse(req.body);
17
  if (evt.event_type !== "tool.invocation") return res.sendStatus(204);
18
19
  // 2. dispatch by tool name
20
  const { name, arguments: args } = evt.tool;
21
  if (name === "track_order") {
22
    const order = await myOrdersApi.get(args.order_id);   // your backend
23
    return res.json(order ? { status: order.status, eta: order.eta }
24
                          : { error: "not_found" });
25
  }
26
  res.json({ error: "unknown_tool" });
27
});
28
app.listen(3000);

The inbound tool.invocation request body:

1
{
2
  "event_type": "tool.invocation",
3
  "call_id": "call_5xk…",
4
  "agent_id": "agt_…",
5
  "workspace_id": "ws_xyz123",
6
  "tool": {
7
    "tool_call_id": "call_9a…",
8
    "name": "track_order",
9
    "arguments": { "order_id": "AC-4821" }
10
  }
11
}

Your response is just the tool’s result as JSON — return 200 with any JSON object:

1
{ "status": "shipped", "eta": "tomorrow" }

Step 4 — Recursive tool loops

The agent can chain tools: call track_order, then based on the result decide to call transfer_call. Voicebip runs the tool loop up to a bounded depth per turn so a misbehaving prompt can’t loop forever. Design tools to resolve in one or two hops.

Test it

tool.invocation is dispatched to your webhook_url in a live call, so it isn’t part of the POST /v1/webhooks/test catalogue (which covers call.* / message.* lifecycle events). Two ways to exercise it:

1. Chat in the playground and watch the model decide to call the tool:

   $
   curl -X POST https://api.voicebip.com/v1/agents/agt_…/playground \
   >
     -H "Authorization: Bearer pk_xxx" -H "Content-Type: application/json" \
   >
     -d '{ "message": "Where is order AC-4821?" }'
2. Unit-test your handler by POSTing a sample tool.invocation body (above) to your own endpoint with a valid signature.

Troubleshooting

Next steps

• Ground the agent’s answers in your docs too → Inbound support + KB.
• Prefer to run your own model end-to-end? → BYOM.
• Route different intents to different specialised agents → Multi-agent routing.