Tool calling | OP Inference docs

Send tools in the OpenAI format. The request is routed only to endpoints that support tool calling. A model's page at opinference.com/models lists tools among its parameters when the model supports them. A model with no such endpoint is refused with 404 no_endpoint_available.

curl https://api.opinference.com/v1/chat/completions \
  -H "Authorization: Bearer $OP_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{
    "model": "openai/gpt-oss-120b",
    "messages": [{"role": "user", "content": "What is the weather in Paris?"}],
    "tools": [{
      "type": "function",
      "function": {
        "name": "get_weather",
        "description": "Get the current weather for a city",
        "parameters": {
          "type": "object",
          "properties": {"city": {"type": "string"}},
          "required": ["city"]
        }
      }
    }],
    "tool_choice": "auto"
  }'

The model answers with tool_calls and finish_reason: "tool_calls":

{
  "choices": [{
    "index": 0,
    "message": {
      "role": "assistant",
      "content": null,
      "tool_calls": [{
        "id": "call_...",
        "type": "function",
        "function": {"name": "get_weather", "arguments": "{\"city\": \"Paris\"}"}
      }]
    },
    "finish_reason": "tool_calls"
  }]
}

Run the function yourself, then send the result back as a tool message with the matching tool_call_id, after the assistant message that asked for it:

import json, os
from openai import OpenAI

client = OpenAI(base_url="https://api.opinference.com/v1", api_key=os.environ["OP_API_KEY"])
tools = [{
    "type": "function",
    "function": {
        "name": "get_weather",
        "description": "Get the current weather for a city",
        "parameters": {
            "type": "object",
            "properties": {"city": {"type": "string"}},
            "required": ["city"],
        },
    },
}]
messages = [{"role": "user", "content": "What is the weather in Paris?"}]

reply = client.chat.completions.create(model="openai/gpt-oss-120b", messages=messages, tools=tools)
call = reply.choices[0].message.tool_calls[0]
messages.append(reply.choices[0].message)
messages.append({
    "role": "tool",
    "tool_call_id": call.id,
    "content": json.dumps({"city": json.loads(call.function.arguments)["city"], "sky": "clear"}),
})

final = client.chat.completions.create(model="openai/gpt-oss-120b", messages=messages, tools=tools)
print(final.choices[0].message.content)

Tool definitions and earlier tool calls count as input tokens on every turn.