Chat completions

POST /v1/chat/completions takes the OpenAI chat completions request and returns the OpenAI response shape. Existing OpenAI code works by pointing it at https://api.opinference.com/v1.

curl https://api.opinference.com/v1/chat/completions \
  -H "Authorization: Bearer $OP_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{
    "model": "openai/gpt-oss-120b",
    "messages": [
      {"role": "system", "content": "Answer in one sentence."},
      {"role": "user", "content": "What is a gateway?"}
    ],
    "max_completion_tokens": 200
  }'

Request

Field	Notes
`model`	Required. A model id from `GET /v1/models`
`messages`	Required. `content` is a string or an array of content parts
`stream`	`true` streams the reply. See Streaming
`max_completion_tokens`	Upper bound on output tokens. `max_tokens` is read too; when both are sent the larger wins. Reasoning models count their reasoning against it, so a small cap can leave no answer, which fails the request
`temperature`	Passed to the model
`tools`, `tool_choice`	See Tool calling
`response_format`	See Structured output
`models`	Other models that may answer. See Fallbacks
`provider`	Routing preferences. See Provider preferences
`op`	Retention, tags and session. See Data retention

The request is routed only to endpoints that support what it asks for. A request with tools goes only to endpoints that support tool calling, and a request whose prompt and output limit do not fit an endpoint's context window is not sent there.

Response

{
  "id": "gen_...",
  "object": "chat.completion",
  "created": 1790940214,
  "model": "openai/gpt-oss-120b",
  "choices": [
    {
      "index": 0,
      "message": {
        "role": "assistant",
        "content": "A gateway is a single entry point that forwards requests to other services."
      },
      "finish_reason": "stop"
    }
  ],
  "usage": {
    "prompt_tokens": 72,
    "completion_tokens": 18,
    "total_tokens": 90
  }
}

id is the generation id. It also arrives in the x-op-generation-id header and is the id shown on your usage and request pages.
model is the model that answered. Without models it is always the model you asked for.
A reasoning model may add message.reasoning and report usage.completion_tokens_details.reasoning_tokens. Reasoning tokens are output tokens and are billed as such.
usage.prompt_tokens_details.cached_tokens, when present, counts input tokens served from a prompt cache.

Errors

A failed request returns an HTTP error with a JSON envelope:

{
  "error": {
    "code": "no_endpoint_available",
    "message": "no endpoint serves model \"no-such/model\"",
    "kind": "terminal",
    "scope": "model",
    "type": "invalid_request_error",
    "param": null,
    "request_id": "req_..."
  }
}

Switch on code. See Errors for every code and which ones are worth retrying.

RequestCopy Anchor Link

ResponseCopy Anchor Link

ErrorsCopy Anchor Link

On this page

Request

Response

Errors