OpenAI SDK
Use the official OpenAI SDK for Python or TypeScript.
The API speaks the OpenAI chat completions format. Point the official SDK at our base URL and use your OP Inference key. Nothing else changes.
pip install openaiChat
import os
from openai import OpenAI
client = OpenAI(
base_url="https://api.opinference.com/v1",
api_key=os.environ["OP_API_KEY"],
)
reply = client.chat.completions.create(
model="openai/gpt-oss-120b",
messages=[{"role": "user", "content": "Say hello."}],
)
print(reply.choices[0].message.content)Streaming, tool calling and structured output work as they do with OpenAI. See Streaming, Tool calling and Structured output.
Our own fields
Fields the OpenAI SDK does not know, such as provider and op, go in extra_body in Python. In
TypeScript, pass them in the request object.
import os
from openai import OpenAI
client = OpenAI(
base_url="https://api.opinference.com/v1",
api_key=os.environ["OP_API_KEY"],
)
reply = client.chat.completions.create(
model="openai/gpt-oss-120b",
messages=[{"role": "user", "content": "Say hello."}],
extra_body={
"provider": {"sort": "price"},
"op": {"retention": "zdr"},
},
)
print(reply.choices[0].message.content)Errors and retries
The SDK raises its usual exceptions: RateLimitError for a 429, AuthenticationError for a
401, and so on. The body's code tells you exactly what happened. See
Errors.
import os
import openai
from openai import OpenAI
client = OpenAI(
base_url="https://api.opinference.com/v1",
api_key=os.environ["OP_API_KEY"],
)
try:
client.chat.completions.create(
model="no-such/model",
messages=[{"role": "user", "content": "Say hello."}],
)
except openai.APIStatusError as e:
print(e.status_code, e.body["code"])The SDK retries 429 and 5xx responses on its own, twice by default, and waits for
Retry-After when we send it. Set max_retries on the client to change that.
Media
Media routes answer with a job, not with the OpenAI response shape, so client.images.generate()
and client.audio.speech.create() do not work. Call the media routes over HTTP instead. See
How media jobs work.