Fallbacks
Let other models answer when your first choice cannot.
Same model, by default
Every request names one model. If the endpoint serving it fails before the reply starts, the
request can move to another endpoint serving the same model. It never moves to a different
model on its own, and no endpoint ever answers under another model's name.
Many models have one endpoint, so this failover is not always available. To accept other models, list them.
Accept other models with models
curl https://api.opinference.com/v1/chat/completions \
-H "Authorization: Bearer $OP_API_KEY" \
-H "Content-Type: application/json" \
-d '{
"model": "openai/gpt-oss-120b",
"models": ["openai/gpt-oss-120b", "openai/gpt-oss-20b"],
"messages": [{"role": "user", "content": "Say hi."}]
}'modelsholds up to 5 model ids.modelis always acceptable too.- Every model you list must be one your key may use, or the whole request is refused with
403 forbiddenbefore anything runs. - The response's
modelfield names the model that answered, and you pay that model's price.
Any model in the list can serve the request, including the first attempt. The list's order and
the model field are not treated as a preference: the gateway picks among all the listed
models' endpoints by its usual ranking. List only models whose answers and prices you accept.
Turn failover off
"provider": {"allow_fallbacks": false} sends the request to one endpoint and returns its answer
or its error. See Provider preferences.
Failover and streaming
Failover is possible only before the first token. See Streaming.