meta-llama/llama-3.1-8b-instruct
Text · Typically $0.05225 per 1M input tokens · $0.0836 per 1M output tokens
Capabilities
StreamingTool callingJSON modeStructured output
Endpoint
POST /v1/chat/completions — OpenAI-compatible chat completions. Any OpenAI SDK works with the base URL changed.
Dependencies
No field depends on another.
Request
curl https://api.opinference.com/v1/chat/completions \
-H "Authorization: Bearer $OP_API_KEY" \
-H "Content-Type: application/json" \
-d '{
"model": "meta-llama/llama-3.1-8b-instruct",
"messages": [
{
"role": "user",
"content": "Say hello."
}
]
}'Response
One chat.completion. With stream: true, a stream of chat.completion.chunk events ending with data: [DONE] instead.
{
"id": "gen_01J9K2…",
"object": "chat.completion",
"model": "meta-llama/llama-3.1-8b-instruct",
"choices": [
{
"index": 0,
"message": {
"role": "assistant",
"content": "Hello!"
},
"finish_reason": "stop"
}
],
"usage": {
"prompt_tokens": 9,
"completion_tokens": 3,
"total_tokens": 12
}
}Input
A JSON body. A key not listed here is refused, and the error names it.
| Field | Description |
|---|---|
| modelstring · required | The id of this model. |
| messagesarray · required | The conversation so far, in the OpenAI chat format. |
| streamboolean | true streams server-sent events; false returns one chat.completion.Default: false |
| max_completion_tokensinteger | The most tokens the model may generate.At most 117964 |
| max_tokensinteger | Older name for max_completion_tokens. |
| temperaturenumber | Sampling randomness. Lower is more deterministic. |
| toolsarray | Functions the model may call, in the OpenAI tools format. |
| response_formatobject | Constrains the reply to JSON.type one of: json_object, json_schema |
| modelsstring[] | Other model ids to try, in order, if this one is unavailable. Up to 5. |
| providerobject | Routing preferences: region, zdr, data_collection, sort, max_price. |
| opobject | Gateway options: retention (may only tighten), tags, session_id. |
Output
A chat completion, in the OpenAI shape.
| Field | Description |
|---|---|
| id | The generation id. |
| object | Always "chat.completion". With stream: true, each event is a "chat.completion.chunk". |
| model | The public id of the model that answered. |
| choices[].message | The reply: role, content and any tool_calls. A chunk carries delta instead. |
| choices[].finish_reason | stop, length, tool_calls or content_filter. |
| usage | prompt_tokens, completion_tokens, total_tokens, and cached and reasoning token details. On the last chunk when streaming. |