Rate limits

Limits todayCopy Anchor Link

The gateway applies no per-key request rate limit today, and there is no setting for one.

What does limit you today:

• Your prepaid credit and any spend cap. See Credits and billing and Key restrictions and spend caps.
• The capacity of the endpoints serving a model. When an endpoint rate limits a request before the reply starts, the gateway moves the request to another endpoint serving the same model. You see 429 only when no endpoint could take it.

The 429 responseCopy Anchor Link

{
  "error": {
    "code": "rate_limited",
    "message": "...",
    "kind": "rate_limit",
    "type": "rate_limit_error",
    "retry_after_ms": 2000
  }
}

When the gateway knows how long to wait, the response carries both retry_after_ms in the body and a Retry-After header in whole seconds, rounded up. The OpenAI SDKs and most HTTP clients read Retry-After and back off on their own.

Backing offCopy Anchor Link

• Wait for Retry-After when it is present.
• Otherwise retry with exponential backoff and jitter, starting at one second.
• Send an Idempotency-Key with media submits, so a retry never starts a second job. See Idempotency.