Rate limits

Per-organization, per-model limits — and how to tell a limit you hit from capacity you did not cause.

Limits are set per organization, per model, by an Atlas operator. They are throughput protection, not a spend control: nothing caps your cumulative usage during the beta.

If normal traffic is being throttled, that is a limit due for review — not a policy you are violating. Ask your Atlas contact to raise it.

Three dimensions

Dimension	What it bounds
Requests per minute	How many requests you may start.
Tokens per minute	Input plus output tokens across the window.
Concurrent streams	How many streamed generations may be open at once, per model.

A new organization starts with conservative seeded defaults — 60 requests per minute, 120,000 tokens per minute, and 4 concurrent streams per model — which an operator raises for your workload. A missing configuration never means unlimited.

How tokens are counted at admission

Admission cannot know your real token count before the model runs, so it reserves an estimate and settles the difference afterwards:

Input is estimated from the request size using a byte heuristic, not a tokenizer. It deliberately errs high; an over-estimate is returned in full when the request settles.
Output is min(the cap you asked for, a per-model default ceiling). The cap is read from max_completion_tokens or the older max_tokens.

Set max_completion_tokens. If you set neither field, a per-model default ceiling applies, and it is usually far larger than what your request will actually generate. Every concurrent request then reserves that ceiling against your token-per-minute budget, and you hit the limit at a fraction of your real throughput.

Settling is idempotent and happens on every exit path, including an aborted stream. If a generation overruns its reservation, the stream is never truncated to make the numbers work — the difference is debited from the next window.

The 429 response

{
  "error": {
    "message": "...",
    "type": "rate_limit_error",
    "param": null,
    "code": "rate_limit_exceeded",
    "request_id": "req_01k4v9m2..."
  }
}

It carries the standard OpenAI headers, so an SDK's built-in backoff works unchanged:

Header	Example
`retry-after`	`2` (whole seconds, never below 1)
`retry-after-ms`	`1450`
`x-ratelimit-limit-requests`	`60`
`x-ratelimit-remaining-requests`	`0`
`x-ratelimit-reset-requests`	`12s`
`x-ratelimit-limit-tokens`	`120000`
`x-ratelimit-remaining-tokens`	`0`
`x-ratelimit-reset-tokens`	`44s`

retry-after-ms is not standard HTTP, but the OpenAI SDKs prefer it — it is the only way to express a sub-second hint, since retry-after: 0 means "immediately" to some intermediaries and "one second" to others after rounding.

These headers appear on the 429 itself, not on successful responses. To track headroom continuously, read your usage rather than scraping headers off every 200 — see Usage and cost.

429 is not 503

This is the distinction worth wiring into your alerting.

429 · rate_limit_exceeded

Your organization is over its configured limit for that model. Carries quota headers and a retry hint. Back off and retry; if it is frequent under normal load, ask for a higher limit.

503 · server_overloaded

Atlas capacity is the binding constraint, and you are within your limits. Carries no quota headers and no retry hint. Retry with your own backoff.

The overload response deliberately carries no rate-limit or retry headers at all. A hint present on one cause and absent on the other would let a client infer quota state from a capacity event — and every caller retrying on the same Atlas-invented beat is a thundering herd against a service that is already saturated.

Practically: a rising 429 rate is a limit to raise; a rising 503 rate is an Atlas capacity problem. If you are alerting on "5xx plus 429" as one series, you cannot tell those apart, and only one of them is actionable by you.

Other backpressure-adjacent responses

Status	Code	Meaning
`413`	`request_too_large`	The body is above the model's published `max_request_bytes`.
`503`	`server_shutting_down`	A deploy is draining. Retryable; in-flight streams get this as an in-band frame.
`503`	`backend_unavailable`	The serving backend could not be reached. Retryable.

See Error codes for the full set.

Staying under the limits

Cap your output. max_completion_tokens is the single most effective change; see the warning above.
Retry with jitter. Honor retry-after-ms when present, and add jitter of your own — the SDKs' default backoff already does.
Split traffic by model. Limits are per model, so moving batch or background work onto atlas-small-1 leaves your interactive budget on atlas-mid-1 intact.
Use separate keys per service so the per-key usage breakdown tells you which one is consuming the budget.