Rate Limits

How fast your workspace can send requests and tokens, how the limits rise, and what to do when you reach one.

Rate limits control how quickly your workspace can send requests and tokens. They exist so a sudden surge, or a loop in your code, can't flood the service, and they rise as you buy credit. They are separate from how much credit you have and from your own spend limit: each of those fails with its own error, described below.

• Requests per minute: how many requests your workspace can send in a minute.
• Input tokens per minute: how many tokens your requests can send in a minute, attached files included.
• Output tokens per minute: how many tokens of answers your workspace can receive in a minute. This limit applies only where a figure is shown for it below.
• Concurrent requests: how many of your requests can be running at the same time.
• Daily and monthly ceilings can also apply.

Loading the current limits…

Limits count everything your workspace sends: every API key and every member, together. An organization workspace shares one set of limits across the whole organization, and your Personal workspace has its own. A request counts against the workspace it was made in.

A rate limit controls how quickly requests can be processed; credit determines how much usage is available to pay for. They are separate, and so is the spend limit you can set on your own account.

The API answers 429 with a Retry-After header, in seconds, and retry_after_ms in the body. The request wasn't processed, so you aren't charged for it. Wait that long, then send it again.

• Spread requests out instead of sending them in a burst.
• Retry with exponential backoff and a little random jitter. The full guidance is on Errors & Retries.
• Queue work on your side, and cap how many requests you run at once.
• If you reach a limit regularly, the Production Readiness card on Usage & Insights shows your recent peak against each limit.

• Under Your limits on the Billing page, when your workspace has a tier.
• GET /v1/quotas returns the limits and usage for the current minute, day and month, and GET /v1/metrics reports token usage over a period. Both need a key with the metrics scope, which a key created without scopes has.