Rate Limit

You might say

I'm getting an error because I called the API too often. What does that limit mean?

A service limit on requests, tokens, or concurrent tasks within a period of timeAfter a limit is reached, a request may wait, slow down, or return 429. The exact limits vary by service, model, and account, so distinguish request, token, and concurrency limits.

Know first

API

How work reaches a queue, a concurrency limit, and a 429

Queue size, concurrency, and wait time are illustrative.

500 summarization tasks#1#2#3#4#5#6#7#8

worker 1Requesting

worker 2Requesting

worker 3Requesting

3 tasks running at once497 tasks remain in the queue

How requests wait and retry within limits

Enter a queue：The product queues work and runs only some tasks at once; that controls concurrency.

A limit returns 429：429 commonly means there are too many requests and an immediate retry is not appropriate.

Wait as instructed：Follow Retry-After when supplied, or use documented backoff while keeping the queue controlled.

Quick checkChoose the best answer

Rapid clicks on Regenerate hit a rate limit. What should the UI do next?

Stop immediate retries, explain when to retry, and prevent duplicate submissionRetry automatically every 100ms until it succeedsHide the error and let people click more

You can say this to an AI Agent

When Regenerate is rate-limited, read any available wait information, disable duplicate clicks, and show when the next attempt is allowed; re-enable it then. Simulate one 429 and confirm only one request is sent, the wait is visible, and retry works after it ends.

Learn next

1Token Usage & Cost→2Agent Loop→3Context Window→