> ## Documentation Index
> Fetch the complete documentation index at: https://docs.booleinference.com/llms.txt
> Use this file to discover all available pages before exploring further.

# API Rate Limits for the Boole AI Cloud: 60 req/sec

> Understand default rate limits on the Boole AI cloud API, handle 429 errors gracefully, and request a higher limit for high-volume workloads.

The Boole AI cloud API enforces rate limits to ensure fair, stable access for all users. By default, each API key is limited to **60 requests per second**. If your application requires higher throughput, you can request an increase — limits are not a hard ceiling for all workloads.

## Default Limits

| Limit                             | Default                            |
| --------------------------------- | ---------------------------------- |
| Requests per second (per API key) | 60                                 |
| Concurrent connections            | Unlimited                          |
| Max tokens per request            | Model-dependent (typically 32,768) |
| Max requests per day              | Unlimited                          |

Rate limits apply per API key, not per account. If you need more than 60 req/sec across a single workload, you can either request a higher limit on one key or distribute traffic across multiple keys.

## Rate Limit Errors

When a request exceeds your limit, the API responds with **HTTP 429 Too Many Requests**:

```json theme={null}
{
  "error": {
    "message": "Rate limit exceeded. Max 60 requests per second.",
    "type": "rate_limit_error",
    "code": 429
  }
}
```

Do not treat a 429 as a fatal error. It signals that you should back off briefly and retry — the API does not penalize retries.

## Handling Rate Limits

Implement exponential backoff with jitter so your application recovers automatically from bursts that exceed the limit:

```python retry.py theme={null}
import time
import random

def call_with_retry(client, **kwargs):
    for attempt in range(5):
        try:
            return client.chat.completions.create(**kwargs)
        except Exception as e:
            if "429" in str(e):
                wait = (2 ** attempt) + random.uniform(0, 1)
                print(f"Rate limited. Retrying in {wait:.2f}s (attempt {attempt + 1}/5)")
                time.sleep(wait)
            else:
                raise
    raise RuntimeError("Max retries exceeded after 5 attempts")
```

The backoff schedule for this implementation:

| Attempt | Base wait | Max wait (with jitter) |
| ------- | --------- | ---------------------- |
| 1       | 2s        | \~3s                   |
| 2       | 4s        | \~5s                   |
| 3       | 8s        | \~9s                   |
| 4       | 16s       | \~17s                  |
| 5       | —         | RuntimeError raised    |

<Tip>
  Use **batching** and **connection pooling** to stay within the limit more easily. Sending multiple messages inside a single `chat/completions` call counts as one request against your rate limit, not one per message.
</Tip>

## Requesting Higher Limits

If 60 req/sec is not enough for your workload, email [support@boole.ai](mailto:support@boole.ai) with:

* **Your account email**
* **The model(s)** you need higher limits for
* **Your expected request volume** (req/sec and daily total)
* **A brief description** of your use case

The team typically responds within one business day. Increases can be applied to an existing key or a new dedicated key, depending on your preference.

<Note>
  Local binary deployments have **no rate limits**. Throughput is constrained only by your GPU hardware — for example, 312 tokens/sec on Llama 3.3 70B on a single NVIDIA A10 GPU.
</Note>