> ## Documentation Index
> Fetch the complete documentation index at: https://docs.booleinference.com/llms.txt
> Use this file to discover all available pages before exploring further.

# Troubleshoot Boole AI: Startup, Auth, and Performance

> Solutions for the most common Boole AI issues — local binary startup failures, API authentication errors, rate limits, and performance problems.

This guide covers the most common issues customers encounter with Boole AI, including local binary startup problems, API authentication errors, and performance troubleshooting. Work through the relevant section below to resolve your issue quickly. If nothing here applies, reach out to [support@boole.ai](mailto:support@boole.ai).

## Local Binary Issues

<Accordion title="The binary won't start">
  If the binary exits immediately or fails to initialize, check the following:

  * **CUDA driver version** — Boole requires CUDA 12.0 or later. Run `nvidia-smi` and confirm the driver version shown in the top-right corner meets this minimum. If not, update your NVIDIA driver.

  * **GPU visibility** — confirm your GPU is visible to the system:

    ```bash theme={null}
    nvidia-smi
    ```

    If this command fails or shows no devices, your GPU driver is not installed correctly or the GPU is not accessible in your current environment (e.g. a container without `--gpus all`).

  * **Disk space** — weights are embedded in the binary and staged to a local cache on first run. Ensure you have enough free disk space for the model you're loading. Llama 3.3 70B requires approximately 40 GB.
</Accordion>

<Accordion title="Slow cold start / high latency">
  Cold start time is expected to be approximately **380 ms** under normal conditions. On the very first invocation after a reboot or cache clear, the runtime must page weights from disk into GPU memory, which adds roughly **104 ms** for weight page-in. Subsequent requests within the same session are significantly faster once weights are resident in GPU memory.

  If cold start consistently exceeds 400 ms, check:

  * Storage throughput — NVMe drives produce the fastest page-in times.
  * Other processes competing for GPU memory, which can force weights to be evicted and reloaded.
  * Whether you're running on a network-mounted filesystem, which can substantially slow weight reads.
</Accordion>

<Accordion title="Port already in use">
  By default, the local server binds to port `8000`. If another process is already using that port, startup will fail with an `address already in use` error.

  Use the `--port` flag to specify a different port:

  ```bash theme={null}
  boole serve --model llama-3.3-70b-instruct --port 8001
  ```

  Then update your client's `base_url` accordingly:

  ```python theme={null}
  base_url="http://localhost:8001/v1"
  ```
</Accordion>

<Accordion title="Out of GPU memory">
  70B-parameter models require approximately **40 GB of VRAM**. If your GPU has less memory available, the runtime will fail with an out-of-memory error during weight loading.

  Two options:

  1. **Switch to a smaller model** — 7B and 13B models fit comfortably on GPUs with 16–24 GB VRAM.
  2. **Enable INT4 quantization** — reduces VRAM usage significantly at a small quality trade-off:

     ```bash theme={null}
     boole serve --model llama-3.3-70b-instruct --quantize int4
     ```

  <Note>
    INT4 quantization is applied by the nightly compiler and is optimized for throughput. Quality degradation is minimal on most task types.
  </Note>
</Accordion>

***

## API Authentication Issues

<Accordion title="401 Unauthorized">
  A `401` response means the server could not authenticate your request. Check the following:

  * **Header format** — Boole requires the standard Bearer token format. Ensure your request includes:

    ```
    Authorization: Bearer YOUR_API_KEY
    ```

    Do not use `API-Key`, `X-API-Key`, or any other header name.

  * **Key validity** — verify the key has not been revoked. Open **Settings → API Keys** in the Boole dashboard to confirm its status.

  * **No extra whitespace** — copy your key carefully; a trailing space or newline will cause authentication to fail.
</Accordion>

<Accordion title="API key not working">
  If your key appears valid but requests still fail, confirm:

  * **Base URL** — you must use `https://api.boole.dev/v1` exactly. A common mistake is omitting `/v1` or using a different subdomain.

    ```python theme={null}
    from openai import OpenAI

    client = OpenAI(
        base_url="https://api.boole.dev/v1",
        api_key="YOUR_BOOLE_API_KEY",
    )
    ```

  * **Environment mismatch** — if you have keys for multiple accounts or environments, ensure you're using the key that corresponds to your active account.
</Accordion>

<Accordion title="Key was working, now returning 401">
  If a key that previously worked is now returning `401`, it has likely been revoked — either manually or as a result of a security rotation.

  To generate a replacement key:

  <Steps>
    <Step title="Open Settings">
      Navigate to **Settings → API Keys** in the Boole dashboard.
    </Step>

    <Step title="Revoke the old key">
      Confirm the original key shows a **Revoked** status.
    </Step>

    <Step title="Create a new key">
      Click **New API Key**, copy the value immediately (it is only shown once), and update your application configuration.
    </Step>
  </Steps>
</Accordion>

***

## Performance Issues

<Accordion title="High latency / slow responses">
  If you're seeing unexpectedly high latency on the cloud API, first check for active incidents:

  **[status.booleinference.com](https://status.booleinference.com)**

  Expected time-to-first-token (TTFT) for Llama 3.3 70B is approximately **62 ms** under normal load. If you're consistently seeing higher values and the status page shows no incidents, include your account email, the model you're using, and example request timestamps when contacting [support@boole.ai](mailto:support@boole.ai).
</Accordion>

<Accordion title="Getting 429 errors">
  A `429 Too Many Requests` response means you've exceeded the default rate limit of **60 requests per second** per API key.

  Recommended mitigations:

  * **Implement exponential backoff** — retry failed requests with increasing delays:

    ```python theme={null}
    import time
    import random

    def request_with_backoff(client, **kwargs):
        for attempt in range(5):
            try:
                return client.chat.completions.create(**kwargs)
            except Exception as e:
                if "429" in str(e):
                    wait = (2 ** attempt) + random.uniform(0, 1)
                    time.sleep(wait)
                else:
                    raise
        raise RuntimeError("Max retries exceeded")
    ```

  * **Raise your limit** — if your workload legitimately requires more than 60 req/sec, contact [support@boole.ai](mailto:support@boole.ai) to request a higher limit.
</Accordion>

***

## Checking Service Status

Visit **[status.booleinference.com](https://status.booleinference.com)** for real-time service status, scheduled maintenance windows, and a full history of past incidents. You can subscribe to status updates to receive email or webhook notifications whenever an incident is opened or resolved.

<Info>
  If you're experiencing an issue not covered here, contact [support@boole.ai](mailto:support@boole.ai) with your account email and a description of the problem. Include any error messages and the model you're using to help us respond faster.
</Info>