> ## Documentation Index
> Fetch the complete documentation index at: https://docs.booleinference.com/llms.txt
> Use this file to discover all available pages before exploring further.

# Deploy the Boole AI Binary — No Python, No Dependencies

> Download and run the Boole AI binary on your GPU — a single executable with no Python runtime, no dependency setup, and a cold start under 400 ms.

The Boole AI local binary is a single self-contained executable with model weights embedded. It requires no Python runtime, no dependency management, and no model zoo. Download once, run anywhere — on an air-gapped server, a workstation, or any Linux or macOS machine with a supported GPU.

## System Requirements

Before you download, confirm your machine meets these requirements:

* **GPU:** NVIDIA GPU with CUDA 12.0+ drivers installed
* **OS:** Linux x86\_64 or macOS (Apple Silicon)
* **Disk:** \~50 GB per 70B-class model; \~14 GB per 8B-class model
* **RAM:** 16 GB system RAM minimum

## Download and Install

Visit [booleinference.com/download](https://booleinference.com/download) and grab the binary for your platform. Always check the download page for the latest release — nightly compiler improvements are shipped as new binaries.

Once downloaded, make the binary executable:

```bash theme={null}
chmod +x boole-latest-linux-x86_64
```

Optionally move it onto your `PATH` so you can invoke it from anywhere:

```bash theme={null}
sudo mv boole-latest-linux-x86_64 /usr/local/bin/boole
```

## Starting the Server

Run `boole serve` with the model slug you want to load:

```bash theme={null}
boole serve --model llama-3.3-70b-instruct
```

The server starts on port `8000` by default. Cold start takes under 400 ms — the nightly compiler pre-fuses kernels and pre-pages the KV cache so the first request is ready almost immediately.

### Common Flags

<ParamField path="--model" type="string" required>
  The model slug to load and serve. Browse available slugs at [booleinference.com/models](https://booleinference.com/models).
</ParamField>

<ParamField path="--port" type="integer" default="8000">
  The port the HTTP server listens on.
</ParamField>

<ParamField path="--host" type="string" default="127.0.0.1">
  The bind address. Set to `0.0.0.0` to accept connections from other machines on your network.
</ParamField>

<ParamField path="--concurrency" type="integer" default="auto">
  Maximum number of concurrent requests. Defaults to a value the compiler selects based on available VRAM.
</ParamField>

<ParamField path="--quantize" type="string" default="auto">
  Quantization level: `int4`, `int8`, or `fp16`. Defaults to the level the nightly compiler chose when it built this binary. Override only when you have a specific quality-vs-memory trade-off in mind.
</ParamField>

## Testing the Server

Once the server is running, send a chat completion request to confirm everything is working:

```bash theme={null}
curl http://localhost:8000/v1/chat/completions \
  -H "Content-Type: application/json" \
  -d '{
    "model": "llama-3.3-70b-instruct",
    "messages": [{"role": "user", "content": "Hello!"}]
  }'
```

A successful response returns a JSON object with a `choices` array containing the model's reply.

## Updating the Binary

The Boole AI nightly compiler continuously improves throughput — the same model weights can run meaningfully faster after a compiler update. To pick up the latest optimizations, download a fresh binary from [booleinference.com/download](https://booleinference.com/download) and replace the existing one:

```bash theme={null}
chmod +x boole-latest-linux-x86_64
sudo mv boole-latest-linux-x86_64 /usr/local/bin/boole
```

No model re-download is required; the weights are embedded in the new binary.

<Note>
  The local binary runs with zero telemetry. No data leaves your machine — your prompts, responses, and model weights stay entirely local.
</Note>

<Tip>
  Prefer a containerized workflow? See [Docker deployment](/deployment/docker) to run the same binary inside a Docker container with GPU passthrough.
</Tip>