Skip to main content
Boole AI is a high-throughput LLM inference platform built for teams that need speed, privacy, and simplicity. You can run models in two ways: download a single self-contained binary and serve models directly on your own NVIDIA GPU, or call the Boole cloud API — a fully OpenAI-compatible endpoint that gets you from signup to first token in minutes. Both modes share the same interface, so switching between them requires nothing more than a one-line config change. Every night the Boole compiler re-ingests every supported model, fuses kernels, quantizes weights, and pages the KV cache — running 2,057 optimization iterations and keeping the 217 best-performing variants. The result is a continuously improving runtime that you consume through the same familiar OpenAI-style API you already know.

​
Two Ways to Run Models

Local Deployment

Download a single executable with weights embedded. No Python, no dependencies, cold start under 400 ms. Runs fully offline after download.

Cloud API

Call https://api.boole.dev/v1 with your existing OpenAI client. Sign up in seconds — no card required, $20 in free credits included.

​
Key Capabilities

312 Tokens / Second

Serve Llama 3.3 70B at 312 tokens/sec on a single NVIDIA A10 GPU — outpacing most multi-GPU setups.

Nightly Compiler Optimization

Kernel fusion, weight quantization, and KV-cache paging rebuilt every night. Your throughput improves automatically, no action needed.

Zero Data Retention

Prompts are never stored. Zero telemetry, fully offline-capable, and SOC 2 Type II compliant — your data stays yours.

OpenAI-Compatible Interface

Drop Boole into any app that already uses the OpenAI SDK. Change one line — base_url — and you’re done.

​
Next Steps

Whether you want to run models on your own hardware or start calling the cloud API right now, the pages below will get you there. Explore the model catalogue to pick the right model for your use case, review pricing before you scale, or jump straight into the API reference.

Models

Browse all 42 available models, including Llama, Qwen, Mixtral, DeepSeek, and Whisper.

Pricing

Per-token and per-minute rates for every model. Prices decrease as the compiler improves.

API Reference

Full reference for every endpoint, parameter, and response field.

FAQ

Answers to common questions about setup, billing, rate limits, and compliance.