Skip to main content
Boole AI is available as a Docker image for teams that prefer containerized deployments. The image bundles the binary and all required CUDA libraries — there is nothing extra to install inside the container. Pass through your GPU and start serving in one command.

​
Prerequisites

Make sure the following are in place before you start:
  • Docker 24+ installed on the host
  • NVIDIA Container Toolkit installed and configured (installation guide)
  • NVIDIA GPU with CUDA 12.0+ drivers on the host

​
Pull and Run

The image is hosted on the GitHub Container Registry. Pull it implicitly by running:
The --gpus all flag passes every available GPU into the container. Replace all with a specific device index (e.g. device=0) if you want to target a single card on a multi-GPU host.

​
Docker Compose

For persistent or multi-service setups, define the Boole service in a Compose file:
docker-compose.yml
Start the service with:

​
Environment Variables

You can configure the server through environment variables instead of CLI flags. Set them with -e in docker run or under environment: in your Compose file.
​
string
required
Model slug to serve. Equivalent to the --model flag.
​
integer
default:"8000"
Port the server listens on inside the container. Remember to update your -p port mapping to match if you change this.
​
string
default:"0.0.0.0"
Bind address. Defaults to 0.0.0.0 inside the container so the port mapping works correctly. Change to 127.0.0.1 if you only want loopback access within the container network.

​
Persistent Weight Cache

By default, the container downloads model weights to a temporary layer that disappears when the container is removed. Mount a host volume to cache weights across container restarts and avoid re-downloading on each run:
On the first run, Boole writes the weights to /data/boole-weights on your host. Subsequent runs load from the cache and skip the download entirely.
Running at scale? See Kubernetes deployment to orchestrate Boole AI across a GPU node pool with automatic scheduling and persistent volume claims.