Chat Completions – Streaming, Tools, and Structured Output

The /v1/chat/completions endpoint generates a response for a conversation. It supports streaming via SSE, tool/function calls, and structured JSON output. It is the primary endpoint for all chat-based interactions and the recommended choice for new integrations.

Endpoint

POST https://api.boole.dev/v1/chat/completions

Request Body

string

required

Model slug to use for the request, e.g. llama-3.3-70b-instruct. See Models for all available slugs.

array

required

Array of message objects that make up the conversation. Each object contains:

role — "system", "user", or "assistant"
content — the message text as a string

boolean

When true, responses are streamed as server-sent events (SSE). The stream ends with data: [DONE]. Default: false.

number

Sampling temperature between 0 and 2. Lower values produce more deterministic output; higher values increase creativity. Default: 1.

integer

Maximum number of tokens to generate in the response. If omitted, the model generates until it reaches a natural stopping point or its context limit.

number

Nucleus sampling probability mass. Only tokens comprising the top top_p probability are considered. Default: 1.

string | array

One or more sequences at which generation stops. The model halts as soon as any sequence in the list is encountered.

array

List of tools the model may call. Each entry has type: "function" and a function object containing name, description, and a JSON Schema parameters object.

string | object

Controls whether and how tools are used. Accepts "auto" (model decides), "none" (no tool calls), or an object targeting a specific function:

{"type": "function", "function": {"name": "my_function"}}

object

Set to {"type": "json_object"} to enable JSON mode. The model is constrained to emit valid JSON.

integer

Number of completion choices to generate. Default: 1.

Example Request

curl https://api.boole.dev/v1/chat/completions \
  -H "Authorization: Bearer $BOOLE_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{
    "model": "llama-3.3-70b-instruct",
    "messages": [
      {"role": "system", "content": "You are a helpful assistant."},
      {"role": "user", "content": "What is the capital of France?"}
    ]
  }'

Example Response

{
  "id": "chatcmpl-abc123",
  "object": "chat.completion",
  "created": 1720000000,
  "model": "llama-3.3-70b-instruct",
  "choices": [
    {
      "index": 0,
      "message": {
        "role": "assistant",
        "content": "The capital of France is Paris."
      },
      "finish_reason": "stop"
    }
  ],
  "usage": {
    "prompt_tokens": 24,
    "completion_tokens": 9,
    "total_tokens": 33
  }
}

string

Unique identifier for the completion. Use this for logging and tracing.

string

The generated text from the assistant for this choice.

string

Reason generation stopped. Common values: "stop" (natural end or stop sequence hit), "length" (max_tokens reached), "tool_calls" (model invoked a tool).

integer

Number of tokens in the input messages.

integer

Number of tokens generated in the response.

Streaming

Set "stream": true to receive the response as a sequence of server-sent events. Each chunk contains a partial delta, and the stream terminates with data: [DONE].

curl https://api.boole.dev/v1/chat/completions \
  -H "Authorization: Bearer $BOOLE_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{
    "model": "llama-3.3-70b-instruct",
    "stream": true,
    "messages": [
      {"role": "user", "content": "Count to five."}
    ]
  }'

Each server-sent event looks like:

data: {"id":"chatcmpl-abc123","object":"chat.completion.chunk","choices":[{"delta":{"content":"One"},"index":0,"finish_reason":null}]}

data: {"id":"chatcmpl-abc123","object":"chat.completion.chunk","choices":[{"delta":{"content":", two"},"index":0,"finish_reason":null}]}

data: [DONE]

Concatenate the delta.content values from each chunk to reconstruct the full response.

Overview

Endpoints

Chat Completions – Streaming, Tools, and Structured Output

Endpoint

Request Body

Example Request

Example Response

Streaming

​Endpoint

​Request Body

​Example Request

​Example Response

​Streaming

Endpoint

Request Body

Example Request

Example Response

Streaming