OpenAI-Compatible API

Polyant exposes its assistants through an HTTP API that mirrors a subset of OpenAI’s Chat Completions API. Any client that speaks that protocol - the openai Python SDK, openai-node, LangChain, LiteLLM, etc. - can call Polyant by changing the base URL.

Base URL

http://<engine-host>:4000/v1

In a default local install, that is http://localhost:4000/v1.

Endpoints

GET /v1/models

Lists the active agents the caller can see as models - the single agent behind a per-agent API key, or, for a signed-in user, the active agents of the workspaces they can reach in the addressed organization:

{
  "object": "list",
  "data": [
    {
      "id": "demo-bot",
      "object": "model",
      "created": 1730390400,
      "owned_by": "polyant"
    },
    {
      "id": "support-bot",
      "object": "model",
      "created": 1730491200,
      "owned_by": "polyant"
    }
  ]
}

The id is the agent slug. Inactive agents, and agents the caller cannot reach, are filtered out. The route requires agent:read. Called with an agent’s own API key, it lists that agent only while its Web channel is on, and nothing otherwise.

POST /v1/chat/completions

The conversation endpoint. Request body:

{
  "model": "demo-bot",
  "messages": [
    {"role": "user", "content": "Hello!"}
  ],
  "stream": false
}

model is the agent slug (lowercase letters, digits, _, . and -, at most 128 characters) and messages holds 1 to 100 entries. Two optional fields are Polyant’s own:

• chat_id (at most 128 characters) - your stable id for the conversation. Requests with the same chat_id continue the same conversation; without one, every request starts a new conversation that does not survive the response.
• context - a JSON object projected onto the conversation state before the turn. See Call context.

Client-supplied system messages are accepted by the schema and dropped: the system prompt is the operator’s.

Response (non-streaming): a standard Chat Completion shape.

{
  "id": "chatcmpl-9f3a8b2c1d4e5f6a7b8c9d0e",
  "object": "chat.completion",
  "created": 1730491200,
  "model": "demo-bot",
  "choices": [
    {
      "index": 0,
      "message": { "role": "assistant", "content": "Hello!" },
      "finish_reason": "stop"
    }
  ],
  "usage": {
    "prompt_tokens": 0,
    "completion_tokens": 0,
    "total_tokens": 0
  }
}

The id field is generated server-side as chatcmpl-<24 hex chars> to match the OpenAI shape. The only emitted finish_reason value is "stop" - tool calls are orchestrated internally by the Supervisor and are never surfaced to the API caller, so tool_calls, length, content_filter, etc. are not returned.

usage reports the provider’s own token counts when it returns them, and falls back to zeros when it does not. The streaming response carries no usage chunk.

Response (streaming, "stream": true): Server-Sent Events with data: lines containing JSON deltas, terminated by data: [DONE]. A request the pipeline refuses before the turn starts is answered with its real HTTP status and error body, never inside a stream that has already opened with 200.

Authentication

GET /v1/models requires authentication - either a session JWT (admin panel users) or a per-instance API key via Authorization: Bearer sk-.... It is not an unauthenticated discovery endpoint; unauthenticated calls return HTTP 401.

POST /v1/chat/completions answers only while the agent’s Web channel is on, switched on the agent’s Channels → Web/API card. While it is off, the answer is 404 Unknown model, the same as for a slug that does not exist. A new agent starts with Web off and with authEnabled = true, and Web can be switched on only once the agent has a key or authentication has been turned off. Agents that existed before the upgrade to 1.2.0 kept Web on and their authEnabled value. A script that creates an agent and then calls it saves the key and enables the channel with PUT /api/instances/<slug>/channels/web and the body {"config":{},"enabled":true}.

With Web on, the endpoint follows per-instance rules. If the instance has authEnabled = false, no header is required: anyone who can reach the engine and knows the slug can talk to the instance. That is meant for public agents, and dangerous for any other.

If the instance has authEnabled = true, every request must include:

Authorization: Bearer <auth-api-key>

The expected key is the API Key set on the agent’s Channels → Web/API card, where Require authentication is switched on (stored AES-256-GCM encrypted in instance_secrets.auth_api_key). The token from the Authorization: Bearer header is compared against the decrypted value using timingSafeEqual to prevent timing attacks (see packages/engine/src/server/openai/instance-api-key-auth.ts).

A request that carries context needs the key even when authEnabled = false, because it writes conversation state.

Errors

Errors are returned as the standard NestJS exception envelope - not the OpenAI {message, type, code} shape:

{
  "statusCode": 401,
  "message": "Invalid API key",
  "error": "Unauthorized"
}

Rate limit

/v1/chat/completions is throttled at 20 requests per minute per client IP, enforced by @Throttle({ default: { limit: 20, ttl: 60_000 } }) on the controller (packages/engine/src/server/openai/openai.controller.ts). This route-level limit is fixed: the installation’s rate-limit policy does not change it. Exceeding the budget returns HTTP 429. For higher volume, rate-limit upstream at the proxy (NGINX, Traefik, Cloudflare) and raise the engine ceiling accordingly.

Streaming

Streaming follows OpenAI’s SSE protocol exactly:

data: {"id":"chatcmpl-xxx","object":"chat.completion.chunk","choices":[{"delta":{"content":"Hel"}}]}
data: {"id":"chatcmpl-xxx","object":"chat.completion.chunk","choices":[{"delta":{"content":"lo"}}]}
data: {"id":"chatcmpl-xxx","object":"chat.completion.chunk","choices":[{"delta":{},"finish_reason":"stop"}]}
data: [DONE]

The first chunk carries role: "assistant"; subsequent chunks carry only content deltas; the last chunk carries finish_reason.

Call context

context hands the agent facts about the caller (a customer id, a phone number, a locale) without routing them through the model. The agent’s Context fields (conversation state) mapping, on the Channels → Web/API card, says which field of context lands on which conversation-state key; dotted paths reach nested fields.

curl http://localhost:4000/v1/chat/completions \
  -H "Content-Type: application/json" \
  -H "Authorization: Bearer sk-myinstancekey" \
  -d '{
    "model": "demo-bot",
    "chat_id": "call-2041",
    "context": { "customer": { "id": "C-881", "phone": "+390212345678" } },
    "messages": [{"role": "user", "content": "Where is my order?"}]
  }'

With the mapping customerId ← customer.id, the conversation state holds customerId: "C-881" before the turn starts, so tools and hooks, including the first turn’s conversation_start hooks, read it from ctx.state. A mapped path that is absent from context is skipped; an unmapped field is ignored.

The rules, and the error each one produces:

The values are written to the state before the turn runs, and they stay there even if the turn is then abandoned or fails. context is accepted by POST /api/instances/{slug}/chat/stream too, with the same rules.

Native streaming endpoint

POST /api/instances/{slug}/chat/stream

The endpoint the CLI and your own streaming clients use. It accepts the same body as /v1/chat/completions (model is overridden by the slug in the URL, and the response always streams), the same per-agent authentication and the same Web channel switch, with POST /api/instances/{slug}/playground/chat/stream as an alias (managed services). The panel’s Playground uses its own route, POST /api/instances/{slug}/playground/stream, which emits the same events but authenticates with the panel session and the agent.playground:use permission, so it works with Web off and needs no agent key. Instead of OpenAI deltas it emits a typed SSE stream that exposes the whole turn as it happens:

A comment line (: ping) every 25 seconds keeps idle proxies from closing the connection during a long tool call.

Errors. An authentication failure is answered with its HTTP status before the stream opens. Anything refused after that, including the call-context 400s above, arrives as an error event followed by done, inside a 200 stream.

What a disconnect does. A client that goes away before the turn has produced anything (no text, no reasoning, no tool call begun) aborts the pipeline: the model stops and the abandoned turn is not stored. Once the turn has produced something, a disconnect only stops the relay: the turn runs to its end and is stored, so a tool that may have written is never left unrecorded.

Buffered turns. When the agent has an enabled output governance gate, or a response_generated hook whose function declares mutatesResponse, the turn is not streamed: it runs to the end, gates and hooks included, and the final text arrives as one text-delta. Such a turn always runs to completion, even if the client disconnects, because its tools may have written before anything reached the relay.

Worked example: cURL

Non-streaming:

curl http://localhost:4000/v1/chat/completions \
  -H "Content-Type: application/json" \
  -H "Authorization: Bearer sk-myinstancekey" \
  -d '{
    "model": "demo-bot",
    "messages": [{"role": "user", "content": "Summarize Polyant in one sentence."}]
  }'

Streaming:

curl -N http://localhost:4000/v1/chat/completions \
  -H "Content-Type: application/json" \
  -H "Authorization: Bearer sk-myinstancekey" \
  -d '{
    "model": "demo-bot",
    "stream": true,
    "messages": [{"role": "user", "content": "Tell me a story about an AI assistant."}]
  }'

Worked example: Python

from openai import OpenAI
 
client = OpenAI(
    base_url="http://localhost:4000/v1",
    api_key="sk-myinstancekey",
)
 
resp = client.chat.completions.create(
    model="demo-bot",
    messages=[{"role": "user", "content": "Hello!"}],
)
print(resp.choices[0].message.content)

Worked example: TypeScript

import OpenAI from "openai";
 
const client = new OpenAI({
  baseURL: "http://localhost:4000/v1",
  apiKey: "sk-myinstancekey",
});
 
const resp = await client.chat.completions.create({
  model: "demo-bot",
  messages: [{ role: "user", content: "Hello!" }],
});
console.log(resp.choices[0].message.content);

What the API does not support

• Function calling on the request side. Tools are configured per instance from the admin panel, not in the request payload.
• Multiple system messages. The system prompt is composed server-side from the seven Prompts sections, and client-supplied system messages are dropped.
• Image input via the API. Multimodal input arrives only through channel adapters today.
• Embeddings or moderations. Those endpoints are not implemented.