Skip to content

API Reference

This section covers the full HTTP API exposed by mlxcel-server.

The server presents OpenAI- and Anthropic-compatible generation interfaces, OpenAI-compatible embeddings, Jina/Cohere/TEI-compatible reranking, and llama-server-native routes. Existing client libraries can normally be pointed at mlxcel with a base URL change, but each route's exact response shape still matters.

The server exposes /v1/embeddings, native /embedding and /embeddings, plus four reranking route spellings. With --image-model, the server also mounts the OpenAI image routes /v1/images/generations and /v1/images/edits; see REST API. See Embedding and Reranking APIs for exact schemas and Embeddings and Reranking for model contracts and retrieval-system design.

Contents

Document Description
REST API Full endpoint reference including request/response schemas, authentication, streaming, and error codes
Embedding and Reranking APIs OpenAI and llama-server embedding shapes, Jina/Cohere/TEI reranking, model loading, flags, and errors
OpenAI Responses API Focused guide for /v1/responses, typed SSE events, stores, and conversation chaining
Anthropic Messages API Focused guide for /v1/messages, content blocks, tools, streaming, and token counting
OpenAI Audio API Whisper transcription/translation and Kokoro speech synthesis, including formats, limits, and admission controls
Realtime VoiceChat API /v1/realtime WebSocket protocol for full-duplex Nemotron VoiceChat speech, with session messages, events, and limits
Supported Models Complete table of supported model architectures organized by family

Quick Start

Start the server:

mlxcel-server --model Qwen3.5-0.8B-4bit --port 8080

Send a chat request:

curl http://localhost:8080/v1/chat/completions \
  -H "Content-Type: application/json" \
  -d '{
    "model": "Qwen3.5-0.8B-4bit",
    "messages": [{"role": "user", "content": "Hello!"}]
  }'

OpenAI SDK Compatibility

Because mlxcel implements the OpenAI REST contract, the official OpenAI Python and JavaScript SDKs work without modification:

from openai import OpenAI

client = OpenAI(base_url="http://localhost:8080/v1", api_key="none")
response = client.chat.completions.create(
    model="Qwen3.5-0.8B-4bit",
    messages=[{"role": "user", "content": "Hello!"}],
)
print(response.choices[0].message.content)