Anthropic Messages API¶

mlxcel exposes an Anthropic-compatible Messages API on both mlxcel serve and mlxcel-server. Official Anthropic SDK calls such as client.messages.create(...) work once the SDK is pointed at the mlxcel server.

Endpoints¶

Method	Path	Description
`POST`	`/v1/messages`	Create a message, synchronously or as SSE.
`POST`	`/v1/messages/count_tokens`	Count the prompt tokens a request would consume.

Aliases without /v1 are also mounted: /messages and /messages/count_tokens.

Quick Example¶

from anthropic import Anthropic

client = Anthropic(base_url="http://127.0.0.1:8080", api_key="sk-no-auth")
msg = client.messages.create(
    model="Qwen3.5-0.8B-4bit",
    max_tokens=64,
    system="Be terse.",
    messages=[{"role": "user", "content": "Reply with: hello"}],
)
print(msg.content[0].text)

curl http://127.0.0.1:8080/v1/messages \
  -H 'content-type: application/json' \
  -H 'anthropic-version: 2023-06-01' \
  -d '{"model":"Qwen3.5-0.8B-4bit","max_tokens":64,"messages":[{"role":"user","content":"Reply with: hello"}]}'

Supported Request Surface¶

Field	Support
`model`	Required by the request schema but not matched against the loaded model; responses report the served model id.
`system`	String or text-block array; rendered before user/assistant turns.
`messages`	`user` and `assistant` turns. Content can be a string or block array. A `system` turn inside the array is accepted and folded into an adjacent user turn.
`max_tokens`	Maximum generated tokens. Falls back to the server default when omitted.
`stop_sequences`	Stops generation and reports `stop_reason: "stop_sequence"`.
`stream`	Emits Anthropic SSE events.
`temperature`, `top_p`, `top_k`	Mapped to mlxcel sampling settings.
`tools` / `tool_choice`	Client-defined function tools are translated to the internal OpenAI-style tool schema.
`thinking`	Enables `thinking` blocks in the response. `budget_tokens` is passed to the shared reasoning-budget resolution.

Unknown top-level fields such as metadata, container, and service_tier are accepted and ignored for SDK compatibility.

Content Blocks¶

Message content can mix text, images, tool calls, and tool results:

[
  {"type": "text", "text": "What is in this image?"},
  {"type": "image", "source": {"type": "url", "url": "https://example.com/x.png"}},
  {"type": "tool_result", "tool_use_id": "toolu_1", "content": "sunny"}
]

Image blocks on user turns are forwarded to the multimodal media pipeline. tool_result blocks can also contain images, which lets a vision model consume tool-produced images on the next turn. Anthropic server tools that lack an input_schema (for example hosted web_search or code_execution) are accepted but dropped because mlxcel does not execute hosted tools.

Response and Streaming¶

Non-streaming responses contain an ordered content array with text, thinking, and tool_use blocks as applicable, plus stop_reason and usage. With extended thinking enabled, templates whose generation prompt already opens a reasoning block (ending in <think>\n) return that reasoning in a thinking block from the first token.

While the prompt cache is on and extended thinking was enabled, the server remembers the thinking block it returned with each plain reply. A later request that replays the assistant turn without that block gets the reasoning re-injected, so the turn renders the way it was generated and the prompt cache still matches.

Streaming follows the Anthropic event sequence: message_start, content_block_start, content_block_delta, content_block_stop, message_delta, and message_stop. Tool calls are streamed as tool_use content blocks.

Token Counting¶

POST /v1/messages/count_tokens renders the request through the chat template and returns {"input_tokens": N} without generating.

Both routes answer 501 Not Implemented in the Anthropic error envelope when the server cannot generate (embedding-only or reranking-only mode, or no chat model loaded).