User Guide¶

This section covers using mlxcel in practice. Each guide explains the concept first, then the configuration.

Pages in This Section¶

• CLI Reference - Complete reference for all seventeen mlxcel subcommands, including offline embedding, reranking, image generation, token classification, and dense prediction, with flags and default values.

• Python HTTP Clients - Calling a separately running mlxcel server with the OpenAI Python SDK or an ordinary HTTP client.

• Memory Estimation - Sizing models before load with mlxcel inspect, --estimate-memory, exact safetensors footprint accounting, KV cache estimates, and memory limit/headroom controls.

• Server Mode - Running mlxcel as a persistent HTTP server: startup, authentication, endpoint overview, lifecycle management, monitoring, and integration examples.

• WebUI - The bundled administrative browser UI: starting it, its security rules, model-free startup, and the Models, Chat, Activity, and Settings screens.

• Image Generation - Generating and editing images with diffusion checkpoints, through mlxcel generate-image and the /v1/images server routes.

• Vision-Language Models - Using models that understand both images and text, including the VLM pipeline, supported model families, vision encoders, and image input formats.

• Token Classification and Span Extraction - Tagging and redacting entities with mlxcel tag, and extracting request-time labels with mlxcel extract.

• Dense Prediction and Vision Tasks - Depth, normals, segmentation, point maps, keypoints, and meshes from mlxcel predict, and how mlxcel detect returns boxes.

• LoRA Adapters - Loading fine-tuned LoRA adapters on top of a base model for task-specific or style-specific inference without retraining the full model.

• TurboQuant KV Cache - Selecting split K/V cache formats and TurboQuant modes for long-context memory reduction.

• Continuous Batching - Serving multiple concurrent requests efficiently with iteration-level scheduling, chunked prefill, and preemptive eviction.

• Performance Tuning - Profiling launch configurations with mlxcel tune, consuming the autotune cache, and understanding automatic paged, cascade, MLA, and sparse attention dispatch.

• Prompt Cache - Reusing prompt-prefix KV state across requests to reduce prefill work for repeated or shared prefixes.

• Language Bias - Controlling the language composition of model outputs by applying additive logit biases to token sets classified by Unicode script, with Conservative and Strict inclusion policies.

Advanced decoding, load-time model transformation, and retrieval-oriented serving now form the independent Advanced Inference and Serving chapter immediately before Distributed Inference.