Development Guide¶

This section covers how to extend and contribute to mlxcel internals.

Contents¶

Document	Description
Adding Models	Complete checklist for adding new model architectures (text and VLM)
mlxcel-core FFI Bridge	Guide to the MLX C++ FFI layer used by all model implementations
Testing	Unit tests, integration tests, distributed E2E tests, and benchmarking
MLXCEL Environment Variables	Runtime, server, build, diagnostic, and CI `MLXCEL_*` shell variables
The OpenXLA Backend	Status of the StableHLO/IREE source, the default-off Cargo features that reach it, and why stock builds expose MLX only

Repository Layout¶

src/
  bin/             # Binary entry points (mlxcel CLI, mlxcel-server)
  commands/        # CLI command implementations
  distributed/     # Distributed inference (TP, PP, disaggregated)
  execution/       # Runtime environment, device selection, sampling
  lib/
    mlxcel-core/   # MLX C++ FFI bridge (cxx-based bindings)
      src/engine/  # Batch-native decode engine (Engine, DirectEngine)
    mlxcel-mlx-pin/ # Unit tests for the MLX pin derivation
    mlxcel-surgery/ # Weight-load surgery framework (model surgery)
    mlxcel-xla/    # OpenXLA/StableHLO emitter and IREE runtime (opt-in)
    mlxcelverse/   # MLX-side C++ layer
      CMakeLists.txt # Fetches the pinned MLX commit, applies overlays, builds libmlx
      overlays/    # Per-backend MLX source overlays: cpu/, metal/, cuda/, rocm/
      kernels/     # mlxcel's own fused kernels and launchers (paged attention, sampling, ...)
  loading/         # Model loading pipeline (config-backed, VLM, special)
  lora/            # LoRA adapter loading
  models/          # Model implementations (one file per architecture)
  multimodal/      # VLM prompt building utilities
  server/          # HTTP server (routes, types, streaming, batch)
  tokenizer/       # Tokenizer wrapper
  vision/          # Vision encoders, connectors, processors
  loaded_model.rs  # LoadedModel enum + dispatch macros
  model_metadata.rs # Centralized model kind/adapter/route policy
tests/             # Integration and E2E tests
docs/              # Documentation

Build and Development Commands¶

# Debug build
make build

# Release build (native CPU optimization)
make release

# Run tests (single-threaded; required for model tests)
make test

# Run tests with verbose output
make test-verbose

# Lint (clippy)
make lint

# Format code
make fmt

# Full development workflow (format + check + test)
make dev

# CI workflow (format check + check + lint + test)
make ci