macOS Memory Guardrails¶
A model that takes most of unified memory behaves differently on a Mac than on a discrete GPU. This page describes the figures mlxcel reports, the memory reserve it leaves for macOS, and the switches that act on them.
Why total RAM is the wrong figure¶
GPU wiring of a buffer ends a few seconds after its last command buffer. While a server sits idle, macOS may compress or swap the model's memory when another process wants memory, and the next request pays for that work. Total RAM and Metal's recommended working set describe what the machine can hold, not what it can hand out now.
Available now and the reserve¶
mlxcel inspect and --estimate-memory print these lines on macOS:
| Line | Meaning |
|---|---|
| Total RAM | hw.memsize |
| Metal working set | Metal's recommended working set |
| Available now | free + purgeable + min(inactive + speculative, file-backed) pages from host_statistics64, with free excluding speculative pages (the kernel's free_count includes them). Inactive anonymous memory of other programs is excluded because macOS frees it only by compressing or swapping it. |
| macOS reserve | max(7 GiB, 15% of RAM) |
| This load | weights + KV cache + activations |
When the load exceeds available-now minus the reserve, a line starting WARNING: memory available now prints the figures. It never refuses a load and does not change the FITS verdict.
The figure and the reserve live in mlxcel_core::host_memory as host_memory_status, HostMemoryStatus, and AvailabilityGate, for any store that grows on demand.
Footprint reporting¶
After a load the server logs Process footprint after load. With --metrics, /metrics adds these gauges on macOS:
| Metric | Meaning |
|---|---|
mlxcel_process_phys_footprint_bytes | phys_footprint of the process |
mlxcel_process_compressed_bytes | Bytes of the process in the compressor or swap. Growth while idle means idle compression. |
mlxcel_mlx_active_bytes, mlxcel_mlx_cache_bytes | MLX allocator counters |
mlxcel_weights_wired_bytes | Bytes wired by MLXCEL_WIRE_WEIGHTS at the last load |
Switches¶
MLXCEL_WIRED_LIMIT keeps its default of max.
MLXCEL_WIRE_WEIGHTS=1 wires the evaluated weights with mlock after load, on macOS only, and is off by default. Weights are wired in 1 GiB chunks. Before the first chunk and after each one the system wired total is compared with RAM minus the reserve, and a chunk that would cross it is unwired and the pass stops. The result, including failures, is logged. The variable needs the default eager MLXCEL_WEIGHT_MATERIALIZATION. Multimodal towers are not wired, and a repack that runs at the first token (expert stacking, dtype conversion) replaces wired sources with unwired results.
Growing stores¶
The external PLE row cache of qwen4_exp checks availability before each 64 MiB growth step and stops growing, evicting instead, rather than push macOS below the reserve. The check reuses one query for 250 ms.
The prompt-cache snapshot store grows the same way: while it is below its byte budget, each snapshot insert that would add to its resident size re-checks availability against the reserve and is skipped instead of stored when the host cannot spare it. A store at its budget only replaces entries, so it is never refused. The KV prompt-cache store is not gated: an entry's arrays already exist when it is inserted and the store is bounded by a byte budget, so there is no growth step to refuse.
Measurement status¶
No soak has run for these guardrails, so none of them is adopted as a default from measurement: MLXCEL_WIRE_WEIGHTS stays off, MLXCEL_WIRED_LIMIT keeps max, and no load is refused. The switches exist so the measurement can be run on a quiet machine, and MLXCEL Environment Variables lists their exact values.