Fix MLX Metal Insufficient Memory & Command Buffer Execution Failed on Apple Silicon

Root Cause & Memory Allocation Mechanics

Unlike discrete PCIe graphics cards with dedicated GDDR memory, Apple Silicon shares a single LPDDR5/LPDDR5X pool between macOS kernel tasks, WindowServer, and the Metal GPU. To prevent a runaway GPU compute shader from starving the XNU kernel and freezing the system, Apple's `IOAccelerator` driver rejects buffer allocations once pinned ('wired') GPU pages exceed `sysctl iogpu.wired_limit_mb` (which defaults to `0`, meaning automatic ~75% dynamic limit). Furthermore, long-prompt prefill passes in `mlx-lm` or `llama.cpp` without `--kv-bits 8` can spike transient Metal command-buffer memory past the wired threshold, triggering error `00000008:kIOGPUCommandBufferCallbackErrorOutOfMemory`.

Exact CLI Remediation Commands

# Raise Metal wired memory limit on a 48GB Mac to 42GB (43008 MB) or 64GB Mac to 56GB (57344 MB)
sudo sysctl iogpu.wired_limit_mb=43008
export OLLAMA_FLASH_ATTENTION=1
export OLLAMA_KV_CACHE_TYPE=q8_0
ollama serve

# Cap prompt batch size (-b 512 -ub 256) to prevent Metal command buffer timeout/OOM
sudo sysctl iogpu.wired_limit_mb=43008
./llama-server -m model.gguf -ngl 99 -fa on -ctk q8_0 -ctv q8_0 -b 512 -ub 256

# Native Apple MLX-LM CLI with quantized 8-bit KV cache and capped KV size:
mlx_lm.generate --model mlx-community/Llama-3.3-70B-Instruct-4bit \
  --kv-bits 8 \
  --max-kv-size 16384 \
  --prompt "Explain unified memory limits"

Frequently Asked Questions