Llama.cpp --n-cpu-moe Flag Guide: Run 120B MoE Models on Consumer GPUs
--n-cpu-moe [N]Root Cause & Memory Allocation Mechanics
Unlike dense models where every weight matrix must be evaluated for every token, Mixture-of-Experts architectures activate only a small subset of experts per token (e.g. 5.1B of 117B in gpt-oss 120B, or 3.3B of 30.5B in Qwen3-Coder). By keeping common attention layers on GPU and streaming only the active expert weights across PCIe from DDR5 RAM, memory bandwidth bottlenecks are significantly mitigated compared to offloading dense models.
Exact CLI Remediation Commands
# In Ollama, use Modelfile parameter or environment flag
PARAMETER num_gpu 99# Offload 18 expert layers to DDR5 RAM while keeping attention in VRAM
./llama-server -m gpt-oss-120b.gguf -ngl 99 --n-cpu-moe 18 -c 8192
# Run with pinned host memory for fastest PCIe streaming
./llama-server -m gpt-oss-120b.gguf -ngl 99 --n-cpu-moe 24 --no-mmap# In vLLM, use CPU offload arguments
vllm serve model --cpu-offload-gb 32Upgrade VRAM on Amazon
If software quantization cannot bridge your memory deficit, explore 16GB–24GB GPUs.
Frequently Asked Questions
What RAM speed is recommended when using --n-cpu-moe?
Dual-channel DDR5-6000 CL30 (or quad-channel workstation DDR5-5600) is strongly recommended. Memory bandwidth over the system bus directly determines expert transfer latency.
How many tok/s can I expect with --n-cpu-moe on an RTX 4090?
On gpt-oss 120B (5.1B active), an RTX 4090 paired with fast DDR5 RAM achieves 22–34 tokens/sec—more than fast enough for interactive reading and reasoning.
Verify Your Hardware Memory Math
Use the GPU Garden Matchmaker to calculate exact KV-cache footprints and avoid future OOM crashes.
Open Interactive Calculator →