# Llama.cpp --n-cpu-moe Flag Guide: Run 120B MoE Models on Consumer GPUs

> **Direct Error Triage**: The --n-cpu-moe flag in llama.cpp allows running massive Mixture-of-Experts (MoE) architectures like OpenAI gpt-oss 120B, Alibaba Qwen 235B, or GLM-4.5-Air 106B on consumer graphics cards. It keeps dense attention heads in fast GPU VRAM while offloading sparse feed-forward expert matrices to system DDR5 RAM, sustaining 20–45 tok/s instead of failing with instant CUDA OOM.

## Error Signature
```
--n-cpu-moe [N]
```

## Root Cause & Memory Allocation Mechanics
Unlike dense models where every weight matrix must be evaluated for every token, Mixture-of-Experts architectures activate only a small subset of experts per token (e.g. 5.1B of 117B in gpt-oss 120B, or 3.3B of 30.5B in Qwen3-Coder). By keeping common attention layers on GPU and streaming only the active expert weights across PCIe from DDR5 RAM, memory bandwidth bottlenecks are significantly mitigated compared to offloading dense models.

## Step-by-Step CLI Remediation
### Ollama
```bash
# In Ollama, use Modelfile parameter or environment flag
PARAMETER num_gpu 99
```

### Llama.cpp
```bash
# Offload 18 expert layers to DDR5 RAM while keeping attention in VRAM
./llama-server -m gpt-oss-120b.gguf -ngl 99 --n-cpu-moe 18 -c 8192

# Run with pinned host memory for fastest PCIe streaming
./llama-server -m gpt-oss-120b.gguf -ngl 99 --n-cpu-moe 24 --no-mmap
```

### vLLM
```bash
# In vLLM, use CPU offload arguments
vllm serve model --cpu-offload-gb 32
```

## Verified Hardware & Cloud Pathways
- Explore 24GB GPUs on Amazon: https://www.amazon.com/s?k=RTX+4090+24GB+graphics+card&tag=sparecables-20 (rel="noopener sponsored")
- Rent on RunPod: https://runpod.io?ref=ynypzu94 (rel="noopener sponsored")
- Rent on Vast.ai: https://cloud.vast.ai/?ref_id=723130 (rel="noopener sponsored")

---
*Interactive Sizing Calculator available at: https://www.gpugarden.com/oom/n-cpu-moe/*