OpenAI gpt-oss 120B (MoE) VRAM Requirements Guide
OpenAI's flagship Apache-2.0 open-weight reasoning Mixture-of-Experts model. Activates just 5.1B parameters per token across 36 layers while fitting natively on single 80GB–128GB systems.
>98% perplexity retention. Standard Ollama/llama.cpp quantization.
Virtually identical to FP16 (~99.9% fidelity). Demands high-capacity VRAM.
Uncompressed weights. Requires multi-GPU clusters or datacenter cards.
Architectural Memory Breakdown & KV-Cache Scaling
OpenAI specifically engineered `gpt-oss-120b` (`116.8B` total parameters, `5.1B` active per token, 36 layers, 8 KV heads, `headDim = 64`) around hardware-friendly MXFP4/4-bit deployment and alternating 1:1 sliding-window (`128` tokens on 18 layers) + global full-context attention (`18` layers) so the entire checkpoint fits inside a single 80GB datacenter GPU or 96GB–128GB unified-memory workstation. Its 8k FP16 KV-cache is a minuscule `0.29 GiB` (`71.17 GB` total Q4_K_M) and just `4.50 GiB` at full 131,072 context (`77.79 GB` total)! Because only 5.1B expert parameters are read per generated token, unified-memory systems like AMD Ryzen AI Max+ 395 Strix Halo (96GB GPU pool, 256 GB/s), NVIDIA DGX Spark (110GB pool, 273 GB/s), and Mac Studio M3 Ultra achieve brisk 35–55 tok/s interactive reasoning.
On a 24GB or 16GB single-GPU desktop, deploy the companion `openai/gpt-oss-20b` (`20.9B` total, `3.6B` active, `14.43 GB` Q4_K_M) for instant local reasoning, or use `llama-server --n-cpu-moe` if your PC has 96GB–128GB DDR5 system RAM.
Provides sufficient headroom for weights, 8k+ context KV-cache, and CUDA runtime buffers without Out-Of-Memory (OOM) crashes.
Run OpenAI gpt-oss 120B (MoE) Locally
ollama run gpt-oss:120bvllm serve openai/gpt-oss-120b --max-model-len 32768 --gpu-memory-utilization 0.95Frequently Asked Questions
Can AMD Strix Halo 128GB or NVIDIA DGX Spark 128GB run gpt-oss-120b locally?
Yes! Both 128GB unified-memory machines allocate 96GB–110GB to GPU compute, comfortably holding the 71.17 GB 4-bit checkpoint plus a full 128k context window (`77.79 GB` total) at silent <140W desktop power.
Does gpt-oss-120b fit on Dual RTX 3090 or Dual RTX 4090 (48GB)?
Purely in VRAM, 48GB falls ~23.2 GB short of the 71.17 GB footprint; even Dual RTX 5090 (64GB) with `IQ4_XS` (`67.20 GB`) needs 4–6 cold MoE layers offloaded to DDR5 RAM via `--n-cpu-moe`.
Test Custom Hardware & Context Windows
Simulate your exact GPU, calculate token throughput speeds, or compare cloud rental break-even costs.
Open in GPU Garden Matchmaker →