# Fix vLLM RuntimeError: Free memory on device is less than desired GPU memory utilization (--gpu-memory-utilization)

> **Direct Error Triage**: vLLM raises Free memory on device is less than desired GPU memory utilization before loading a single weight tensor whenever another process (an active Ollama daemon, a zombie Python worker, Xorg/Windows DWM desktop compositing, or a previous crashed vLLM EngineCore) is already occupying more than 10% of total GPU memory. Fix it by killing zombie GPU processes, stopping background Ollama models, or lowering --gpu-memory-utilization from 0.90 to 0.80–0.85.

## Error Signature
```
ValueError: Free memory on device (X/Y GiB) on startup is less than desired GPU memory utilization (0.9, Z GiB)
```

## Root Cause & Memory Allocation Mechanics
Unlike relative memory flags that take 90% of *currently free* VRAM, vLLM's `--gpu-memory-utilization` parameter (default `0.90`) specifies a fraction of *total physical* GPU capacity (`total_gpu_memory * 0.90`). On a 24.0 GiB RTX 3090 or RTX 4090, `0.90` demands at least `21.60 GiB` of completely untouched VRAM at startup. If Ubuntu `gnome-shell`/`Xorg`, Windows Desktop Window Manager, or a lingering `VLLM::EngineCore` child process from a `Ctrl+C` interrupt is holding `2.8 GiB` (`21.2 GiB` free), vLLM's pre-flight check (`free_gpu_memory < total_gpu_memory * utilization`) immediately aborts with a `ValueError`.

## Step-by-Step CLI Remediation
### Ollama
```bash
# Inspect and unload background Ollama models occupying VRAM
ollama ps
ollama stop <model_name>
# Or stop the background service completely before launching vLLM:
sudo systemctl stop ollama
```

### Llama.cpp
```bash
# Find and terminate zombie CUDA processes holding GPU memory on Linux
nvidia-smi
sudo fuser -v /dev/nvidia* | awk '{for(i=1;i<=NF;i++) if ($i ~ /^[0-9]+$/) print $i}' | xargs -r kill -9
```

### vLLM
```bash
# Lower --gpu-memory-utilization to match actual free VRAM after OS overhead
vllm serve model \
  --gpu-memory-utilization 0.82 \
  --max-model-len 8192 \
  --kv-cache-dtype fp8
```

## Verified Hardware & Cloud Pathways
- Explore 24GB GPUs on Amazon: https://www.amazon.com/s?k=RTX+4090+24GB+graphics+card&tag=sparecables-20 (rel="noopener sponsored")
- Rent on RunPod: https://runpod.io?ref=ynypzu94 (rel="noopener sponsored")
- Rent on Vast.ai: https://cloud.vast.ai/?ref_id=723130 (rel="noopener sponsored")

---
*Interactive Sizing Calculator available at: https://www.gpugarden.com/oom/vllm-gpu-memory-utilization-oom/*