# OLLAMA_KV_CACHE_TYPE Guide: Reduce Ollama VRAM Usage by 50% to 75%

> **Direct Error Triage**: By default, Ollama allocates uncompressed FP16 (16-bit floating point, 2 bytes/token/layer) KV-cache buffers for every active conversation stream. On models with 32k or 128k context, this KV buffer consumes up to 16GB–40GB VRAM on its own! Setting OLLAMA_KV_CACHE_TYPE=q8_0 cuts KV memory by 50% with near-zero perplexity loss, while q4_0 cuts KV memory by 75%, allowing 32k context on 16GB–24GB GPUs.

## Error Signature
```
OLLAMA_KV_CACHE_TYPE=q8_0 / OLLAMA_KV_CACHE_TYPE=q4_0
```

## Root Cause & Memory Allocation Mechanics
In autoregressive transformers, key and value tensors for every past token must be retained in memory for attention computation. In dense models like LLaMA 3.3 70B (80 layers, 8 KV heads, headDim 128), FP16 KV-cache takes 320 KB per token. At 32,768 context, KV-cache alone takes 10.0 GB; at 131,072 context, it takes 40.0 GB! Quantizing key and value tensors to 8-bit integers (`q8_0`) reduces storage to 170 KB/token with >99.8% reasoning fidelity.

## Step-by-Step CLI Remediation
### Ollama
```bash
# Linux / macOS Terminal
export OLLAMA_KV_CACHE_TYPE=q8_0
export OLLAMA_FLASH_ATTENTION=1
ollama serve

# Windows PowerShell 5.1
$env:OLLAMA_KV_CACHE_TYPE="q8_0"; $env:OLLAMA_FLASH_ATTENTION="1"; ollama serve

# Linux systemd Service (/etc/systemd/system/ollama.service)
[Service]
Environment="OLLAMA_KV_CACHE_TYPE=q8_0"
Environment="OLLAMA_FLASH_ATTENTION=1"
```

### Llama.cpp
```bash
# Native llama.cpp equivalents
./llama-server -m model.gguf -ctk q8_0 -ctv q8_0 -fa on
```

### vLLM
```bash
# Native vLLM FP8 KV Cache
vllm serve model --kv-cache-dtype fp8
```

## Verified Hardware & Cloud Pathways
- Explore 24GB GPUs on Amazon: https://www.amazon.com/s?k=RTX+4090+24GB+graphics+card&tag=sparecables-20 (rel="noopener sponsored")
- Rent on RunPod: https://runpod.io?ref=ynypzu94 (rel="noopener sponsored")
- Rent on Vast.ai: https://cloud.vast.ai/?ref_id=723130 (rel="noopener sponsored")

---
*Interactive Sizing Calculator available at: https://www.gpugarden.com/oom/ollama-kv-cache-type/*