# Fix llama_kv_cache_init: failed to allocate buffer for kv cache in Llama.cpp

> **Direct Error Triage**: When llama.cpp prints llama_kv_cache_init: failed to allocate buffer for kv cache, your GPU had enough VRAM to load the GGUF model weights, but ran out of remaining memory when pre-allocating the key-value attention ring buffer right afterward. Fix it at zero cost by passing -fa on -ctk q8_0 -ctv q8_0 (cutting KV memory by 50%), explicitly setting -c 8192 instead of defaulting to the model's 128k training metadata, or reducing -np parallel slots.

## Error Signature
```
llama_kv_cache_init: failed to allocate buffer for kv cache
```

## Root Cause & Memory Allocation Mechanics
Modern GGUF checkpoints (such as LLaMA 3.1/3.3, Qwen 2.5/3, and Mistral) declare `llama.context_length = 131072` inside their file header. In recent releases of `llama-cli` and `llama-server`, omitting `-c` (or passing `-c 0`) instructs `llama_kv_cache_init()` to allocate the full 131,072-token KV buffer in 16-bit FP16 upfront. On a 32-layer 8B model, 128k FP16 KV-cache consumes 16.0 GiB; on an 80-layer 70B model, it demands 40.0 GiB! Consequently, the GGUF weight tensors load cleanly into VRAM first, and the process crashes milliseconds later during `llama_new_context_with_model()` when the KV allocator requests tens of gigabytes of non-existent VRAM.

## Step-by-Step CLI Remediation
### Ollama
```bash
# Override 128k default context in Modelfile or API payload and quantize KV
export OLLAMA_FLASH_ATTENTION=1
export OLLAMA_KV_CACHE_TYPE=q8_0
# Inside Modelfile:
PARAMETER num_ctx 8192
```

### Llama.cpp
```bash
# Explicitly clamp -c to 8192/16384 and compress K/V cache to 8-bit or 4-bit
./llama-server -m model.gguf -ngl 99 -c 16384 -np 1 -fa on -ctk q8_0 -ctv q8_0

# Ultra-low VRAM mode (75% KV reduction via q4_0)
./llama-server -m model.gguf -ngl 99 -c 32768 -np 1 -fa on -ctk q4_0 -ctv q4_0
```

### vLLM
```bash
# In vLLM, cap --max-model-len so KV cache blocks fit remaining VRAM
vllm serve model --max-model-len 16384 --kv-cache-dtype fp8 --gpu-memory-utilization 0.92
```

## Verified Hardware & Cloud Pathways
- Explore 24GB GPUs on Amazon: https://www.amazon.com/s?k=RTX+4090+24GB+graphics+card&tag=sparecables-20 (rel="noopener sponsored")
- Rent on RunPod: https://runpod.io?ref=ynypzu94 (rel="noopener sponsored")
- Rent on Vast.ai: https://cloud.vast.ai/?ref_id=723130 (rel="noopener sponsored")

---
*Interactive Sizing Calculator available at: https://www.gpugarden.com/oom/llama-kv-cache-init-failed/*