# Fix ggml_backend_cuda_buffer_type_alloc_ctx: failed to allocate buffer in Llama.cpp & Ollama

> **Direct Error Triage**: This fatal error occurs when llama.cpp or Ollama requests a CUDA memory block that exceeds free physical VRAM. The allocation fails during weight tensor loading, KV-cache creation, or scratchpad allocation. Fix it immediately by decreasing GPU offloaded layers (-ngl), enabling FP8/INT8 KV-cache (-ctk q8_0), clamping context window (-c 4096), or closing GPU-heavy desktop applications.

## Error Signature
```
ggml_backend_cuda_buffer_type_alloc_ctx: failed to allocate buffer
```

## Root Cause & Memory Allocation Mechanics
When the CUDA runtime executes `cudaMalloc` or `cudaMallocHost`, the NVIDIA driver verifies that contiguous unfragmented memory exists. If Windows Desktop Window Manager (DWM) or background Chrome tabs reserve 1.5 GB of VRAM, an allocation for a 22.8 GB model on a 24GB card will fail with error code 2 (`cudaErrorMemoryAllocation`). Additionally, memory fragmentation from repeated model reloads without restarting the server daemon can trigger this error even when total reported free VRAM appears sufficient.

## Step-by-Step CLI Remediation
### Ollama
```bash
# Option A: Force FP8 KV-cache
export OLLAMA_KV_CACHE_TYPE=q8_0
ollama run model:q4_K_M

# Option B: Reduce context window via Modelfile
PARAMETER num_ctx 4096
```

### Llama.cpp
```bash
# 1. Reduce offloaded layers from 99 to 32
./llama-server -m model.gguf -ngl 32 -c 4096

# 2. Enable Flash Attention & Quantized KV-cache
./llama-server -m model.gguf -ngl 99 -fa on -ctk q8_0 -ctv q8_0
```

### vLLM
```bash
# Cap GPU memory utilization and enforce FP8 KV
vllm serve model --gpu-memory-utilization 0.90 --kv-cache-dtype fp8 --max-model-len 8192
```

## Verified Hardware & Cloud Pathways
- Explore 24GB GPUs on Amazon: https://www.amazon.com/s?k=RTX+4090+24GB+graphics+card&tag=sparecables-20 (rel="noopener sponsored")
- Rent on RunPod: https://runpod.io?ref=ynypzu94 (rel="noopener sponsored")
- Rent on Vast.ai: https://cloud.vast.ai/?ref_id=723130 (rel="noopener sponsored")

---
*Interactive Sizing Calculator available at: https://www.gpugarden.com/oom/ggml-backend-cuda/*