# Fix FlashAttention-2 is only supported on Ampere, Ada, or Hopper GPUs (Turing / Pascal / Consumer Fix)

> **Direct Error Triage**: Tri Dao's FlashAttention-2 CUDA kernels rely on hardware warp-level matrix-multiply instructions (mma.sync.aligned.m16n8k16 in BF16/FP16) introduced with NVIDIA Ampere (compute capability sm_80, RTX 30-series / A100). Running attn_implementation='flash_attention_2' on Turing (RTX 2080 Ti, T4, Quadro RTX, sm_75), Volta (V100, sm_70), or Pascal (GTX 1080 Ti, P40, sm_61) triggers an immediate RuntimeError. Fix it by switching to PyTorch Native SDPA (attn_implementation='sdpa'), xFormers, or vLLM --dtype half.

## Error Signature
```
RuntimeError: FlashAttention only supports Ampere GPUs or newer (sm_80+)
```

## Root Cause & Memory Allocation Mechanics
Older GPU architectures lack two hardware primitives required by the `flash-attn` Python wheel: first, native `bfloat16` (`BF16`) Tensor Core pipelines; second, asynchronous shared-memory copy instructions (`cp.async`) and `sm_80` PTX MMA shapes. However, you do NOT have to fall back to quadratic $O(N^2)$ eager attention on an RTX 2080 Ti, Tesla T4, or Titan RTX! PyTorch 2.2+ includes `torch.nn.functional.scaled_dot_product_attention` (`sdpa`) backed by Memory-Efficient Attention (Cutlass / xFormers kernels compiled for `sm_70` and `sm_75` in FP16) and llama.cpp includes custom Turing/Volta/Pascal vectorized `-fa on` kernels that deliver linear $O(N)$ memory scaling without requiring `sm_80`.

## Step-by-Step CLI Remediation
### Ollama
```bash
# Ollama's GGUF FlashAttention backend works on Turing/Pascal via custom CUDA kernels
export OLLAMA_FLASH_ATTENTION=1
ollama serve
```

### Llama.cpp
```bash
# llama.cpp supports -fa on across sm_61 (Pascal), sm_75 (Turing), and sm_80+
./llama-server -m model.gguf -ngl 99 -fa on -ctk q8_0 -ctv q8_0
```

### vLLM
```bash
# Force FP16 (--dtype half) and XFORMERS/FLASHINFER backend on Turing (RTX 2080 Ti / T4)
export VLLM_ATTENTION_BACKEND=XFORMERS
vllm serve model --dtype half --max-model-len 8192

# In HuggingFace Transformers Python code:
# model = AutoModelForCausalLM.from_pretrained(id, torch_dtype=torch.float16, attn_implementation="sdpa")
```

## Verified Hardware & Cloud Pathways
- Explore 24GB GPUs on Amazon: https://www.amazon.com/s?k=RTX+4090+24GB+graphics+card&tag=sparecables-20 (rel="noopener sponsored")
- Rent on RunPod: https://runpod.io?ref=ynypzu94 (rel="noopener sponsored")
- Rent on Vast.ai: https://cloud.vast.ai/?ref_id=723130 (rel="noopener sponsored")

---
*Interactive Sizing Calculator available at: https://www.gpugarden.com/oom/flash-attn-2-not-supported/*