Dual RTX 3090 (48GB) 128k Context Limits & Setup Guide
How to Maximize Context on the r/LocalLLaMA Sovereign Workstation
🧮 Root Cause & Mathematical Reality
Dual RTX 3090s provide 48 GB pooled VRAM. For 8B and 14B models, 48GB easily runs full 128k context. However, for 70B models, 37.9 GB weights leave only 7.6 GB for KV cache and drivers—capping 70B context at 32k with Q8_0 KV cache.
- Dual RTX 3090s deliver 48 GB total VRAM across two 384-bit memory buses for ~$1,400 used.
- 8B models at Q4 consume ~5.5 GB weights, leaving 40GB+ headroom to run full 128k context with multiple concurrent streams.
- 70B models at Q4 consume 37.9 GB weights, strictly capping reliable context to 32,768 tokens using Q8_0 KV cache.
🛠️ Actionable Fix & Tuning Steps
- Launch 70B models with context capped at 32k: 'llama-server -c 32768 -fa on -ctk q8_0 -ctv q8_0'.
- Distribute layers across both cards: Set tensor split '--tensor-split 1,1' or use llama.cpp automatic layer splitting.
- For 128k document analysis, downsize from 70B to Qwen 2.5 Coder 32B (Q4 ~19.5 GB weights + 18 GB KV cache = 39.5 GB, fits 48GB).
- Ensure PCIe slot spacing on your motherboard provides at least 3 slots of breathing room between cards.
- Use a minimum 1000W 80+ Gold or Platinum power supply with independent 8-pin PCIe power cables.
⚖️ The 3-Tier Honesty Funnel
Enable FP8/Q8_0 KV-cache or clamp context tokens to fit 100% within your local GPU memory.
If model weights or 128k context exceed physical VRAM, rent an 80GB GPU by the hour.
❓ Frequently Asked Questions
Do I need an NVLink bridge for dual RTX 3090 inference?
No! In llama.cpp, vLLM, and Ollama, tensor parallelism and layer offloading run over PCIe 4.0 x8/x16 bus without needing physical NVLink bridges.
How fast does LLaMA 3.3 70B run on dual 3090s?
Expect 18–24 tokens per second at Q4_K_M, providing a smooth, real-time conversational experience.
Explore the Full OOM Troubleshooting Hub
Master VRAM requirements for LLaMA 3.3 70B, DeepSeek V3, Qwen 2.5, and vLLM concurrency.
View All 12 GPU & VRAM Fixers →