Can My GPU or Mac Run This LLM?
The naive estimate — “7 billion parameters at 4 bits is 4 GB, it fits” — breaks on long context, where the KV cache can outweigh the model itself. This one counts it.
No signup, no email
On RTX 3090 24GB at 8K context, 8 of 12 listed models fit in memory.
| Model | Parameters | Best quantization that fits | Memory needed | Verdict |
|---|---|---|---|---|
| Qwen3 4BAlibaba | 4B | FP16 | 9.5 GBweights 7.5 GB + KV cache 1.1 GB + overhead 1.0 GB | fits |
| Gemma 3 4BGoogle | 4.3B | FP16 | 10.1 GBweights 8.0 GB + KV cache 1.1 GB + overhead 1.0 GB | fits |
| Llama 3.1 8BMeta | 8B | FP16 | 17.2 GBweights 14.9 GB + KV cache 1.0 GB + overhead 1.3 GB | fits |
| Qwen3 8BAlibaba | 8.2B | FP16 | 17.8 GBweights 15.3 GB + KV cache 1.1 GB + overhead 1.4 GB | fits |
| Gemma 3 12BGoogle | 12.2B | Q8 | 15.5 GBweights 11.4 GB + KV cache 3.0 GB + overhead 1.2 GB | fits |
| Mistral Small 3.2 24BMistral AI | 24B | Q5_K_M | 18.3 GBweights 15.6 GB + KV cache 1.3 GB + overhead 1.4 GB | fits |
| Gemma 3 27BGoogle | 27.4B | Q4_K_M | 19.7 GBweights 14.5 GB + KV cache 3.9 GB + overhead 1.3 GB | fits |
| Qwen3 32BAlibaba | 32.8B | — | 20.9 GBweights 17.4 GB + KV cache 2.0 GB + overhead 1.5 GB | tight |
| Qwen3 30B-A3B (MoE)Alibaba | 30.5B3.3B active | Q4_K_M | 18.4 GBweights 16.2 GB + KV cache 0.8 GB + overhead 1.4 GB | fits |
| Llama 3.3 70BMeta | 70.6B | — | 42.5 GBweights 37.5 GB + KV cache 2.5 GB + overhead 2.5 GB | does not fit |
| Qwen3 235B-A22B (MoE)Alibaba | 235B22B active | — | 133.1 GBweights 124.8 GB + KV cache 1.5 GB + overhead 6.8 GB | does not fit |
| DeepSeek V3 671B (MoE)DeepSeek | 671B37B active | — | 405.1 GBweights 356.2 GB + KV cache 30.5 GB + overhead 18.4 GB | does not fit |
Pricing verified 2026-08-12
How this is calculated
Memory = weights (parameters × bytes per parameter at the chosen quantization) + KV cache (2 × layers × KV heads × head dimension × context tokens × 2 bytes) + overhead for buffers and the runtime. “Tight” means over 85% utilisation — technically it fits, but the first longer request will likely run out of memory. We make no claim about speed: that depends on the runtime, the quantization and memory bandwidth.
Now the harder question
Counting tokens is the easy part. The hard part is deciding which processes in your company should go to AI agents, and in what order. That is what our AI-role org chart is for.
FAQ
- What is quantization, and how much quality does it cost?
- Quantization lowers the precision of the weights: 8, 6, 5 or 4 bits per parameter instead of 16. Q8 is nearly indistinguishable from the original, Q5 and Q6 lose a little, and Q4 degrades noticeably on hard tasks — but it fits a model twice the size.
- Why is less memory available than my Mac has installed?
- Mac memory is shared between the system and the GPU. macOS and your apps always hold part of it, so we count roughly two thirds on machines up to 24 GB and three quarters above that.
- If a model does not fit, will it refuse to run?
- It will run, but some layers spill to system RAM or disk and throughput drops several-fold. That is usually unacceptable for chat, and sometimes acceptable for overnight batch work.
en