So I stopped estimating and pulled the numbers from the source for 35 current open models:

• Weights: the real size of the published file — GGUF builds by unsloth, ggml-org, bartowski and the model authors (Q4_K_M unless the model ships quantised natively, like GPT-OSS in MXFP4), read from the Hugging Face API.
• Context (KV) cache: computed from each model's own config.json — number of layers, KV heads and head size — including the architectures where the naive formula is wrong (more on that below).
• Runtime: CUDA context and buffers, about 0.75 GiB plus the logits buffer of a 512-token batch (it adds up for 256k-token vocabularies).

All figures are GiB (1024³ bytes), the unit GPU memory is sold in, so they compare directly with "24 GB". Hugging Face lists decimal gigabytes, which is why its file sizes look ~7% larger.

The table

One GPU, one request, FP16 cache. "Smallest single card" is the smallest common card size where the total fits in 97.5% of the memory (the driver and CUDA take the rest).

What surprised me

Hybrid and sliding-window attention change the picture. Qwen3 32B and Qwen3.8 27B are about the same size on disk, but going from an 8k to a 32k context costs Qwen3 32B +6 GiB and Qwen3.8 27B +1.5 GiB. Qwen3.8 keeps a KV cache on only 16 of its 64 layers; the other 48 are linear-attention layers with a fixed state per sequence. Gemma 4 does something similar with a 1024-token sliding window on 5 of every 6 layers. If you plan for long contexts, the architecture matters as much as the parameter count.

MoE models need all of their weights in memory. Qwen3 30B-A3B uses 3.3B parameters per token, but all 30.5B have to be loaded: it needs the same 24 GB card as a dense 27B model. It just runs much faster on it. (llama.cpp can keep experts in system RAM with --n-cpu-moe, at a speed cost.)

GPT-OSS is the best fit for its size class. GPT-OSS 20B ships in MXFP4 and fits a 16 GB card with room for context; GPT-OSS 120B fits a single 80 GB H100.

70B still means 48 GB. Llama 3.3 70B in Q4_K_M is 43 GiB at 8k context: a 48 GB card, or two 24 GB cards with tensor parallelism. At 32k it no longer fits in 48 GB without a quantised KV cache.

DeepSeek's MLA cache is tiny. DeepSeek V3.2 and R1 cache one 576-wide latent per token and layer instead of full keys and values: about 70 KB per token for the whole model. The weights are the problem there, not the context.

The formula, if you want to check a model yourself

weights   = file size of the quantisation you run (or params × bits ÷ 8)
kv_cache  = tokens × layers_with_full_attention × 2 × kv_heads × head_dim × 2 bytes
          + min(tokens, window) × sliding_layers × 2 × kv_heads × head_dim × 2 bytes
runtime   ≈ 0.75 GiB + 512 × vocab × 4 bytes
total     = weights + kv_cache × parallel_requests + runtime

A q8_0 KV cache (-ctk q8_0 -ctv q8_0 in llama.cpp, --kv-cache-dtype fp8 in vLLM) roughly halves the second line. vLLM preallocates most of the card for its cache by default (--gpu-memory-utilization), so nvidia-smi will always look full there; the numbers above are what the model needs, not what vLLM grabs.

Calculator

I turned this into a free calculator: pick a model (35 LLMs plus FLUX, Qwen-Image, Wan 2.2, HunyuanVideo and other image and video models), a quantisation, the context length and the number of parallel requests, and it shows the memory breakdown and the GPUs that fit: gpu.nz/en/tools/vram. Every model also has a page with the full quantisation table: gpu.nz/en/models.

Disclosure: I run gpu.nz, a per-minute GPU rental, so the calculator also lists hourly prices of the cards that fit. The numbers themselves don't depend on that — if you spot a model where they look wrong, tell me and I'll check it against the files.