Falcon180B params2k context
Falcon 180B locally
Falcon 180B is an open-weight LLM from the Falcon family with 180B parameters. Main use: chat, RAG and general assistance. Detected minimum hardware: Instinct MI250 (128 GB).
Technical facts
Parameters180B
Max context2k
Q4_K_M113.2 GB
Q5_K_M138.3 GB
Q8201.2 GB
FP16402.3 GB
FamilyFalcon
Last sync2026-05-12
Available quantizations
GGUF weightsQ4_K_M
113.2GB
Acceptable. Good compromise when VRAM is limited.
Q5_K_M
138.3GB
Good quality. Sweet spot for size and precision.
Q8
201.2GB
Near-FP16 quality. Comfortable for production.
FP16
402.3GB
Reference precision. Maximum quality, doubled VRAM.
Compatible GPUs
12 single-GPUGPUs that can run Falcon 180B on a single card, ranked by VRAM headroom.
Instinct MI250
AMD128 GB · Instinct CDNA 1-2
113.2 / 128 GBtight · Q4
Instinct MI250X
AMD128 GB · Instinct CDNA 1-2
113.2 / 128 GBtight · Q4
MacBook Pro 14 M3 Max (128GB)
Apple128 GB · MacBook Pro 14
113.2 / 128 GBtight · Q4
MacBook Pro 14 M4 Max (128GB)
Apple128 GB · MacBook Pro 14
113.2 / 128 GBtight · Q4
MacBook Pro 16 M3 Max (128GB)
Apple128 GB · MacBook Pro 16
113.2 / 128 GBtight · Q4
MacBook Pro 16 M4 Max (128GB)
Apple128 GB · MacBook Pro 16
113.2 / 128 GBtight · Q4
Mac Studio M1 Ultra (128GB)
Apple128 GB · Mac Studio
113.2 / 128 GBtight · Q4
Mac Studio M2 Ultra (128GB)
Apple128 GB · Mac Studio
113.2 / 128 GBtight · Q4
Mac Studio M4 Max (128GB)
Apple128 GB · Mac Studio
113.2 / 128 GBtight · Q4
Mac Pro M2 Ultra (128GB)
Apple128 GB · Mac Pro
113.2 / 128 GBtight · Q4
Gaudi 3
Intel128 GB · Gaudi
113.2 / 128 GBtight · Q4
NVIDIA H200 141GB
NVIDIA141 GB · Hopper
113.2 / 141 GBcomfortable · Q4
Recommended multi-GPU rigs
2x / 4x consumer GPUsFor Falcon 180B at higher quantization or with more context, a multi-GPU rig gives more headroom.
Recommended rig
4× RTX 5090
Falcon 180B with Ubuntu, vLLM, Open WebUI and the model already downloaded.
Similar models
VRAM estimates: parameters x bits/8 plus margin. Real performance varies by engine, context length and batch size.
sync: 2026-05-12