CodeLlama34B params16k context
CodeLlama 34B locally
CodeLlama 34B is an open-weight LLM from the CodeLlama family with 34B parameters. Main use: code and developer agents. Detected minimum hardware: TITAN RTX (24 GB).
Technical facts
Parameters34B
Max context16k
Q4_K_M21.4 GB
Q5_K_M26.1 GB
Q838.0 GB
FP1676.0 GB
FamilyCodeLlama
Last sync2026-05-12
Available quantizations
GGUF weightsQ4_K_M
21.4GB
Acceptable. Good compromise when VRAM is limited.
Q5_K_M
26.1GB
Good quality. Sweet spot for size and precision.
Q8
38.0GB
Near-FP16 quality. Comfortable for production.
FP16
76.0GB
Reference precision. Maximum quality, doubled VRAM.
Compatible GPUs
12 single-GPUGPUs that can run CodeLlama 34B on a single card, ranked by VRAM headroom.
TITAN RTX
NVIDIA24 GB · RTX 20
21.4 / 24 GBtight · Q4
RTX 3090
NVIDIA24 GB · RTX 30
21.4 / 24 GBtight · Q4
RTX 3090 Ti
NVIDIA24 GB · RTX 30
21.4 / 24 GBtight · Q4
RTX 4090
NVIDIA24 GB · RTX 40
21.4 / 24 GBtight · Q4
Quadro RTX 6000
NVIDIA24 GB · Quadro RTX
21.4 / 24 GBtight · Q4
RTX A5000
NVIDIA24 GB · RTX A (Ampere)
21.4 / 24 GBtight · Q4
RTX A5500
NVIDIA24 GB · RTX A (Ampere)
21.4 / 24 GBtight · Q4
RTX 4500 Ada
NVIDIA24 GB · RTX Ada
21.4 / 24 GBtight · Q4
NVIDIA A10
NVIDIA24 GB · Ampere DC
21.4 / 24 GBtight · Q4
NVIDIA A30
NVIDIA24 GB · Ampere DC
21.4 / 24 GBtight · Q4
NVIDIA L4
NVIDIA24 GB · Lovelace DC
21.4 / 24 GBtight · Q4
Radeon RX 7900 XTX
AMD24 GB · RDNA 3
21.4 / 24 GBtight · Q4
Recommended multi-GPU rigs
2x / 4x consumer GPUsFor CodeLlama 34B at higher quantization or with more context, a multi-GPU rig gives more headroom.
2× GTX 1080 Ti
NVIDIA22 GB · GTX 10
21.4 / 22 GBtight · Q4
2× RTX 2080 Ti
NVIDIA22 GB · RTX 20
21.4 / 22 GBtight · Q4
4× GTX 1060 6GB
NVIDIA24 GB · GTX 10
21.4 / 24 GBtight · Q4
4× GTX 1660
NVIDIA24 GB · GTX 16
21.4 / 24 GBtight · Q4
4× GTX 1660 Super
NVIDIA24 GB · GTX 16
21.4 / 24 GBtight · Q4
4× GTX 1660 Ti
NVIDIA24 GB · GTX 16
21.4 / 24 GBtight · Q4
4× RTX 2060 6GB
NVIDIA24 GB · RTX 20
21.4 / 24 GBtight · Q4
2× RTX 2060 12GB
NVIDIA24 GB · RTX 20
21.4 / 24 GBtight · Q4
Recommended rig
2× GTX 1080 Ti
CodeLlama 34B with Ubuntu, vLLM, Open WebUI and the model already downloaded.
Similar models
VRAM estimates: parameters x bits/8 plus margin. Real performance varies by engine, context length and batch size.
sync: 2026-05-12