Aya35B params8k context
Aya 23 35B locally
Aya 23 35B is an open-weight LLM from the Aya family with 35B parameters. Main use: chat, RAG and general assistance. Detected minimum hardware: TITAN RTX (24 GB).
Technical facts
Parameters35B
Max context8k
Q4_K_M22.0 GB
Q5_K_M26.9 GB
Q839.1 GB
FP1678.2 GB
FamilyAya
Last sync2026-05-12
Available quantizations
GGUF weightsQ4_K_M
22.0GB
Acceptable. Good compromise when VRAM is limited.
Q5_K_M
26.9GB
Good quality. Sweet spot for size and precision.
Q8
39.1GB
Near-FP16 quality. Comfortable for production.
FP16
78.2GB
Reference precision. Maximum quality, doubled VRAM.
Compatible GPUs
12 single-GPUGPUs that can run Aya 23 35B on a single card, ranked by VRAM headroom.
TITAN RTX
NVIDIA24 GB · RTX 20
22.0 / 24 GBtight · Q4
RTX 3090
NVIDIA24 GB · RTX 30
22.0 / 24 GBtight · Q4
RTX 3090 Ti
NVIDIA24 GB · RTX 30
22.0 / 24 GBtight · Q4
RTX 4090
NVIDIA24 GB · RTX 40
22.0 / 24 GBtight · Q4
Quadro RTX 6000
NVIDIA24 GB · Quadro RTX
22.0 / 24 GBtight · Q4
RTX A5000
NVIDIA24 GB · RTX A (Ampere)
22.0 / 24 GBtight · Q4
RTX A5500
NVIDIA24 GB · RTX A (Ampere)
22.0 / 24 GBtight · Q4
RTX 4500 Ada
NVIDIA24 GB · RTX Ada
22.0 / 24 GBtight · Q4
NVIDIA A10
NVIDIA24 GB · Ampere DC
22.0 / 24 GBtight · Q4
NVIDIA A30
NVIDIA24 GB · Ampere DC
22.0 / 24 GBtight · Q4
NVIDIA L4
NVIDIA24 GB · Lovelace DC
22.0 / 24 GBtight · Q4
Radeon RX 7900 XTX
AMD24 GB · RDNA 3
22.0 / 24 GBtight · Q4
Recommended multi-GPU rigs
2x / 4x consumer GPUsFor Aya 23 35B at higher quantization or with more context, a multi-GPU rig gives more headroom.
4× GTX 1060 6GB
NVIDIA24 GB · GTX 10
22.0 / 24 GBtight · Q4
4× GTX 1660
NVIDIA24 GB · GTX 16
22.0 / 24 GBtight · Q4
4× GTX 1660 Super
NVIDIA24 GB · GTX 16
22.0 / 24 GBtight · Q4
4× GTX 1660 Ti
NVIDIA24 GB · GTX 16
22.0 / 24 GBtight · Q4
4× RTX 2060 6GB
NVIDIA24 GB · RTX 20
22.0 / 24 GBtight · Q4
2× RTX 2060 12GB
NVIDIA24 GB · RTX 20
22.0 / 24 GBtight · Q4
4× RTX 3050 6GB
NVIDIA24 GB · RTX 30
22.0 / 24 GBtight · Q4
2× RTX 3060 12GB
NVIDIA24 GB · RTX 30
22.0 / 24 GBtight · Q4
Recommended rig
4× GTX 1060 6GB
Aya 23 35B with Ubuntu, vLLM, Open WebUI and the model already downloaded.
Similar models
VRAM estimates: parameters x bits/8 plus margin. Real performance varies by engine, context length and batch size.
sync: 2026-05-12