Open the calculator →
NVIDIADatacenterAmpere DC

NVIDIA A40 for local AI

NVIDIA A40 provides 48 GB of VRAM for local AI. In the LocalIA catalog, 210 out of 242 models run comfortably on a single card.

View all compatible models →Rig around the A40 ↗
VRAM
48GB
Category
Datacenter
Series
Ampere DC
Vendor
NVIDIA

Models that run comfortably

These models fit in 48 GB with room for context and stable inference.

★Mixtral 8x7Bmistral36.1 GBcomfortableQ5 · / 48 GB
Falcon 40Bfalcon30.7 GBcomfortableQ5 · / 48 GB
Command R 35Bcommand39.1 GBcomfortableQ8 · / 48 GB
Aya 23 35Baya39.1 GBcomfortableQ8 · / 48 GB
CodeLlama 34Bcodellama38.0 GBcomfortableQ8 · / 48 GB
Yi 1.5 34Byi38.0 GBcomfortableQ8 · / 48 GB
★dolphin 2.9.1 yi 1.5 34byi38.0 GBcomfortableQ8 · / 48 GB
★Qwen 2.5 32Bqwen35.8 GBcomfortableQ8 · / 48 GB
★Qwen 2.5 Coder 32Bqwen35.8 GBcomfortableQ8 · / 48 GB
★Qwen 3 32Bqwen35.8 GBcomfortableQ8 · / 48 GB
★QwQ 32Bqwq35.8 GBcomfortableQ8 · / 48 GB
★DeepSeek R1 Distill 32Bdeepseek35.8 GBcomfortableQ8 · / 48 GB
Qwen 2.5 VL 32Bqwen35.8 GBcomfortableQ8 · / 48 GB
★Granite 4 H-Small 32B-A9Bgranite35.8 GBcomfortableQ8 · / 48 GB
GLM-4.6glm35.8 GBcomfortableQ8 · / 48 GB
★GLM-4.7glm35.8 GBcomfortableQ8 · / 48 GB
★GLM-5glm35.8 GBcomfortableQ8 · / 48 GB
★GLM-5.1glm35.8 GBcomfortableQ8 · / 48 GB
★Qwen3 32Bqwen35.8 GBcomfortableQ8 · / 48 GB
★Qwen2.5 32B Instructqwen35.8 GBcomfortableQ8 · / 48 GB
★Qwen2.5 Coder 32B Instructqwen35.8 GBcomfortableQ8 · / 48 GB
★DeepSeek R1 Distill Qwen 32Bqwen35.8 GBcomfortableQ8 · / 48 GB
★Gemma 4 31Bgemma34.6 GBcomfortableQ8 · / 48 GB
★Qwen 3 30B A3Bqwen33.5 GBcomfortableQ8 · / 48 GB
MPT 30Bmpt33.5 GBcomfortableQ8 · / 48 GB
★Qwen3 30B A3Bqwen33.5 GBcomfortableQ8 · / 48 GB
★Qwen3 30B A3B Instruct 2507qwen33.5 GBcomfortableQ8 · / 48 GB
★NVIDIA Nemotron 3 Nano 30B A3B BF16nemotron33.5 GBcomfortableQ8 · / 48 GB
★Qwen3 Coder 30B A3B Instructqwen33.5 GBcomfortableQ8 · / 48 GB
★granite 4.1 30bgranite33.5 GBcomfortableQ8 · / 48 GB

Tight models

These models barely fit. They can run, but context and speed will be limited.

★Qwen 2.5 72Bqwen45.3 GBtightQ4 · / 48 GB
Qwen 2.5 VL 72Bqwen45.3 GBtightQ4 · / 48 GB
★Qwen 72Bqwen45.3 GBtightQ4 · / 48 GB
Llama 2 70Bllama44.0 GBtightQ4 · / 48 GB
Llama 3 70Bllama44.0 GBtightQ4 · / 48 GB
Llama 3.1 70Bllama44.0 GBtightQ4 · / 48 GB
★Llama 3.3 70Bllama44.0 GBtightQ4 · / 48 GB
CodeLlama 70Bcodellama44.0 GBtightQ4 · / 48 GB
★DeepSeek R1 Distill 70Bdeepseek44.0 GBtightQ4 · / 48 GB
Hermes 3 70Bhermes44.0 GBtightQ4 · / 48 GB
★Llama 3.1 Nemotron 70Bnemotron44.0 GBtightQ4 · / 48 GB
Athene 70Bathene44.0 GBtightQ4 · / 48 GB
★Llama 3.3 70B Instructllama44.0 GBtightQ4 · / 48 GB

Unlocked in a 2x rig

With two cards in parallel (96 GB total), larger models become reachable.

Mixtral 8x22Bmistral88.6 GBtightQ4 · / 96 GB
★Mistral Large 123Bmistral77.3 GBcomfortableQ4 · / 96 GB
★NVIDIA Nemotron 3 Super 120B A12B BF16nemotron75.4 GBcomfortableQ4 · / 96 GB
★Llama 4 Scout 17Bx16llama68.5 GBcomfortableQ4 · / 96 GB
★Command R+ 104Bcommand79.9 GBcomfortableQ5 · / 96 GB

Unlocked in a 4x rig

Server-style configuration (192 GB total) for the largest open-weight models.

DeepSeek V2deepseek148.4 GBcomfortableQ4 · / 192 GB
DeepSeek Coder V2deepseek148.4 GBcomfortableQ4 · / 192 GB
★Qwen 3 235B A22Bqwen147.7 GBcomfortableQ4 · / 192 GB
★Qwen3 235B A22Bqwen147.7 GBcomfortableQ4 · / 192 GB
Falcon 180Bfalcon138.3 GBcomfortableQ5 · / 192 GB

Similar GPUs

VRAM estimates updated 2026-09-27.