NVIDIAVendor documented

A40

Ampere · Ampere · PCIe · 2020

Professional-visualization and inference GPU with 48 GB GDDR6. Common for graphics, Omniverse, and mixed inference.

GraphicsVisionLLM inferenceDev sandbox
Precision fingerprint
6432t3216BF168i8
Memory

48 GB

GDDR6

Bandwidth

0.696 TB/s

peak

TDP

300 W

air

Max model

~13B

FP16, planning est.

Compute throughput

FP64—
FP3237.4 TFLOPS
TF3274.8 TFLOPS
FP16149.7 TFLOPS
BF16149.7 TFLOPS
FP8—
INT8299 TFLOPS

Platform & software

InterconnectNVLink — 112 GB/s
PCIePCIe 4.0 x16
Coolingair
MIGNot supported
VirtualizationvGPU
FrameworksCUDA, TensorRT, Triton, Omniverse
AvailabilityAzure, bare-metal
Known limitations
  • ·GDDR6 bandwidth well below HBM parts
  • ·No MIG
Related GPUs
NVIDIA

H100 SXM5

80 GB · 989 TFLOPS FP16

NVIDIA

H100 PCIe

80 GB · 756 TFLOPS FP16

NVIDIA

H100 NVL

94 GB · 989 TFLOPS FP16