NVIDIAVendor documented

A100 40GB PCIe

Ampere · Ampere · PCIe · 2020

Entry A100 for mainstream servers. Good for mid-size inference and fine-tuning where 40 GB is sufficient.

LLM inferenceFine-tuningVisionHPC
Precision fingerprint
6432t3216BF168i8
Memory

40 GB

HBM2e

Bandwidth

1.555 TB/s

peak

TDP

250 W

air

Max model

~13B

FP16, planning est.

Compute throughput

FP6419.5 TFLOPS
FP3219.5 TFLOPS
TF32156 TFLOPS
FP16312 TFLOPS
BF16312 TFLOPS
FP8—
INT8624 TFLOPS

Platform & software

InterconnectNVLink bridge — 600 GB/s
PCIePCIe 4.0 x16
Coolingair
MIGSupported
PartitioningUp to 7× MIG
VirtualizationvGPU, MIG
FrameworksCUDA, TensorRT, Triton, vLLM
AvailabilityAWS, Azure, GCP, bare-metal
Known limitations
  • ·40 GB limits model size and batch
  • ·No FP8
Related GPUs
NVIDIA

H100 SXM5

80 GB · 989 TFLOPS FP16

NVIDIA

H100 PCIe

80 GB · 756 TFLOPS FP16

NVIDIA

H100 NVL

94 GB · 989 TFLOPS FP16