NVIDIAVendor documented

H100 SXM5

Hopper · Hopper · SXM5 · 2022

The workhorse training/inference GPU of the generation. Transformer Engine and FP8 make it the baseline for large-model work.

LLM trainingLLM inferenceFine-tuningHPCMultimodal
Precision fingerprint
6432t3216BF168i8
Memory

80 GB

HBM3

Bandwidth

3.35 TB/s

peak

TDP

700 W

air or liquid

Max model

~30B

FP16, planning est.

Compute throughput

FP6467 TFLOPS
FP3267 TFLOPS
TF32494 TFLOPS
FP16989 TFLOPS
BF16989 TFLOPS
FP81.98 PFLOPS
INT81.98 PFLOPS

Platform & software

InterconnectNVLink 4 — 900 GB/s
PCIePCIe 5.0 x16
Coolingair or liquid
MIGSupported
PartitioningUp to 7× MIG
VirtualizationvGPU, MIG
FrameworksCUDA, TensorRT-LLM, Triton, NeMo, vLLM
AvailabilityAWS, Azure, GCP, OCI, bare-metal
Known limitations
  • ·700 W demands rack-level power and thermal planning
  • ·HBM3 supply-constrained through much of its life
Related GPUs
NVIDIA

H100 PCIe

80 GB · 756 TFLOPS FP16

NVIDIA

H100 NVL

94 GB · 989 TFLOPS FP16

NVIDIA

H200 SXM

141 GB · 989 TFLOPS FP16