NVIDIAVendor documented

H100 PCIe

Hopper · Hopper · PCIe · 2022

PCIe-form Hopper for standard servers. Easier to deploy than SXM at the cost of bandwidth and NVLink scale.

LLM inferenceFine-tuningRAGHPC
Precision fingerprint
6432t3216BF168i8
Memory

80 GB

HBM3

Bandwidth

2 TB/s

peak

TDP

350 W

air

Max model

~30B

FP16, planning est.

Compute throughput

FP6451 TFLOPS
FP3251 TFLOPS
TF32378 TFLOPS
FP16756 TFLOPS
BF16756 TFLOPS
FP81.51 PFLOPS
INT81.51 PFLOPS

Platform & software

InterconnectNVLink bridge — 600 GB/s
PCIePCIe 5.0 x16
Coolingair
MIGSupported
PartitioningUp to 7× MIG
VirtualizationvGPU, MIG
FrameworksCUDA, TensorRT-LLM, Triton, vLLM
AvailabilityAWS, Azure, GCP, bare-metal
Known limitations
  • ·Lower clocks and bandwidth than SXM
  • ·NVLink limited to bridge pairs
Related GPUs
NVIDIA

H100 SXM5

80 GB · 989 TFLOPS FP16

NVIDIA

H100 NVL

94 GB · 989 TFLOPS FP16

NVIDIA

H200 SXM

141 GB · 989 TFLOPS FP16