NVIDIAVendor documented

H100 NVL

Hopper · Hopper · PCIe (NVL pair) · 2023

Near-SXM Hopper compute in a PCIe part with 94 GB HBM3, tuned for large-model inference; typically deployed as bridged NVL pairs.

LLM inferenceLLM trainingFine-tuningRAG
Precision fingerprint
6432t3216BF168i8
Memory

94 GB

HBM3

Bandwidth

3.9 TB/s

peak

TDP

400 W

air

Max model

~35B

FP16, planning est.

Compute throughput

FP6467 TFLOPS
FP3267 TFLOPS
TF32494 TFLOPS
FP16989 TFLOPS
BF16989 TFLOPS
FP81.98 PFLOPS
INT81.98 PFLOPS

Platform & software

InterconnectNVLink bridge — 600 GB/s
PCIePCIe 5.0 x16
Coolingair
MIGSupported
PartitioningUp to 7× MIG
VirtualizationvGPU, MIG
FrameworksCUDA, TensorRT-LLM, Triton, NeMo, vLLM
AvailabilityAzure, GCP, OCI, bare-metal
Known limitations
  • ·Sold and deployed as bridged NVL pairs
  • ·Higher per-card power than H100 PCIe
Related GPUs
NVIDIA

H100 SXM5

80 GB · 989 TFLOPS FP16

NVIDIA

H100 PCIe

80 GB · 756 TFLOPS FP16

NVIDIA

H200 SXM

141 GB · 989 TFLOPS FP16