NVIDIAVendor documented

GH200 Grace Hopper

Hopper + Grace · Hopper · Superchip · 2023

Hopper GPU joined to a Grace CPU over 900 GB/s NVLink-C2C, giving huge coherent CPU+GPU memory for big-context inference.

LLM trainingLLM inferenceRAGHPC
Precision fingerprint
6432t3216BF168i8
Memory

141 GB

HBM3e

Bandwidth

4.9 TB/s

peak

TDP

1000 W

air or liquid

Max model

~55B

FP16, planning est.

Compute throughput

FP6467 TFLOPS
FP3267 TFLOPS
TF32494 TFLOPS
FP16989 TFLOPS
BF16989 TFLOPS
FP81.98 PFLOPS
INT81.98 PFLOPS

Platform & software

InterconnectNVLink-C2C — 900 GB/s to Grace
PCIePCIe 5.0
Coolingair or liquid
MIGSupported
PartitioningUp to 7× MIG
VirtualizationMIG
FrameworksCUDA, TensorRT-LLM, Triton, NeMo, vLLM
AvailabilityOCI, bare-metal
Known limitations
  • ·Coupled Grace CPU changes host-memory and NUMA planning
Related GPUs
NVIDIA

H100 SXM5

80 GB · 989 TFLOPS FP16

NVIDIA

H100 PCIe

80 GB · 756 TFLOPS FP16

NVIDIA

H100 NVL

94 GB · 989 TFLOPS FP16