Knowledge Base
Curated concepts, guides, and best practices for building and operating GPU platforms, plus a glossary. Living, source-tracked content is maintained by KnowledgeOps.
What is MIG (Multi-Instance GPU)?
MIG partitions a single NVIDIA GPU into up to seven hardware-isolated instances, each with dedicated memory and compute.
ReadHow collective communication (NCCL/RCCL) works
Distributed training synchronizes gradients with collective operations like all-reduce, run by NCCL (NVIDIA) or RCCL (AMD).
ReadChoosing a numeric precision (FP8, BF16, FP16)
Lower precision raises throughput and cuts memory — but support and accuracy vary by hardware and workload.
ReadCheckpointing without stalling training
Size storage for the checkpoint burst and overlap I/O with compute so checkpoints never pause the GPUs.
ReadWhat the GPU Operator does
The GPU Operator automates drivers, the container runtime, the device plugin, and telemetry on Kubernetes.
ReadMulti-tenancy patterns for shared GPU clusters
Fair sharing, isolation, and cost attribution across teams on a finite GPU pool.
ReadRunning GPU platforms air-gapped
No egress changes everything: mirror the supply chain, verify at admission, and design for in-cluster resilience.
ReadCutting GPU cost without cutting capability
Raise utilization, right-size, and use the right pricing — GPUs are the most expensive resource in the building.
ReadThe GPU health signals that matter
ECC errors, Xid codes, thermal throttling, and utilization tell you whether a GPU is healthy, degrading, or failing.
ReadFrequently asked questions
Quick answers to common GPU-platform questions.
ReadMixing GPU types and vendors in one cluster
Kubernetes happily runs heterogeneous GPU node pools — different types, and even NVIDIA + AMD together. Steer workloads with labels, taints, and affinity.
ReadHPC, Slurm, and Kubernetes together
Slurm still rules tightly-coupled HPC/MPI; Kubernetes rules services and AI. You can run both — and increasingly bridge them.
ReadMIG
Multi-Instance GPU — hardware partitioning of one NVIDIA GPU into up to 7 isolated instances.
NVLink / NVSwitch
NVIDIA's high-bandwidth intra-node GPU interconnect (and the switch that fabrics it).
Infinity Fabric
AMD's equivalent high-bandwidth GPU interconnect.
InfiniBand
A low-latency, native-RDMA networking fabric common in large training clusters.
RoCEv2
RDMA over Converged Ethernet — RDMA on a lossless (PFC/ECN) Ethernet fabric.
RDMA
Remote Direct Memory Access — network transfers that bypass the CPU.
GPUDirect RDMA
NICs reading/writing GPU memory directly, bypassing the CPU bounce buffer.
GPUDirect Storage
Storage DMA directly into GPU memory, cutting data-loading overhead.
NCCL / RCCL
NVIDIA / AMD collective-communication libraries (all-reduce, etc.) for distributed training.
all-reduce
The collective that sums and distributes gradients across all GPUs each step.
DCGM
NVIDIA Data Center GPU Manager — the telemetry source for GPU metrics.
Xid error
A driver-reported GPU error code; some (79, 48, 94) are fatal hardware faults.
ECC error
Memory error; correctable ones warn, uncorrectable ones mean drain + RMA.
HBM
High-Bandwidth Memory — the stacked memory on data-center GPUs.
FP8 / BF16 / TF32
Reduced-precision formats that trade accuracy for throughput and memory.
MPS
Multi-Process Service — concurrent spatial GPU sharing across cooperative processes.
time-slicing
Software oversubscription of a GPU with no memory isolation.
SR-IOV / vGPU
Virtualization of a NIC/GPU into functions/profiles for VMs.
GPU Operator
Kubernetes operator that manages the GPU driver/runtime/device-plugin stack.
vLLM
A high-throughput LLM serving engine with continuous batching.
KServe / Triton
Model-serving platforms for inference on Kubernetes.
Volcano / Kueue
Batch schedulers adding gang scheduling and fair-share to Kubernetes.
LoRA / QLoRA
Parameter-efficient fine-tuning with low-rank adapters (QLoRA adds 4-bit quantization).
SHARP
In-network reduction on InfiniBand that offloads part of all-reduce to the switch.
bisection bandwidth
The bandwidth across the narrowest cut of a fabric — the ceiling for collectives.
rail-optimized
A topology giving each GPU its own NIC/rail so collectives spread across paths.