AI Training Cluster

Click a node to drill in, or click any component to learn about it.

IB Spine 0 · InfiniBand HDR 200IB Spine 1 · InfiniBand HDR 200DGX A100 NODE 0EPYC7742EPYC7742A100A100A100A100A100A100A100A1001TB DDR4 ECC · 640GB HBM2e · 30TB NVMeDGX A100 NODE 1EPYC7742EPYC7742A100A100A100A100A100A100A100A1001TB DDR4 ECC · 640GB HBM2e · 30TB NVMeDGX A100 NODE 2EPYC7742EPYC7742A100A100A100A100A100A100A100A1001TB DDR4 ECC · 640GB HBM2e · 30TB NVMeDGX A100 NODE 3EPYC7742EPYC7742A100A100A100A100A100A100A100A1001TB DDR4 ECC · 640GB HBM2e · 30TB NVMeSHARED STORAGENVMe · 122 TB · 100 GB/sMANAGEMENT NETWORK · 1 GbEBMC · IPMI · Cluster Orchestrator (Slurm / Kubernetes)InfiniBand HDR · 200 Gb/s per link · 8 links per node = 200 GB/s total
Cluster

AI Training Cluster

Nodes (DGX A100)4
GPUs total32 (8 per node)
CUDA Cores total221,184
GPU memory total2,560 GB HBM2e
Peak FP16 Tensor9,984 TFLOPS
System RAM total2 TB DDR4 ECC
Inter-node fabricInfiniBand HDR 200
Intra-node GPU BW600 GB/s (NVLink 3.0)
Storage122 TB NVMe

This cluster is modeled on the NVIDIA DGX SuperPOD — the reference architecture for large-scale AI training. Each of the 4 DGX A100 nodes houses 8 A100 GPUs connected in a full NVLink mesh via NVSwitch. Nodes connect to each other via InfiniBand HDR (200 Gb/s per link). A real SuperPOD has 20 nodes; we show 4 here for clarity. Training GPT-3 (175B parameters) required clusters of this scale running for weeks.