Overview
Domain 2 is about what an AI data center is built from and why. Its ten objectives run from the GPU outward: hardware requirements for training use cases, scaling GPU infrastructure, power and cooling concepts, on-premises versus cloud trade-offs, the parts of an accelerated cluster, facility requirements, networking requirements for AI, data center networking protocols, high-speed network options, and the purpose of a data processing unit (DPU).
At 40%, this is the heaviest domain and the one with the most objectives. Questions stay at the associate level: you identify the right interconnect, cooling approach or deployment model for a described situation, not the cabling plan or switch configuration. Reference designs such as DGX BasePOD and DGX SuperPOD give useful mental models for how compute, storage and networks fit together.
How it's tested
Domain 2 questions describe an AI workload or a data center constraint and ask which hardware, network, cooling approach or deployment model fits. Many hinge on one keyword, such as low latency between nodes, rack power density, data sovereignty or offloading security, so read for the constraint before looking at options.
-
Pick the interconnect
A communication pattern is described, inside a server or between servers, and you choose NVLink, InfiniBand, RoCE Ethernet or a standard TCP/IP network.
-
Scale up or scale out
A model or workload outgrows its current hardware, and you decide whether to add GPUs within a node or add nodes to the cluster.
-
Name the constraint
A facility, cost or compliance limit is described, and you identify the deciding factor, such as power per rack, cooling method or keeping data on premises.
A team training a large model across 64 servers sees GPUs idle while waiting on gradient exchanges over standard Ethernet. Adding GPUs per server does not help because the bottleneck is between nodes. The right move is a low-latency RDMA fabric, such as InfiniBand or RoCE-capable Ethernet, with GPUDirect RDMA. The clue is idle GPUs during communication across nodes, which points to the scale-out network.
Objectives 10
Official objectives from the NVIDIA outline, in outline order.
2.1 Identify hardware requirements for specific AI training task use cases.GPU count, GPU memory, interconnect and storage for training jobs.
- What it means
- This objective covers sizing hardware to a training task. Relevant factors include model size and the GPU memory needed to hold weights, gradients and optimizer state, dataset size and storage throughput, the number of GPUs needed to finish in acceptable time, and whether GPUs must communicate heavily.
- What to understand
- Small models or fine-tuning may fit on a single GPU or workstation. Large language model pretraining needs many high-memory GPUs, such as H100, H200 or Blackwell-generation parts, joined by NVLink inside the server and InfiniBand or Spectrum-X Ethernet between servers. Fast shared storage keeps GPUs fed. Undersized memory forces smaller batches or model sharding; slow storage leaves expensive GPUs idle.
- How it shows up
- A training task is described, and you identify which hardware attribute is the limiting requirement.
- Common confusion
- Focusing only on GPU count and ignoring GPU memory and storage throughput.
2.2 Scale a GPU infrastructure for different use cases.Scale-up within a node versus scale-out across nodes, plus partitioning for small jobs.
- What it means
- This covers growing GPU capacity to match a use case. Scale-up adds GPUs inside one server, linked by NVLink and NVSwitch, or uses rack-scale NVLink domains such as GB200 NVL72. Scale-out adds servers connected by a high-speed fabric. For many small inference or development workloads, the opposite move applies: partitioning GPUs with Multi-Instance GPU (MIG).
- What to understand
- Scale-up gives the fastest GPU-to-GPU bandwidth but is limited by the NVLink domain size. Scale-out reaches thousands of GPUs but depends on network bandwidth and latency. Reference architectures like DGX BasePOD and DGX SuperPOD describe tested ways to scale in units. Inference often scales horizontally by adding replicas, while training jobs scale by distributing data or model across GPUs.
- How it shows up
- A workload's growth is described, and you choose scale-up, scale-out or partitioning.
- Common confusion
- Assuming adding servers always helps even when the network becomes the bottleneck.
2.3 Identify key concepts and high-level specifications related to power and cooling requirements within a data center.Rack power density, air versus liquid cooling and efficiency metrics.
- What it means
- This objective covers why AI changes power and cooling plans. A single eight-GPU server can draw around 10 kW or more, and rack-scale systems like GB200 NVL72 draw well over 100 kW per rack, far above the single-digit kilowatts of a typical enterprise rack. Heat removal must keep pace.
- What to understand
- Air cooling works for moderate densities but struggles as rack power climbs. Direct-to-chip liquid cooling moves heat through cold plates and is required for the densest NVIDIA rack-scale systems. Know power usage effectiveness (PUE), total facility power divided by IT power, where lower is better. Redundant power feeds and N+1 cooling keep training jobs running through component failures.
- How it shows up
- A rack density or cooling limit is described, and you identify the consequence or the cooling approach required.
- Common confusion
- Assuming existing air-cooled racks can host any GPU system without power or cooling upgrades.
2.4 Articulate the key advantages, challenges, and considerations related to on-prem vs cloud infrastructures.Cost model, control, speed, data location and utilization trade-offs.
- What it means
- This covers choosing where AI infrastructure lives. On-premises gives control, predictable cost at steady high utilization, and data that stays in your facility, but needs capital, space, power and skilled staff. Cloud, including DGX Cloud and GPU instances from major providers, offers fast start, elastic capacity and operating-expense billing, but costs can climb with constant use.
- What to understand
- Key factors are utilization, data gravity and egress costs, data sovereignty and compliance, lead time for hardware, and in-house expertise. Hybrid models are common: experiment or burst in the cloud and run steady training or sensitive workloads on premises. The same NVIDIA AI Enterprise software runs in both, which eases moving workloads.
- How it shows up
- A business constraint is described, and you pick on-prem, cloud or hybrid as the best fit.
- Common confusion
- Assuming cloud is always cheaper; for constant high utilization, owned infrastructure can cost less.
2.5 Identify key components and considerations of a cluster of an accelerated infrastructure.Compute nodes, management nodes, storage and the separate networks that join them.
- What it means
- This objective covers the building blocks of a GPU cluster: GPU compute nodes, head or management nodes running cluster software, high-performance shared storage, and several networks, typically a compute fabric for GPU traffic, a storage network, an in-band management network and an out-of-band network for baseboard management controllers.
- What to understand
- Separating networks keeps GPU-to-GPU traffic from competing with storage and management traffic. Management software such as NVIDIA Base Command Manager provisions and monitors nodes. Storage must deliver high parallel throughput, often through parallel file systems and GPUDirect Storage. Balance matters: fast GPUs with slow storage or an oversubscribed fabric waste capacity.
- How it shows up
- A cluster diagram or description is given, and you identify a missing or misused component.
- Common confusion
- Putting all traffic on one network and expecting full GPU performance.
2.6 Identify facility requirements.Space, floor loading, power delivery, cooling plant and cabling in the facility.
- What it means
- This covers what the building must provide for AI systems: enough floor space and weight capacity for heavy GPU racks, power distribution sized for high-density loads with redundancy, a cooling plant capable of air or liquid heat removal, and physical paths for large volumes of copper and optical cabling.
- What to understand
- Facility planning often sets the real limit on AI deployments. Liquid-cooled racks need coolant distribution units and facility water loops. Dense racks can exceed floor loading limits. Cable lengths and runs affect whether copper or optical links can be used. Colocation providers advertising AI readiness are judged on kilowatts per rack and liquid cooling support.
- How it shows up
- A site constraint is described, and you identify which facility requirement is missing.
- Common confusion
- Treating a facility as ready because it has rack space, without checking power, cooling and weight.
2.7 Determine networking requirements for AI workloads.Bandwidth, latency, east-west traffic and lossless behaviour.
- What it means
- This objective covers what AI traffic needs from the network. Distributed training generates heavy east-west traffic between GPUs on different nodes, with collective operations like all-reduce that are sensitive to both bandwidth and tail latency. Inference and storage traffic add their own demands.
- What to understand
- AI fabrics are usually designed non-blocking or close to it, often in a rail-optimized topology where each GPU's network adapter connects to its own leaf switch. RDMA lets data move between servers without involving the CPU, and GPUDirect RDMA lets network adapters read and write GPU memory directly. Packet loss and congestion stall collective operations, so lossless or congestion-controlled networking matters.
- How it shows up
- A traffic pattern or performance problem is described, and you identify the network requirement it reveals.
- Common confusion
- Sizing an AI network for north-south user traffic instead of east-west GPU traffic.
2.8 Identify and describe data center networking protocols and key concepts.RDMA, InfiniBand, RoCE, TCP/IP, congestion control and in-network computing.
- What it means
- This covers protocols used in AI data centers. TCP/IP carries general traffic. RDMA allows direct memory access between hosts with low CPU overhead. InfiniBand is a native RDMA fabric with credit-based lossless flow control. RoCE (RDMA over Converged Ethernet) runs RDMA on Ethernet and relies on mechanisms such as Priority Flow Control and ECN.
- What to understand
- Know the purpose of each term rather than packet formats. Subnet managers control InfiniBand fabrics. SHARP performs collective reductions inside InfiniBand switches, reducing data movement. Spectrum-X adds adaptive routing and congestion control to Ethernet for AI. Leaf-spine topologies give predictable hop counts. VLANs and VXLANs segment tenants in shared environments.
- How it shows up
- A protocol or concept is named or described, and you identify its role or benefit.
- Common confusion
- Thinking RDMA requires InfiniBand; RoCE delivers RDMA over Ethernet.
2.9 Identify high-speed data center network options and their use cases.NVLink, InfiniBand Quantum, Spectrum-X Ethernet and ConnectX adapters.
- What it means
- This objective covers the main NVIDIA networking products and where each fits. NVLink and NVSwitch connect GPUs within a server or NVLink domain. Quantum InfiniBand switches with ConnectX adapters provide high-bandwidth, low-latency scale-out fabrics at 400 Gb/s NDR and 800 Gb/s XDR. Spectrum-X Ethernet targets AI clouds that prefer Ethernet operations.
- What to understand
- Choose by scope and operating model. NVLink is the fastest option but only within its domain. InfiniBand is the established choice for large dedicated training clusters. Spectrum-X brings near-InfiniBand behaviour to Ethernet for multi-tenant AI clouds. Standard Ethernet remains suitable for management, storage in some designs and user access. Optical transceivers and cables link the faster speeds over distance.
- How it shows up
- A use case is described, and you choose the network option that matches its scope and requirements.
- Common confusion
- Using NVLink as if it were a data center-wide network between racks of servers.
2.10 Explain the purpose and benefits of a DPU in a data center.BlueField DPUs offload networking, storage and security from host CPUs.
- What it means
- This covers the data processing unit, NVIDIA's BlueField line. A DPU combines a high-speed network interface, Arm cores and hardware accelerators on one card. It takes over infrastructure tasks such as virtual switching, encryption, firewalling and storage virtualization so host CPUs can focus on applications.
- What to understand
- Benefits are offload, acceleration and isolation. Because security services run on the DPU rather than the host operating system, a compromised host has a harder time tampering with them, supporting zero-trust designs. Cloud providers use DPUs to run their infrastructure software separate from tenant workloads. Developers program BlueField with the DOCA software framework.
- How it shows up
- A host burdened with infrastructure tasks or needing isolated security is described, and you identify the DPU as the fix.
- Common confusion
- Confusing a DPU with a GPU; a DPU handles infrastructure tasks, not model training.
High-yield concepts
The terms, rules and formulas this domain turns into questions most often.
-
Scale-up vs scale-out
Scale-up adds GPUs within a node or NVLink domain; scale-out adds nodes over a fabric.
- Remember
- NVLink is scale-up; InfiniBand or Ethernet is scale-out.
- Tested as
- Scaling questions.
- Trap
- Mixing up which interconnect serves which.
-
NVLink and NVSwitch
High-bandwidth GPU-to-GPU links and the switch that connects them.
- Remember
- Fastest, but limited to its domain.
- Tested as
- Interconnect questions.
- Trap
- Using it between distant racks.
-
InfiniBand
Lossless, low-latency RDMA fabric for clusters.
- Remember
- Native RDMA and in-switch SHARP reductions.
- Tested as
- Network option questions.
- Trap
- Thinking it is just faster Ethernet.
-
RoCE
RDMA over Converged Ethernet.
- Remember
- Needs lossless or congestion-controlled Ethernet.
- Tested as
- Protocol questions.
- Trap
- Assuming RDMA needs InfiniBand.
-
Spectrum-X
NVIDIA Ethernet platform tuned for AI with adaptive routing.
- Remember
- Ethernet operations with AI-grade performance.
- Tested as
- Network option questions.
- Trap
- Treating it as InfiniBand.
-
GPUDirect RDMA and Storage
Direct data paths between GPU memory and network or storage, bypassing the CPU.
- Remember
- Cuts latency and CPU load.
- Tested as
- Networking and storage questions.
- Trap
- Assuming data must stage in system memory.
-
BlueField DPU
Card with networking, Arm cores and accelerators for infrastructure offload.
- Remember
- Offload, acceleration and isolation.
- Tested as
- DPU questions.
- Trap
- Using it to train models.
-
Rack power density
Power drawn per rack, now tens to over 100 kW for AI.
- Remember
- Drives cooling and facility design.
- Tested as
- Power and cooling questions.
- Trap
- Assuming legacy rack budgets apply.
-
Direct-to-chip liquid cooling
Cold plates carry heat from GPUs and CPUs to a coolant loop.
- Remember
- Required for the densest rack-scale systems.
- Tested as
- Cooling questions.
- Trap
- Thinking air cooling scales indefinitely.
-
PUE
Total facility power divided by IT equipment power.
- Remember
- Closer to 1.0 is more efficient.
- Tested as
- Efficiency questions.
- Trap
- Reading a higher PUE as better.
-
Rail-optimized topology
Each GPU's adapter connects to a dedicated leaf switch rail.
- Remember
- Shortens paths for collective traffic.
- Tested as
- Cluster network questions.
- Trap
- Treating it as a storage design.
-
DGX SuperPOD
NVIDIA reference architecture for large-scale AI clusters.
- Remember
- Built from scalable units of DGX systems.
- Tested as
- Cluster questions.
- Trap
- Thinking it is a single server.
-
Out-of-band management
Separate network for BMCs to manage hardware independent of the OS.
- Remember
- Works even when a node is down.
- Tested as
- Cluster component questions.
- Trap
- Mixing it with the compute fabric.
Side by side
Concepts candidates mix up, compared on the point that decides the answer.
AI interconnect options
| Concept | Meaning | Key difference | Common trap |
|---|---|---|---|
| NVLink / NVSwitch | Direct GPU-to-GPU links within a node or rack domain | Highest bandwidth, limited reach | Using it across the data center |
| InfiniBand | Native RDMA fabric with lossless flow control | Established choice for dedicated training clusters | Calling it a type of Ethernet |
| Spectrum-X Ethernet | Ethernet with AI-tuned routing and congestion control | Ethernet tooling with AI performance | Assuming plain Ethernet behaves the same |
| Standard Ethernet TCP/IP | General-purpose networking | Fine for management and user access | Using it for heavy all-reduce traffic |
On-premises vs cloud vs hybrid
| Concept | Meaning | Key difference | Common trap |
|---|---|---|---|
| On-premises | Owned GPU infrastructure in your facility | Control and lower cost at high steady utilization | Ignoring power, space and staffing needs |
| Cloud | Rented GPU capacity, including DGX Cloud | Fast start and elastic capacity | Assuming it is always cheaper |
| Hybrid | Steady work on premises, burst in cloud | Balances control and flexibility | Overlooking data movement costs |
Air vs liquid cooling
| Concept | Meaning | Key difference | Common trap |
|---|---|---|---|
| Air cooling | Fans and room air handlers remove heat | Suits moderate rack density | Applying it to the densest racks |
| Direct-to-chip liquid | Cold plates and coolant loops at the components | Handles very high rack power | Forgetting facility water and CDUs |
CPU, GPU and DPU roles
| Concept | Meaning | Key difference | Common trap |
|---|---|---|---|
| CPU | General-purpose processing and control | Runs the OS and application logic | Overloading it with infrastructure tasks |
| GPU | Parallel compute for AI math | Accelerates training and inference | Expecting it to run network services |
| DPU | Infrastructure processor on the network card | Offloads networking, storage and security | Confusing it with a GPU |
Exam traps
Each trap is an answer that looks right. Learn the clue that should change your mind.
-
More GPUs per node
- What candidates pick
- Add GPUs inside each server when nodes wait on each other.
- Why it looks right
- More compute sounds like the fix.
- Why it's wrong
- If GPUs idle while exchanging data between servers, the bottleneck is the scale-out network, not compute inside the node.
Clue to watch for Low utilization during cross-node communication.
-
NVLink across racks
- What candidates pick
- Pick NVLink to connect hundreds of servers in different rows.
- Why it looks right
- NVLink is the fastest interconnect.
- Why it's wrong
- NVLink serves GPUs within its domain; inter-node fabrics such as InfiniBand or Spectrum-X connect servers across the data center.
Clue to watch for The scenario spans many servers or racks.
-
RDMA means InfiniBand
- What candidates pick
- Reject an Ethernet design because RDMA is required.
- Why it looks right
- RDMA is closely associated with InfiniBand.
- Why it's wrong
- RoCE provides RDMA over Ethernet when the network is configured for lossless or controlled congestion.
Clue to watch for The organization wants RDMA but standardizes on Ethernet.
-
Cloud is always cheaper
- What candidates pick
- Move a constantly busy training platform to on-demand cloud to save money.
- Why it looks right
- Cloud avoids capital purchases.
- Why it's wrong
- At sustained high utilization, owned infrastructure often costs less over time, and data egress adds cost.
Clue to watch for Workload runs around the clock for years.
-
Air cooling is enough
- What candidates pick
- Deploy a rack-scale liquid-cooled GPU system in an air-only room.
- Why it looks right
- Existing rooms already cool servers.
- Why it's wrong
- The densest NVIDIA systems need direct-to-chip liquid cooling and facility water; air cannot remove that heat.
Clue to watch for Rack power well beyond typical enterprise levels.
-
DPU as accelerator for AI math
- What candidates pick
- Choose a BlueField DPU to speed up model training.
- Why it looks right
- It is also called an accelerator.
- Why it's wrong
- DPUs accelerate infrastructure work like networking, storage and security, freeing CPUs; GPUs do the model math.
Clue to watch for The problem is CPU time lost to infrastructure tasks.
-
Higher PUE is better
- What candidates pick
- Pick the facility with PUE 1.8 over 1.2.
- Why it looks right
- Bigger numbers look stronger.
- Why it's wrong
- PUE divides total facility power by IT power, so values closer to 1.0 mean less overhead.
Clue to watch for A question compares facility efficiency.
Answer framework
Apply these steps to infrastructure scenarios to find the constraint the question is really testing.
- Locate the problem: inside a GPU server, between servers, in storage, in the facility, or in a business decision such as where to host.
- For compute problems, compare GPU memory, GPU count and interconnect against the workload, and decide between scale-up within an NVLink domain or scale-out across nodes.
- For network problems, identify east-west GPU traffic and choose an RDMA fabric such as InfiniBand or RoCE-capable Ethernet, adding GPUDirect where CPU staging is the issue.
- For facility problems, check rack power density, cooling method, floor loading and cabling before assuming hardware can be installed as is.
- For hosting and offload questions, weigh utilization, data location and expertise for on-prem versus cloud, and pick a DPU when host CPUs are burdened with infrastructure work.
Practice question
Want more like this? The NCA-AIIO practice test has questions for every domain.
Checklist
0 of 14
Tick each one you can do without notes. Your ticks stay in this browser.
Study strategy
This is the biggest domain, so give it the largest block of study time and split it into three parts. First, compute: learn what limits a training job (GPU memory, GPU count, interconnect and storage) and the difference between scaling up inside an NVLink domain and scaling out across nodes.
Second, the physical layer. Learn rough numbers rather than exact specifications: an eight-GPU server at around 10 kW, rack-scale systems above 100 kW, and why that forces liquid cooling. Add PUE, redundancy and facility checks, then the on-prem versus cloud trade-offs.
Third, networking, which covers four objectives on its own. Draw a cluster with separate compute, storage, in-band and out-of-band networks. Build a table comparing NVLink, InfiniBand, RoCE on Spectrum-X and standard Ethernet by reach, latency and typical use. Finish with the DPU and how it differs from CPUs and GPUs. NVIDIA's DGX SuperPOD and BasePOD reference architecture documents are useful reading for seeing how the pieces combine. Test yourself with scenario questions and, for each miss, name the constraint you overlooked.
FAQ
How much of NCA-AIIO is AI Infrastructure?
AI Infrastructure is 40% of the exam, about 20 of 50 questions, and is the largest domain with ten official objectives.
Do I need to know exact GPU specifications?
No. Know relative capabilities and rough figures, such as typical rack power for GPU servers, rather than memorizing data sheet numbers.
What is the difference between InfiniBand and RoCE?
Both carry RDMA. InfiniBand is a dedicated fabric with built-in lossless flow control. RoCE runs RDMA over Ethernet and depends on Ethernet congestion handling.
When should I choose NVLink over a network fabric?
NVLink connects GPUs within a server or rack-scale NVLink domain at the highest bandwidth. Communication between servers beyond that domain uses InfiniBand or Ethernet.
What does a DPU do that a SmartNIC or CPU does not?
A BlueField DPU adds programmable Arm cores and accelerators to the network card, so it can run full infrastructure services in isolation from the host.
Why is liquid cooling covered?
GPU rack power has grown beyond what air can remove, and the densest NVIDIA rack-scale systems are designed for direct-to-chip liquid cooling.
Is this domain the same as the NCP-AII professional exam?
No. NCA-AIIO tests recognition of concepts and choices. The professional NCP-AII exam goes into installation, validation and hands-on configuration.
Other Artificial Intelligence & Machine Learning certifications
Where the same subject shows up in other exams. Cards marked "Same subject" open the matching domain.
Test your Domain 2 knowledge
You have covered GPU sizing, scaling, power and cooling, hosting models, cluster design, AI networking and DPUs. Practise picking the right infrastructure choice for a scenario before exam day.
Keep going
- Training Training Built on your Weak areas
- Practice test Questions by domain, with explanations
- Mock exam Full length, timed like the real exam
- Readiness test Find your weakest domain in minutes
- Cheat sheet Key terms and formulas on one page
- Ask the NCA-AIIO AI tutor Get any concept explained your way