Overview
Domain 3 shifts from building AI infrastructure to keeping it productive. It has four objectives: AI data center management and monitoring essentials, cluster orchestration and job scheduling, the measures used to monitor GPUs, and considerations for virtualizing accelerated infrastructure.
At 22%, it is the smallest domain, but each objective is dense. You should recognize what tools such as nvidia-smi, DCGM, Base Command Manager, Kubernetes with the NVIDIA GPU Operator, Slurm and Run:ai are for, which GPU metrics reveal which problems, and how Multi-Instance GPU, NVIDIA vGPU and passthrough differ when sharing GPUs. Questions describe an operational symptom or goal and ask for the right tool, metric or sharing approach, not the exact command.
How it's tested
Domain 3 items usually present an operational situation, such as idle GPUs, a job queue backlog, an overheating node or teams that need to share hardware, and ask which tool, metric or virtualization approach applies. Expect tool-to-task matching and metric interpretation rather than syntax or step-by-step procedures.
-
Choose the operations tool
A management or monitoring goal is described, and you choose between nvidia-smi, DCGM, Base Command Manager, Prometheus and Grafana, or a BMC interface.
-
Pick the scheduler
A workload mix is described, such as batch HPC jobs or containerized microservices, and you select Slurm, Kubernetes or a GPU-aware scheduling layer.
-
Read the metric
A GPU symptom such as low utilization, high temperature or rising error counts is given, and you identify the likely cause or the next metric to check.
Several research teams each need a small, isolated slice of GPU capacity for inference tests on an H100 server, with reserved memory and no noisy-neighbour effects. Time-sliced sharing would let one team's load slow another. Multi-Instance GPU splits the GPU into up to seven hardware-isolated instances, each with dedicated memory and compute. The words reserved and isolated point to MIG.
Objectives 4
Official objectives from the NVIDIA outline, in outline order.
3.1 Describe AI data center management and monitoring essentials.Provisioning, health monitoring, telemetry and alerting for AI clusters.
- What it means
- This objective covers how operators keep an AI data center healthy. It includes provisioning and configuring nodes, monitoring hardware and software health, collecting telemetry, alerting on problems, and managing firmware and driver versions consistently across many GPU servers.
- What to understand
- Know the tool layers. nvidia-smi gives a quick per-node view. NVIDIA Data Center GPU Manager (DCGM) collects GPU telemetry and runs health checks and diagnostics at cluster scale, and its exporter feeds Prometheus and Grafana dashboards. NVIDIA Base Command Manager provisions and manages clusters. Baseboard management controllers, reached through IPMI or Redfish over the out-of-band network, report power, fans and hardware events.
- How it shows up
- An operations goal is described, and you choose the tool or practice that achieves it.
- Common confusion
- Relying on nvidia-smi for fleet-wide monitoring; DCGM is designed for cluster scale.
3.2 Describe AI cluster orchestration and job scheduling essentials.Kubernetes, Slurm, GPU Operator and GPU-aware scheduling.
- What it means
- This covers how work is placed onto GPUs. Slurm is a batch scheduler common in HPC and large training clusters, queueing jobs and allocating whole nodes or GPUs. Kubernetes orchestrates containers and suits inference services and MLOps pipelines. The NVIDIA GPU Operator automates drivers, the device plugin and monitoring on Kubernetes nodes.
- What to understand
- Match the scheduler to the workload. Long multi-node training jobs fit Slurm's queues, priorities and fair-share policies. Containerized, always-on services fit Kubernetes with autoscaling. Run:ai, now part of NVIDIA, adds GPU pooling, quotas and fractional allocation on Kubernetes. Gang scheduling, which starts all parts of a distributed job together, avoids partially started jobs holding GPUs idle.
- How it shows up
- A workload or team requirement is described, and you choose the scheduler or orchestration feature.
- Common confusion
- Assuming Kubernetes recognizes GPUs out of the box; it needs the device plugin, usually installed by the GPU Operator.
3.3 Articulate the key measures and criteria related to monitoring GPUs.Utilization, memory, temperature, power, clocks, errors and interconnect health.
- What it means
- This objective covers which GPU metrics operators watch and what they mean: GPU and Tensor Core utilization, memory used and memory bandwidth, temperature, power draw against the limit, clock speeds and throttle reasons, ECC memory errors, XID error events, and NVLink and PCIe throughput.
- What to understand
- Read metrics together. Low GPU utilization with high CPU or I/O wait suggests a data pipeline bottleneck rather than a slow GPU. High temperature plus reduced clocks indicates thermal throttling. Rising correctable ECC errors can foreshadow failure, while uncorrectable errors or certain XID codes call for draining the node. Memory nearly full with low compute often means oversized batches or a memory leak.
- How it shows up
- A metric reading or symptom is described, and you identify the cause or the action.
- Common confusion
- Equating high memory usage with high compute utilization; they measure different things.
3.4 Identify the key considerations for virtualizing accelerated infrastructure.GPU passthrough, NVIDIA vGPU, MIG, licensing and performance trade-offs.
- What it means
- This covers ways to share or assign GPUs in virtualized environments. Passthrough gives one virtual machine a whole physical GPU. NVIDIA vGPU software lets several VMs share a GPU through time slicing with defined profiles. Multi-Instance GPU partitions supported data center GPUs into isolated hardware instances, which can also be combined with vGPU.
- What to understand
- Consider isolation, performance, density and features. Passthrough is closest to bare metal but serves one VM per GPU. vGPU raises density and supports features like live migration on supported hypervisors but shares compute over time. MIG gives dedicated memory and compute slices with fault isolation. vGPU requires NVIDIA licensing, often through NVIDIA AI Enterprise, and hypervisor support must be confirmed.
- How it shows up
- A sharing requirement is described, and you choose passthrough, vGPU or MIG.
- Common confusion
- Treating MIG and vGPU as the same; MIG partitions hardware, vGPU time-slices it.
High-yield concepts
The terms, rules and formulas this domain turns into questions most often.
-
nvidia-smi
Command-line utility showing GPU status on one node.
- Remember
- Good for quick checks, not fleet monitoring.
- Tested as
- Tool selection questions.
- Trap
- Using it as a cluster monitoring system.
-
DCGM
Data Center GPU Manager for telemetry, health checks and diagnostics.
- Remember
- Scales across clusters and exports to Prometheus.
- Tested as
- Monitoring questions.
- Trap
- Confusing it with a scheduler.
-
Base Command Manager
NVIDIA software for provisioning and managing AI clusters.
- Remember
- Handles node images, configuration and monitoring.
- Tested as
- Management questions.
- Trap
- Treating it as a model server.
-
Slurm
Batch workload manager widely used for HPC and training.
- Remember
- Queues, priorities and node allocation.
- Tested as
- Scheduling questions.
- Trap
- Using it for always-on microservices.
-
Kubernetes with GPU Operator
Container orchestration made GPU-aware by NVIDIA's operator.
- Remember
- Operator installs drivers, device plugin and monitoring.
- Tested as
- Orchestration questions.
- Trap
- Expecting GPUs to appear without the device plugin.
-
Run:ai
NVIDIA GPU orchestration layer for Kubernetes with pooling and quotas.
- Remember
- Improves sharing and utilization across teams.
- Tested as
- Scheduling questions.
- Trap
- Thinking it replaces Kubernetes.
-
Thermal throttling
GPU lowers clocks when it gets too hot or hits power limits.
- Remember
- Shows as high temperature and reduced clocks.
- Tested as
- Metric interpretation.
- Trap
- Blaming the application code.
-
ECC and XID errors
ECC tracks memory errors; XID events log driver-reported GPU faults.
- Remember
- Rising or uncorrectable errors signal hardware trouble.
- Tested as
- Health questions.
- Trap
- Ignoring correctable errors as harmless.
-
Multi-Instance GPU
Hardware partitioning of a GPU into up to seven isolated instances.
- Remember
- Dedicated memory and compute per instance.
- Tested as
- Virtualization questions.
- Trap
- Calling it time slicing.
-
NVIDIA vGPU
Software that time-slices a GPU among VMs using profiles.
- Remember
- Requires NVIDIA licensing.
- Tested as
- Virtualization questions.
- Trap
- Assuming hardware isolation.
-
GPU passthrough
Assigns an entire physical GPU to one VM.
- Remember
- Near bare-metal performance, lowest density.
- Tested as
- Virtualization questions.
- Trap
- Expecting multiple VMs to share it.
Side by side
Concepts candidates mix up, compared on the point that decides the answer.
GPU sharing options
| Concept | Meaning | Key difference | Common trap |
|---|---|---|---|
| Passthrough | Whole GPU dedicated to one VM | Best performance, no sharing | Expecting high density |
| NVIDIA vGPU | Time-sliced sharing across VMs | High density, licensed, supports migration | Assuming reserved compute |
| MIG | Hardware partitions with dedicated memory and compute | Strong isolation and predictable performance | Confusing it with time slicing |
Scheduling and orchestration
| Concept | Meaning | Key difference | Common trap |
|---|---|---|---|
| Slurm | Batch job scheduler | Queues and allocates nodes for long jobs | Using it for web-style services |
| Kubernetes | Container orchestrator | Runs and scales long-lived services | Forgetting the GPU device plugin |
| Run:ai | GPU-aware scheduling layer on Kubernetes | Pools and shares GPUs across teams | Treating it as a separate orchestrator |
Monitoring tools
| Concept | Meaning | Key difference | Common trap |
|---|---|---|---|
| nvidia-smi | Per-node GPU status utility | Quick local view | Using it for fleet monitoring |
| DCGM | Cluster-scale GPU telemetry and diagnostics | Health checks and exporter for dashboards | Calling it a scheduler |
| BMC via IPMI or Redfish | Out-of-band hardware management | Works when the OS is down | Expecting GPU kernel metrics from it |
Exam traps
Each trap is an answer that looks right. Learn the clue that should change your mind.
-
nvidia-smi for the fleet
- What candidates pick
- Use nvidia-smi to monitor hundreds of GPU nodes centrally.
- Why it looks right
- It is the best-known GPU tool.
- Why it's wrong
- nvidia-smi reports on a single node; DCGM gathers telemetry, health checks and diagnostics across clusters.
Clue to watch for The scenario mentions many nodes or dashboards.
-
Low utilization means slow GPUs
- What candidates pick
- Replace GPUs when utilization is low.
- Why it looks right
- Low utilization looks like underperforming hardware.
- Why it's wrong
- Low GPU utilization is often caused by data loading, storage or CPU bottlenecks starving the GPU.
Clue to watch for CPU or I/O wait is high while the GPU sits idle.
-
MIG is time slicing
- What candidates pick
- Choose vGPU when teams need reserved, isolated GPU memory.
- Why it looks right
- Both let several users share a GPU.
- Why it's wrong
- vGPU time-slices a GPU among VMs; MIG provides hardware-isolated instances with dedicated memory and compute.
Clue to watch for Words like reserved, isolated or predictable.
-
Kubernetes sees GPUs natively
- What candidates pick
- Deploy GPU pods without any NVIDIA components.
- Why it looks right
- Kubernetes schedules CPU and memory by default.
- Why it's wrong
- GPUs must be advertised by the NVIDIA device plugin, typically installed with the GPU Operator along with drivers.
Clue to watch for Pods stay pending on GPU nodes.
-
Correctable errors are harmless
- What candidates pick
- Ignore rising correctable ECC counts.
- Why it looks right
- The errors were corrected.
- Why it's wrong
- A rising trend can signal failing memory and should trigger investigation before uncorrectable errors stop jobs.
Clue to watch for Error counts increase steadily on one GPU.
-
Slurm for microservices
- What candidates pick
- Run an autoscaling inference API under Slurm batch jobs.
- Why it looks right
- Slurm is the standard in many GPU clusters.
- Why it's wrong
- Slurm is built for queued batch jobs, while long-lived, autoscaling services suit Kubernetes.
Clue to watch for The workload is an always-on service.
Answer framework
Work through these steps for any operations question about monitoring, scheduling or GPU sharing.
- Classify the question as monitoring and management, scheduling and orchestration, metric interpretation, or virtualization and sharing.
- For monitoring, pick the tool by scope: nvidia-smi for one node, DCGM with Prometheus and Grafana for the cluster, BMC interfaces for hardware out of band.
- For scheduling, identify whether work is queued batch jobs, which favours Slurm, or containerized services, which favour Kubernetes with the GPU Operator and optionally Run:ai.
- For metrics, read several signals together, such as utilization with I/O wait or temperature with clock speed, before blaming the GPU itself.
- For sharing, match the isolation need: passthrough for a whole GPU, vGPU for dense time-sliced VMs, MIG for reserved hardware-isolated slices.
Practice question
Want more like this? The NCA-AIIO practice test has questions for every domain.
Checklist
0 of 12
Tick each one you can do without notes. Your ticks stay in this browser.
Study strategy
This domain is compact, so aim for complete coverage rather than depth in any single tool. Begin with a monitoring map that has three levels: nvidia-smi on a single node, DCGM across the cluster feeding Prometheus and Grafana, and BMCs reached out of band for hardware events. Add Base Command Manager as the provisioning and management layer.
Next, compare Slurm and Kubernetes in a two-column table covering workload type, unit of work, and how each becomes GPU-aware. Note what the GPU Operator and Run:ai contribute.
Then study GPU metrics by symptom rather than by list. For each of low utilization, high temperature, falling clocks, rising ECC counts and XID events, write the likely cause and the first check. Finally, learn the three GPU sharing models and practise choosing between them from a short requirement. If you can access a GPU machine or a cloud instance, running nvidia-smi and the DCGM diagnostics once makes the metrics far easier to remember. Close with mixed questions across all three domains, because operations items often reuse infrastructure vocabulary.
FAQ
How much of NCA-AIIO is AI Operations?
AI Operations is 22% of the exam, around 11 of 50 questions, spread across four official objectives.
Do I need to memorize nvidia-smi or DCGM commands?
No. Know what each tool is for and which metrics it reports. The associate exam does not test command syntax.
What is the difference between MIG and vGPU?
MIG splits a supported GPU into hardware-isolated instances with dedicated memory and compute. vGPU shares a GPU among virtual machines by time slicing under a license.
When is Slurm preferred over Kubernetes?
Slurm suits queued batch jobs such as multi-node training, with priorities and fair-share policies. Kubernetes suits containerized, long-running services and pipelines.
Which GPU metrics matter most?
Utilization, memory use, temperature, power, clock speeds and throttle reasons, ECC and XID errors, and interconnect throughput. Interpret them together rather than one at a time.
What does the NVIDIA GPU Operator do?
It automates installing and managing the NVIDIA driver, container toolkit, device plugin and DCGM exporter on Kubernetes nodes.
Other Artificial Intelligence & Machine Learning certifications
Where the same subject shows up in other exams. Cards marked "Same subject" open the matching domain.
Test your Domain 3 knowledge
You have covered AI data center monitoring, job scheduling, GPU health metrics and GPU virtualization. See whether you can pick the right tool or sharing model for an operational scenario.
Keep going
- Training Training Built on your Weak areas
- Practice test Questions by domain, with explanations
- Mock exam Full length, timed like the real exam
- Readiness test Find your weakest domain in minutes
- Cheat sheet Key terms and formulas on one page
- Ask the NCA-AIIO AI tutor Get any concept explained your way