Home / artificial-intelligence-machine-learning / NVIDIA-Certified Professional: AI Infrastructure (NCP-AII) / Cheat Sheet
NVIDIA-Certified Professional: AI Infrastructure (NCP-AII)

NCP-AII Cheat Sheet

NCP-AII Tests Bringing a GPU Cluster Up and Proving It Works — Not Buying One

A hands-on professional exam for engineers who rack, image, test and repair DGX and HGX clusters. Think in deployment order: firmware baseline before any test, single-node before multi-node, evidence before swapping parts.

Check Your Readiness →
Among the harder certs
Avg: Not published by NVIDIA
Pass: 750 / 1000
Most candidates understand NVIDIA-Certified Professional: AI Infrastructure (NCP-AII) concepts — and still fail. This exam tests how you apply knowledge under pressure.

NCP-AII Domain Map (SEP25 study guide)

Questions are operational: a stage of a DGX or HGX rollout, an nccl-tests table, an HPL summary or an XID line, and you choose the next step or the faulty layer. Testing and bring-up together make up 64%, so sequencing and output reading decide the result.

  1. 01
    Cluster Test and Verification (33%) — single-node stress, HPL, nccl-tests, cable signal quality, firmware checks, ClusterKit, burn-in
  2. 02
    System and Server Bring-Up (31%) — deployment order, BMC, Redfish, TPM, validated firmware bundles, power, cooling and cabling
  3. 03
    Control Plane Installation and Configuration (19%) — Base Command Manager HA, categories, Slurm with Enroot and Pyxis, drivers, Container Toolkit, NGC CLI
  4. 04
    Troubleshooting and Optimization (12%) — XID errors, BMC SEL, part replacement, AMD NPS vs. Intel SNC, storage paths
  5. 05
    Physical Layer Management (5%) — BlueField DPU vs. NIC mode, rshim and BFB images, mlxconfig, MIG partitioning

Wrong instinct vs correct approach

Single-node all_reduce_perf on a DGX shows bus bandwidth far below expected
✕ Wrong instinct

Investigate the InfiniBand compute fabric and cabling

✓ Correct approach

A single-node NCCL test never touches the network; traffic runs over NVLink and NVSwitch. Check nvidia-smi nvlink link state and speed, and confirm Fabric Manager is running at the driver's version.

nvidia-smi lists seven GPUs on an eight-GPU server and the BMC inventory also shows seven
✕ Wrong instinct

Reinstall the GPU driver and reboot the node

✓ Correct approach

If the BMC, which works without the host OS, also misses the device, the cause is hardware, seating, power or firmware. Check the SEL and XID logs, run nvidia-bug-report.sh, drain the node, then locate the GPU by bus ID rather than index.

A containerized training job sees no GPUs although nvidia-smi works on the host
✕ Wrong instinct

Install the GPU driver inside the container image

✓ Correct approach

The NVIDIA Container Toolkit mounts the host driver into containers; a driver in the image conflicts with the kernel module. Configure and restart the runtime, then pass --gpus. Under Slurm, Pyxis adds --container-image to srun but needs Enroot installed.

An HPL run reports high TFLOPS but the output shows FAILED beside the residual
✕ Wrong instinct

Record the TFLOPS figure as the node's acceptance result

✓ Correct approach

A failed residual means the computation was wrong, so the run is invalid regardless of speed. Investigate the node, retune N, NB and the P x Q grid within memory, and rerun before burn-in.

Jobs trip rack breakers only once GPUs reach full load in a new deployment
✕ Wrong instinct

Replace the PSUs, since idle power readings already passed validation

✓ Correct approach

Peak draw appears only under full GPU load, and an eight-GPU server can draw around 10 kW. Plan density by rack kilowatts and cooling, not rack units, and validate power under stress.

Know these cold

  • ▸ Order — ack, cable, power, BMC, firmware baseline, OS, single-node tests, multi-node tests, burn-in
  • ▸ The BMC works with the host off; Redfish is the scriptable standard, IPMI the legacy protocol
  • ▸ OOB carries BMC and switch management only, never storage, job or provisioning data
  • ▸ Passive DAC shortest reach, then active copper, then AOC, then transceiver plus fiber
  • ▸ BlueField DPU mode runs Arm services in the path; NIC mode takes the Arm cores out
  • ▸ mlxconfig settings are non-volatile and apply only after a firmware reset or power cycle
  • ▸ Pyxis is the srun plugin; Enroot is the daemonless runtime it depends on
  • ▸ nvcr.io login — sername $oauthtoken, password is the NGC API key, not your account password
  • ▸ Active links can still fail — heck mlxlink errors, speed and width, and ibdiagnet topology versus plan
  • ▸ One correctable ECC error or pending row remap needs a GPU reset, not a replacement

Can you answer these without checking your notes?

In this scenario: "Single-node all_reduce_perf on a DGX shows bus bandwidth far below expected" — what should you do first?
A single-node NCCL test never touches the network; traffic runs over NVLink and NVSwitch. Check nvidia-smi nvlink link state and speed, and confirm Fabric Manager is running at the driver's version.
In this scenario: "nvidia-smi lists seven GPUs on an eight-GPU server and the BMC inventory also shows seven" — what should you do first?
If the BMC, which works without the host OS, also misses the device, the cause is hardware, seating, power or firmware. Check the SEL and XID logs, run nvidia-bug-report.sh, drain the node, then locate the GPU by bus ID rather than index.
In this scenario: "A containerized training job sees no GPUs although nvidia-smi works on the host" — what should you do first?
The NVIDIA Container Toolkit mounts the host driver into containers; a driver in the image conflicts with the kernel module. Configure and restart the runtime, then pass --gpus. Under Slurm, Pyxis adds --container-image to srun but needs Enroot installed.

Common Exam Mistakes — What candidates get wrong

Running performance tests before every node matches the validated firmware bundle

NVIDIA validates BMC, BIOS, GPU, NVSwitch and NIC firmware as one set. Updating one component from a release note creates an untested combination, and spare parts may ship with other versions, so align firmware before acceptance tests and after any replacement.

Comparing nccl-tests algbw directly against NVLink or NIC link speed

algbw depends on the collective; busbw is normalized so it can be compared with hardware bandwidth. Run all_reduce_perf on one node first, which exercises NVLink and NVSwitch only, then multi-node across the rails.

Upgrading the GPU driver alone on an NVSwitch-based HGX system

Fabric Manager must run the same version as the GPU driver, or the NVSwitch fabric does not initialize and multi-GPU work fails. The DOCA host stack for ConnectX and BlueField is a separate install from the GPU driver.

Patching compute nodes by hand on a Base Command Manager cluster

BCM reprovisions nodes from their software image, so manual edits vanish. Change the node category or the software image instead. BCM HA counts as verified only after status checks pass and a controlled head-node failover succeeds.

Creating MIG instances straight after enabling MIG mode

MIG mode takes effect only after a GPU reset or reboot. Profiles are fixed sizes such as 1g.10gb, 3g.40gb and 7g.80gb on an 80 GB GPU, up to seven instances. Time-slicing shares a GPU without memory or fault isolation.

Resources to help you pass NVIDIA AI Infrastructure Professional on your first attempt

NCP-AII rewards doing cluster bring-up steps in the right order. Find out where your sequencing slips.