Overview
Software Development has seven official objectives and the second-largest weight. Three restate Domain 1 bullets from an engineering angle: deploying and evaluating models for scalability, performance and reliability, building RAG systems, chatbots and summarizers, and using Python packages for language work and classic analyses. The other three are new: identifying the data, hardware and software components a solution needs, monitoring data collection, experiments and software processes, and writing software components or scripts under supervision.
The study guide's readings set the scope: TensorRT, NeMo best practices, LLM customization, INT8 quantization-aware training, NCCL and all-reduce for multi-GPU communication, Hugging Face datasets, scaling laws and BERT. Expect questions about how a model gets from a checkpoint to a fast, observable service on NVIDIA GPUs.
How it's tested
These questions describe a deployment or development task and ask which NVIDIA component, hardware configuration or coding practice fits. Some need quick arithmetic, such as estimating weight memory. Others test whether you can place a tool correctly in the stack, from CUDA and NCCL through TensorRT-LLM and Triton to NIM microservices and application frameworks.
-
Place the tool
A serving or optimization need is described, and you choose among TensorRT-LLM, Triton Inference Server, NIM, NeMo or a framework.
-
Size the hardware
A model size and precision are given and you estimate GPU memory or choose a parallelism strategy.
-
Keep it observable
A production or experiment process misbehaves, and you choose the metric, log or monitoring tool that reveals the cause.
A team wants to run a Llama-based model behind an OpenAI-compatible API with optimized inference and minimal setup. Building a TensorRT-LLM engine and configuring Triton manually would work but takes effort. An NVIDIA NIM microservice packages an optimized engine, a serving runtime and a standard API in one container. The clues are minimal setup and a standard API, which point to NIM.
Objectives 7
Official objectives from the NVIDIA outline, in outline order.
4.1 Assist in the deployment and evaluations of model scalability, performance, and reliability under the supervision of a senior team member.The NVIDIA serving stack and inference optimization.
- What it means
- This covers the engineering side of deployment: optimizing a model for inference, serving it at scale and confirming it stays reliable as load grows.
- What to understand
- TensorRT-LLM compiles models into optimized engines with fused kernels, in-flight batching, paged KV cache and FP8 or INT8 quantization. Triton Inference Server hosts models from many backends, adds dynamic batching, model ensembles and metrics. NVIDIA NIM wraps optimized engines and a server into a container with an industry-standard API. Kubernetes autoscaling adds replicas as traffic grows.
- How it shows up
- A deployment requirement is given and you choose the component or optimization that meets it.
- Common confusion
- Treating TensorRT-LLM as a server; it optimizes the model, while Triton or NIM serves it.
4.2 Build LLM use cases such as RAGs, chatbots, and summarizers.Coding LLM applications with frameworks and NVIDIA services.
- What it means
- Where Domain 1 explains these patterns, this objective covers wiring them together in code: calling model endpoints, orchestrating retrieval, managing state and adding safeguards.
- What to understand
- Frameworks such as LangChain and LlamaIndex chain loaders, splitters, retrievers and LLM calls. NIM endpoints accept OpenAI-style requests, so standard client libraries work. Streaming tokens improves perceived latency in chat interfaces. Tool or function calling lets a model trigger external actions. NeMo Guardrails adds programmable rails around a conversation, and error handling with retries and timeouts keeps applications stable.
- How it shows up
- An application requirement is described and you pick the component or code pattern that implements it.
- Common confusion
- Expecting the LLM to manage chat history by itself; the application must store and resend it.
4.3 Familiarity with the capabilities of Python natural language packages (spaCy, NumPy, vector databases, etc.) (also objective 1.6)Integrating NLP libraries and vector stores in application code.
- What it means
- From a developer's view, this objective is about choosing and calling the right library inside a pipeline, and connecting it to data sources and vector stores.
- What to understand
- Hugging Face Transformers and Tokenizers load models and their matching tokenizers, and the Datasets library streams large corpora efficiently. spaCy handles preprocessing steps such as sentence splitting before chunking. Vector database clients for Milvus or pgvector insert embeddings with metadata and run filtered similarity queries. NumPy and CuPy arrays move data between steps, with CuPy running on GPUs.
- How it shows up
- A pipeline step is missing and you choose the library call or component that fills it.
- Common confusion
- Loading a model with a tokenizer from a different checkpoint.
4.4 Identify system data, hardware, or software components required to meet user needs.GPU memory sizing, parallelism and the software stack.
- What it means
- This objective asks you to translate a requirement into components: how much GPU memory, how many GPUs, which interconnect and which software layers.
- What to understand
- Weight memory is roughly parameters times bytes per parameter: two bytes in FP16 or BF16, one in INT8 or FP8. Add KV cache, activations and overhead for serving, and optimizer states for training. Models too large for one GPU use tensor or pipeline parallelism; data parallelism copies the model to scale throughput. NVLink and NCCL all-reduce move gradients and activations between GPUs. CUDA and cuDNN sit beneath every framework.
- How it shows up
- A model size, precision and workload are given and you estimate memory or choose the hardware and parallelism.
- Common confusion
- Confusing data parallelism, which replicates the model, with tensor parallelism, which splits it.
4.5 Monitor functioning of data collection, experiments, and other software processes.Logging, metrics and alerts for pipelines and services.
- What it means
- Running systems need observation. This covers watching data collection jobs, training experiments and inference services so failures and drift are caught early.
- What to understand
- Triton and NIM expose Prometheus-format metrics such as request counts, queue time and latency that dashboards like Grafana display. NVIDIA DCGM and nvidia-smi report GPU utilization, memory and temperature. Experiment trackers such as MLflow, Weights & Biases or TensorBoard record runs. For data pipelines, monitor row counts, schema changes and error rates, and alert on thresholds.
- How it shows up
- A process degrades silently and you choose the metric or tool that would have revealed it.
- Common confusion
- Watching only GPU utilization while queue time and error rate reveal the real problem.
4.6 Use Python packages (spaCy, NumPy, Keras, etc.) to implement specific traditional machine learning analyses (also objective 1.10)Packaging classic analyses as reusable code.
- What it means
- In this domain the same analyses are treated as software: reusable functions, pinned dependencies and code that runs reliably beyond a notebook.
- What to understand
- Wrap preprocessing and model steps in functions or scikit-learn style pipelines so training and inference transform data identically. Pin package versions and use NVIDIA NGC containers for consistent CUDA and framework versions. Save trained models with their preprocessing objects. GPU versions such as cuML can replace CPU estimators with few code changes.
- How it shows up
- A model behaves differently in production than in testing and you identify the packaging fault.
- Common confusion
- Fitting a vectorizer again at inference time instead of reusing the one fitted on training data.
4.7 Write software components or scripts under the supervision of a senior team member.Writing, testing and reviewing scripts and components.
- What it means
- Associates write data loaders, API wrappers, evaluation scripts and automation that others review and maintain.
- What to understand
- Use version control, small functions with clear inputs and outputs, unit tests and code review. Read configuration such as endpoints and keys from environment variables rather than hard-coding secrets. Handle API errors with retries and backoff, batch requests to use GPUs efficiently, and log enough context to debug failures.
- How it shows up
- A script has a reliability or security flaw and you identify the coding practice that fixes it.
- Common confusion
- Hard-coding API keys in scripts committed to version control.
High-yield concepts
The terms, rules and formulas this domain turns into questions most often.
-
TensorRT-LLM
NVIDIA library that compiles LLMs into optimized inference engines.
- Remember
- Provides in-flight batching, paged KV cache and quantization.
- Tested as
- Optimization questions.
- Trap
- Calling it a serving platform.
-
Triton Inference Server
Open-source server hosting models from many frameworks.
- Remember
- Offers dynamic batching, ensembles and metrics endpoints.
- Tested as
- Serving questions.
- Trap
- Thinking it only serves TensorRT models.
-
NVIDIA NIM
Prebuilt inference microservices with optimized engines and standard APIs.
- Remember
- Fastest route to a production endpoint on NVIDIA GPUs.
- Tested as
- Deployment choice questions.
- Trap
- Confusing it with a training framework.
-
NeMo framework
NVIDIA's framework for building, customizing and aligning generative models.
- Remember
- Covers training and fine-tuning, not low-latency serving.
- Tested as
- Tool placement questions.
- Trap
- Choosing it to serve production traffic.
-
Weight memory rule
Parameters times bytes per parameter.
- Remember
- FP16 uses 2 bytes, so 7B needs about 14 GB.
- Tested as
- Sizing questions.
- Trap
- Forgetting KV cache and overhead on top.
-
KV cache
Stored attention keys and values for tokens already processed.
- Remember
- Grows with context length and concurrent requests.
- Tested as
- Inference memory questions.
- Trap
- Ignoring it when sizing a server.
-
In-flight batching
Adds and removes requests from a running batch at each generation step.
- Remember
- Raises GPU utilization for variable-length outputs.
- Tested as
- Throughput questions.
- Trap
- Confusing it with static batching.
-
Quantization
Lower-precision weights and activations such as FP8, INT8 or INT4.
- Remember
- Saves memory and speeds inference, with possible accuracy loss.
- Tested as
- Optimization questions.
- Trap
- Assuming it is lossless.
-
Tensor vs. data parallelism
Tensor parallelism splits layers across GPUs; data parallelism copies the model.
- Remember
- Use tensor parallelism when a model will not fit on one GPU.
- Tested as
- Multi-GPU questions.
- Trap
- Using data parallelism to fit a large model.
-
NCCL and all-reduce
NVIDIA's collective communication library; all-reduce sums and shares values across GPUs.
- Remember
- Used to synchronize gradients in distributed training.
- Tested as
- Distributed training questions.
- Trap
- Thinking it is a storage protocol.
-
Prometheus metrics
Text-format metrics exposed by Triton and NIM for scraping.
- Remember
- Track latency, queue time and request counts.
- Tested as
- Monitoring questions.
- Trap
- Relying on logs alone.
-
OpenAI-compatible API
Request format many NIM LLM endpoints accept.
- Remember
- Standard client libraries work with little change.
- Tested as
- Integration questions.
- Trap
- Assuming NVIDIA endpoints need a custom SDK.
Side by side
Concepts candidates mix up, compared on the point that decides the answer.
NVIDIA inference stack
| Concept | Meaning | Key difference | Common trap |
|---|---|---|---|
| TensorRT-LLM | Optimizes and compiles the model | Engine building and kernels | Treating it as the API server |
| Triton Inference Server | Serves models with batching and metrics | Multi-framework hosting | Thinking it optimizes model weights itself |
| NVIDIA NIM | Packaged microservice with engine, server and API | Least setup for production | Treating it as a training tool |
| NeMo framework | Builds, customizes and aligns models | Training side of the lifecycle | Choosing it for low-latency serving |
Multi-GPU strategies
| Concept | Meaning | Key difference | Common trap |
|---|---|---|---|
| Data parallelism | Full model copy per GPU, data split | Scales throughput, not model size | Using it for a model too big for one GPU |
| Tensor parallelism | Each layer's matrices split across GPUs | Fits larger models, needs fast NVLink | Running it over slow interconnects |
| Pipeline parallelism | Different layers on different GPUs | Fits very deep models across nodes | Ignoring idle pipeline bubbles |
Precision and memory per parameter
| Concept | Meaning | Key difference | Common trap |
|---|---|---|---|
| FP32 | 4 bytes per parameter | Training baseline, largest footprint | Using it for inference by default |
| FP16 or BF16 | 2 bytes per parameter | Standard for training and serving | Forgetting BF16 has a wider range |
| FP8 or INT8 | 1 byte per parameter | Faster, smaller inference | Assuming no accuracy check is needed |
Exam traps
Each trap is an answer that looks right. Learn the clue that should change your mind.
-
NeMo for serving
- What candidates pick
- Serve production inference with the NeMo training framework.
- Why it looks right
- NeMo is NVIDIA's LLM framework.
- Why it's wrong
- NeMo builds and customizes models; NIM, Triton and TensorRT-LLM handle optimized serving.
Clue to watch for The requirement is low-latency production inference.
-
Weights are the whole bill
- What candidates pick
- Size a server using only parameter memory.
- Why it looks right
- Weights are the obvious large number.
- Why it's wrong
- KV cache, activations and runtime overhead also need memory and grow with context and concurrency.
Clue to watch for The scenario mentions long contexts or many concurrent users.
-
Data parallel fits big models
- What candidates pick
- Spread a too-large model across GPUs with data parallelism.
- Why it looks right
- It is the most familiar multi-GPU method.
- Why it's wrong
- Data parallelism puts a full copy on every GPU, so it cannot fit a model that exceeds one GPU's memory.
Clue to watch for The model will not fit on a single GPU.
-
Triton optimizes weights
- What candidates pick
- Expect Triton alone to quantize and fuse the model.
- Why it looks right
- Triton is NVIDIA's inference product.
- Why it's wrong
- Triton serves models; engine optimization and quantization come from TensorRT or TensorRT-LLM.
Clue to watch for The question asks how to speed up the model itself.
-
GPU busy means healthy
- What candidates pick
- Declare a service healthy because GPU utilization is high.
- Why it looks right
- Busy GPUs seem productive.
- Why it's wrong
- Requests can still queue or fail; latency, queue time and error-rate metrics show user impact.
Clue to watch for Users report slow responses despite high utilization.
Answer framework
Apply these steps to Domain 4 questions about building, deploying or sizing an LLM system.
- Locate the lifecycle stage named in the scenario, whether build, customize, optimize, serve or monitor, since each maps to different NVIDIA tools.
- For sizing, multiply parameters by bytes per parameter for the stated precision, then add room for the KV cache and overhead.
- If the model exceeds one GPU, choose tensor or pipeline parallelism; if throughput is the issue, add replicas or data parallelism.
- For serving, pick TensorRT-LLM to optimize, Triton to host flexibly, or NIM when a packaged, standard-API microservice is preferred.
- For reliability, choose the metric that matches the symptom, such as queue time, latency, error rate or GPU memory, and log it.
Practice question
Want more like this? The NCA-GENL practice test has questions for every domain.
Checklist
0 of 13
Tick each one you can do without notes. Your ticks stay in this browser.
Study strategy
Start by drawing the NVIDIA stack on one page, from CUDA, cuDNN and NCCL at the bottom, through TensorRT-LLM and Triton, to NIM and application frameworks at the top, with NeMo off to the side for training and customization. Most tool questions become easy once each product has a clear place on that diagram.
Next, drill memory arithmetic. Work out weight memory for 7, 13 and 70 billion parameter models at FP32, FP16 and INT8, then reason about when a model needs tensor parallelism across several GPUs. Add the KV cache idea so you remember that context length and concurrency also consume memory.
Then get hands-on if possible. Call a hosted NIM endpoint from Python with an OpenAI-style client, stream a response, and build a minimal RAG chain. Look at a Triton or NIM metrics page to see what latency and queue metrics look like. Finish by reviewing good scripting habits: tests, retries, logging and keeping secrets in environment variables. Two weeks of part-time study suits most candidates.
FAQ
How much of NCA-GENL is the Software Development domain?
Software Development is 24% of the exam, about twelve to fourteen questions, the second-largest share after Core Machine Learning and AI Knowledge.
What is the difference between TensorRT-LLM and Triton?
TensorRT-LLM optimizes and compiles a model into a fast engine. Triton Inference Server hosts models, including TensorRT-LLM engines, and adds batching, scheduling and metrics.
What is NVIDIA NIM?
NIM is a set of containerized inference microservices that bundle optimized models and a serving runtime behind standard APIs, so teams can deploy with minimal setup on NVIDIA GPUs.
How do I estimate GPU memory for a model?
Multiply the parameter count by bytes per parameter: four for FP32, two for FP16 or BF16, one for INT8 or FP8. Then add headroom for KV cache and runtime overhead.
Will I need to write code on the exam?
No. Questions are multiple choice, but they assume you can read simple Python and know what common libraries and API calls do.
Why does NCCL appear in the readings?
NCCL provides the collective operations, such as all-reduce, that synchronize data across GPUs in distributed training and multi-GPU inference.
What should be monitored on an inference service?
Request rate, latency, time to first token, queue time, error rate and GPU memory and utilization, typically scraped as Prometheus metrics and shown in dashboards.
Other Artificial Intelligence & Machine Learning certifications
Where the same subject shows up in other exams. Cards marked "Same subject" open the matching domain.
Test your Domain 4 knowledge
You have covered the NVIDIA serving stack, GPU sizing, parallelism, application code and monitoring. Check whether you can place each tool and size a deployment quickly.
Keep going
- Training Training Built on your Weak areas
- Practice test Questions by domain, with explanations
- Mock exam Full length, timed like the real exam
- Readiness test Find your weakest domain in minutes
- Cheat sheet Key terms and formulas on one page
- Ask the NCA-GENL AI tutor Get any concept explained your way