Overview
Domain 4 covers what happens after a GenAI application is live, across three task statements. Cost optimization covers token estimation and tracking, context pruning, prompt compression and response limits, tiered model use by query complexity, batching, provisioned throughput sizing, and caching through semantic caching, request hashing, edge caching and Bedrock prompt caching. Performance optimization covers latency-optimized models, pre-computation, parallel requests, streaming, retrieval speed through index tuning and hybrid search, batch inference, temperature and top-p settings, and capacity planning for token workloads. Monitoring covers observability with CloudWatch dashboards and tracing, GenAI-specific metrics such as token usage, hallucination rate and response drift, Bedrock model invocation logs, tool-call tracking for agents, vector store health and troubleshooting with golden datasets.
At 12%, this is a smaller domain, but cost and latency trade-offs also decide many answers in the larger domains.
How it's tested
Domain 4 questions give operating data such as cost, latency, token counts or quality trends and ask for the optimization or monitoring change with the best effect. Usually several answers help a little, and the correct one targets the specific driver named in the scenario.
-
Find the cost driver
Spend is rising, and you identify whether input tokens, output tokens, model tier or repeated calls drive it.
-
Cut the latency
Users wait too long, and you choose streaming, a latency-optimized model, pre-computation or parallel calls.
-
Detect silent degradation
Quality or usage shifts without errors, and you choose metrics, invocation logs or anomaly detection to reveal it.
An internal assistant sends the same 6,000-token policy preamble on every request, and input tokens dominate the bill. Shortening the user question saves little, and switching models changes quality. Bedrock prompt caching stores the repeated prefix so later calls reuse it at lower cost and latency. The clue is a large, identical prefix across requests, which is exactly what prompt caching targets.
Task statements 3
Official task statements from the Amazon Web Services outline, in outline order.
4.1 Implement cost optimization and resource efficiency strategiesToken efficiency, model tiering, throughput utilization and caching.
- What it means
- This task covers reducing FM spend: estimating and tracking tokens, optimizing context windows, compressing prompts, pruning context and limiting response length, selecting models by price-to-performance, routing by query complexity, batching and capacity planning, sizing provisioned throughput, and caching with semantic caches, result fingerprints, request hashing, edge caching and Bedrock prompt caching.
- What to understand
- Cost scales with input plus output tokens multiplied by the model's price. Smaller models handle routine queries at a fraction of the cost. Semantic caching returns stored answers for similar questions and skips the FM entirely, while prompt caching reuses a repeated prompt prefix inside the model call. Batch inference costs less for jobs without real-time needs.
- How it shows up
- A cost problem is described with usage details, and you choose the token, model, batching or caching change that cuts it most.
- Common confusion
- Confusing prompt caching with semantic caching; one reuses a prompt prefix in the call, the other avoids the call.
4.2 Optimize application performanceLatency, retrieval speed, throughput, inference parameters and resource allocation.
- What it means
- This task covers improving responsiveness with pre-computation, latency-optimized Bedrock models, parallel requests and streaming, speeding retrieval with index tuning, query preprocessing and hybrid search, managing throughput with token processing and batch strategies, choosing temperature and top-k or top-p settings, planning capacity for token workloads, and profiling API calls and vector queries.
- What to understand
- Latency has parts: retrieval time, time to first token and generation time. Streaming helps the second, shorter outputs help the third and index tuning helps the first. Lower temperature gives more deterministic output for extraction tasks; higher values add variety for creative ones. Independent sub-tasks can run in parallel instead of one long chain.
- How it shows up
- A latency, throughput or output-quality issue is described, and you choose the tuning technique that addresses the slowest part.
- Common confusion
- Raising temperature to fix inaccurate answers; higher temperature adds randomness and usually hurts factual tasks.
4.3 Implement monitoring systems for GenAI applicationsObservability, GenAI metrics, invocation logs, tool tracking, vector store health and failure modes.
- What it means
- This task covers observability for FM applications: operational metrics, tracing and business dashboards, CloudWatch tracking of token usage, prompt effectiveness, hallucination rates and response quality, anomaly detection for token bursts and drift, Bedrock Model Invocation Logs, tool call and multi-agent tracking, vector database monitoring and index maintenance, and troubleshooting with golden datasets, output diffing and reasoning path tracing.
- What to understand
- GenAI can fail silently: the API returns success while answers get worse. That is why quality metrics, golden datasets and drift detection matter alongside latency and errors. Invocation logs give request and response detail for analysis. Tool call baselines reveal agents that loop or misuse tools. Vector store monitoring catches slow queries and stale indexes.
- How it shows up
- A production issue with no obvious errors is described, and you choose the metric, log or detection method that reveals it.
- Common confusion
- Monitoring only latency and error rates; GenAI quality problems often appear with healthy infrastructure metrics.
High-yield concepts
The terms, rules and formulas this domain turns into questions most often.
-
Token-based pricing
Cost equals input plus output tokens times the per-token rate.
- Remember
- Output tokens usually cost more than input.
- Tested as
- Cost questions.
- Trap
- Optimizing only request count.
-
Amazon Bedrock prompt caching
Reuses a repeated prompt prefix across calls.
- Remember
- Cuts cost and latency for long shared context.
- Tested as
- Repeated system prompt questions.
- Trap
- Confusing it with response caching.
-
Semantic caching
Returns a stored response for a semantically similar query.
- Remember
- Avoids the FM call entirely.
- Tested as
- Repeated question questions.
- Trap
- Serving stale answers without expiry.
-
Model tiering and cascading
Routing simple queries to small models and hard ones to large models.
- Remember
- Bedrock intelligent prompt routing automates part of this.
- Tested as
- Model selection questions.
- Trap
- Sending every request to the largest model.
-
Batch inference
Processing many prompts as one asynchronous job.
- Remember
- Lower cost for offline workloads.
- Tested as
- Throughput and cost questions.
- Trap
- Using it for interactive chat.
-
Latency-optimized inference
Bedrock options that reduce response time for supported models.
- Remember
- For time-sensitive applications.
- Tested as
- Latency questions.
- Trap
- Assuming every model supports it.
-
Context pruning
Removing irrelevant history or passages from the prompt.
- Remember
- Saves tokens and can improve focus.
- Tested as
- Token efficiency questions.
- Trap
- Cutting context the answer needs.
-
Temperature and top-p
Sampling parameters that control randomness.
- Remember
- Low for factual tasks, higher for creative ones.
- Tested as
- Parameter tuning questions.
- Trap
- Raising temperature to fix accuracy.
-
Time to first token
Delay before the first output token appears.
- Remember
- Streaming improves perceived latency.
- Tested as
- Responsiveness questions.
- Trap
- Measuring only total latency.
-
Bedrock model invocation logs
Full request and response records for analysis.
- Remember
- Basis for quality and cost investigations.
- Tested as
- Monitoring questions.
- Trap
- Leaving logging disabled.
-
Golden dataset
A fixed set of questions with expected answers.
- Remember
- Detects hallucination and regression over time.
- Tested as
- Troubleshooting questions.
- Trap
- Relying on ad hoc spot checks.
-
Token anomaly detection
Alerting on unusual token bursts or cost spikes.
- Remember
- Catches loops, abuse and misconfiguration.
- Tested as
- Monitoring questions.
- Trap
- Discovering spikes only on the monthly bill.
Side by side
Concepts candidates mix up, compared on the point that decides the answer.
Caching approaches
| Concept | Meaning | Key difference | Common trap |
|---|---|---|---|
| Prompt caching | Reuse repeated prompt prefix | Call still runs, cheaper | Expecting zero FM calls |
| Semantic caching | Reuse answers to similar questions | Skips the FM call | No freshness policy |
| Deterministic request hashing | Exact-match cache key | Only identical requests hit | Expecting paraphrases to match |
| Edge caching | Cache responses near users | Static or shared answers | Caching personalized output |
Latency levers
| Concept | Meaning | Key difference | Common trap |
|---|---|---|---|
| Streaming | Deliver tokens as produced | Improves perceived latency | Expecting lower total time |
| Smaller or latency-optimized model | Faster generation | Lower total time | Losing needed quality |
| Parallel requests | Run independent steps at once | Shortens multi-step workflows | Parallelizing dependent steps |
| Pre-computation | Generate predictable answers ahead | Near-instant reads | Pre-computing unpredictable queries |
Monitoring signal types
| Concept | Meaning | Key difference | Common trap |
|---|---|---|---|
| Operational metrics | Latency, errors, throttles | Infrastructure health | Missing quality drops |
| GenAI quality metrics | Hallucination rate, relevance, drift | Answer health | Never establishing a baseline |
| Cost and token metrics | Tokens per request, spend | Efficiency and abuse | No anomaly alerts |
Exam traps
Each trap is an answer that looks right. Learn the clue that should change your mind.
-
Bigger model for everything
- What candidates pick
- Route all traffic to the most capable model.
- Why it looks right
- It maximizes quality.
- Why it's wrong
- Most routine queries do not need it; routing by complexity keeps quality where it matters and cuts cost sharply.
Clue to watch for Many simple, repetitive queries and a cost complaint.
-
Prompt caching skips the call
- What candidates pick
- Expect prompt caching to avoid invoking the model.
- Why it looks right
- Caching usually means no call.
- Why it's wrong
- Prompt caching reuses a stored prefix within the invocation; skipping the call entirely needs a response or semantic cache.
Clue to watch for The goal is avoiding duplicate FM calls.
-
Streaming lowers total time
- What candidates pick
- Enable streaming to shorten end-to-end processing.
- Why it looks right
- Users see text sooner.
- Why it's wrong
- Streaming improves time to first token, but total generation time and token count stay the same.
Clue to watch for A downstream system needs the complete response faster.
-
Temperature fixes accuracy
- What candidates pick
- Increase temperature when answers are wrong.
- Why it looks right
- It sounds like making the model try harder.
- Why it's wrong
- Higher temperature increases randomness; factual tasks benefit from low temperature plus better context.
Clue to watch for Inconsistent or inaccurate factual answers.
-
Errors equal health
- What candidates pick
- Treat zero errors and normal latency as a healthy app.
- Why it looks right
- Classic monitoring works that way.
- Why it's wrong
- FMs can return confident but wrong answers with successful API calls, so quality metrics and golden datasets are needed.
Clue to watch for Users complain while dashboards look green.
-
Batch for chat
- What candidates pick
- Use batch inference to cut chat costs.
- Why it looks right
- Batch is cheaper.
- Why it's wrong
- Batch jobs are asynchronous and unsuitable for interactive users who need immediate responses.
Clue to watch for Real-time user interaction is required.
Answer framework
Use these steps for a Domain 4 optimization or monitoring scenario.
- Identify the target metric the scenario cares about: cost, perceived latency, total latency, throughput or answer quality.
- Find the driver: input tokens, output tokens, model tier, repeated calls, retrieval time or capacity limits, using the numbers given.
- Pick the lever that hits that driver, such as caching for repetition, tiering for simple queries, streaming for perceived delay or batching for offline work.
- Check side effects on quality and freshness, avoiding changes like aggressive pruning or high temperature that degrade answers.
- Add monitoring that would have caught the issue earlier: token anomalies, quality baselines, invocation logs and tool or vector store metrics.
Practice question
Want more like this? The AIP-C01 practice test has questions for every domain.
Checklist
0 of 12
Tick each one you can do without notes. Your ticks stay in this browser.
Study strategy
This domain is smaller, so aim for precision rather than breadth. Begin with token arithmetic: take a sample prompt, count input and output tokens and compute cost on two Bedrock models of different sizes. That one exercise makes tiering, pruning and response limits intuitive.
Next, learn the caching family as a set and be able to say which one skips the model call and which one only cheapens it. Try Bedrock prompt caching on a long system prompt if your account supports it, and sketch a semantic cache using embeddings and a similarity threshold.
Then study latency as three parts, retrieval, first token and generation, and map each optimization to the part it improves. Review temperature and top-p settings with a short experiment.
Finish with monitoring: enable model invocation logs, build a CloudWatch dashboard for token usage and set an anomaly alarm. Keep a short list of GenAI failure modes that return HTTP 200, since those are favorite exam scenarios, and revisit it before your mock exam.
FAQ
How much of AIP-C01 is Domain 4?
Operational Efficiency and Optimization for GenAI Applications is 12% of the scored content, with three task statements on cost, performance and monitoring.
What is the difference between prompt caching and semantic caching?
Prompt caching reuses a repeated prompt prefix during a model call. Semantic caching returns a saved answer for a similar question so the model is not called at all.
How do I cut FM costs without hurting quality?
Route simple queries to smaller models, trim unnecessary context, limit output length, cache repeated answers and batch offline work.
Does streaming reduce cost?
No. Streaming changes when tokens reach the user, which improves perceived speed, but the same tokens are generated and billed.
Which temperature suits factual extraction?
A low temperature, which makes output more deterministic. Higher temperatures add variety and suit creative writing more than factual tasks.
What should a GenAI monitoring dashboard show?
Latency and errors, plus token usage, cost, hallucination or grounding scores, response drift, tool call patterns and vector store query performance.
What is a golden dataset?
A fixed set of prompts with expected answers that you rerun to detect hallucinations, regressions and drift after model or prompt changes.
Other Artificial Intelligence & Machine Learning certifications
Where the same subject shows up in other exams. Cards marked "Same subject" open the matching domain.
Test your Domain 4 knowledge
You have covered token cost control, caching, model tiering, latency tuning and GenAI monitoring. Test whether you can find the true cost or latency driver in a scenario.
Keep going
- Training Training Built on your Weak areas
- Practice test Questions by domain, with explanations
- Mock exam Full length, timed like the real exam
- Readiness test Find your weakest domain in minutes
- Cheat sheet Key terms and formulas on one page
- Ask the AIP-C01 AI tutor Get any concept explained your way