Overview
Data Operations and Support has four task statements about keeping pipelines running and useful. Task 3.1 covers automating processing: orchestrating with Amazon MWAA and Step Functions, troubleshooting managed workflows, calling SDKs, using features of Amazon EMR, Redshift and AWS Glue, preparing data with AWS Glue DataBrew and Amazon SageMaker Unified Studio, querying with Amazon Athena, and running Lambda and EventBridge automation. Task 3.2 covers analysis: SQL in Redshift and Athena, Athena notebooks with Apache Spark, visualization in DataBrew and Amazon Quick Sight, and aggregation concepts such as rolling averages and pivots.
Task 3.3 covers monitoring and maintenance with Amazon CloudWatch, CloudWatch Logs Insights, AWS CloudTrail, Amazon OpenSearch Service and alert notifications. Task 3.4 covers data quality: rules, consistency checks, sampling and handling data skew.
How it's tested
Domain 3 questions start after a pipeline exists. A job fails, runs slowly, produces duplicates, or an analyst needs a quick answer from new data. You choose the AWS feature, query, metric, log source or quality rule that resolves the situation with the least manual effort, often comparing serverless and provisioned options on cost and administration.
-
Diagnose the failure
Logs, metrics or error messages are described, and you identify the cause, such as memory pressure, skew, throttling or a permissions gap, and its fix.
-
Pick the query tool
An analysis need, data location and usage frequency are given, and you choose Athena, Redshift, Athena Spark notebooks or Quick Sight.
-
Enforce quality
Bad or inconsistent records reach consumers, and you choose where and how to apply quality rules, quarantine or sampling.
An AWS Glue Spark job that joins orders to customers now fails with out-of-memory errors on a single executor, while other executors sit idle. The pattern points to data skew: one customer key holds a large share of rows. Salting the hot key or using a broadcast join for the smaller table spreads the work. Simply adding workers would not help, because the oversized partition still lands on one executor.
Task statements 4
Official task statements from the Amazon Web Services outline, in outline order.
3.1 Automate data processing by using AWS servicesOrchestration, SDK calls, service features, data preparation, Athena queries, Lambda and EventBridge.
- What it means
- This task statement covers orchestrating pipelines with Amazon MWAA and AWS Step Functions, troubleshooting managed workflows, calling AWS SDKs from code, using processing features of Amazon EMR, Amazon Redshift and AWS Glue, consuming and maintaining data APIs, preparing data with AWS Glue DataBrew and Amazon SageMaker Unified Studio, querying with Amazon Athena, automating with AWS Lambda, and managing events and schedulers in Amazon EventBridge.
- What to understand
- Automation means no person starts the job. EventBridge rules react to events such as a Glue job state change or run on cron schedules. Step Functions execution history shows which state failed and with what input, which speeds troubleshooting. The Redshift Data API runs SQL from Lambda without managing connections. DataBrew offers more than 250 built-in transforms for analysts who prefer a visual tool. SageMaker Unified Studio brings data preparation, SQL and notebooks into one governed workspace. Athena CTAS and INSERT INTO statements can transform and write data as a pipeline step.
- How it shows up
- A manual or brittle process is described, and you choose the event, service feature or API that automates it.
- Common confusion
- Opening JDBC connections from Lambda to run Redshift SQL when the Redshift Data API handles calls without connection management.
3.2 Analyze data by using AWS servicesSQL in Redshift and Athena, Spark notebooks, visualization, cleaning and aggregations.
- What it means
- This task statement covers visualizing data with AWS Glue DataBrew and Amazon Quick Sight (QuickSight in the guide), verifying and cleaning data with Lambda, Athena, Jupyter notebooks and Amazon SageMaker Data Wrangler, writing SQL in Redshift and Athena to query data and create views, exploring data with Athena notebooks for Apache Spark, weighing provisioned against serverless services, and defining aggregation, rolling averages, grouping and pivoting.
- What to understand
- Athena charges by data scanned, so it suits sporadic queries over S3; provisioned Redshift clusters suit steady, heavy workloads, and Redshift Serverless charges for compute used. A window function such as AVG() OVER (ORDER BY day ROWS BETWEEN 6 PRECEDING AND CURRENT ROW) computes a rolling average. GROUP BY aggregates rows, while PIVOT turns row values into columns. Athena for Apache Spark gives interactive notebooks without a cluster. Quick Sight uses SPICE in-memory storage for fast dashboards. Data Wrangler profiles and cleans data for ML preparation.
- How it shows up
- An analysis question, data size and usage frequency are given, and you choose the tool, SQL construct or pricing model.
- Common confusion
- Assuming serverless is always cheaper; constant heavy workloads can cost less on provisioned capacity.
3.3 Maintain and monitor data pipelinesLogging, alerting, CloudTrail, performance troubleshooting and log analysis.
- What it means
- This task statement covers extracting logs for audits, deploying logging and monitoring for traceability, sending alerts from monitoring, troubleshooting performance, tracking API calls with AWS CloudTrail, maintaining AWS Glue and Amazon EMR pipelines, configuring and automating Amazon CloudWatch Logs, and analyzing logs with Athena, Amazon EMR, Amazon OpenSearch Service and CloudWatch Logs Insights.
- What to understand
- CloudWatch metrics and alarms watch pipeline health, for example Kinesis GetRecords.IteratorAgeMilliseconds rising when consumers fall behind, or Glue job duration growing. Alarms publish to SNS for notification. Glue writes driver and executor logs to CloudWatch Logs, and the Spark UI shows stages and skew. EMR can archive logs to S3. CloudWatch Logs Insights queries recent log groups interactively, Athena suits large historical logs in S3, and OpenSearch Service supports search and dashboards. CloudTrail records who called which API, which answers questions about changes.
- How it shows up
- A pipeline is slow, failing or under review, and you choose the metric, log source, alarm or analysis tool.
- Common confusion
- Looking in CloudTrail for application errors; CloudTrail records API activity, while job output and errors go to CloudWatch Logs.
3.4 Ensure data qualityQuality checks, rules, consistency, sampling and data skew.
- What it means
- This task statement covers running data quality checks during processing, such as checking for empty fields, defining data quality rules in AWS Glue DataBrew, investigating data consistency, describing data sampling techniques, and implementing mechanisms that handle data skew.
- What to understand
- Quality checks belong inside the pipeline so bad records are caught before consumers see them. DataBrew profiles show missing values, distributions and outliers, and its rulesets validate thresholds. AWS Glue Data Quality evaluates Data Quality Definition Language (DQDL) rules in jobs and can route failing rows to a quarantine location. Random, stratified and systematic sampling let you test large datasets cheaply. Skew is handled by salting hot keys, repartitioning, broadcast joins for small tables, or adaptive query execution in Spark.
- How it shows up
- Incorrect, missing or unevenly distributed data is described, and you pick the rule, check placement, sampling method or skew fix.
- Common confusion
- Treating data skew as a cluster size problem; adding nodes does not split one oversized key.
High-yield concepts
The terms, rules and formulas this domain turns into questions most often.
-
Athena pricing
Billed per data scanned.
- Remember
- Parquet, compression and partitions cut cost.
- Tested as
- Cost questions.
- Trap
- Running SELECT * on raw CSV.
-
Redshift Data API
HTTP API for running SQL against Redshift asynchronously.
- Remember
- No drivers or persistent connections.
- Tested as
- Automation questions.
- Trap
- Managing JDBC pools in Lambda.
-
EventBridge rules
Route service events or run schedules to targets.
- Remember
- Glue and Step Functions emit state-change events.
- Tested as
- Event automation questions.
- Trap
- Polling job status in a loop.
-
Step Functions execution history
Record of each state's input, output and errors.
- Remember
- First place to look when a workflow fails.
- Tested as
- Troubleshooting questions.
- Trap
- Searching unrelated logs first.
-
CloudWatch Logs Insights
Query language for CloudWatch log groups.
- Remember
- Interactive analysis of recent logs.
- Tested as
- Log analysis questions.
- Trap
- Using it for years of archived S3 logs.
-
AWS CloudTrail
Records AWS API calls with identity and time.
- Remember
- Answers who changed what.
- Tested as
- Change tracking questions.
- Trap
- Expecting application errors in it.
-
Iterator age
Kinesis metric showing how far consumers lag behind the stream.
- Remember
- A rising value means consumers fall behind.
- Tested as
- Monitoring questions.
- Trap
- Watching only incoming record counts.
-
AWS Glue Data Quality
DQDL rules evaluated in Glue jobs and on catalog tables.
- Remember
- Can quarantine failing records.
- Tested as
- Quality questions.
- Trap
- Validating only after data reaches dashboards.
-
DataBrew profiling and rulesets
Visual profiling and rule-based validation.
- Remember
- No code needed for analysts.
- Tested as
- Quality and preparation questions.
- Trap
- Writing Spark for simple checks.
-
Data skew
Uneven distribution of keys across partitions.
- Remember
- Salting or broadcast joins fix it.
- Tested as
- Performance questions.
- Trap
- Adding workers without changing the key.
-
Window functions
Calculations across related rows, such as rolling averages.
- Remember
- Use OVER with PARTITION BY and ORDER BY.
- Tested as
- SQL analysis questions.
- Trap
- Using GROUP BY when row-level results are needed.
-
Provisioned vs. serverless
Fixed capacity versus paying for usage.
- Remember
- Steady heavy load favors provisioned.
- Tested as
- Trade-off questions.
- Trap
- Choosing serverless for 24/7 workloads by habit.
Side by side
Concepts candidates mix up, compared on the point that decides the answer.
Log and audit tools
| Concept | Meaning | Key difference | Common trap |
|---|---|---|---|
| CloudWatch Logs | Stores application and job logs | Retention settings per log group | Expecting API caller identity here |
| CloudWatch Logs Insights | Queries log groups | Fast, interactive | Using it for archived S3 logs |
| AWS CloudTrail | Records AWS API calls | Who did what and when | Looking here for stack traces |
| Amazon OpenSearch Service | Search and dashboards over logs | Full-text search at scale | Choosing it for one-off queries |
| Athena on S3 logs | SQL over stored log files | Cheap for large history | Querying unpartitioned logs |
Query and analysis services
| Concept | Meaning | Key difference | Common trap |
|---|---|---|---|
| Athena SQL | Serverless SQL over S3 | Pay per scan | Ignoring partitions and formats |
| Athena for Apache Spark | Serverless Spark notebooks | Interactive exploration | Launching EMR for quick exploration |
| Redshift provisioned | Fixed-size warehouse cluster | Steady heavy workloads | Running it idle most of the day |
| Redshift Serverless | Warehouse billed by compute used | Variable or unpredictable demand | Assuming it is always cheaper |
Data quality options
| Concept | Meaning | Key difference | Common trap |
|---|---|---|---|
| AWS Glue Data Quality | DQDL rules in jobs and on tables | Quarantine in the pipeline | Running it only after loads |
| DataBrew rulesets | Visual profiling and rules | Analyst-friendly, no code | Expecting streaming validation |
| Custom Lambda checks | Code-based validation | Full flexibility | Writing code a managed rule covers |
Exam traps
Each trap is an answer that looks right. Learn the clue that should change your mind.
-
More workers for skew
- What candidates pick
- Double the Glue worker count after one executor runs out of memory.
- Why it looks right
- More capacity usually speeds Spark.
- Why it's wrong
- A hot key sends its rows to one partition regardless of worker count; salting or a broadcast join redistributes the work.
Clue to watch for One task runs far longer than the rest.
-
CloudTrail for job errors
- What candidates pick
- Search CloudTrail to find why a Glue job failed.
- Why it looks right
- CloudTrail is a well-known AWS logging service.
- Why it's wrong
- CloudTrail shows API calls such as StartJobRun, but the exception and stack trace are in the job's CloudWatch Logs.
Clue to watch for The question asks why the code failed, not who started it.
-
Polling for job completion
- What candidates pick
- Have Lambda check the Glue job status every minute.
- Why it looks right
- Simple to write.
- Why it's wrong
- Glue publishes job state changes to EventBridge, so a rule can trigger the next step or an alert without polling cost.
Clue to watch for React when a job finishes or fails.
-
Logs Insights for archives
- What candidates pick
- Use CloudWatch Logs Insights to analyze three years of logs stored in S3.
- Why it looks right
- It is a log query tool.
- Why it's wrong
- Logs Insights queries CloudWatch log groups; historical logs in S3 are queried with Athena or processed with EMR.
Clue to watch for Large, old logs already sit in S3.
-
Serverless is always cheaper
- What candidates pick
- Move a heavily used, always-on Redshift cluster to serverless to save money.
- Why it looks right
- Serverless removes idle cost.
- Why it's wrong
- With constant high utilization, provisioned capacity or reserved nodes usually cost less; serverless saves money for intermittent workloads.
Clue to watch for Usage is steady and predictable.
-
Quality checks after load
- What candidates pick
- Let analysts report bad rows they find in dashboards.
- Why it looks right
- No pipeline changes needed.
- Why it's wrong
- Glue Data Quality or DataBrew rules inside the pipeline stop bad records and can quarantine them before they reach consumers.
Clue to watch for Bad data has already caused downstream errors.
Answer framework
Apply these steps to Domain 3 scenarios about automating, analyzing, monitoring or validating data in a running pipeline.
- Identify whether the problem is automation, analysis, monitoring or quality, because each points to a different family of services.
- For automation, prefer events and managed APIs: EventBridge rules, Step Functions integrations, the Redshift Data API and Lambda instead of polling scripts.
- For analysis, weigh frequency and data size: Athena for sporadic SQL on S3, Redshift for heavy repeated workloads, Spark notebooks for exploration.
- For failures, read the right evidence: CloudWatch Logs and the Spark UI for job errors, CloudWatch metrics for lag, and CloudTrail for API actions.
- For bad data, place Glue Data Quality or DataBrew rules inside the pipeline, quarantine failing rows, and fix skew by changing keys rather than adding capacity.
Practice question
Want more like this? The DEA-C01 practice test has questions for every domain.
Checklist
0 of 12
Tick each one you can do without notes. Your ticks stay in this browser.
Study strategy
Treat this domain as operations practice. Take a pipeline you built for Domain 1 and break it on purpose: remove an IAM permission, feed it a malformed file, and skew one join key. For each failure, find the evidence in CloudWatch Logs, the Spark UI, Step Functions execution history or CloudTrail, and write down which tool told you what.
Next, automate the same pipeline without polling. Add EventBridge rules for Glue job state changes, an SNS topic for failures and a CloudWatch alarm on job duration. Call the Redshift Data API from a Lambda function to run a post-load SQL statement.
Spend a separate session on SQL and analysis. Write a rolling seven-day average with a window function, a PIVOT query in Redshift, and a CTAS statement in Athena that writes Parquet. Compare the scanned bytes before and after. Then add a Glue Data Quality ruleset with completeness and uniqueness checks and route failing rows to a quarantine prefix. In practice questions, look first for the verb: automate, analyze, monitor or validate.
FAQ
How much of DEA-C01 does Domain 3 cover?
Data Operations and Support is 22% of scored content, about 11 of the 50 scored questions, under four task statements.
Is Amazon QuickSight still the right name to study?
The exam guide says QuickSight. AWS has since folded it into Amazon Quick, where the BI capability is Amazon Quick Sight. The dashboard and SPICE concepts are the same under either name.
Do I need to know Apache Spark internals?
Know enough to read symptoms: stages, partitions, shuffles, executor memory and skew. The exam asks which fix applies, not how to tune every configuration property.
Which data quality tools does the guide name?
Task 3.4 names AWS Glue DataBrew for rules and consistency checks. AWS Glue Data Quality is also worth knowing because it applies rules inside Glue jobs.
What is the difference between CloudTrail and CloudWatch Logs?
CloudTrail records AWS API calls, such as who started or deleted a job. CloudWatch Logs stores application and job output, including errors and custom log lines.
How much SQL is on the exam?
Expect SQL reading rather than writing: joins, views, CTAS, window functions for rolling averages, GROUP BY, pivoting and filters on partition columns.
Does this domain overlap with logging in Domain 4?
Yes. Domain 3 uses logs to run and fix pipelines, while Domain 4 prepares logs for audit and adds CloudTrail Lake and long-term retention.
Other Data & Analytics certifications
Where the same subject shows up in other exams. Cards marked "Same subject" open the matching domain.
Test your Domain 3 knowledge
You have reviewed pipeline automation, SQL analysis, monitoring and data quality. Find out whether you can diagnose a failing or slow pipeline and choose the right fix under time pressure.
Keep going
- Training Training Built on your Weak areas
- Practice test Questions by domain, with explanations
- Mock exam Full length, timed like the real exam
- Readiness test Find your weakest domain in minutes
- Cheat sheet Key terms and formulas on one page
- Ask the DEA-C01 AI tutor Get any concept explained your way