Overview
Data Ingestion and Transformation is the heaviest DEA-C01 domain and has four task statements. Task 1.1 covers reading from streaming and batch sources such as Amazon Kinesis, Amazon MSK, DynamoDB Streams, AWS DMS and Amazon AppFlow, plus schedulers, event triggers, throttling, fan-in and fan-out, and replayability. Task 1.2 covers transforming data with AWS Glue, Amazon EMR, Lambda and Amazon Redshift, converting formats such as CSV to Parquet, and integrating LLMs into processing. Task 1.3 covers orchestration with Step Functions, Amazon MWAA, EventBridge and Glue workflows, and Task 1.4 covers code, Lambda tuning, infrastructure as code and CI/CD.
As of October 31, 2026, DEA-C01 is still the current exam and AWS has announced no DEA-C02. The guide in force is version 1.1, dated December 12, 2025, which merged the older knowledge and skills lists into one skills list per task.
How it's tested
Domain 1 questions read like design reviews. A scenario gives a data source, arrival pattern, latency target, volume and often a cost or operations constraint, then asks for the service, configuration or pipeline order that meets every requirement. Several options usually work technically, so the deciding factor is the least operational overhead or the lowest cost that still meets the stated latency.
-
Streaming or batch choice
Arrival rate and freshness requirement are given, and you choose between Kinesis Data Streams, Amazon Data Firehose, Amazon MSK, scheduled Glue jobs or AWS DMS.
-
Transformation engine
A dataset size, format and team skill set are described, and you pick AWS Glue, Amazon EMR, Lambda or Redshift SQL for the transformation.
-
Orchestration fit
A multi-step workflow with dependencies, retries or existing Airflow DAGs is described, and you pick Step Functions, Amazon MWAA, Glue workflows or EventBridge.
A company receives clickstream events at 20,000 records per second and must land them in Amazon S3 as Parquet within five minutes, with no servers to manage. Amazon Data Firehose fits: it buffers by size or time, can convert JSON to Parquet using a Glue Data Catalog schema, and writes to S3 without consumer code. Kinesis Data Streams with custom consumers also works but adds code and shard management that the requirement does not need.
Task statements 4
Official task statements from the Amazon Web Services outline, in outline order.
1.1 Perform data ingestionStreaming and batch sources, schedulers, triggers, throttling and replay.
- What it means
- This task statement covers reading from streaming sources (Amazon Kinesis, Amazon MSK, DynamoDB Streams, AWS DMS) and batch sources (Amazon S3, AWS Glue, Amazon EMR, Amazon Redshift, Lambda, Amazon AppFlow), consuming data APIs, setting schedulers and event triggers, invoking Lambda from Kinesis, IP allowlists, throttling and rate limits, fan-in and fan-out, replayability, and stateful versus stateless transactions.
- What to understand
- Kinesis Data Streams keeps records for 24 hours by default, extendable up to 365 days, so consumers can replay from a stored position; Firehose only delivers and cannot replay. Enhanced fan-out gives each registered consumer its own 2 MB per second per shard of read throughput. ProvisionedThroughputExceeded errors call for more shards, a better partition key or exponential backoff. AWS DMS with change data capture streams ongoing database changes. Amazon AppFlow pulls from SaaS applications such as Salesforce. EventBridge schedules or S3 Event Notifications start Glue jobs and crawlers.
- How it shows up
- A source type, rate and freshness target are described, and you choose the ingestion service, trigger and retry approach.
- Common confusion
- Assuming Amazon Data Firehose can replay old records; only Kinesis Data Streams and Amazon MSK retain data for consumers to re-read.
1.2 Transform and process dataChoosing engines, format conversion, cost, troubleshooting, data APIs and LLMs.
- What it means
- This task statement covers choosing a transformation service (Amazon EMR, AWS Glue, Lambda, Amazon Redshift), running containers on Amazon ECS or EKS, connecting through JDBC and ODBC, joining data from many sources, converting formats such as CSV to Apache Parquet, controlling processing cost, debugging failed or slow jobs, exposing data through APIs, describing volume, velocity and variety, and using large language models in processing.
- What to understand
- AWS Glue is serverless Spark priced by DPU-hour and suits most ETL without cluster management. Amazon EMR gives control over frameworks, instance types and Spot capacity for large or custom workloads. Lambda handles small, short transformations within its 15-minute limit. Columnar formats such as Parquet with compression cut Athena and Redshift Spectrum scan costs. Out-of-memory errors in Spark often trace to skewed keys or too few workers. An LLM called through Amazon Bedrock can classify, extract or summarize unstructured text inside a pipeline step.
- How it shows up
- A dataset size, format, runtime limit and cost goal are given, and you pick the engine and output format.
- Common confusion
- Choosing Lambda for multi-hour transformations; its 15-minute timeout points to AWS Glue or Amazon EMR instead.
1.3 Orchestrate data pipelinesWorkflow services, resilient pipeline design, serverless workflows and alerts.
- What it means
- This task statement covers building ETL workflows with Lambda, Amazon EventBridge, Amazon Managed Workflows for Apache Airflow (Amazon MWAA), AWS Step Functions and AWS Glue workflows, designing pipelines for performance, availability, scalability, resiliency and fault tolerance, running serverless workflows, and sending alerts through Amazon SNS and Amazon SQS.
- What to understand
- Step Functions is a serverless state machine with built-in retries, catch blocks, parallel and map states, and direct integrations with Glue, EMR, Lambda and Athena. Amazon MWAA runs Apache Airflow DAGs and suits teams that already write Python DAGs or need many open-source operators, but it keeps an environment running. Glue workflows chain Glue crawlers and jobs only. EventBridge routes events and schedules runs. SNS pushes notifications to many subscribers, while SQS buffers messages so consumers can process at their own pace and retry.
- How it shows up
- A workflow with dependencies, retries and failure alerts is described, and you choose the orchestrator and notification path.
- Common confusion
- Treating SQS as an alerting service; SNS pushes alerts, while SQS queues work for consumers to poll.
1.4 Apply programming conceptsCode optimization, Lambda tuning, IaC, AWS SAM, CI/CD and distributed computing.
- What it means
- This task statement covers optimizing ingestion and transformation code, configuring Lambda concurrency and performance, using Python, SQL, Scala, Java and shell languages, applying version control, testing, logging and monitoring, deploying with infrastructure as code (AWS CloudFormation, AWS CDK, AWS SAM), mounting storage in Lambda, describing CI/CD for pipelines, distributed computing, and data structures such as graphs and trees.
- What to understand
- Lambda memory also scales CPU, so raising memory can shorten runtime. Reserved concurrency caps a function so it cannot overwhelm a downstream database, and provisioned concurrency removes cold starts. Lambda can mount Amazon EFS for shared files larger than its ephemeral /tmp storage. AWS SAM templates package Lambda, Step Functions and DynamoDB resources and deploy through CloudFormation. The AWS CDK defines the same resources in a general-purpose language. Distributed engines such as Spark split data into partitions processed in parallel across workers.
- How it shows up
- A code, deployment or scaling problem is described, and you choose the Lambda setting, IaC tool or coding practice that fixes it.
- Common confusion
- Confusing reserved and provisioned concurrency; reserved limits a function's scale, provisioned keeps instances warm.
High-yield concepts
The terms, rules and formulas this domain turns into questions most often.
-
Kinesis Data Streams vs. Amazon Data Firehose
Streams stores records for custom consumers; Firehose buffers and delivers to destinations.
- Remember
- Replay and custom processing mean Streams.
- Tested as
- Streaming ingestion questions.
- Trap
- Picking Streams when no custom consumer is needed.
-
Shards and partition keys
Each shard accepts up to 1 MB or 1,000 records per second of writes.
- Remember
- A hot partition key overloads one shard.
- Tested as
- Throughput and throttling questions.
- Trap
- Adding shards without fixing a skewed key.
-
Enhanced fan-out
A dedicated 2 MB per second per shard read pipe for each registered consumer.
- Remember
- Use it when several consumers share one stream.
- Tested as
- Fan-out questions.
- Trap
- Adding shards to solve a shared read limit.
-
AWS DMS with CDC
Full load plus ongoing change data capture from source databases.
- Remember
- CDC keeps targets in near sync.
- Tested as
- Database ingestion questions.
- Trap
- Using nightly exports when near-real-time sync is required.
-
Amazon AppFlow
Managed transfers from SaaS apps such as Salesforce into S3 or Redshift.
- Remember
- No-code connectors for SaaS sources.
- Tested as
- Batch source questions.
- Trap
- Writing custom API scripts for a supported SaaS source.
-
AWS Glue vs. Amazon EMR
Serverless Spark ETL versus managed clusters you configure.
- Remember
- Least operations points to Glue.
- Tested as
- Engine selection questions.
- Trap
- Choosing EMR when no custom framework is needed.
-
Apache Parquet
Columnar, compressed file format for analytics.
- Remember
- Cuts scan cost for Athena and Spectrum.
- Tested as
- Format conversion questions.
- Trap
- Keeping CSV for analytical queries.
-
Glue job bookmarks
Track processed data so reruns pick up only new files.
- Remember
- Prevents reprocessing old data.
- Tested as
- Incremental batch questions.
- Trap
- Building custom state tables instead.
-
Step Functions vs. Amazon MWAA
Serverless state machine versus a managed Airflow environment.
- Remember
- Existing Airflow DAGs mean MWAA.
- Tested as
- Orchestration questions.
- Trap
- Picking MWAA for a short serverless chain.
-
Lambda concurrency
Reserved concurrency caps scale; provisioned concurrency pre-warms instances.
- Remember
- Protect downstream databases with reserved concurrency.
- Tested as
- Lambda tuning questions.
- Trap
- Mixing up the two settings.
-
Exponential backoff with jitter
Retry pattern for throttled API and database calls.
- Remember
- Spreads retries to avoid synchronized bursts.
- Tested as
- Rate limit questions.
- Trap
- Retrying immediately in a tight loop.
-
Infrastructure as code
CloudFormation, AWS CDK and AWS SAM define repeatable deployments.
- Remember
- SAM is a CloudFormation extension for serverless.
- Tested as
- Deployment questions.
- Trap
- Building each environment by hand in the console.
Side by side
Concepts candidates mix up, compared on the point that decides the answer.
Streaming ingestion services
| Concept | Meaning | Key difference | Common trap |
|---|---|---|---|
| Kinesis Data Streams | Durable stream with shards and custom consumers | Replay within the retention period | Picking it for simple delivery to S3 |
| Amazon Data Firehose | Managed buffering and delivery to destinations | No consumer code and no replay | Expecting it to keep records for re-reading |
| Amazon MSK | Managed Apache Kafka | Kafka APIs and ecosystem compatibility | Choosing it with no Kafka requirement |
| DynamoDB Streams | Item-level change log for one table | 24-hour retention, table changes only | Treating it as a general event stream |
Transformation engines
| Concept | Meaning | Key difference | Common trap |
|---|---|---|---|
| AWS Glue | Serverless Spark and Python shell ETL | No cluster management | Assuming it cannot handle large jobs |
| Amazon EMR | Managed Spark, Hive, Presto and Hadoop clusters | Full control and Spot capacity | Choosing it when operations must be minimal |
| AWS Lambda | Event-driven functions | 15-minute limit, small payloads | Using it for long batch jobs |
| Amazon Redshift (ELT) | SQL transforms inside the warehouse | Data already loaded | Exporting data out just to transform it |
Orchestration options
| Concept | Meaning | Key difference | Common trap |
|---|---|---|---|
| AWS Step Functions | Serverless state machines | Built-in retries and service integrations | Assuming it needs servers |
| Amazon MWAA | Managed Apache Airflow | Python DAGs and Airflow operators | Using it for a two-step serverless chain |
| AWS Glue workflows | Chains of Glue crawlers and jobs | Glue resources only | Expecting it to orchestrate EMR or Lambda |
| Amazon EventBridge | Event routing and schedules | Starts work, does not manage state | Using it alone for complex branching |
Exam traps
Each trap is an answer that looks right. Learn the clue that should change your mind.
-
Firehose for replay
- What candidates pick
- Use Amazon Data Firehose so a new consumer can reprocess last week's events.
- Why it looks right
- Firehose is the simplest streaming option.
- Why it's wrong
- Firehose delivers and does not retain records for consumers; replay requires Kinesis Data Streams retention or Amazon MSK topics.
Clue to watch for The scenario mentions reprocessing, replay or several independent consumers.
-
Lambda for heavy ETL
- What candidates pick
- Process a 500 GB nightly file with a Lambda function.
- Why it looks right
- Lambda is serverless and cheap per call.
- Why it's wrong
- Lambda stops at 15 minutes and has limited memory and storage, while AWS Glue scales Spark workers for large batch files.
Clue to watch for Large files, long runtimes or joins across big datasets.
-
EMR for least overhead
- What candidates pick
- Launch an EMR cluster for a standard Spark ETL job.
- Why it looks right
- EMR is the well-known Spark platform.
- Why it's wrong
- When no custom framework or instance tuning is required, a serverless Glue job meets the need without cluster administration.
Clue to watch for The question says least operational overhead.
-
More shards for a hot key
- What candidates pick
- Double the shard count to stop write throttling.
- Why it looks right
- Shards add write capacity.
- Why it's wrong
- If one partition key carries most records, those records still map to one shard; a well-distributed key removes the hotspot.
Clue to watch for Throttling continues although total throughput is below stream capacity.
-
SQS for alerts
- What candidates pick
- Send pipeline failure alerts to an SQS queue for the on-call team.
- Why it looks right
- SQS is a messaging service.
- Why it's wrong
- SQS holds messages until a consumer polls; SNS pushes email, SMS or HTTP notifications to subscribers immediately.
Clue to watch for People must be told when a job fails.
-
Full reloads for incremental data
- What candidates pick
- Reprocess the entire S3 prefix every hour.
- Why it looks right
- Simple to implement.
- Why it's wrong
- Glue job bookmarks or S3 event triggers process only new objects, cutting runtime and DPU cost.
Clue to watch for New files arrive continuously and cost or runtime matters.
Answer framework
Work through these steps when a Domain 1 scenario asks how data should enter, change and move through a pipeline.
- Classify the source and arrival pattern: continuous events point to Kinesis or Amazon MSK, database changes to AWS DMS with CDC, SaaS data to Amazon AppFlow, and files to S3 triggers.
- Check the freshness target and whether replay or several consumers are needed, since that separates Kinesis Data Streams from Amazon Data Firehose.
- Pick the transformation engine by size and runtime: Lambda for small, short work, AWS Glue for serverless Spark, Amazon EMR when custom frameworks or tuning matter.
- Choose the orchestrator: Step Functions for serverless steps with retries, Amazon MWAA for existing Airflow DAGs, Glue workflows for Glue-only chains.
- Confirm cost and resilience details such as Parquet output, job bookmarks, backoff on throttling and SNS alerts on failure.
Practice question
Want more like this? The DEA-C01 practice test has questions for every domain.
Checklist
0 of 13
Tick each one you can do without notes. Your ticks stay in this browser.
Study strategy
Spend the largest share of your study time here, because this domain carries a third of the scored content. Begin with ingestion: draw a table of Kinesis Data Streams, Amazon Data Firehose, Amazon MSK, DynamoDB Streams, AWS DMS and Amazon AppFlow with columns for source type, latency, retention, replay and consumer code. Then build a small stream in a sandbox account, push test records with the SDK, and watch what happens when one partition key dominates.
Next, run the same CSV-to-Parquet conversion three ways: a Lambda function on a small file, an AWS Glue job with bookmarks on a folder, and an EMR step if budget allows. Compare runtime, setup effort and cost so the trade-offs stick.
Finish with orchestration and code. Chain a crawler, a Glue job and an Athena query in Step Functions with a retry and an SNS alert on failure, then read an Airflow DAG so MWAA questions feel familiar. Deploy one pipeline from an AWS SAM template. Close each session with timed scenario questions and note which requirement word, such as replay, least overhead or cost, decided the answer.
FAQ
How much of DEA-C01 is Domain 1?
Data Ingestion and Transformation is 34% of scored content, the largest of the four domains, or about 17 of the 50 scored questions, spread across four task statements.
Is there a DEA-C02 version of the Data Engineer exam?
Not as of October 31, 2026. AWS has not announced DEA-C02 or a retirement date for DEA-C01. The current guide is version 1.1 from December 2025.
How much experience does AWS recommend before taking DEA-C01?
The exam guide suggests two to three years of data engineering experience and one to two years of hands-on AWS experience. It is a recommendation, not a prerequisite.
Do I need to write code on the exam?
No code is typed, but questions show SQL, Python or configuration snippets and expect you to recognize correct logic, Lambda settings and IaC choices.
Is Amazon Data Firehose the same as Kinesis Data Firehose?
Yes. AWS renamed Kinesis Data Firehose to Amazon Data Firehose in 2024, so study material may use either name.
How deep should I go on Apache Airflow?
Know what a DAG is, that Amazon MWAA runs Airflow as a managed service, and when it beats Step Functions. Writing operators from memory is not required.
Why does the guide mention large language models?
Task 1.2 lists integrating LLMs for data processing, such as calling a model through Amazon Bedrock to extract fields or classify text during transformation.
Other Data & Analytics certifications
Where the same subject shows up in other exams. Cards marked "Same subject" open the matching domain.
Test your Domain 1 knowledge
You have reviewed streaming and batch ingestion, transformation engines, orchestration and pipeline code. Test whether you can pick the right service when a scenario names latency, volume and cost limits.
Keep going
- Training Training Built on your Weak areas
- Practice test Questions by domain, with explanations
- Mock exam Full length, timed like the real exam
- Readiness test Find your weakest domain in minutes
- Cheat sheet Key terms and formulas on one page
- Ask the DEA-C01 AI tutor Get any concept explained your way