Overview
AI/ML Integration Overview is 5% of the ABDE exam. It covers building data infrastructure for machine learning workflows: implementing feature stores, integrating with ML pipelines, creating data infrastructure for model deployment, formatting data for AI-specific needs, optimizing data for training and inference, and establishing data feeds for model monitoring. DASCA names MLflow, an open-source platform for tracking experiments, packaging models and managing a model registry.
The domain exists because most production ML failures are data failures: features computed differently in training and serving, future information leaking into training sets, or no feed to detect that inputs have drifted. The data engineer does not tune models here but makes sure features are consistent, reproducible and available at the right latency. It draws on pipelines from Domains 1 and 3, governance from Domain 4 and streaming from Domain 6.
How it's tested
Questions describe a model's data supply problem, such as different results in production than in testing, suspiciously high validation accuracy or no visibility into degrading predictions, and ask for the BEST data engineering fix. Look for clues about timing, consistency between training and serving, and latency of feature access.
-
Keep features consistent
Choose a shared feature definition or feature store to remove differences between training and serving.
-
Build a correct training set
Spot leakage and apply point-in-time joins, proper splits and reproducible dataset versions.
-
Feed monitoring
Pick the data feeds, such as logged inputs, predictions and delayed labels, that let teams detect drift and performance loss.
A churn model scored 97% in validation but performs poorly in production. The training set joined each customer to their latest support-ticket count, including tickets filed after the prediction date. That is leakage: training saw future information unavailable at prediction time. A point-in-time join that uses only feature values recorded before each label's timestamp produces an honest training set.
Practical applications 6
Official practical applications from the DASCA outline, in outline order.
7.1 Implementing feature storesShared, consistent features for training and serving.
- What it means
- A feature store is a system, such as Feast or a platform-managed equivalent, that defines, computes, stores and serves ML features, with an offline store holding history for training and an online store holding the latest values for low-latency inference.
- What to understand
- Its main value is consistency: one feature definition is used to build training data and to serve predictions, which prevents training-serving skew. The offline store supports point-in-time correct retrieval for training, while the online store, often a key-value database, returns features in milliseconds. Feature reuse across teams also avoids duplicated, slightly different calculations.
- How it shows up
- Two teams compute 'average order value over 30 days' differently and production results diverge from testing, which points to a shared feature store definition.
- Common confusion
- Treating a feature store as just another table, missing the online/offline split and point-in-time retrieval it provides.
7.2 Integrating with ML pipelinesConnecting data pipelines to training and experiment tracking.
- What it means
- Integration links data pipelines to ML workflows so that new data triggers feature computation, training or evaluation, and so each training run records the exact data version, code and parameters used, for example through MLflow tracking.
- What to understand
- Reproducibility depends on versioning data as well as code: snapshot or time-travel versions of training tables let a team rebuild any past model. Orchestrators such as Airflow can schedule retraining, while MLflow records runs and registers resulting models. Data validation should run before training so bad data does not silently produce a bad model.
- How it shows up
- An auditor asks which data trained a deployed model, and the answer is logging dataset versions with each tracked run.
- Common confusion
- Versioning model code and parameters but not the training data, which makes results impossible to reproduce.
7.3 Creating data infrastructure for model deploymentSupplying data to models running in production.
- What it means
- Deployment infrastructure provides the data paths models need in production: batch scoring pipelines that write predictions to tables, or online serving where an API retrieves features and returns predictions in real time, supported by a model registry that tracks which version is in which stage.
- What to understand
- Choose batch inference when predictions can be computed ahead of time, such as nightly churn scores, and online inference when they depend on fresh context, such as fraud checks during checkout. Online serving needs low-latency feature lookup and fallbacks if a feature is missing. Registries such as MLflow's support controlled promotion and rollback of model versions.
- How it shows up
- A scenario states that recommendations can be refreshed daily, so batch scoring into a table is simpler and cheaper than a real-time endpoint.
- Common confusion
- Defaulting to real-time serving when the use case tolerates precomputed predictions.
7.4 Formatting data for AI-specific needsShaping data into the forms models consume.
- What it means
- AI-specific formatting turns raw data into model-ready inputs: encoding categorical values, scaling numeric features, handling missing values, building fixed-shape tensors or sequences, labeling data and storing it in efficient formats such as Parquet or TFRecord.
- What to understand
- Transformations fitted on data, such as scaling parameters or category vocabularies, must be computed on training data only and reused unchanged at inference. Labels need clear definitions and timestamps. Unstructured data such as images and text needs consistent preprocessing and metadata so samples can be traced to their source and license.
- How it shows up
- A model performs differently in production because the serving pipeline rescales features using live data statistics instead of the saved training statistics.
- Common confusion
- Fitting preprocessing on the full dataset before splitting, which leaks information from the test set into training.
7.5 Optimizing data for training and inferenceCorrect splits, efficient reads and fast lookups.
- What it means
- Optimization covers building training sets that are correct and efficient, such as time-aware splits, point-in-time joins, sampling or rebalancing, and columnar storage, and serving data fast enough for inference, through precomputed features and caching.
- What to understand
- Random splits on time-dependent data let the model learn from the future; split by time when predicting future events. Class imbalance may call for stratified sampling or reweighting, but evaluation data should reflect the real distribution. Training reads benefit from large, columnar, partitioned files, while inference benefits from precomputed features in a low-latency store.
- How it shows up
- A demand forecasting dataset was split randomly and validation looks excellent, but production accuracy drops; a time-based split is the fix.
- Common confusion
- Using random train-test splits for time-series problems.
7.6 Establishing data feeds for model monitoringLogging inputs, predictions and outcomes to detect drift.
- What it means
- Monitoring feeds capture model inputs, predictions, model version and, when available later, the true outcomes, so teams can compare production data with training data and measure live performance.
- What to understand
- Data drift is a change in input distributions, and concept drift is a change in the relationship between inputs and the target. Drift can be measured with statistics such as the population stability index, but performance can only be confirmed once ground-truth labels arrive, often with delay, so feeds must join predictions to later outcomes by a shared ID. Pipeline issues, such as a feature suddenly null, often look like drift and should be checked first.
- How it shows up
- A team cannot tell whether a fraud model is degrading because it never stored predictions with transaction IDs, which prevents joining them to later chargeback labels.
- Common confusion
- Equating data drift with lower accuracy; inputs can shift without hurting performance, and performance can fall without obvious input drift.
High-yield concepts
The terms, rules and formulas this domain turns into questions most often.
-
Feature store
A feature store manages feature definitions, computes them and serves them from an offline store for training and an online store for inference.
- Remember
- Its core purpose is one definition used in both training and serving, which prevents skew.
- Tested as
- Consistency and reuse scenarios point here.
- Trap
- Thinking a regular warehouse table provides millisecond online serving.
-
Training-serving skew
Training-serving skew is a difference between how features are computed or distributed during training and during production inference.
- Remember
- Share transformation code or a feature store between both paths and log served features for comparison.
- Tested as
- Classic cause when production results trail validation.
- Trap
- Blaming the model algorithm before checking feature pipelines.
-
Point-in-time correctness
Point-in-time correctness means each training example uses only feature values that were known at the moment of the event being predicted.
- Remember
- Join features as of the label timestamp, not their latest values.
- Tested as
- Leakage scenarios with unrealistically high validation scores.
- Trap
- Joining current feature values to historical labels.
-
Data leakage
Leakage occurs when training data includes information that would not be available at prediction time, inflating validation results.
- Remember
- Common sources are future data in joins, preprocessing fitted before splitting and random splits on time series.
- Tested as
- Any 'great in testing, poor in production' scenario.
- Trap
- Fitting scalers or encoders on the whole dataset before the split.
-
MLflow
MLflow is an open-source platform with components for experiment tracking, model packaging and a model registry with versioned stages.
- Remember
- Log data versions and parameters with every run so models are reproducible and auditable.
- Tested as
- DASCA names MLflow for this domain.
- Trap
- Thinking MLflow stores or serves features like a feature store.
-
Model registry
A model registry stores model versions with metadata and lifecycle stages, supporting controlled promotion and rollback.
- Remember
- Deployment pipelines should pull approved versions from the registry rather than from ad hoc files.
- Tested as
- Deployment and rollback questions.
- Trap
- Copying model files manually between environments.
-
Batch vs. online inference
Batch inference precomputes predictions on a schedule and stores them, while online inference computes predictions on request.
- Remember
- Use batch when predictions can be prepared ahead of need; use online when fresh request context matters.
- Tested as
- Architecture questions about serving predictions.
- Trap
- Building real-time endpoints for daily reports.
-
Data drift vs. concept drift
Data drift is a shift in input distributions, while concept drift is a change in how inputs relate to the target.
- Remember
- Data drift is detectable from inputs alone; concept drift needs ground-truth labels to confirm.
- Tested as
- Monitoring scenarios ask which drift is present or what feed detects it.
- Trap
- Assuming stable inputs mean a stable model.
-
Time-based split
A time-based split trains on earlier data and validates on later data.
- Remember
- Use it whenever the model predicts future events, to mimic real deployment.
- Tested as
- Forecasting and churn scenarios with suspicious validation scores.
- Trap
- Shuffling time-series data before splitting.
-
Data and dataset versioning
Dataset versioning records the exact snapshot of data used for a training run, for example through table time travel or snapshot IDs.
- Remember
- Reproducibility needs data version, code version and parameters together.
- Tested as
- Audit and reproducibility questions.
- Trap
- Retraining on a table that has since been overwritten and calling the result reproducible.
Side by side
Concepts candidates mix up, compared on the point that decides the answer.
Offline vs. online feature store
| Concept | Meaning | Key difference | Common trap |
|---|---|---|---|
| Offline store | Historical feature values in a warehouse or lake | Supports point-in-time training sets and batch scoring | Using it for millisecond lookups during requests |
| Online store | Latest feature values in a low-latency key-value store | Serves features to real-time inference | Expecting it to hold full history for training |
Data drift vs. concept drift vs. training-serving skew
| Concept | Meaning | Key difference | Common trap |
|---|---|---|---|
| Data drift | Production input distributions shift over time | Detected by comparing input statistics with training data | Assuming it always lowers accuracy |
| Concept drift | Relationship between inputs and outcome changes | Confirmed only with ground-truth labels | Trying to detect it without storing predictions and outcomes |
| Training-serving skew | Features differ between training and serving pipelines | Present from day one, caused by inconsistent code or data | Mistaking it for gradual drift |
Batch vs. online inference
| Concept | Meaning | Key difference | Common trap |
|---|---|---|---|
| Batch inference | Scheduled scoring of many records, results stored in tables | Cheaper and simpler, predictions may be hours old | Using it when the prediction depends on in-session data |
| Online inference | Prediction computed per request through an API | Fresh context and low latency, but needs online features and high availability | Choosing it when daily refreshed predictions suffice |
Exam traps
Each trap is an answer that looks right. Learn the clue that should change your mind.
-
Blame the algorithm
- What candidates pick
- Retrain with a more complex model when production accuracy is far below validation.
- Why it looks right
- A better model seems the natural response to poor performance.
- Why it's wrong
- A large gap at launch usually means leakage or training-serving skew; a new algorithm trained on the same flawed data repeats the problem.
Clue to watch for Validation scores look unusually high, or the gap appeared immediately after deployment.
-
Latest values for training
- What candidates pick
- Join each historical label to the customer's current feature values.
- Why it looks right
- Current values are easy to query and complete.
- Why it's wrong
- They include information from after the prediction moment; point-in-time joins use only values known at each label's timestamp.
Clue to watch for Features that change over time, such as counts, balances or statuses.
-
Real-time by habit
- What candidates pick
- Deploy an online endpoint for weekly marketing propensity scores.
- Why it looks right
- Real-time serving sounds more advanced.
- Why it's wrong
- Batch scoring into a table meets the need with less infrastructure, cost and operational risk.
Clue to watch for Predictions are consumed on a schedule, such as a weekly campaign.
-
Drift equals failure
- What candidates pick
- Retrain immediately whenever an input distribution shifts.
- Why it looks right
- Drift alerts seem to demand action.
- Why it's wrong
- Input shifts do not always reduce performance, and a sudden shift may be a pipeline bug; check data quality and labeled performance before retraining.
Clue to watch for A sudden change in one feature, such as many nulls, right after a pipeline change.
-
Version code only
- What candidates pick
- Rely on Git commits to reproduce last quarter's model.
- Why it looks right
- Version control is standard engineering practice.
- Why it's wrong
- The training table has changed since then; reproducibility also needs the data snapshot logged with the run.
Clue to watch for Audit or reproducibility requirements for past models.
-
Fit preprocessing on everything
- What candidates pick
- Normalize the full dataset before splitting into training and test sets.
- Why it looks right
- It is one simple step and the code is shorter.
- Why it's wrong
- Test-set statistics leak into training; fit preprocessing on the training split and apply the same parameters elsewhere.
Clue to watch for Preprocessing described as happening before the split.
Answer framework
For ABDE questions on supporting machine learning, use these steps to find the data engineering answer.
- Check for leakage first by confirming every training feature was known at the time of its label, using point-in-time joins and time-based splits.
- Make features consistent by sharing one definition between training and serving, ideally through a feature store.
- Match inference to need: batch scoring when predictions can be prepared ahead of time, online serving when request context matters.
- Version datasets with each tracked training run, and promote models through a registry such as MLflow's so any deployed version can be traced and rolled back.
- Log inputs, predictions, model versions and IDs so delayed labels can be joined, and check pipeline quality before treating changes as drift.
Practice question
Want more like this? The ABDE practice test has questions for every domain.
Checklist
0 of 11
Tick each one you can do without notes. Your ticks stay in this browser.
Study strategy
This domain is 5% of the exam, so aim for a compact study block built around a few concepts that generate most questions.
Start with leakage and point-in-time correctness, since they explain most scenarios where a model looks excellent in testing and fails in production. Work through one example by hand: a label dated 1 March, features recorded in February and April, and decide which values the training row may use.
Next, study feature stores and training-serving skew together, focusing on why one shared definition matters and how offline and online stores differ. Then review MLflow at a conceptual level: tracking runs, logging parameters and dataset versions, and promoting models through the registry. Candidates often underestimate dataset versioning and assume code versioning is enough.
Finish with deployment and monitoring: when to choose batch or online inference, and what to log so delayed labels can be joined to predictions for performance measurement.
Afterwards, attempt timed questions for this domain and review misses by asking whether the issue was timing, consistency or monitoring. Then move to Domain 8, which applies the same reasoning to embeddings, vector stores and retrieval pipelines.
FAQ
How much of the ABDE exam is AI/ML integration?
DASCA weights AI/ML Integration Overview at 5%, which suggests about 4 of the 75 questions. DASCA does not publish per-domain counts.
Do I need to know how to train machine learning models?
The domain is about the data infrastructure around ML: features, training and inference data, deployment data paths and monitoring feeds. Basic ML vocabulary helps, but model tuning is not the focus of this domain's outline.
What is training-serving skew?
It is a mismatch between how features are computed or distributed in training and in production. It usually appears from the first day of deployment and is fixed by sharing feature definitions, for example through a feature store.
Why does DASCA mention MLflow?
MLflow is named as the reference tool for this domain. Know its role in tracking experiment runs, logging parameters and data versions, and managing model versions in a registry.
What is point-in-time correctness?
It means each training example uses only feature values that existed when the predicted event happened. Joining current values to past labels leaks future information and inflates validation results.
How do data engineers support model monitoring?
By building feeds that log model inputs, predictions, model version and an ID, then joining later ground-truth outcomes. This enables drift statistics and real performance measurement.
How does this domain connect to Generative AI & LLM Operations?
Domain 8 applies the same principles to embeddings and retrieval: consistent preprocessing, versioned data, fresh indexes and monitoring. Understanding feature consistency here makes embedding pipeline questions easier.
Other Data & Analytics certifications
Where the same subject shows up in other exams. Cards marked "Same subject" open the matching domain.
Test your Domain 7 knowledge
You have covered feature stores, ML pipeline integration, deployment data paths, AI-ready formatting, training and inference data, and monitoring feeds. Test whether you can spot leakage and skew in exam scenarios.