Overview
Data Preparation covers the retrieval side of a RAG application before any prompt is written. Its eight official objectives start with chunking for a given document structure and model limit, removing boilerplate and other noise that degrades answers, and picking the right Python package to extract text from PDFs, HTML, Office files or scans. You also need the order of operations for writing chunked text into Delta Lake tables governed by Unity Catalog, and the judgment to identify which source documents actually contain the knowledge the application needs.
The last three objectives move into retrieval quality: using tools and metrics to evaluate retrieval, designing advanced chunking such as parent-child or semantic splitting, and explaining how re-ranking reorders candidates after the first search. At 14% this section is worth about six questions, and its ideas reappear when Application Development asks you to choose a chunking strategy from evaluation results.
How it's tested
Questions describe a document set, a model constraint or a retrieval symptom and ask for the preparation step that fixes it. Some show short PySpark or Python snippets for writing chunks. Read for document structure, token limits and whether the problem is missing context, noisy context or poorly ordered context.
-
Symptom to data fix
Answers are incomplete or noisy and you identify the cause, such as chunks too small, boilerplate left in or a missing source.
-
Pick the parser
A file type and layout are described and you choose the extraction package or approach suited to it.
-
Measure and improve retrieval
Retrieval quality is in question and you pick the metric, chunking change or re-ranking step that addresses it.
A RAG app over product manuals often returns answers that cite a step but omit the warning printed just after it. Retrieval finds the right page, so the source is fine. Small fixed-size chunks are separating the step from its warning. A structure-aware or parent-child strategy that keeps sections together, retrieving small chunks but passing the larger parent section, preserves that context.
Objectives 8
Official objectives from the Databricks outline, in outline order.
2.1 Apply a chunking strategy for a given document structure and model constraintsChoosing chunk size, overlap and boundaries for the documents and models.
- What it means
- Chunking splits documents into retrievable pieces. This objective asks you to pick the method and size given the document layout (headings, tables, short FAQs, long narrative) and the limits of the embedding model and the LLM context window.
- What to understand
- Each chunk must fit the embedding model's maximum input tokens, or text is truncated and lost. Overlap preserves meaning across boundaries. Structured documents chunk well on headings or sections; FAQs chunk well per question. Smaller chunks raise precision but can lose context; larger chunks keep context but dilute relevance and use more of the LLM window.
- How it shows up
- A document type and model limit are given and you select chunk size, overlap or boundary method.
- Common confusion
- Sizing chunks by characters while the embedding limit is counted in tokens, so chunks overflow silently.
2.2 Filter extraneous content in source documents that degrades quality of a RAG applicationRemoving headers, footers, navigation, duplicates and irrelevant text.
- What it means
- Source files often contain repeated page headers, legal footers, navigation menus, cookie banners, tables of contents and duplicate versions. This objective is about removing that noise before chunking so the retriever does not return it and the LLM does not quote it.
- What to understand
- Noise competes with real content for similarity, so a footer repeated on every page can match many queries. Remove boilerplate, deduplicate near-identical documents, drop outdated versions and strip markup. Keep metadata such as title, section and date in separate columns where it helps filtering instead of mixing it into the text.
- How it shows up
- Retrieved chunks contain repeated boilerplate and you choose the cleaning step that improves quality.
- Common confusion
- Assuming a larger top-k will compensate for noisy chunks; more noise usually makes answers worse.
2.3 Choose the appropriate Python package to extract document content from provided source data and formatMatching parsers to PDFs, HTML, Office files and scanned images.
- What it means
- Different formats need different extraction tools. This objective asks you to match a package to the source: a PDF text library for digital PDFs, an HTML parser for web pages, Office libraries for Word files, OCR for scanned images, and multi-format partitioners for mixed collections.
- What to understand
- Digital PDFs can be read with libraries such as pypdf or PyMuPDF, while table-heavy PDFs may need layout-aware tools. BeautifulSoup parses HTML. python-docx reads Word documents. Scans need OCR such as Tesseract. The unstructured library handles many formats and splits by element type. Databricks also offers ai_parse_document in SQL for parsing documents at scale.
- How it shows up
- A source format is described, sometimes scanned or table-heavy, and you choose the extraction package.
- Common confusion
- Using a plain PDF text extractor on scanned images, which contain no text layer to extract.
2.4 Define operations and sequence to write given chunked text into Delta Lake tables in Unity CatalogRead, parse, chunk, add IDs and write a governed Delta table.
- What it means
- This objective covers the data engineering steps: load raw files from a Unity Catalog volume, parse them, split into chunks, explode the chunks into rows with a unique ID and metadata, and save the result as a Delta table in a catalog and schema.
- What to understand
- A typical sequence reads binary files from a volume, applies a parsing and chunking function (often a pandas UDF), explodes the chunk array into rows, adds a primary key column, and writes with saveAsTable using a three-level name. Enabling Change Data Feed on the table lets a Delta Sync Vector Search index pick up changes incrementally.
- How it shows up
- You are given steps or code and choose the correct order or the missing operation.
- Common confusion
- Forgetting a unique primary key or Change Data Feed, both needed before a Delta Sync index can track the table.
2.5 Identify needed source documents that provide necessary knowledge and quality for a given RAG applicationChoosing authoritative, current and relevant sources.
- What it means
- Retrieval can only return what is indexed. This objective asks you to judge which documents to include: authoritative, current, relevant to expected questions and of usable quality, and which to exclude, such as drafts, superseded versions or unrelated material.
- What to understand
- Start from the questions users will ask and confirm each can be answered from a source. Prefer official, maintained documents over copies. Exclude outdated or conflicting versions, which produce contradictory answers. Check access rights and licensing. If users ask about topics no source covers, add sources rather than tuning prompts.
- How it shows up
- A RAG app gives wrong or missing answers and you identify the missing or bad source as the cause.
- Common confusion
- Indexing every available document, which adds conflicting and outdated material to retrieval.
2.6 Use tools and metrics to evaluate retrieval performancePrecision, recall, ranking metrics and MLflow retrieval judges.
- What it means
- This objective covers measuring whether the retriever returns the right chunks. Classic metrics include precision at k, recall at k, mean reciprocal rank and NDCG, computed against labeled relevant documents. MLflow also provides LLM judges that score retrieved context.
- What to understand
- Recall at k asks whether the relevant chunks appear in the top k; precision at k asks how many retrieved chunks are relevant. MRR and NDCG reward ranking relevant results higher. In MLflow, retrieval relevance and groundedness judges score context without labels, while retrieval sufficiency needs expected facts. Evaluate retrieval separately from generation to locate the fault.
- How it shows up
- A retrieval problem is described and you select the metric or MLflow judge that measures it.
- Common confusion
- Judging retrieval by final answer quality alone, which cannot tell retrieval faults from generation faults.
2.7 Design retrieval systems using advanced chunking strategiesSemantic, hierarchical, parent-child and context-enriched chunking.
- What it means
- Beyond fixed-size splitting, advanced strategies include semantic chunking that breaks where meaning shifts, hierarchical or parent-child chunking that indexes small units but returns larger parents, sentence-window retrieval, and adding section titles or summaries to each chunk.
- What to understand
- Parent-child gives precise matching on small chunks with full context for the LLM. Semantic chunking keeps related sentences together at the cost of extra embedding work. Prepending document title and heading paths to each chunk improves matching for short chunks. Metadata columns support filtering by product, date or region. Hybrid keyword and vector search helps with codes and exact terms.
- How it shows up
- A retrieval weakness is described and you choose the advanced chunking design that resolves it.
- Common confusion
- Assuming advanced chunking is always better; it adds cost and complexity that simple documents do not need.
2.8 Explain the role of re-ranking in the information retrieval processA second scoring pass that reorders retrieved candidates.
- What it means
- Re-ranking runs after the first vector search. The retriever fetches a larger candidate set quickly, then a more accurate model, usually a cross-encoder that reads the query and chunk together, reorders them so the best few go to the LLM.
- What to understand
- First-stage retrieval is tuned for recall and speed; re-ranking improves precision at the top. It costs extra latency per query, so it is applied to a limited candidate list such as the top 50. Mosaic AI Vector Search can apply a reranking step at query time. Re-ranking cannot recover documents the first stage never retrieved.
- How it shows up
- Relevant chunks are retrieved but ranked too low and you choose re-ranking as the fix.
- Common confusion
- Expecting re-ranking to fix low recall; it only reorders what retrieval already returned.
High-yield concepts
The terms, rules and formulas this domain turns into questions most often.
-
Embedding token limit
The maximum input an embedding model accepts per chunk.
- Remember
- Text beyond it is truncated and never embedded.
- Tested as
- Chunk sizing questions.
- Trap
- Counting characters instead of tokens.
-
Chunk overlap
Repeated text between adjacent chunks.
- Remember
- It protects meaning that crosses a boundary.
- Tested as
- Chunking questions.
- Trap
- Setting overlap so high that duplicates dominate results.
-
Structure-aware chunking
Splitting on headings, sections or list items.
- Remember
- It keeps related content together.
- Tested as
- Document structure questions.
- Trap
- Using fixed size on well-structured manuals.
-
Parent-child chunking
Index small chunks, return their larger parent section.
- Remember
- Precise matching with full context.
- Tested as
- Advanced chunking questions.
- Trap
- Embedding only the large parents.
-
Boilerplate filtering
Removing headers, footers, menus and duplicates before chunking.
- Remember
- Repeated noise matches many queries.
- Tested as
- Data quality questions.
- Trap
- Raising top-k instead of cleaning.
-
OCR for scans
Optical character recognition for image-only documents.
- Remember
- Scanned PDFs have no text layer.
- Tested as
- Parser selection questions.
- Trap
- Using a plain PDF text library on scans.
-
ai_parse_document
Databricks SQL function that parses documents into structured content.
- Remember
- It scales parsing inside SQL or pipelines.
- Tested as
- Extraction questions.
- Trap
- Assuming parsing must happen outside Databricks.
-
Unity Catalog volume
Governed storage location for raw files such as PDFs.
- Remember
- Source files are read from volumes.
- Tested as
- Ingestion sequence questions.
- Trap
- Reading from ungoverned paths.
-
Change Data Feed
A Delta table setting that records row-level changes.
- Remember
- Required for a Delta Sync Vector Search index.
- Tested as
- Delta table write questions.
- Trap
- Forgetting to enable it on the chunk table.
-
Recall at k vs precision at k
Whether relevant chunks appear, versus how many retrieved are relevant.
- Remember
- Recall first, then precision.
- Tested as
- Retrieval metric questions.
- Trap
- Swapping the two definitions.
-
MRR and NDCG
Ranking metrics rewarding relevant results near the top.
- Remember
- They measure order, not just presence.
- Tested as
- Retrieval evaluation questions.
- Trap
- Using them without relevance labels.
-
Re-ranking
Second-stage reordering of candidates by a more accurate model.
- Remember
- Improves top-result precision, adds latency.
- Tested as
- Retrieval design questions.
- Trap
- Expecting it to fix missing documents.
-
Hybrid search
Combining keyword and vector similarity in one query.
- Remember
- Helps with product codes and exact terms.
- Tested as
- Advanced retrieval questions.
- Trap
- Relying on pure vectors for identifiers.
Side by side
Concepts candidates mix up, compared on the point that decides the answer.
Chunking strategies
| Concept | Meaning | Key difference | Common trap |
|---|---|---|---|
| Fixed size with overlap | Equal token windows with shared edges | Simple, ignores structure | Splitting tables and steps |
| Structure-aware | Split on headings or sections | Follows document layout | Uneven chunk sizes past token limits |
| Semantic | Split where meaning shifts | Embedding-driven boundaries | Extra compute during preparation |
| Parent-child | Small chunks indexed, parents returned | Separates matching from context | Sending children only to the LLM |
Retrieval metrics
| Concept | Meaning | Key difference | Common trap |
|---|---|---|---|
| Recall at k | Share of relevant chunks found in top k | Coverage | Confusing it with precision |
| Precision at k | Share of top k that is relevant | Noise level | Optimizing it while recall is low |
| MRR | Rank of the first relevant result | Rewards early hits | Ignoring later relevant results |
| MLflow retrieval judges | LLM scoring of context relevance and sufficiency | Sufficiency needs expected facts | Assuming none need labels |
Document parsers
| Concept | Meaning | Key difference | Common trap |
|---|---|---|---|
| pypdf or PyMuPDF | Text from digital PDFs | Needs an existing text layer | Using it on scans |
| BeautifulSoup | Parse HTML pages | Strips markup and menus | Keeping navigation text |
| Tesseract OCR | Text from images and scans | Recognizes characters from pixels | Skipping it for scanned files |
| unstructured | Multi-format partitioning by element | Handles mixed collections | Assuming it removes all noise |
Exam traps
Each trap is an answer that looks right. Learn the clue that should change your mind.
-
Bigger top-k fixes noise
- What candidates pick
- Retrieve more chunks to work around boilerplate.
- Why it looks right
- More context sounds safer.
- Why it's wrong
- Extra chunks bring more noise into the prompt and can push out the useful content; cleaning the source is the fix.
Clue to watch for Retrieved text is full of repeated headers or menus.
-
Characters equal tokens
- What candidates pick
- Set chunk size to 2,000 characters for a 512-token embedding model.
- Why it looks right
- Character counts are easy to compute.
- Why it's wrong
- Token counts vary by text, and anything beyond the model's token limit is truncated before embedding.
Clue to watch for The scenario states a token limit for the embedding model.
-
Text extraction on scans
- What candidates pick
- Parse scanned contracts with a PDF text library.
- Why it looks right
- The files are PDFs.
- Why it's wrong
- Scanned PDFs are images, so a text library returns nothing useful; OCR is required.
Clue to watch for The documents are described as scanned or photographed.
-
Index everything
- What candidates pick
- Load every file in the shared drive into the knowledge base.
- Why it looks right
- More data seems to raise coverage.
- Why it's wrong
- Outdated, duplicate and irrelevant documents produce conflicting answers and lower retrieval precision.
Clue to watch for Answers contradict each other or cite old policies.
-
Re-rank to fix recall
- What candidates pick
- Add a reranker because the right document never appears.
- Why it looks right
- Re-ranking is an advanced retrieval feature.
- Why it's wrong
- A reranker only reorders retrieved candidates and cannot surface documents the first stage missed.
Clue to watch for The relevant chunk is absent from the candidate list.
-
Answer quality as retrieval metric
- What candidates pick
- Measure retrieval with overall answer correctness.
- Why it looks right
- It is the outcome users see.
- Why it's wrong
- Answer quality mixes retrieval and generation, so it cannot isolate whether the retriever failed.
Clue to watch for The question asks specifically about retrieval performance.
Answer framework
For a Data Preparation scenario, trace the problem from source documents through to ranked retrieval results.
- Confirm the right sources are present: authoritative, current, relevant to expected questions and allowed for use, with outdated versions removed.
- Check extraction: match the parser to the format, using OCR for scans and layout-aware tools for tables, and strip boilerplate and duplicates.
- Choose chunking from document structure and the embedding token limit, adding overlap, structure-aware or parent-child splitting where context is lost.
- Write chunks to a Unity Catalog Delta table with a primary key, metadata columns and Change Data Feed enabled for Vector Search syncing.
- Measure retrieval with recall, precision or ranking metrics and MLflow judges, then add hybrid search or re-ranking when ranking is the weakness.
Practice question
Want more like this? The Databricks GenAI Assoc practice test has questions for every domain.
Checklist
0 of 12
Tick each one you can do without notes. Your ticks stay in this browser.
Study strategy
Learn this section by building a small pipeline once. Put a handful of PDFs and HTML pages in a Unity Catalog volume, parse them in a notebook, and compare a plain PDF library with OCR on a scanned page so the difference is concrete. Strip headers and footers, then chunk the text two ways: fixed size with overlap, and by headings.
Write the chunks to a Delta table with an ID column and Change Data Feed enabled. Doing this by hand fixes the order of operations in memory, which is what code-sequence questions test.
Next, write ten test questions with the chunk IDs that answer them and compute recall at 5 and precision at 5 for each chunking approach. Then read the MLflow documentation on retrieval judges and note which need expected facts. Finally, read about parent-child chunking, hybrid search and re-ranking, and for each one write the symptom it fixes. The exam frames these as symptoms, so linking each technique to the problem it solves is the most useful preparation.
FAQ
How much of the Databricks GenAI Associate exam is Data Preparation?
Data Preparation is 14% of the exam, roughly six scored questions, focused on getting documents ready for retrieval.
What chunk size should I use?
There is no single answer. Fit chunks within the embedding model's token limit, follow document structure where possible, and tune size using retrieval evaluation results.
Why enable Change Data Feed on the chunk table?
A Delta Sync Vector Search index reads the table's change feed to update embeddings incrementally when rows are added, changed or deleted.
Which package should I use for scanned PDFs?
An OCR tool such as Tesseract, because scanned pages are images with no extractable text layer.
What is the difference between recall and precision at k?
Recall at k measures whether relevant chunks appear in the top k. Precision at k measures how many of those top k are relevant.
When does re-ranking help?
When relevant chunks are retrieved but ranked too low to reach the LLM. It does not help when they are never retrieved.
Is parent-child chunking always better?
No. It suits long, structured documents where small matches need surrounding context, but it adds complexity that short FAQs do not need.
Other Artificial Intelligence & Machine Learning certifications
Where the same subject shows up in other exams. Cards marked "Same subject" open the matching domain.
Test your Domain 2 knowledge
You have covered sources, parsing, cleaning, chunking, Delta table writes, retrieval metrics and re-ranking. Test whether you can trace a weak answer back to the right data step.
Keep going
- Training Training Built on your Weak areas
- Practice test Questions by domain, with explanations
- Mock exam Full length, timed like the real exam
- Readiness test Find your weakest domain in minutes
- Cheat sheet Key terms and formulas on one page
- Ask the Databricks GenAI Assoc AI tutor Get any concept explained your way