Overview
Generative AI & LLM Operations Overview is 5% of the ABDE exam. It covers designing data systems for generative AI: implementing vector database solutions, creating embedding generation pipelines, building retrieval augmented generation (RAG) systems, preparing data for large language model (LLM) fine-tuning, developing prompt management infrastructure, optimizing semantic search and managing token usage and costs. DASCA names no specific tool for this domain, so focus on concepts that apply across products.
The domain exists because most enterprise LLM applications depend on the organization's own documents and data. Engineers must chunk and embed content, keep indexes fresh as documents change, enforce access rights at retrieval time and control spending on tokens. It extends the feature consistency and monitoring ideas of Domain 7 and depends on the ingestion, governance and orchestration skills of Domains 1, 3 and 4.
How it's tested
Questions describe an LLM application problem, such as outdated or wrong answers, poor retrieval quality, leaked confidential content or high cost, and ask for the BEST data system fix. Decide first whether the problem is about knowledge (retrieval and freshness), behavior (fine-tuning or prompts), security or cost.
-
RAG or fine-tune
Choose between retrieval augmentation for changing factual knowledge and fine-tuning for consistent style, format or task behavior.
-
Improve retrieval
Fix poor answers by adjusting chunking, embedding consistency, metadata filters, hybrid search or re-ranking.
-
Run it responsibly
Control token cost, version prompts and enforce document permissions at retrieval time.
An internal assistant gives outdated policy answers. The policy library changes weekly, and the team proposes fine-tuning the model on the documents each month. Fine-tuning shapes behavior but is a poor way to inject frequently changing facts, and it cannot cite sources. A RAG pipeline that re-embeds changed documents and retrieves current chunks at query time keeps answers current and traceable.
Practical applications 7
Official practical applications from the DASCA outline, in outline order.
8.1 Implementing vector database solutionsStoring and searching embeddings at scale.
- What it means
- A vector database, or a vector index in an existing database such as pgvector in PostgreSQL, stores embeddings with metadata and finds the nearest vectors to a query embedding using similarity measures such as cosine similarity, dot product or Euclidean distance.
- What to understand
- Exact nearest-neighbor search compares every vector and becomes slow at scale, so approximate nearest neighbor (ANN) indexes such as HNSW or IVF trade a little recall for much faster queries. Metadata stored with vectors enables filtering, for example by department, date or permission group. The index must use the same similarity metric the embedding model was designed for.
- How it shows up
- Search latency grows as the collection reaches tens of millions of vectors, and the right answer is an ANN index rather than brute-force comparison.
- Common confusion
- Assuming a vector database replaces the system of record; it is a search index that must be kept in sync with source documents.
8.2 Creating embedding generation pipelinesTurning documents into vectors and keeping them current.
- What it means
- An embedding pipeline extracts text from sources, cleans it, splits it into chunks, generates an embedding for each chunk with an embedding model and writes the vectors with metadata to the index.
- What to understand
- Chunk size and overlap shape retrieval quality: chunks that are too large dilute meaning, while chunks that are too small lose context. Queries and documents must be embedded with the same model and version, so changing models requires re-embedding the whole corpus. Incremental pipelines re-embed only changed documents, detected by content hashes or update timestamps, and remove vectors for deleted documents.
- How it shows up
- Search results became poor right after the team switched to a new embedding model for queries only, which points to mixing vector spaces.
- Common confusion
- Treating embeddings from different models as compatible because they have the same number of dimensions.
8.3 Building RAG (Retrieval Augmented Generation) systemsGrounding LLM answers in retrieved enterprise data.
- What it means
- RAG retrieves relevant chunks from a knowledge source for each query and adds them to the LLM prompt, so answers draw on current, specific information and can cite their sources.
- What to understand
- Answer quality depends mostly on retrieval quality: if the right chunk is not retrieved, the model cannot use it. Improvements include better chunking, metadata filters, hybrid search combining keyword and vector results, and re-ranking the top candidates with a more precise model. Retrieval must enforce the user's permissions so restricted content never enters the prompt.
- How it shows up
- A RAG assistant reveals salary data to an employee without HR access, and the fix is permission-aware filtering at retrieval time.
- Common confusion
- Relying on a prompt instruction such as 'do not reveal confidential data' instead of preventing restricted chunks from being retrieved at all.
8.4 Preparing data for LLM fine-tuningCurating training examples that change model behavior.
- What it means
- Fine-tuning data preparation builds a curated set of examples, often instruction and response pairs in JSONL format, to adapt a model's style, format or task behavior.
- What to understand
- Quality matters more than volume: deduplicate, remove personal data and secrets, check licensing, fix inconsistent labels and keep a held-out evaluation set. Fine-tuning suits consistent behavior, such as a fixed output format or domain tone, while RAG suits frequently changing facts. Parameter-efficient methods such as LoRA reduce compute, but they do not reduce the need for clean data.
- How it shows up
- A team wants model outputs to follow a strict JSON schema every time, which fine-tuning on well-formed examples supports better than adding more documents to retrieval.
- Common confusion
- Using fine-tuning to keep a model updated with weekly changing facts.
8.5 Developing prompt management infrastructureVersioning, testing and tracking prompts as code.
- What it means
- Prompt management stores prompt templates centrally with versions, parameters, owners and associated evaluation results, so changes can be reviewed, tested, rolled out gradually and rolled back.
- What to understand
- Prompts change model behavior as much as code changes do, so they need version control, regression testing against evaluation sets and logs linking each response to the prompt version used. Separating templates from application code allows controlled updates. Logged prompts and responses also feed monitoring and cost analysis, but they may contain sensitive data and need retention and access controls.
- How it shows up
- Answer quality dropped after someone edited a prompt directly in production, and no one can tell what changed; the fix is versioned prompts with evaluation before release.
- Common confusion
- Treating prompts as informal text rather than versioned, tested artifacts.
8.6 Optimizing semantic search capabilitiesFinding results by meaning, precisely and quickly.
- What it means
- Semantic search retrieves results by meaning using embeddings rather than exact keyword matches, and optimization tunes relevance, latency and recall through hybrid search, re-ranking, filtering and index parameters.
- What to understand
- Pure vector search can miss exact identifiers such as product codes or error numbers, which keyword search (such as BM25) finds easily, so hybrid search often performs best. Re-rankers improve precision on a short candidate list at extra latency cost. Measure retrieval quality with metrics such as recall@k on a labeled query set rather than relying on impressions.
- How it shows up
- Users searching for exact part numbers get semantically similar but wrong parts, which points to adding keyword matching in a hybrid search.
- Common confusion
- Assuming semantic search is strictly better than keyword search for every query type.
8.7 Managing token usage and costsControlling LLM spend and context limits.
- What it means
- Tokens are the units LLMs process and bill for, counting both input (prompt plus retrieved context) and output; token management keeps usage within context window limits and budget.
- What to understand
- Cost and latency rise with tokens, so send fewer, more relevant chunks, trim prompt boilerplate, cap output length, cache repeated responses or embeddings, and route simple requests to smaller, cheaper models. Track tokens per request, user and feature to find what drives spending. Larger context windows allow more input but do not guarantee the model uses it well.
- How it shows up
- Monthly LLM costs tripled after the team increased retrieval from 5 to 30 chunks per query, and the fix is better ranking with fewer, more relevant chunks.
- Common confusion
- Treating embedding dimensions or document count as the main cost driver of generation, when input and output tokens per request drive it.
High-yield concepts
The terms, rules and formulas this domain turns into questions most often.
-
Embedding
An embedding is a numeric vector representing the meaning of text or other content, so similar meanings sit close together in vector space.
- Remember
- Only vectors from the same model and version are comparable.
- Tested as
- Foundation for vector search and RAG questions.
- Trap
- Mixing embeddings from different models in one index.
-
Approximate nearest neighbor (ANN) index
An ANN index, such as HNSW or IVF, finds close vectors quickly by searching a structure rather than comparing every vector.
- Remember
- It trades a small loss in recall for large gains in speed, tunable through index parameters.
- Tested as
- Scaling and latency questions for vector search.
- Trap
- Expecting exact results from an approximate index.
-
Chunking
Chunking splits documents into passages sized for embedding and retrieval, often with overlap between neighbors.
- Remember
- Chunk by meaningful structure such as sections, and tune size and overlap against retrieval quality.
- Tested as
- Retrieval quality problems frequently trace to chunking.
- Trap
- Embedding whole long documents as single vectors.
-
Retrieval augmented generation (RAG)
RAG retrieves relevant content at query time and includes it in the prompt so the LLM answers from current, specific sources.
- Remember
- It suits changing knowledge and supports citations; it does not change the model's weights.
- Tested as
- RAG versus fine-tuning is a core decision in this domain.
- Trap
- Fine-tuning to add frequently changing facts.
-
Fine-tuning
Fine-tuning further trains a model on curated examples to change its behavior, style or output format.
- Remember
- Data quality, deduplication and removing personal data matter more than volume.
- Tested as
- Used when the scenario needs consistent behavior rather than new facts.
- Trap
- Expecting it to make the model cite sources.
-
Hybrid search
Hybrid search combines keyword ranking such as BM25 with vector similarity, merging the results.
- Remember
- It catches exact identifiers that vector search alone may miss.
- Tested as
- Search questions involving codes, names or rare terms.
- Trap
- Replacing keyword search entirely with vector search.
-
Re-ranking
Re-ranking applies a more precise but slower model, such as a cross-encoder, to reorder a short list of retrieved candidates.
- Remember
- Retrieve broadly, then re-rank to send fewer, better chunks to the LLM.
- Tested as
- Precision and cost optimization questions.
- Trap
- Re-ranking the entire corpus, which is too slow.
-
Permission-aware retrieval
Permission-aware retrieval filters candidate chunks by the requesting user's access rights before they enter the prompt.
- Remember
- Store access metadata with each chunk and keep it in sync with the source system.
- Tested as
- Security scenarios with leaked confidential content.
- Trap
- Relying on prompt instructions to withhold confidential information.
-
Tokens and context window
Tokens are the pieces of text an LLM processes and bills for, and the context window is the maximum tokens per request.
- Remember
- Cost and latency scale with input plus output tokens.
- Tested as
- Cost-control and context-overflow questions.
- Trap
- Sending more retrieved chunks to fix quality and multiplying cost.
-
Prompt versioning
Prompt versioning tracks each change to prompt templates with an identifier, owner and evaluation results.
- Remember
- Log the prompt version with every response so regressions can be traced and rolled back.
- Tested as
- Quality regression scenarios after prompt edits.
- Trap
- Editing prompts directly in production without testing.
-
Semantic caching
Semantic caching reuses stored responses for new queries whose embeddings are very similar to earlier ones.
- Remember
- It cuts token cost and latency for repeated questions but needs expiry when source data changes.
- Tested as
- Cost optimization questions.
- Trap
- Serving stale cached answers after the underlying documents are updated.
Side by side
Concepts candidates mix up, compared on the point that decides the answer.
RAG vs. fine-tuning vs. prompt engineering
| Concept | Meaning | Key difference | Common trap |
|---|---|---|---|
| RAG | Retrieve relevant content and add it to the prompt at query time | Supplies current, citable knowledge without changing the model | Expecting it to fix output style or format issues |
| Fine-tuning | Further train the model on curated examples | Changes behavior, tone or format consistently | Using it to keep fast-changing facts current |
| Prompt engineering | Adjust instructions and examples in the prompt | Fastest and cheapest change, but limited by context length and model ability | Using prompt instructions as a security control |
Keyword vs. semantic vs. hybrid search
| Concept | Meaning | Key difference | Common trap |
|---|---|---|---|
| Keyword search (BM25) | Ranks documents by term matches | Precise for exact terms and identifiers, weak on synonyms | Missing paraphrased queries |
| Semantic (vector) search | Ranks by embedding similarity | Handles meaning and paraphrase, weaker on exact codes | Returning similar but wrong items for part numbers |
| Hybrid search | Combines keyword and vector rankings | Balances exact matching with meaning | Skipping evaluation of how results are merged |
Exact vs. approximate nearest neighbor search
| Concept | Meaning | Key difference | Common trap |
|---|---|---|---|
| Exact (brute force) | Compare the query with every stored vector | Perfect recall, latency grows with collection size | Using it for tens of millions of vectors |
| Approximate (HNSW, IVF) | Search an index structure for likely neighbors | Fast at scale with slightly lower recall | Not tuning index parameters against recall targets |
Exam traps
Each trap is an answer that looks right. Learn the clue that should change your mind.
-
Fine-tune for fresh facts
- What candidates pick
- Fine-tune the LLM monthly on updated product documentation.
- Why it looks right
- Training on the documents seems to teach the model the content directly.
- Why it's wrong
- Facts change faster than retraining cycles, fine-tuned knowledge cannot be cited, and retraining is costly; RAG with incremental re-embedding keeps answers current.
Clue to watch for Content that changes frequently or answers that must cite sources.
-
Prompt as security
- What candidates pick
- Add 'never reveal confidential HR data' to the system prompt.
- Why it looks right
- It is quick and seems to address the problem directly.
- Why it's wrong
- Prompt instructions can be bypassed and the data is already in the context; restricted chunks must be filtered out at retrieval based on user permissions.
Clue to watch for Users receiving information from documents they cannot access in the source system.
-
Partial model upgrade
- What candidates pick
- Switch query embeddings to a newer model while keeping the existing document index.
- Why it looks right
- It improves query understanding without the cost of a rebuild.
- Why it's wrong
- Vectors from different models live in different spaces, so similarity scores become meaningless; the whole corpus must be re-embedded with the new model.
Clue to watch for Retrieval quality collapsed immediately after an embedding model change.
-
More context, better answers
- What candidates pick
- Retrieve 40 chunks per question instead of 5.
- Why it looks right
- More context seems to raise the chance of including the right passage.
- Why it's wrong
- It multiplies token cost and latency and can bury relevant content among noise; better chunking, hybrid search and re-ranking improve precision instead.
Clue to watch for Rising costs or answers that ignore relevant passages despite long prompts.
-
Vector search for everything
- What candidates pick
- Replace keyword search with pure vector search for an internal parts catalog.
- Why it looks right
- Semantic search is the newer technology.
- Why it's wrong
- Exact part numbers and codes are better matched by keyword search; hybrid search combines both strengths.
Clue to watch for Queries containing exact identifiers, SKUs or error codes.
-
Index never updated
- What candidates pick
- Build the vector index once at launch and leave it.
- Why it looks right
- Initial results are good and rebuilding seems unnecessary.
- Why it's wrong
- Source documents change and get deleted, so the index drifts and may serve outdated or removed content; incremental re-embedding and deletion are needed.
Clue to watch for Answers referencing retired policies or documents that no longer exist.
Answer framework
For ABDE generative AI questions, classify the problem and then apply these steps.
- Decide whether the problem is knowledge, behavior, security or cost, using RAG for changing knowledge and fine-tuning for consistent behavior or format.
- Check the embedding pipeline for consistent models, sensible chunking and incremental updates that also remove deleted content.
- Improve retrieval with metadata filters, hybrid search and re-ranking, and measure recall on a labeled query set, before adding more context to prompts.
- Enforce document permissions at retrieval time so restricted content never reaches the prompt, and protect logged prompts and responses.
- Control cost by measuring tokens per request, trimming context, caching repeats and routing simple requests to smaller models, while versioning prompts with evaluation.
Practice question
Want more like this? The ABDE practice test has questions for every domain.
Checklist
0 of 11
Tick each one you can do without notes. Your ticks stay in this browser.
Study strategy
This domain is 5% of the exam and DASCA names no specific tool, so study the concepts rather than any one product.
Start with embeddings and vector search: what an embedding represents, why models cannot be mixed, and how ANN indexes such as HNSW speed up search. Then build a mental model of a RAG pipeline end to end: extract, chunk, embed, index, retrieve, re-rank, assemble the prompt and generate. If you can, build a tiny version with an open-source embedding model and pgvector or a local vector store, and observe how chunk size changes results.
Next, focus on the decision candidates most often get wrong: RAG for changing knowledge versus fine-tuning for behavior. Then cover permission-aware retrieval, which connects to Domain 4 and is easy to underestimate.
Finish with operations: prompt versioning, hybrid search, semantic caching and token cost drivers. Write down three ways to lower cost without hurting quality.
Afterwards, attempt timed practice questions and classify each miss as a knowledge, behavior, security or cost problem you misidentified. Then review Domain 7, since feature consistency and monitoring follow the same reasoning as embeddings and retrieval.
FAQ
How much of the ABDE exam is generative AI?
DASCA weights Generative AI & LLM Operations Overview at 5%, which suggests about 4 of the 75 questions. DASCA does not publish per-domain counts.
Which vector database should I learn?
DASCA does not name a tool for this domain, so focus on concepts that apply everywhere: embeddings, similarity metrics, ANN indexes, metadata filtering and keeping indexes in sync with sources.
When should I use RAG instead of fine-tuning?
Use RAG when answers depend on knowledge that changes or must be cited. Use fine-tuning when you need consistent behavior, tone or output format that prompts alone cannot achieve reliably.
Why must I re-embed everything when changing embedding models?
Each model maps text into its own vector space, so similarity between vectors from different models is meaningless. Queries and documents must share the same model and version.
How do I stop a RAG system from exposing confidential documents?
Store access metadata with each chunk and filter retrieval by the user's permissions, so restricted content never enters the prompt. Prompt instructions alone are not a reliable control.
What drives LLM costs in a RAG application?
Mainly the input and output tokens per request, which include retrieved context. Fewer, better-ranked chunks, shorter prompts, output limits, caching and routing to smaller models all reduce cost.
How does this domain build on earlier ABDE domains?
Embedding pipelines are ingestion pipelines (Domain 1) that need orchestration (Domain 3), governance (Domain 4) and the consistency and monitoring ideas from AI/ML Integration (Domain 7).
Other Data & Analytics certifications
Where the same subject shows up in other exams. Cards marked "Same subject" open the matching domain.
Test your Domain 8 knowledge
You have covered vector databases, embedding pipelines, RAG, fine-tuning data, prompt management, semantic search and token costs. Test whether you can diagnose LLM data problems under exam timing.