Overview
Deep Learning has two topics. The introduction covers neural network architecture (layers of neurons with weights, biases and activation functions, trained by backpropagation and gradient descent) and contrasts deep learning with classic machine learning. Deep learning techniques then covers convolutional neural networks for grid data such as images, recurrent neural networks for sequences, transfer learning for reusing pretrained models, and generative adversarial networks for generating realistic data.
The module builds on Module 2's loss functions and gradient descent and on Module 5's generative models, and it connects to Module 7 because training deep networks on big data needs GPUs and distributed infrastructure. On the exam, the main skill is matching architecture to data and recognizing training problems from their symptoms, not deriving equations.
How it's tested
Questions are mainly architecture-choice and training-diagnosis scenarios. The data type (image, sequence, tabular), the amount of labeled data and the symptom in the training curves usually determine the answer. Expect a few conceptual items on activations, backpropagation and the generator-discriminator relationship in GANs.
-
Match architecture to data
The scenario describes images, time series, text or tables and you choose a CNN, an RNN or LSTM, another architecture or a classic model.
-
Diagnose training
Loss curves or behavior are described, such as validation loss rising while training loss falls, or gradients vanishing, and you choose the fix.
-
Use what already exists
Little labeled data is available in a domain with pretrained models, and you choose transfer learning and how much of the network to fine-tune.
A team trains a deep network on 600 rows of customer data with 12 numeric columns and gets worse validation results than a gradient-boosted tree. With few rows, structured features and no images or text, deep learning has little to learn from and overfits easily. The better answer keeps the tree ensemble; deep learning earns its cost on large volumes of unstructured data.
Topics 2
Official topics from the Enterprise Big Data Framework Alliance outline, in outline order.
8.1 Introduction to Deep LearningNeural network architecture and how deep learning differs from classic machine learning.
- What it means
- A neural network stacks layers of neurons; each computes a weighted sum of its inputs plus a bias and passes it through a non-linear activation function. Training runs a forward pass, measures loss, and uses backpropagation (the chain rule) to compute gradients that an optimizer such as stochastic gradient descent or Adam uses to update weights. Deep learning means networks with many hidden layers that learn features from raw data.
- What to understand
- Without non-linear activations, stacked layers collapse into one linear model. ReLU is the common hidden-layer activation because it eases vanishing gradients; sigmoid suits binary outputs and softmax multi-class outputs. Overfitting is controlled with dropout, early stopping, weight decay and data augmentation. Compared with classic machine learning, deep learning needs more data and compute, learns features automatically and is harder to interpret.
- How it shows up
- A scenario asks why a network with only linear activations cannot learn a curved boundary, or whether deep learning or gradient-boosted trees suits a small tabular dataset.
- Common confusion
- Assuming deep learning always outperforms classic machine learning; on small or tabular data it often does not.
8.2 Deep Learning TechniquesConvolutional neural networks, recurrent neural networks, transfer learning and generative adversarial networks.
- What it means
- CNNs apply learned filters across grid data such as images, sharing weights to detect local patterns. RNNs carry a hidden state across time steps to model sequences, with LSTM and GRU gates handling longer dependencies. Transfer learning reuses a network pretrained on a large dataset for a new task. GANs train a generator to create data and a discriminator to tell real from generated data, in competition.
- What to understand
- A convolution layer's output width is (W - F + 2P) / S + 1 for input width W, filter size F, padding P and stride S, and depth equals the number of filters. Plain RNNs suffer vanishing gradients over long sequences, which LSTMs mitigate. For transfer learning, freeze early layers and retrain the head when data is scarce and similar; fine-tune more layers with a low learning rate when data is larger or more different. GAN training can be unstable and suffer mode collapse, where the generator produces little variety.
- How it shows up
- A manufacturer has 800 labeled defect images and you choose fine-tuning a pretrained CNN; or a GAN produces near-identical outputs and you identify mode collapse.
- Common confusion
- Thinking transfer learning means copying a model's predictions; it reuses learned weights and features, then adapts them to the new task.
High-yield concepts
The terms, rules and formulas this domain turns into questions most often.
-
Neuron and activation function
Each neuron computes a weighted sum of inputs plus a bias and applies a non-linear activation such as ReLU, sigmoid or tanh.
- Remember
- Non-linearity is what lets stacked layers model curved relationships.
- Tested as
- Conceptual items on why networks need activations.
- Trap
- Believing more linear layers increase what the network can represent.
-
Backpropagation
Uses the chain rule to compute the gradient of the loss with respect to every weight, layer by layer from output to input.
- Remember
- Backpropagation computes gradients; the optimizer (SGD, Adam) uses them to update weights.
- Tested as
- Training-process questions.
- Trap
- Calling backpropagation an optimizer.
-
Output activations
Sigmoid gives one probability for binary tasks, softmax gives a probability distribution over multiple classes, and a linear output suits regression.
- Remember
- Pair softmax with categorical cross-entropy and sigmoid with binary cross-entropy.
- Tested as
- Architecture setup items.
- Trap
- Using softmax for multi-label problems where several classes can be true at once.
-
Vanishing gradients
Gradients shrink as they are multiplied back through many layers or time steps, so early layers learn very slowly.
- Remember
- ReLU activations, careful initialization, residual connections and LSTM gates reduce the problem.
- Tested as
- Diagnosis of deep or recurrent networks that stop learning.
- Trap
- Responding by simply training for more epochs.
-
Regularizing neural networks
Dropout randomly disables neurons in training, early stopping halts when validation loss rises, weight decay penalizes large weights, and augmentation creates varied training examples.
- Remember
- Validation loss rising while training loss falls signals overfitting.
- Tested as
- Training-curve scenarios.
- Trap
- Adding layers to fix overfitting.
-
Deep learning vs. classic machine learning
Deep learning learns features from raw data and excels on large unstructured datasets; classic methods rely more on engineered features and often win on small or tabular data.
- Remember
- Deep models need more data, compute and effort to explain.
- Tested as
- Method-choice scenarios.
- Trap
- Choosing deep learning because it is the most advanced option.
-
Convolution and pooling
Convolution slides shared filters across an input to detect local patterns; pooling downsamples feature maps to reduce size and add tolerance to small shifts.
- Remember
- Output width = (W - F + 2P) / S + 1; depth = number of filters.
- Tested as
- Image scenarios and short size calculations.
- Trap
- Forgetting that weight sharing makes CNNs far smaller than fully connected networks on images.
-
RNN, LSTM and GRU
RNNs pass a hidden state from step to step; LSTMs and GRUs add gates that control what to keep and forget.
- Remember
- Gates let LSTMs keep information over longer sequences than plain RNNs.
- Tested as
- Sequence and time-series scenarios.
- Trap
- Choosing a plain RNN for long sequences with distant dependencies.
-
Transfer learning
Reusing a network pretrained on a large dataset as the starting point for a new, related task.
- Remember
- Small, similar data: freeze most layers and train a new head. More or different data: fine-tune deeper layers with a low learning rate.
- Tested as
- Scenarios with few labeled examples.
- Trap
- Training from scratch on a few hundred images.
-
Generative adversarial network
A generator produces synthetic samples and a discriminator classifies real versus generated; each improves by trying to beat the other.
- Remember
- At the ideal equilibrium the discriminator can do no better than chance.
- Tested as
- Generative modeling and synthetic data scenarios.
- Trap
- Thinking the discriminator is the model that produces the output.
-
Mode collapse
A GAN failure where the generator produces only a few kinds of outputs that fool the discriminator.
- Remember
- Low variety in generated samples, even if each looks realistic, signals mode collapse.
- Tested as
- GAN diagnosis items.
- Trap
- Judging the GAN successful because individual samples look real.
Side by side
Concepts candidates mix up, compared on the point that decides the answer.
CNN vs. RNN vs. fully connected network
| Concept | Meaning | Key difference | Common trap |
|---|---|---|---|
| CNN | Shared convolutional filters over grid-structured data | Captures local spatial patterns; efficient on images | Using it by default for long text dependencies |
| RNN (LSTM, GRU) | Hidden state carried across sequence steps | Models order and temporal dependence | Using a plain RNN for very long sequences |
| Fully connected network | Every neuron connected to every input | General-purpose; no built-in structure for space or time | Applying it to raw images, which explodes parameter counts |
Feature extraction vs. fine-tuning in transfer learning
| Concept | Meaning | Key difference | Common trap |
|---|---|---|---|
| Feature extraction | Freeze the pretrained layers and train only a new output head | Fast and safe with small, similar datasets | Using it when the new domain is very different |
| Fine-tuning | Unfreeze some or all layers and continue training with a low learning rate | Adapts features to the new domain; needs more data | Using a high learning rate that destroys pretrained features |
Generator vs. discriminator
| Concept | Meaning | Key difference | Common trap |
|---|---|---|---|
| Generator | Maps random noise to synthetic samples | Tries to make samples the discriminator accepts as real | Evaluating it on a few realistic samples only |
| Discriminator | Classifies samples as real or generated | Provides the training signal to the generator | Treating its accuracy near 50% as failure; at equilibrium that is expected |
Exam traps
Each trap is an answer that looks right. Learn the clue that should change your mind.
-
Deep learning on small tabular data
- What candidates pick
- A deep neural network for 600 customer records with 12 numeric features.
- Why it looks right
- Deep learning is the newest and most powerful-sounding option.
- Why it's wrong
- Deep networks need large data to learn useful features and overfit small tabular sets; gradient-boosted trees or logistic regression are usually stronger and easier to explain.
Clue to watch for Few rows, structured columns, and no images, audio or text.
-
Training from scratch
- What candidates pick
- Design and train a new CNN on 800 labeled defect images.
- Why it looks right
- A custom model sounds more tailored to the task.
- Why it's wrong
- So few images cannot train a deep CNN reliably; a model pretrained on a large image dataset already knows edges and textures and only needs adapting.
Clue to watch for A small labeled dataset in a domain where pretrained models exist.
-
More capacity for overfitting
- What candidates pick
- Add layers when validation loss rises while training loss keeps falling.
- Why it looks right
- Deeper networks are associated with better results.
- Why it's wrong
- The diverging curves show overfitting; dropout, early stopping, weight decay or more data help, while extra capacity worsens it.
Clue to watch for Training loss falls steadily while validation loss turns upward.
-
Plain RNN for long sequences
- What candidates pick
- A simple RNN to use words from the start of long documents.
- Why it looks right
- RNNs are the classic sequence model.
- Why it's wrong
- Plain RNNs suffer vanishing gradients and lose distant information; LSTMs or GRUs keep longer dependencies.
Clue to watch for Long sequences where early inputs matter for later outputs.
-
Realistic samples mean success
- What candidates pick
- Accept a GAN because each generated product image looks real.
- Why it looks right
- Visual realism is the obvious goal.
- Why it's wrong
- If the outputs are nearly identical, the generator has collapsed to a few modes and does not represent the data's variety.
Clue to watch for Generated samples look alike or cover only part of the real categories.
-
High fine-tuning learning rate
- What candidates pick
- Fine-tune all layers of a pretrained model with the original training learning rate.
- Why it looks right
- Faster learning seems efficient.
- Why it's wrong
- A large rate overwrites the useful pretrained features; fine-tuning uses a much lower learning rate, often after first training only the new head.
Clue to watch for Accuracy drops sharply after fine-tuning starts.
Answer framework
Use these steps when a scenario asks whether and how to apply deep learning.
- Identify the data type and volume: images, sequences or text favor deep learning, while small structured tables often favor classic models such as gradient-boosted trees.
- Match the architecture to structure: CNNs for spatial grids, LSTMs or GRUs for ordered sequences, and GANs or other generative models for synthetic data.
- Check labeled data availability and whether a pretrained model exists; if data is scarce, use transfer learning before considering training from scratch.
- Read the training curves to diagnose overfitting, underfitting, vanishing gradients or mode collapse.
- Apply the matching fix, such as dropout, early stopping, gated units, a lower learning rate or a smaller model, rather than adding layers by default.
Practice question
Want more like this? The EBDS practice test has questions for every domain.
Checklist
0 of 12
Tick each one you can do without notes. Your ticks stay in this browser.
Study strategy
Start with topic 8.1 and make sure the training loop is concrete: forward pass, loss, backpropagation, optimizer step. Build a tiny network in a notebook with a framework such as Keras or PyTorch, remove the activations to watch it fail on a curved boundary, then add dropout and early stopping and compare loss curves. High-priority concepts are activations, vanishing gradients and overfitting signals.
Move to topic 8.2 by pairing each architecture with its data: train a small CNN on images, an LSTM on a short time series, and fine-tune a pretrained image model on a few hundred examples. Practice the convolution output-size formula until it is quick.
Commonly underestimated areas are the deep-versus-classic judgment and GAN failure modes, because they are tested as reasoning rather than definitions. Afterwards, drill architecture-choice and training-diagnosis scenarios under time pressure, then continue to Module 9, where deep learning and the rest of the toolkit meet real case studies and data ethics.
FAQ
How mathematical is the EBDS deep learning module?
The outline focuses on architecture and techniques. Expect concepts such as activations, backpropagation and gradients, and simple calculations like convolution output size, not derivations.
How many questions cover deep learning?
No module weights are published. An even split of the 80 questions suggests about nine, as a planning estimate only.
When should I choose deep learning over classic machine learning?
When you have large volumes of unstructured data such as images, audio or text, or a pretrained model to adapt. For small tabular datasets, classic models are usually stronger and more interpretable.
What is the difference between a CNN and an RNN?
A CNN applies shared filters to capture local spatial patterns in grid data such as images. An RNN carries a hidden state across steps to model ordered sequences.
Why use transfer learning?
It reuses features learned from a large dataset, so a new task needs far less labeled data and compute and usually reaches better accuracy than training from scratch.
What is mode collapse in a GAN?
The generator produces only a narrow range of outputs that fool the discriminator, so the generated data lacks the variety of the real data.
How does deep learning connect to the black-box topic in Module 3?
Deep networks are among the least interpretable models, so explanation methods such as SHAP and LIME from Module 3 are often needed before stakeholders act on their outputs.
Other Data & Analytics certifications
Where the same subject shows up in other exams. Cards marked "Same subject" open the matching domain.
Test your Domain 8 knowledge
You have covered neural network architecture, the deep-versus-classic decision, CNNs, RNNs, transfer learning and GANs. Check whether you can choose and troubleshoot deep models in exam scenarios.