Overview
Statistical Methods for Data Science and Machine Learning covers three blocks. Probability distributions take you from discrete and continuous distributions through sampling, fuzzy logic, Bayes' theorem and Bayesian networks to feature scaling. Parametric models contrast fixed-parameter models with nonparametric ones and introduce the calculus and loss functions that let a model learn by minimizing error. Assessing model quality closes with the bias-variance tradeoff, the curse of dimensionality and confusion matrices.
The module exists because every later technique is a statistical model with assumptions. Logistic regression and linear discriminant analysis in Module 4, PCA in Module 5 and neural network training in Module 8 all reuse these ideas: a loss to minimize, a distribution to assume, and a way to tell underfitting from overfitting. Learn the concepts here once and the rest of the course reads as applications.
How it's tested
Questions mix small calculations with interpretation. Expect a Bayes' theorem or confusion-matrix computation, a choice of distribution for a described process, and diagnosis items where training and validation error are given and you must name the problem and the fix. Work numbers on paper and check which quantity the stem asks for.
-
Compute and interpret
A scenario gives prevalence and test accuracy, or a confusion matrix, and asks for a posterior probability, precision or recall. The trap answer is usually the number given in the stem.
-
Pick the distribution
A data-generating process is described, such as counts of events per hour or the number of successes in fixed trials, and you choose Poisson, binomial, normal or another distribution.
-
Diagnose the fit
Training and validation errors are described, and you identify high bias or high variance and the remedy that addresses it.
A model scores 99% training accuracy and 71% validation accuracy. Low training error with a large gap means high variance, so the fix reduces flexibility or adds data: regularization, fewer features or more training examples. Adding more polynomial terms would widen the gap. Had both errors been high and close together, the diagnosis would be high bias and the fix a more flexible model.
Topics 3
Official topics from the Enterprise Big Data Framework Alliance outline, in outline order.
2.1 Probability DistributionsDiscrete and continuous distributions, sampling, fuzzy logic, Bayes' theorem, Bayesian networks and feature scaling.
- What it means
- A distribution describes how likely each value of a variable is: a probability mass function for discrete variables and a density for continuous ones. The topic also covers how samples are drawn, how fuzzy logic assigns degrees of membership between 0 and 1, how Bayes' theorem updates beliefs with evidence, and how features are rescaled.
- What to understand
- Match processes to distributions: binomial for successes in n fixed trials, Poisson for event counts in an interval, normal for symmetric continuous measurements, exponential for waiting times. Apply P(A|B) = P(B|A)P(A) / P(B) and respect the prior. Use min-max scaling to map to [0, 1] and standardization z = (x - mean) / standard deviation when distance- or gradient-based methods are involved.
- How it shows up
- A stem gives a 1% prevalence and a 90% sensitive test and asks the probability that a positive result is a true case, or asks which scaling suits k-means.
- Common confusion
- Reading P(positive | disease) as P(disease | positive), and treating fuzzy membership as a probability.
2.2 Parametric ModelsParametric versus nonparametric models, the calculus behind fitting, and loss functions.
- What it means
- A parametric model assumes a functional form with a fixed number of parameters, such as linear or logistic regression, while a nonparametric model such as k-nearest neighbors or a decision tree lets its complexity grow with the data. Fitting means choosing parameters that minimize a loss function, usually by following its gradient.
- What to understand
- Gradient descent updates parameters as theta minus learning rate times the gradient of the loss; too large a rate diverges, too small converges slowly. Mean squared error penalizes large errors quadratically and is sensitive to outliers, mean absolute error is more robust, and cross-entropy (log loss) is the standard loss for probabilistic classification.
- How it shows up
- A scenario with heavy outliers asks which loss to prefer, or a training curve that oscillates and grows asks what to change.
- Common confusion
- Thinking nonparametric means no parameters at all; it means the number of parameters is not fixed in advance.
2.3 Assessing Model QualityThe bias-variance tradeoff, the curse of dimensionality and confusion matrices.
- What it means
- Expected prediction error splits into bias squared, variance and irreducible noise. Adding dimensions makes data sparse and distances less informative. A confusion matrix tabulates true positives, false positives, false negatives and true negatives for a classifier.
- What to understand
- High bias shows as high training and validation error; high variance shows as low training error with a large validation gap. Precision is TP / (TP + FP), recall (sensitivity) is TP / (TP + FN), specificity is TN / (TN + FP), and accuracy misleads on imbalanced classes.
- How it shows up
- A fraud model with 99.5% accuracy on data that is 0.5% fraud is described, and you must see that it may catch no fraud at all.
- Common confusion
- Swapping precision and recall: precision asks how many flagged cases were real, recall asks how many real cases were flagged.
High-yield concepts
The terms, rules and formulas this domain turns into questions most often.
-
Bayes' theorem
P(A|B) = P(B|A) x P(A) / P(B) updates a prior belief P(A) into a posterior after observing evidence B.
- Remember
- With a rare condition, even an accurate test gives a low posterior because false positives from the large negative group dominate.
- Tested as
- Diagnostic-test and spam-filter calculations are the classic computation items in this module.
- Trap
- Answering with the test's sensitivity instead of computing the posterior.
-
Bayesian network
A directed acyclic graph whose nodes are variables and whose edges carry conditional probability tables, so the joint distribution factorizes into local pieces.
- Remember
- Each node depends only on its parents, which keeps inference tractable.
- Tested as
- Questions ask what an edge represents or why the structure reduces the probabilities you must estimate.
- Trap
- Reading edges as proven causation rather than modeled conditional dependence.
-
Common distributions
Binomial counts successes in n independent trials, Poisson counts events in a fixed interval, normal models symmetric continuous values, exponential models time between events.
- Remember
- Poisson mean equals its variance; binomial mean is np.
- Tested as
- Distribution-choice items describe a process and ask which fits.
- Trap
- Choosing normal for small non-negative counts that are better modeled as Poisson.
-
Sampling methods
Simple random sampling gives each unit equal chance, stratified sampling samples within subgroups, cluster sampling samples whole groups, systematic sampling takes every k-th unit.
- Remember
- Stratified sampling preserves the proportion of a rare class, which matters for imbalanced data.
- Tested as
- Scenarios ask which design keeps a minority segment represented.
- Trap
- Assuming a large convenience sample removes selection bias.
-
Fuzzy logic
Truth values range continuously from 0 to 1, so an item can be partly a member of a set, for example 'warm' to degree 0.7.
- Remember
- Fuzzy membership measures vagueness of a category, not the probability of an event.
- Tested as
- Contrast items ask how fuzzy logic differs from Boolean logic or from probability.
- Trap
- Interpreting a membership of 0.7 as a 70% chance.
-
Feature scaling
Min-max scaling maps values to [0, 1]; standardization subtracts the mean and divides by the standard deviation.
- Remember
- Distance-based methods (k-means, k-NN, SVM) and gradient-based training need scaling; tree-based models do not.
- Tested as
- Preprocessing questions ask when scaling changes results.
- Trap
- Fitting the scaler on the whole dataset before splitting, which leaks test information.
-
Loss functions
Mean squared error, mean absolute error, cross-entropy and hinge loss each quantify prediction error differently.
- Remember
- MSE squares errors so outliers dominate; MAE is more robust; cross-entropy fits probabilistic classifiers; hinge loss underlies SVMs.
- Tested as
- Scenarios with outliers or probabilistic outputs ask which loss suits them.
- Trap
- Using MSE as the default when outliers would distort the fit.
-
Gradient descent
An iterative optimizer that moves parameters a step proportional to the negative gradient of the loss.
- Remember
- Learning rate too high causes oscillation or divergence; too low causes slow convergence.
- Tested as
- Training-curve diagnosis items reference it directly.
- Trap
- Fixing a diverging loss by training for more epochs.
-
Bias-variance tradeoff
Expected error equals bias squared plus variance plus irreducible error; flexible models lower bias but raise variance.
- Remember
- High bias: both errors high and close. High variance: low training error, large validation gap.
- Tested as
- Diagnosis questions with given error figures test it in almost every modeling module.
- Trap
- Adding model complexity to fix overfitting.
-
Curse of dimensionality
As features increase, data becomes sparse and distances between points become similar, so neighborhood methods lose meaning.
- Remember
- The data needed to cover the space grows exponentially with dimensions.
- Tested as
- Questions ask why k-NN degrades with many features and what helps: feature selection or PCA.
- Trap
- Assuming more features always add information.
-
Precision, recall and specificity
Precision = TP / (TP + FP); recall = TP / (TP + FN); specificity = TN / (TN + FP).
- Remember
- Favor recall when missing a positive is costly, precision when false alarms are costly.
- Tested as
- Confusion-matrix calculations and metric-choice scenarios.
- Trap
- Reporting accuracy on heavily imbalanced data.
Side by side
Concepts candidates mix up, compared on the point that decides the answer.
Parametric vs. nonparametric models
| Concept | Meaning | Key difference | Common trap |
|---|---|---|---|
| Parametric | Fixed functional form and number of parameters, such as linear regression or logistic regression | Strong assumptions; fast and data-efficient; can underfit | Thinking it cannot model curves; polynomial terms keep it parametric |
| Nonparametric | Complexity grows with data, such as k-NN or decision trees | Few assumptions; flexible; needs more data and can overfit | Believing it has no parameters at all |
Precision vs. recall vs. accuracy
| Concept | Meaning | Key difference | Common trap |
|---|---|---|---|
| Precision | Share of predicted positives that are truly positive | Penalizes false positives | Using it when missed cases are the main cost |
| Recall | Share of actual positives the model catches | Penalizes false negatives | Confusing it with precision in fraud or disease scenarios |
| Accuracy | Share of all predictions that are correct | Ignores class balance | Trusting 99% accuracy when positives are under 1% of data |
Min-max scaling vs. standardization
| Concept | Meaning | Key difference | Common trap |
|---|---|---|---|
| Min-max scaling | (x - min) / (max - min), mapping to [0, 1] | Bounded range; one extreme value compresses the rest | Using it on data with strong outliers |
| Standardization | (x - mean) / standard deviation | Zero mean, unit variance; unbounded | Believing it makes data normally distributed |
High bias vs. high variance
| Concept | Meaning | Key difference | Common trap |
|---|---|---|---|
| High bias (underfitting) | Model too simple for the pattern | Training and validation error both high and close | Adding data, which rarely helps an underfit model |
| High variance (overfitting) | Model fits noise in training data | Low training error, much higher validation error | Increasing model complexity instead of regularizing |
Exam traps
Each trap is an answer that looks right. Learn the clue that should change your mind.
-
Ignoring the base rate
- What candidates pick
- 90% as the chance a positive result is a true case, because the test is 90% sensitive.
- Why it looks right
- The number is given directly and sounds like the test's reliability.
- Why it's wrong
- Sensitivity is P(positive | condition); the question asks P(condition | positive), which depends heavily on prevalence and the false-positive rate.
Clue to watch for A low prevalence such as 1% or 1 in 1,000 appears in the stem.
-
Accuracy on imbalanced data
- What candidates pick
- The model with 99.5% accuracy as the best fraud detector.
- Why it looks right
- It is the highest number in the options.
- Why it's wrong
- If 0.5% of cases are fraud, predicting 'not fraud' every time scores 99.5% accuracy with zero recall.
Clue to watch for The positive class is rare and the cost of missing it is stated.
-
Complexity cures overfitting
- What candidates pick
- Add more features or a deeper model when validation error is far above training error.
- Why it looks right
- More capacity sounds like more accuracy.
- Why it's wrong
- A large gap signals high variance; extra capacity widens it. Regularization, simpler models or more data narrow it.
Clue to watch for Training error is very low while validation error is much higher.
-
Scaling everything
- What candidates pick
- Standardize features before training a random forest to improve accuracy.
- Why it looks right
- Scaling is described as standard preprocessing.
- Why it's wrong
- Tree splits depend on value order, not magnitude, so scaling leaves a tree model unchanged; it matters for distance and gradient methods.
Clue to watch for The model named is a decision tree, random forest or boosted trees.
-
Fuzzy membership as probability
- What candidates pick
- A temperature with 'hot' membership 0.8 has an 80% chance of being hot.
- Why it looks right
- Both are numbers between 0 and 1.
- Why it's wrong
- Fuzzy membership expresses the degree to which a value fits a vague category; it is certain, not a chance of an event.
Clue to watch for The scenario uses linguistic categories such as hot, tall or high risk with graded membership.
-
MSE with outliers
- What candidates pick
- Mean squared error to fit delivery times that include rare extreme delays.
- Why it looks right
- MSE is the default regression loss.
- Why it's wrong
- Squaring makes a few extreme errors dominate the fit; mean absolute error or a robust loss gives a more representative model.
Clue to watch for The stem mentions outliers, extreme values or heavy tails.
-
Scaler leakage
- What candidates pick
- Standardize the full dataset, then split into training and test sets.
- Why it looks right
- It is one tidy preprocessing step.
- Why it's wrong
- Test-set statistics leak into training, so evaluation is optimistic; fit the scaler on training data only and apply it to the test set.
Clue to watch for The order of preprocessing and splitting is described.
Answer framework
Use these steps on any statistics or model-quality item in this module.
- Write down exactly which quantity is asked for, such as P(A|B) versus P(B|A), or precision versus recall, before touching any numbers in the stem.
- List the given values with symbols, including prevalence, true and false positive rates, or the four confusion-matrix cells.
- Apply the formula explicitly and sanity-check the result against the base rate or class balance described in the scenario.
- For diagnosis items, compare training and validation error to label the problem as high bias or high variance.
- Choose the remedy that targets that problem, and reject options that only change scaling or loss when the issue is model capacity or data quantity.
Practice question
Want more like this? The EBDS practice test has questions for every domain.
Checklist
0 of 12
Tick each one you can do without notes. Your ticks stay in this browser.
Study strategy
Begin with topic 2.1 and build the habit of writing the conditional probability you need before calculating. Work five or six Bayes' theorem problems with different base rates until the frequency method (imagine 1,000 people) is automatic; it is faster and less error-prone than the formula under time pressure. Then make a one-page distribution table with use cases, mean and variance.
Move to topic 2.2 with a notebook open: fit a linear model by gradient descent with three learning rates and watch the loss curves, then fit the same data with MSE and MAE after adding outliers. Seeing the behavior makes the exam's diagnosis questions straightforward.
Topic 2.3 is the highest-leverage block because bias-variance and confusion matrices recur in Modules 3, 4, 5 and 8. Commonly underestimated areas are fuzzy logic and Bayesian networks, which are short in the syllabus but easy to confuse with probability. Afterwards, drill timed calculation sets that mix confusion matrices and Bayes items, then continue to Module 3, where these metrics are applied to full machine learning pipelines.
FAQ
How much math does the EBDS statistics module require?
Expect arithmetic-level calculations such as Bayes' theorem, confusion-matrix metrics and z-scores, plus conceptual understanding of derivatives and gradients. Proofs are not part of the published outline.
How many questions come from statistical methods?
No module weights are published. An even split of 80 questions suggests about nine, but this is a planning estimate, and the concepts also appear inside questions from other modules.
What is the difference between precision and recall?
Precision is the share of predicted positives that are correct, TP / (TP + FP). Recall is the share of actual positives that the model finds, TP / (TP + FN).
When should I scale features?
Scale for distance-based methods such as k-means, k-NN and SVMs, and for gradient-trained models. Tree-based models are unaffected by monotonic rescaling.
Is fuzzy logic the same as probability?
No. Fuzzy logic measures how strongly something belongs to a vague category, while probability measures the likelihood of an uncertain event.
How do I tell overfitting from underfitting quickly?
Compare training and validation error. Both high and close means underfitting (high bias); low training error with a large gap means overfitting (high variance).
How does this module connect to later modules?
Loss functions and gradients return in deep learning, confusion matrices in model performance measurement, the curse of dimensionality in PCA, and Bayes' theorem in linear discriminant analysis.
Other Data & Analytics certifications
Where the same subject shows up in other exams. Cards marked "Same subject" open the matching domain.
Test your Domain 2 knowledge
You have worked through distributions, Bayes' theorem, loss functions, bias-variance and confusion matrices. Check whether you can compute and diagnose them accurately under time pressure.