Machine Learning Interview Questions and Answers
Last updated 2026-11-19
Machine learning interviews ask you to explain how algorithms work, to compute small examples by hand, and to diagnose a model that is not performing. The most common failure is knowing the names but not the mechanics, so every answer below includes the mechanism and, where possible, a number you can verify yourself.
This page covers classical machine learning. For network-based models see deep learning interview questions, and for evaluation and modelling cases in a role context see data scientist interview questions.
| Typical rounds | ML theory, hand calculations, coding in Python, project discussion |
|---|---|
| Core skills | Algorithms, statistics, numpy, scikit-learn |
| Questions here | 15 worked questions |
Hand calculations worth practising
- One step of gradient descent on a tiny dataset.
- Gini impurity and entropy for a split.
- Precision, recall and ROC AUC from a few scored examples.
- Bayes rule for naive Bayes and conditional probability.
- Explained variance ratio from PCA eigenvalues.
Algorithm choice in one paragraph
For tabular data start with a regularised linear or logistic model as a baseline, then try gradient boosting. Use k-nearest neighbours or SVM when the data is small and the feature space is meaningful. Use clustering and PCA for exploration, not as a substitute for labelled evaluation. State why you chose a model in terms of data size, interpretability needs and training cost.
Machine Learning Interview Questions and Answers: questions with worked answers
1 Derive the closed-form solution of linear regression and state when you would not use it.
mediumMinimising the squared error ||y - Xw||^2 and setting the gradient to zero gives the normal equation w = (X^T X)^-1 X^T y. Inverting a d by d matrix costs about O(d^3), and the matrix is singular when features are perfectly collinear. For large d or large n, use gradient descent or stochastic gradient descent. With L2 regularisation the solution becomes (X^T X + lambda I)^-1 X^T y, which is always invertible for lambda above zero.
2 Do one step of gradient descent for y = w x on the points (1, 2) and (2, 4), starting at w = 0 with learning rate 0.1.
easyUse mean squared error L = (1/n) sum (w x - y)^2. The gradient is (2/n) sum (w x - y) x. At w = 0 the errors are -2 and -4, so the gradient is (2/2)(-2 x 1 + -4 x 2) = -10. The update is w = 0 - 0.1 x (-10) = 1.0. A second step: errors -1 and -2, gradient = -5, so w = 1.5. The values approach the true slope 2. Too large a learning rate would overshoot and diverge.
pythonimport numpy as np x = np.array([1.0, 2.0]); y = np.array([2.0, 4.0]); w, lr = 0.0, 0.1 for _ in range(2): grad = 2 * np.mean((w * x - y) * x) w -= lr * grad print(w) # 1.0 then 1.53 Compute the Gini impurity and the gain of a split. A node has 6 positives and 4 negatives; the split gives (4 positives, 0 negatives) and (2 positives, 4 negatives).
mediumParent Gini = 1 - (0.6^2 + 0.4^2) = 1 - 0.52 = 0.48. Left child is pure, so Gini = 0. Right child has 6 samples: 1 - ((1/3)^2 + (2/3)^2) = 1 - 5/9 = 0.444. The weighted child impurity is 0.4 x 0 + 0.6 x 0.444 = 0.267. The gain is 0.48 - 0.267 = 0.213. A tree picks the split with the largest gain. With entropy the parent is 0.971 bits.
4 How do entropy and information gain differ from Gini?
mediumEntropy is -sum p log2 p and Gini is 1 - sum p^2. Both are zero for a pure node and maximal for an even split (1 bit and 0.5 for two classes). Entropy penalises mixed nodes slightly more and costs a logarithm to compute. In practice they usually choose the same or very similar splits, so Gini is the common default because it is cheaper. Information gain is parent entropy minus weighted child entropy.
5 Why do decision trees overfit and how do you control it?
easyAn unconstrained tree keeps splitting until every leaf is pure, so it memorises noise. Control it with max depth, a minimum number of samples per leaf or per split, a minimum gain, or cost-complexity pruning (ccp_alpha), and tune these with cross-validation. Ensembles such as random forests reduce variance further by averaging many trees trained on different samples.
6 Compute ROC AUC for positives scored 0.9, 0.8, 0.4 and negatives scored 0.7, 0.3.
mediumAUC is the probability that a random positive is scored above a random negative. There are 3 x 2 = 6 pairs. Score 0.9 beats both negatives (2), 0.8 beats both (2), 0.4 beats 0.3 but not 0.7 (1). That gives 5 of 6 correct pairs, AUC = 0.833. Ties count as half. AUC is threshold independent, but on heavily imbalanced data PR AUC is more informative because ROC AUC is dominated by the many true negatives.
7 Apply naive Bayes: 40 percent of mails are spam, a word appears in 50 percent of spam and 10 percent of ham. What is P(spam | word)?
easyBy Bayes rule: P(spam | word) = 0.4 x 0.5 / (0.4 x 0.5 + 0.6 x 0.1) = 0.20 / 0.26 = 0.769. The naive assumption is that features are conditionally independent given the class, so for several words you multiply their likelihoods (or add their logs). The assumption is wrong for correlated words, but the model still ranks classes well and trains in one pass. Use Laplace smoothing to avoid zero probabilities.
8 Explain PCA. Eigenvalues are 4, 2, 1, 1. What fraction of variance do the first two components keep?
mediumPCA centres the data and finds orthogonal directions of maximum variance, the eigenvectors of the covariance matrix, ordered by eigenvalue. The total variance is 4 + 2 + 1 + 1 = 8, and the first two keep (4 + 2) / 8 = 75 percent. Scale features before PCA, because it is variance based. Components are linear combinations and less interpretable. Use the cumulative explained variance curve to choose how many to keep.
pythonfrom sklearn.decomposition import PCA from sklearn.preprocessing import StandardScaler Xs = StandardScaler().fit_transform(X) pca = PCA(n_components=0.95).fit(Xs) # keep 95% of variance print(pca.explained_variance_ratio_.cumsum())9 How does k-means work, and how do you choose k?
easyInitialise k centroids (k-means++ spreads them out), assign every point to its nearest centroid, move each centroid to the mean of its points, and repeat until assignments stop changing. Each iteration costs O(n k d). It finds a local minimum, so run several initialisations. Choose k with the elbow of inertia, or the silhouette score, which ranges from -1 to 1. It assumes round clusters of similar size and needs scaled features.
pythonfrom sklearn.cluster import KMeans from sklearn.metrics import silhouette_score for k in range(2, 7): km = KMeans(n_clusters=k, n_init=10, random_state=0).fit(Xs) print(k, km.inertia_, silhouette_score(Xs, km.labels_))10 What is multicollinearity and how do you detect it?
mediumIt is when predictors are strongly correlated, so coefficient estimates become unstable and have large standard errors, although predictions stay fine. Detect it with the variance inflation factor, VIF = 1 / (1 - R^2), where R^2 comes from regressing that feature on the others. If R^2 = 0.9 then VIF = 10, a common warning level. Fix by dropping or combining features, using PCA, or regularising with ridge.
11 How does k-nearest neighbours work and what are its weaknesses?
easyTo predict, find the k closest training points by a distance such as Euclidean, then average their targets or take a majority vote. There is no training, but prediction costs O(n d) per query without an index. It suffers from the curse of dimensionality because distances become similar in high dimensions, and it needs feature scaling. A small k has high variance; a large k has high bias. Use an odd k for binary classification to avoid ties.
12 Explain the SVM margin and the role of the kernel.
hardA linear SVM finds the hyperplane with the largest margin, the gap between it and the nearest points of each class, which are the support vectors. The soft-margin parameter C trades margin width against misclassified points: a large C fits training data harder. A kernel such as RBF computes inner products in a high-dimensional space without building it explicitly, so a nonlinear boundary in the original space is linear in the kernel space. Training scales badly beyond tens of thousands of rows.
13 How do you handle missing values?
easyFirst understand why values are missing: random, or related to the value itself. Options are dropping rows (only when few), simple imputation with the median or mode, model-based imputation such as kNN or iterative imputation, and adding a missing-indicator column when missingness carries signal. Tree-based libraries such as XGBoost and LightGBM handle missing values natively. Always fit the imputer on training data only, inside the pipeline.
14 How would you tune hyperparameters efficiently?
mediumStart with a sensible default and a small search space on the most influential parameters, for example learning rate, depth and number of trees for boosting. Random search usually beats grid search for the same budget because only a few parameters really matter. Use cross-validation inside the search, and keep a final test set untouched. Bayesian optimisation or successive halving help when each fit is expensive.
pythonfrom sklearn.model_selection import RandomizedSearchCV from sklearn.ensemble import GradientBoostingClassifier search = RandomizedSearchCV( GradientBoostingClassifier(random_state=0), {'learning_rate': [0.01, 0.05, 0.1], 'max_depth': [2, 3, 4], 'n_estimators': [100, 300]}, n_iter=10, cv=5, scoring='roc_auc', random_state=0).fit(X_train, y_train)
Frequently asked questions
How much maths do machine learning interviews need?
Linear algebra (matrix products, eigenvectors), probability (Bayes rule), and calculus for gradients. Be able to derive linear regression and one gradient descent step.
Which algorithms should I know best?
Linear and logistic regression, decision trees, random forest, gradient boosting, k-means, kNN, naive Bayes, SVM and PCA.
Are coding questions asked in machine learning interviews?
Yes, usually Python with numpy or pandas, and sometimes implementing k-means or logistic regression from scratch. See the numpy page.
How do I explain a model I used in a project?
Say what problem it solved, why you chose it over simpler options, how you validated it and what its limits were.
Related guides
More in Role-wise Interview Questions by Job and Tech Stack
- Manual Testing Interview Questions and Answers
- MERN Stack Interview Questions and Answers
- Next.js Interview Questions and Answers (Next.js 15)
- NLP Interview Questions and Answers
- Node.js Interview Questions and Answers (Node 22)
- NumPy Interview Questions and Answers
- Pandas Interview Questions and Answers
- Performance Testing and JMeter Interview Questions
- Power BI Interview Questions and Answers
- Probability for Data Science Interview Questions and Answers
- Product Analyst Interview Questions and Answers
- PySpark Interview Questions and Answers