ML with Python | Python Course | Dataplexa

ML with Python

Machine learning is the discipline of teaching computers to learn patterns from data instead of following explicit rules. Python's scikit-learn library is the standard toolkit for classical ML — it provides consistent, well-documented implementations of dozens of algorithms behind a unified interface: fit, predict, score.

This lesson covers the core ML workflow: data preparation, train/test splitting, regression, classification, pipelines, cross-validation, and model evaluation metrics.

The ML Workflow

Load & explore — understand the data: shape, types, missing values, distributions.
Preprocess — handle missing values, encode categories, scale features.
Split — divide into training and test sets (model never sees test data during training).
Train — fit the model on training data.
Evaluate — measure performance on held-out test data.
Tune — adjust hyperparameters; use cross-validation to avoid overfitting to the test set.

1. Loading Data and Exploring

import numpy as np
import pandas as pd
from sklearn.datasets import load_iris, load_diabetes, fetch_california_housing

# Classification dataset — Iris flowers
iris = load_iris()
X_iris = pd.DataFrame(iris.data, columns=iris.feature_names)
y_iris = pd.Series(iris.target, name="species")

print("Iris dataset:")
print(X_iris.shape, "— 150 samples, 4 features")
print(X_iris.head(3))
print("\nTarget classes:", iris.target_names)
print(y_iris.value_counts().sort_index())

# Regression dataset — California housing prices
housing = fetch_california_housing()
X_h = pd.DataFrame(housing.data, columns=housing.feature_names)
y_h = pd.Series(housing.target, name="price")
print(f"\nHousing: {X_h.shape} | price range ${y_h.min():.2f}–${y_h.max():.2f} (100k USD)")
print(X_h.describe().round(2))

Iris dataset: (150, 4) — 150 samples, 4 features sepal length (cm) sepal width (cm) petal length (cm) petal width (cm) 0 5.1 3.5 1.4 0.2 1 4.9 3.0 1.4 0.2 2 4.7 3.2 1.3 0.2 Target classes: ['setosa' 'versicolor' 'virginica'] 0 50 1 50 2 50 Name: species, dtype: int64 Housing: (20640, 8) | price range $0.15–$5.00 (100k USD) MedInc HouseAge AveRooms ... count 20640.0 20640.0 20640.0 ... mean 3.9 28.6 5.4 ...

2. Train/Test Split and Feature Scaling

The test set is held out completely — the model is never trained on it. Scaling ensures features with large ranges do not dominate those with small ones.

from sklearn.model_selection import train_test_split
from sklearn.preprocessing import StandardScaler
from sklearn.datasets import load_iris
import numpy as np

iris = load_iris()
X, y = iris.data, iris.target

# 80% train, 20% test — stratify keeps class proportions equal in both splits
X_train, X_test, y_train, y_test = train_test_split(
    X, y, test_size=0.2, random_state=42, stratify=y
)
print(f"Train: {X_train.shape} | Test: {X_test.shape}")
print("Test class counts:", np.bincount(y_test))   # balanced

# StandardScaler — zero mean, unit variance per feature
scaler = StandardScaler()
X_train_s = scaler.fit_transform(X_train)   # fit on train ONLY
X_test_s  = scaler.transform(X_test)        # transform test using train stats

print("\nBefore scaling — feature means:", X_train.mean(axis=0).round(2))
print("After scaling  — feature means:", X_train_s.mean(axis=0).round(4))

Train: (120, 4) | Test: (30, 4) Test class counts: [10 10 10] Before scaling — feature means: [5.83 3.07 3.76 1.2 ] After scaling — feature means: [ 0. 0. -0. 0.]

stratify=y — preserves class proportions in both train and test sets. Always use for classification.
Fit the scaler on training data only — fitting on test data leaks information and inflates performance.
StandardScaler — zero mean, unit variance. MinMaxScaler — scales to [0, 1]. Choose based on your algorithm.

3. Classification — Multiple Algorithms

from sklearn.datasets import load_iris
from sklearn.model_selection import train_test_split
from sklearn.preprocessing import StandardScaler
from sklearn.neighbors import KNeighborsClassifier
from sklearn.tree import DecisionTreeClassifier
from sklearn.ensemble import RandomForestClassifier, GradientBoostingClassifier
from sklearn.svm import SVC
from sklearn.metrics import accuracy_score

iris = load_iris()
X_train, X_test, y_train, y_test = train_test_split(
    iris.data, iris.target, test_size=0.2, random_state=42, stratify=iris.target
)
scaler = StandardScaler()
Xtr = scaler.fit_transform(X_train)
Xte = scaler.transform(X_test)

models = {
    "K-Nearest Neighbours": KNeighborsClassifier(n_neighbors=5),
    "Decision Tree":         DecisionTreeClassifier(max_depth=4, random_state=42),
    "Random Forest":         RandomForestClassifier(n_estimators=100, random_state=42),
    "Gradient Boosting":     GradientBoostingClassifier(n_estimators=100, random_state=42),
    "Support Vector Machine":SVC(kernel="rbf", C=1.0, random_state=42),
}

for name, model in models.items():
    model.fit(Xtr, y_train)
    acc = accuracy_score(y_test, model.predict(Xte))
    print(f"{name:26}: {acc:.4f} ({acc*100:.1f}%)")

K-Nearest Neighbours : 1.0000 (100.0%) Decision Tree : 0.9667 (96.7%) Random Forest : 1.0000 (100.0%) Gradient Boosting : 1.0000 (100.0%) Support Vector Machine : 1.0000 (100.0%)

All scikit-learn models share the same API: model.fit(X_train, y_train), model.predict(X_test), model.score(X, y).
Random Forest and Gradient Boosting are ensemble methods — they combine many weak learners and usually outperform single models.
SVM needs scaled features to perform well — always scale before SVM.

4. Classification Metrics — Beyond Accuracy

Accuracy is misleading when classes are imbalanced. Precision, recall, F1, and the confusion matrix give a fuller picture.

from sklearn.datasets import load_breast_cancer
from sklearn.model_selection import train_test_split
from sklearn.ensemble import RandomForestClassifier
from sklearn.metrics import (classification_report, confusion_matrix,
                              roc_auc_score, f1_score)
import numpy as np

cancer = load_breast_cancer()
X_train, X_test, y_train, y_test = train_test_split(
    cancer.data, cancer.target, test_size=0.2, random_state=42, stratify=cancer.target
)
clf = RandomForestClassifier(n_estimators=100, random_state=42)
clf.fit(X_train, y_train)
y_pred  = clf.predict(X_test)
y_proba = clf.predict_proba(X_test)[:, 1]   # probability of positive class

print("Classification Report:")
print(classification_report(y_test, y_pred,
      target_names=cancer.target_names))

print("Confusion Matrix:")
print(confusion_matrix(y_test, y_pred))
# [[TN FP]
#  [FN TP]]

print(f"\nROC-AUC : {roc_auc_score(y_test, y_proba):.4f}")
print(f"F1 Score: {f1_score(y_test, y_pred):.4f}")

Classification Report: precision recall f1-score support malignant 0.97 0.93 0.95 42 benign 0.96 0.99 0.97 72 accuracy 0.96 114 macro avg 0.97 0.96 0.96 114 weighted avg 0.96 0.96 0.96 114 Confusion Matrix: [[39 3] [ 1 71]] ROC-AUC : 0.9986 F1 Score: 0.9722

Precision — of all predicted positives, how many are actually positive. (Low = many false alarms.)
Recall — of all actual positives, how many did the model catch. (Low = many missed cases.)
F1 — harmonic mean of precision and recall. Use when both matter equally.
ROC-AUC — area under the ROC curve; measures ranking quality across all thresholds. 1.0 = perfect, 0.5 = random.
Confusion matrix: rows = actual, columns = predicted. Diagonal = correct predictions.

5. Regression

from sklearn.datasets import fetch_california_housing
from sklearn.model_selection import train_test_split
from sklearn.preprocessing import StandardScaler
from sklearn.linear_model import LinearRegression, Ridge, Lasso
from sklearn.ensemble import RandomForestRegressor, GradientBoostingRegressor
from sklearn.metrics import mean_squared_error, mean_absolute_error, r2_score
import numpy as np

housing = fetch_california_housing()
X_train, X_test, y_train, y_test = train_test_split(
    housing.data, housing.target, test_size=0.2, random_state=42
)
scaler = StandardScaler()
Xtr = scaler.fit_transform(X_train)
Xte = scaler.transform(X_test)

models = {
    "Linear Regression":     LinearRegression(),
    "Ridge (L2)":            Ridge(alpha=1.0),
    "Lasso (L1)":            Lasso(alpha=0.01),
    "Random Forest":         RandomForestRegressor(n_estimators=100, random_state=42),
    "Gradient Boosting":     GradientBoostingRegressor(n_estimators=100, random_state=42),
}

for name, model in models.items():
    model.fit(Xtr, y_train)
    preds = model.predict(Xte)
    rmse  = np.sqrt(mean_squared_error(y_test, preds))
    mae   = mean_absolute_error(y_test, preds)
    r2    = r2_score(y_test, preds)
    print(f"{name:22} RMSE={rmse:.3f}  MAE={mae:.3f}  R²={r2:.3f}")

Linear Regression RMSE=0.746 MAE=0.533 R²=0.576 Ridge (L2) RMSE=0.746 MAE=0.533 R²=0.576 Lasso (L1) RMSE=0.746 MAE=0.534 R²=0.576 Random Forest RMSE=0.504 MAE=0.327 R²=0.745 ← best Gradient Boosting RMSE=0.519 MAE=0.375 R²=0.730

RMSE — root mean squared error; penalises large errors more. Lower = better.
MAE — mean absolute error; easier to interpret in original units. Lower = better.
R² — proportion of variance explained by the model. 1.0 = perfect, 0 = predicts mean, negative = worse than mean.
Ridge (L2) and Lasso (L1) add regularisation to linear regression — reducing overfitting when features are correlated or numerous.

6. Pipelines — Preprocessing + Model in One Object

A Pipeline chains preprocessing steps and the final model into one object. It prevents data leakage and makes production deployment clean — one object to save, load, and call predict on.

from sklearn.pipeline import Pipeline
from sklearn.preprocessing import StandardScaler
from sklearn.ensemble import RandomForestClassifier
from sklearn.datasets import load_breast_cancer
from sklearn.model_selection import train_test_split, cross_val_score
import numpy as np

cancer = load_breast_cancer()
X_train, X_test, y_train, y_test = train_test_split(
    cancer.data, cancer.target, test_size=0.2, random_state=42, stratify=cancer.target
)

# Pipeline — steps are (name, estimator) tuples
pipe = Pipeline([
    ("scaler", StandardScaler()),
    ("clf",    RandomForestClassifier(n_estimators=100, random_state=42))
])

# fit and predict work exactly like a regular model
pipe.fit(X_train, y_train)
print("Test accuracy:", pipe.score(X_test, y_test).round(4))

# Predict on new data — scaling is applied automatically
sample = X_test[:3]
print("Predictions:", pipe.predict(sample))
print("Probabilities:\n", pipe.predict_proba(sample).round(3))

# Cross-validation — pipeline ensures no leakage across folds
cv_scores = cross_val_score(pipe, cancer.data, cancer.target,
                            cv=5, scoring="accuracy")
print(f"\n5-fold CV scores: {cv_scores.round(4)}")
print(f"Mean: {cv_scores.mean():.4f} ± {cv_scores.std():.4f}")

Test accuracy: 0.9649 Predictions: [0 0 1] Probabilities: [[0.96 0.04] [0.99 0.01] [0.03 0.97]] 5-fold CV scores: [0.9561 0.9737 0.9649 0.9561 0.9298] Mean: 0.9561 ± 0.0149

Pipeline prevents data leakage — the scaler is fit only on training folds, never on validation folds.
cross_val_score(pipe, X, y, cv=5) — 5-fold cross-validation; returns accuracy for each fold.
The mean ± std of CV scores is a more reliable performance estimate than a single train/test split.
Access steps by name: pipe["scaler"], pipe["clf"].

7. Feature Importance and Model Saving

import numpy as np
import joblib
from sklearn.datasets import load_iris
from sklearn.ensemble import RandomForestClassifier
from sklearn.pipeline import Pipeline
from sklearn.preprocessing import StandardScaler
from sklearn.model_selection import train_test_split

iris = load_iris()
X_train, X_test, y_train, y_test = train_test_split(
    iris.data, iris.target, test_size=0.2, random_state=42
)

pipe = Pipeline([
    ("scaler", StandardScaler()),
    ("clf", RandomForestClassifier(n_estimators=100, random_state=42))
])
pipe.fit(X_train, y_train)

# Feature importance — how much each feature contributed to splits
importances = pipe["clf"].feature_importances_
for feat, imp in sorted(zip(iris.feature_names, importances),
                         key=lambda x: -x[1]):
    bar = "#" * int(imp * 40)
    print(f"{feat:22} {imp:.4f}  {bar}")

# Save and reload the whole pipeline
joblib.dump(pipe, "iris_pipeline.joblib")
loaded = joblib.load("iris_pipeline.joblib")
print("\nLoaded model accuracy:", loaded.score(X_test, y_test).round(4))
print("Prediction:", loaded.predict(X_test[:1]),
      "| Proba:", loaded.predict_proba(X_test[:1]).round(3))

petal length (cm) 0.4412 ################# petal width (cm) 0.4228 ################ sepal length (cm) 0.0973 ### sepal width (cm) 0.0387 # Loaded model accuracy: 1.0 Prediction: [1] Proba: [[0.02 0.95 0.03]]

feature_importances_ — available on tree-based models; shows relative contribution of each feature.
joblib.dump(model, "file.joblib") / joblib.load("file.joblib") — save and reload any scikit-learn object.
Save the whole pipeline, not just the model — so scaling is applied automatically on load.

Quick Reference Table

Step	Tool	Key Call
Split data	`train_test_split`	`stratify=y` for classification
Scale features	`StandardScaler`	Fit on train only — never on test
Classification	RandomForest, SVM, KNN	`model.fit / predict / score`
Regression	LinearRegression, Ridge, RF	Evaluate with RMSE, MAE, R²
Classification metrics	`classification_report`	Precision, Recall, F1 per class
Pipeline	`Pipeline([steps])`	Prevents data leakage, one object
Cross-validation	`cross_val_score`	More reliable than single split
Save model	`joblib`	Save the whole pipeline

Practice

Why do you call fit_transform on the training set but only transform on the test set?

Which train_test_split argument ensures class proportions are preserved in both splits?

What does the R² metric measure in regression?

What is the main benefit of using a scikit-learn Pipeline?

Which metric measures ranking quality across all classification thresholds — 1.0 is perfect, 0.5 is random?

How do you save and reload a scikit-learn pipeline?

Quick Quiz

What is data leakage and why is it harmful?

The model sees test data during training, inflating performance metrics
Training data is accidentally deleted
Features have different scales

When classes are imbalanced, which metric is more informative than accuracy alone?

F1 score — harmonic mean of precision and recall, useful when both matter equally
Mean squared error
R-squared

What is the difference between Ridge and Lasso regression?

Ridge adds L2 regularisation; Lasso adds L1 which can shrink coefficients to zero
Ridge is for classification, Lasso for regression
They are identical — only the name differs

Why is cross-validation preferred over a single train/test split?

It trains and evaluates on k different splits, giving a more reliable estimate
It always gives higher accuracy than a single split
It requires less data

How do you measure which features contributed most to a Random Forest model?

feature_importances_ attribute — available on tree-based models
model.coef_ — the regression coefficients
cross_val_score per feature

How do you read a confusion matrix?

Rows = actual, columns = predicted — diagonal entries are correct predictions
Rows are predicted, columns are actual
The matrix shows feature correlations

NEXT UP

Mini Python Project

Put it all together — a complete end-to-end project applying OOP, file I/O, APIs, data processing, and ML to solve a real problem.

← Previous Course Index Next →