Logistic Regression

Logistic Regression is a supervised machine learning algorithm used for classification problems. Even though the name contains the word “regression”, it is actually used to predict categories rather than continuous values.

In this lesson, you will learn how logistic regression works, why it is different from linear regression, where it is used in real life, and how to implement it using code.

Why Do We Need Logistic Regression?

Linear regression predicts continuous values such as price, temperature, or sales. But many real-world problems require classification instead of prediction.

Examples include:

• Email spam detection (spam or not spam)
• Loan approval (approved or rejected)
• Disease prediction (positive or negative)
• User churn (will leave or stay)

Logistic regression is designed specifically for these yes/no or binary outcomes.

What Is Logistic Regression?

Logistic regression predicts the probability that an input belongs to a particular class. The output is always between 0 and 1.

Instead of predicting a value directly, it predicts the likelihood of an event occurring.

• Output close to 0 → negative class
• Output close to 1 → positive class

Real-World Connection

Consider an email spam filter. The model analyzes features such as keywords, sender reputation, and message length.

Logistic regression calculates the probability that the email is spam. If the probability crosses a threshold (for example, 0.5), the email is marked as spam.

How Logistic Regression Works

Logistic regression uses the same linear equation as linear regression but applies a special function called the sigmoid function.

The sigmoid function converts any value into a number between 0 and 1.

Sigmoid function:

σ(x) = 1 / (1 + e⁻ˣ)

This transformation allows the model to interpret outputs as probabilities.

Decision Boundary Concept

Logistic regression separates classes using a decision boundary. Points on one side of the boundary belong to class 0, and points on the other side belong to class 1.

The decision boundary is learned automatically from data.

Simple Logistic Regression Example

The following example demonstrates how logistic regression predicts probabilities.

import math

def sigmoid(x):
    return 1 / (1 + math.exp(-x))

values = [-4, -2, 0, 2, 4]

for v in values:
    print("Input:", v, "Probability:", round(sigmoid(v), 3))
  

This output shows how logistic regression converts values into probabilities. Larger values push the probability closer to 1, while smaller values push it closer to 0.

How Predictions Are Made

Once the probability is calculated, a threshold is applied.

• If probability ≥ 0.5 → Class 1
• If probability < 0.5 → Class 0

This threshold can be adjusted depending on the problem.

Advantages of Logistic Regression

• Simple and efficient
• Easy to interpret
• Works well for binary classification

Limitations of Logistic Regression

• Cannot model complex nonlinear relationships
• Requires feature engineering
• Performance drops with overlapping classes

Practice Questions

Quick Quiz

Coming up next: Support Vector Machines (SVM) — powerful classifiers with strong mathematical foundations.