Linear Regression

Linear Regression is one of the most fundamental and widely used algorithms in Machine Learning. It is often the first algorithm learned because it introduces how machines model relationships between variables.

In this lesson, you will understand what linear regression is, how it works, where it is used in real life, and how to implement a simple version using code.

What Is Linear Regression?

Linear Regression is a supervised learning algorithm used to predict a continuous numerical value based on one or more input features.

It works by finding a straight-line relationship between input variables and the output variable.

• Input variables are called features
• The output variable is a continuous value
• The relationship is modeled using a straight line

Real-World Connection

A very common real-world example is house price prediction.

Based on features like house size, number of rooms, and location, linear regression can estimate the price of a house.

Businesses also use linear regression for:

• Sales forecasting
• Revenue prediction
• Demand estimation

How Linear Regression Works

Linear regression tries to find the best-fitting line that minimizes the difference between predicted values and actual values.

The equation of a simple linear regression line is:

y = mx + b

• x = input feature
• y = predicted output
• m = slope (weight)
• b = intercept (bias)

The algorithm adjusts the slope and intercept to reduce prediction errors.

Training a Linear Regression Model

During training, the model:

• Makes predictions
• Calculates the error between prediction and actual value
• Updates parameters to reduce error

This process repeats until the error is minimized.

Simple Linear Regression Example

The example below demonstrates a basic linear regression idea using Python.

# Simple linear regression example
x = [1, 2, 3, 4, 5]
y = [2, 4, 6, 8, 10]

def predict(x_value):
    return 2 * x_value

for value in x:
    print("Input:", value, "Prediction:", predict(value))
  

In this example, the model learns a linear relationship where the output is always double the input.

Real linear regression algorithms automatically learn this relationship instead of hardcoding it.

Assumptions of Linear Regression

Linear regression works best when certain assumptions are met.

• Linear relationship between variables
• Independence of observations
• Constant variance of errors
• Minimal outliers

Violating these assumptions can reduce model accuracy.

Advantages of Linear Regression

Linear regression is popular because it is simple and interpretable.

• Easy to understand
• Fast to train
• Works well for many real-world problems

Limitations of Linear Regression

Despite its usefulness, linear regression has limitations.

• Cannot model complex nonlinear relationships
• Sensitive to outliers
• Assumes linearity

Practice Questions

Quick Quiz

Coming up next: Logistic Regression — using regression ideas for classification problems.