Autoregressive (AR) Models

Up to now, we have focused on understanding time series behavior — trend, seasonality, noise, smoothing.

From this lesson onward, we start doing something powerful:

Using the past to mathematically predict the future.

Autoregressive models are the first true forecasting models in time series.

What Does “Autoregressive” Mean?

The word auto means “self” and regressive means “predicting”.

An Autoregressive (AR) model predicts a value using its own past values.

In simple words:

Today depends on yesterday.

Real-World Intuition

Autoregression appears everywhere in real life:

• Yesterday’s stock price influences today’s price
• Yesterday’s temperature affects today’s temperature
• Yesterday’s website traffic impacts today’s traffic

If yesterday was unusually high, today often stays relatively high. That dependence is exactly what AR models capture.

AR(1): The Simplest Autoregressive Model

AR models are written as AR(p), where:

• p = number of past values used

AR(1) uses just one past value:

Today = constant + (coefficient × yesterday) + noise

Python Example: Generating an AR(1) Series

We’ll first create an artificial AR(1) time series to clearly understand the behavior.

import numpy as np
import matplotlib.pyplot as plt

np.random.seed(0)
n = 100
phi = 0.7
noise = np.random.normal(0, 1, n)

series = [0]
for t in range(1, n):
    series.append(phi * series[t-1] + noise[t])

plt.figure(figsize=(8,4))
plt.plot(series)
plt.title("Autoregressive Series AR(1)")
plt.xlabel("Time")
plt.ylabel("Value")
plt.show()

Below is the actual AR(1) series visualized:

What to observe:

• Values don’t jump randomly
• Each point depends on the previous one
• Series shows smooth continuity

Understanding the Coefficient (φ)

The coefficient φ controls how strong the memory is.

• φ close to 0 → weak memory
• φ close to 1 → strong memory

Let’s visually compare two AR(1) series with different φ values.

Interpretation:

• Higher φ → smoother, more persistent behavior
• Lower φ → more randomness

Stationarity and AR Models

This is extremely important:

AR models require the series to be stationary.

• Mean must be constant
• Variance must be constant
• No long-term trend

If the series is not stationary, AR models will fail. That’s why Lessons 4–6 were critical.

AR Models in the Real World

AR models are used when:

• Short-term forecasting is needed
• Recent past strongly influences future
• No strong seasonality exists

Examples:

• Short-term demand forecasting
• Sensor signal prediction
• Financial micro-movements

Common Mistakes Beginners Make

• Using AR on trending data
• Ignoring stationarity checks
• Using too many lags blindly

Always ask:

Does the past really influence the present?

Practice Questions

Big Picture

Autoregressive models formalize what we intuitively believe:

The recent past matters.

But they only work when the data respects certain rules.

Next Lesson

In the next lesson, we’ll study Moving Average (MA) Models — which model noise instead of past values.