ARMA Models (Autoregressive Moving Average)

So far, we learned two different ways time series behave.

• AR models → depend on past values
• MA models → react to past shocks (errors)

Real-world data rarely follows just one of these behaviors. Most of the time, both memory and shocks exist together.

That is exactly why ARMA models exist.

What Is an ARMA Model?

ARMA combines:

• AR(p) → past values (memory)
• MA(q) → past errors (shocks)

In simple terms:

Today depends on yesterday AND how wrong we were yesterday.

Real-World Intuition

Think about daily product sales:

• Yesterday’s sales influence today (momentum)
• A surprise discount causes a sudden spike (shock)
• That spike affects the next day slightly

AR captures momentum. MA captures shock correction. ARMA captures both at the same time.

ARMA(p, q) Notation

ARMA models are written as:

ARMA(p, q)

• p → number of past values
• q → number of past errors

Example:

• ARMA(1,1) → one past value + one past error

Python Example: Simulating an ARMA(1,1) Series

Let’s generate an ARMA(1,1) time series to see how it behaves.

import numpy as np
import matplotlib.pyplot as plt

np.random.seed(2)
n = 100
phi = 0.6
theta = 0.7

noise = np.random.normal(0, 1, n)
series = np.zeros(n)

for t in range(1, n):
    series[t] = phi * series[t-1] + noise[t] + theta * noise[t-1]

plt.figure(figsize=(8,4))
plt.plot(series)
plt.title("ARMA(1,1) Time Series")
plt.xlabel("Time")
plt.ylabel("Value")
plt.show()

Here is the actual ARMA series plotted:

What you should notice:

• Smoother than pure MA
• More reactive than pure AR
• Shocks influence but do not dominate

Comparing AR, MA, and ARMA (Visually)

Let’s compare all three behaviors side by side.

Visual interpretation:

• AR → smooth, memory-driven
• MA → sharp spikes, short-lived
• ARMA → balanced behavior

When Should You Use ARMA?

ARMA models work well when:

• The series is stationary
• No strong trend or seasonality
• Both memory and noise matter

Typical use cases:

• Short-term demand forecasting
• Financial return modeling
• Signal processing

Important Limitation

ARMA cannot handle:

• Trends
• Seasonality

If your data has these, ARMA alone will fail.

This limitation leads directly to the next model.

Practice Questions

Big Picture

AR remembers the past. MA reacts to mistakes. ARMA does both.

But real-world data often trends over time.

That’s why the next model adds one more idea — differencing.