Time Series Cheat Sheet — ARIMA, LSTM, Stationarity, ACF, Forecasting | Dataplexa

Time Series Cheat Sheet

ARIMA · SARIMA · LSTM · stationarity · ACF/PACF · lag features · decomposition · forecasting

Sheet 3 of 4 Specialized ML Intermediate Printable

Time Series Fundamentals

Core Concepts

A time series is a sequence of observations indexed in chronological order. Understanding its components is the first step before any modelling.

Additive Decomposition

Y_t = T_t + S_t + R_t

T = Trend, S = Seasonality, R = Residual. Use additive when seasonal variation is constant over time.

Multiplicative Decomposition

Y_t = T_t × S_t × R_t

Use multiplicative when seasonal variation grows with the trend level (e.g. retail sales, stock prices). Log-transform converts multiplicative → additive.

Component	Description	Example
Trend	Long-run increase or decrease	Population growth
Seasonality	Fixed-period repeating pattern	Holiday sales spike
Cyclical	Irregular multi-year waves	Economic boom/bust
Residual	Random noise after decomposition	Weather anomalies
Irregular	One-off shocks	Pandemic, earthquake

STL Decomposition (statsmodels)

from statsmodels.tsa.seasonal import STL

stl = STL(series, period=12).fit()
stl.trend      # trend component
stl.seasonal   # seasonal component
stl.resid      # residuals
stl.plot()     # 4-panel decomposition

Term	Meaning
Lag k	Value at time t−k
Differencing	∇Y_t = Y_t − Y_t−1
Rolling mean	Moving average over window w
Autocorrelation	Corr(Y_t, Y_t−k)
White noise	iid, zero mean, constant var
Random walk	Y_t = Y_t−1 + ε_t

Always plot first: Before any modelling, visualise the raw series, rolling mean, and rolling std. Patterns (trend, seasonality, outliers) are usually visible to the eye before any test confirms them.

Stationarity

ADF · KPSS Tests

A stationary series has constant mean, variance, and autocovariance over time. Required by ARIMA and most classical models.

Strict Stationarity Conditions

E[Y_t] = μ · Var(Y_t) = σ² · Cov(Y_t, Y_t−k) = f(k)

Mean and variance are constant. Covariance depends only on lag k, not on time t. Also called "weak" or "covariance" stationarity.

ADF & KPSS Tests

from statsmodels.tsa.stattools import adfuller, kpss

# ADF: H0 = unit root (non-stationary)
adf, p, _, _, _, _ = adfuller(series)
print(f"ADF p={p:.4f}")
# p < 0.05 → reject H0 → stationary ✓

# KPSS: H0 = stationary
stat, p, _, _ = kpss(series, regression='c')
print(f"KPSS p={p:.4f}")
# p > 0.05 → fail to reject H0 → stationary ✓

# Make stationary: 1st difference
diff1 = series.diff().dropna()

ADF p-value	KPSS p-value	Conclusion
< 0.05	> 0.05	Stationary ✓
> 0.05	< 0.05	Non-stationary → difference
< 0.05	< 0.05	Trend-stationary
> 0.05	> 0.05	Ambiguous → inspect plot

ACF & PACF

Autocorrelation

Autocorrelation Function (ACF)

ACF(k) = Corr(Y_t, Y_t−k) = γ_k / γ₀

Total correlation between Y_t and Y_t−k including indirect paths via intermediate lags. Use to identify MA(q) order.

Partial Autocorrelation (PACF)

PACF(k) = Corr(Y_t, Y_t−k | Y_t−1,…,Y_t−k+1)

Direct correlation after removing effects of shorter lags. Use to identify AR(p) order.

Pattern	ACF	PACF	Model
AR(p)	Tails off slowly	Cuts off after lag p	ARIMA(p,d,0)
MA(q)	Cuts off after lag q	Tails off slowly	ARIMA(0,d,q)
ARMA(p,q)	Tails off	Tails off	ARIMA(p,d,q)
White noise	All within ±1.96/√n	All within bands	No model needed

Plot ACF & PACF

from statsmodels.graphics.tsaplots import (
  plot_acf, plot_pacf)
import matplotlib.pyplot as plt

fig, (ax1, ax2) = plt.subplots(2, 1)
plot_acf(series, lags=40, ax=ax1)
plot_pacf(series, lags=40, ax=ax2)
plt.tight_layout()

ARIMA & SARIMA

Classical Forecasting

ARIMA(p, d, q) Model

ΔᵈY_t = c + Σφ_iY_t−i + Σθ_jε_t−j + ε_t

p = AR order (PACF cuts off), d = differencing order (for stationarity), q = MA order (ACF cuts off). ε_t = white noise error.

SARIMA(p,d,q)(P,D,Q)[m]

ARIMA × Seasonal ARIMA at period m

P, D, Q = seasonal AR, differencing, MA orders. m = seasonal period (12=monthly, 4=quarterly, 7=daily-weekly). Handles recurring seasonal patterns.

Parameter	Meaning	How to choose
p	AR lags	PACF cutoff
d	Differences	ADF test (usually 0–2)
q	MA lags	ACF cutoff
P,D,Q	Seasonal orders	Seasonal ACF/PACF
m	Season period	Domain knowledge

ARIMA / SARIMA (statsmodels)

from statsmodels.tsa.arima.model import ARIMA
from statsmodels.tsa.statespace.sarimax import SARIMAX

# ARIMA(1,1,1)
model = ARIMA(train, order=(1,1,1))
fit = model.fit()
print(fit.summary())

# Forecast 12 steps ahead
fc = fit.forecast(steps=12)

# SARIMA(1,1,1)(1,1,1,12)
model = SARIMAX(train,
  order=(1,1,1),
  seasonal_order=(1,1,1,12))
fit = model.fit(disp=False)

# Auto-select orders: pmdarima
from pmdarima import auto_arima
model = auto_arima(train,
  seasonal=True, m=12,
  information_criterion='aic',
  stepwise=True)

Model Check	Good Sign
Residual ACF	All lags within ±1.96/√n bands
Ljung-Box test	p > 0.05 (no autocorrelation)
AIC/BIC	Lower is better across models
Residual dist.	Approx. normal, zero mean

Model Diagnostics

# Residual diagnostics
fit.plot_diagnostics()

# Ljung-Box test
from statsmodels.stats.diagnostic import acorr_ljungbox
lb = acorr_ljungbox(fit.resid, lags=[10])
# p > 0.05 → residuals are white noise ✓

AIC vs BIC: AIC favours more complex models (lower penalty). BIC penalises complexity more — better for short series. Use AIC for forecasting, BIC for model selection.

Lag Features & Feature Engineering

ML Approach

Convert a time series into a supervised ML problem by engineering features from past observations.

Creating Lag & Rolling Features (pandas)

import pandas as pd

df['lag_1']  = df['value'].shift(1)
df['lag_7']  = df['value'].shift(7)
df['lag_30'] = df['value'].shift(30)

# Rolling statistics
df['roll_mean_7'] = (
  df['value'].rolling(7).mean())
df['roll_std_7']  = (
  df['value'].rolling(7).std())
df['ewm_7'] = (
  df['value'].ewm(span=7).mean())

# Calendar features
df['hour']       = df.index.hour
df['dayofweek']  = df.index.dayofweek
df['month']      = df.index.month
df['is_weekend'] = df.index.dayofweek >= 5

Feature	Captures
`lag_k`	Direct autocorrelation at lag k
`rolling_mean`	Local trend / smoothed level
`rolling_std`	Local volatility / spread
`ewm`	Exponentially weighted trend
`diff`	Change from previous step
Calendar	Day/month/hour cyclical effects

LSTM for Time Series

Deep Learning

LSTM Gate Equations

f_t = σ(W_f·[h_t−1, x_t] + b_f) ← forget gate
i_t = σ(W_i·[h_t−1, x_t] + b_i) ← input gate
C_t = f_t⊙C_t−1 + i_t⊙tanh(W_c·[h_t−1,x_t]+b_c)
o_t = σ(W_o·[h_t−1, x_t] + b_o) ← output gate
h_t = o_t ⊙ tanh(C_t)

C_t = cell state (long-term memory). h_t = hidden state (short-term). ⊙ = element-wise multiply. Forget gate decides what to erase from memory.

LSTM Forecasting (PyTorch)

import torch.nn as nn

class LSTMForecaster(nn.Module):
  def __init__(self, n_features,
               hidden=64, n_layers=2):
    super().__init__()
    self.lstm = nn.LSTM(
      n_features, hidden,
      num_layers=n_layers,
      batch_first=True,
      dropout=0.2)
    self.fc = nn.Linear(hidden, 1)

  def forward(self, x):
    out, _ = self.lstm(x)
    return self.fc(out[:, -1, :])

Input shape: LSTM expects [batch, seq_len, features]. Use a sliding window to create sequences. Typical seq_len = 30–90 days for daily data.

Forecasting Models Comparison

Model Selection Guide

Model	Type	Handles Seasonality	Best For
Naïve	Baseline	Seasonal naïve	Benchmarking
Exponential Smoothing	Statistical	Holt-Winters (triple)	Smooth trends, fast
ARIMA	Statistical	With SARIMA	Stationary, univariate
Prophet	Decomposition	Multiple seasonalities	Business data, holidays
XGBoost	ML (tabular)	Via lag features	Many features, tabular
LSTM	Deep Learning	Learned implicitly	Long sequences, multivariate
Temporal Fusion Transformer	Deep Learning	Attention-based	Multi-horizon, interpretable
N-BEATS	Deep Learning	Explicit blocks	Pure TS, no covariates

Prophet — Quick Forecast

from prophet import Prophet

# df must have columns: ds, y
m = Prophet(
  yearly_seasonality=True,
  weekly_seasonality=True,
  daily_seasonality=False,
  changepoint_prior_scale=0.05
)
m.add_country_holidays(country_name='US')
m.fit(df)

future = m.make_future_dataframe(
  periods=365)
forecast = m.predict(future)
m.plot(forecast)
m.plot_components(forecast)

When to use what: Start with Naïve + ETS baselines. Try ARIMA/SARIMA for univariate with clear patterns. Use Prophet for business data with holidays. Use LSTM/TFT when you have multivariate inputs or very long sequences (>1000 steps).

Train / Test Split

Validation Strategy

Never shuffle time series data. Always respect temporal order — future data must not leak into training.

Correct Time Series Split

# Simple chronological split
split = int(len(df) * 0.8)
train, test = df[:split], df[split:]

# Walk-forward cross-validation
from sklearn.model_selection import TimeSeriesSplit
tscv = TimeSeriesSplit(n_splits=5)
for train_idx, test_idx in tscv.split(X):
  X_tr, X_te = X[train_idx], X[test_idx]
  # fit → predict → score each fold

Strategy	When	Notes
Hold-out	Quick baseline	Last 20% as test
Walk-forward	Robust evaluation	Expanding window
Sliding window	Fixed training size	Same window size each fold
Purged k-fold	Overlapping features	Gap between train/test folds

Data leakage: Computing rolling features on the full dataset before splitting leaks future information into training. Always compute rolling stats after the train/test split.

Forecasting Evaluation Metrics

MAE · RMSE · MAPE · SMAPE

Key Error Metrics

MAE = (1/n) Σ|y_t − ŷ_t|
RMSE = √[(1/n) Σ(y_t − ŷ_t)²]
MAPE = (100/n) Σ|y_t−ŷ_t| / |y_t|
SMAPE = (200/n) Σ|y_t−ŷ_t| / (|y_t|+|ŷ_t|)

Metric	Unit	Use When	Weakness
MAE	Same as y	Interpretable error	Doesn't penalise large errors
RMSE	Same as y	Large errors matter more	Sensitive to outliers
MAPE	%	Comparing series	Undefined when y=0
SMAPE	%	Symmetric % error	Can be misleading near 0
MASE	Relative	Scale-free comparison	Needs in-sample naïve MAE

Always compare to a naïve baseline: A naïve model (predict last value) or seasonal naïve (predict same period last year) sets the minimum bar. If your model doesn't beat naïve, it's not useful.

End-to-End Forecasting Pipeline

Workflow

Load & inspect — parse datetime index, check frequency, plot raw series, identify gaps and outliers
Clean — handle missing values (forward fill, interpolate, or model-based imputation), remove outliers
Decompose — STL or classical decomposition to separate trend, seasonality, residual
Stationarity test — ADF + KPSS tests. Difference until stationary (d=1 usually sufficient)
ACF/PACF analysis — identify AR(p) and MA(q) orders for ARIMA, or choose lag window for ML
Feature engineering — lag features, rolling stats, calendar variables, Fourier terms for seasonality
Train / validate — walk-forward cross-validation, never shuffle, watch for leakage
Model & tune — fit ARIMA/Prophet/LSTM, tune hyperparameters, compare against naïve baseline
Evaluate — report MAE, RMSE, MAPE on hold-out test set; plot forecast vs actuals with CI
Monitor — track residuals and retrain when distribution shifts (concept drift)

XGBoost on Lag Features

import xgboost as xgb
from sklearn.metrics import mean_absolute_error

# Build lag feature matrix
lags = [1, 7, 14, 30]
for lag in lags:
  df[f'lag_{lag}'] = df['y'].shift(lag)
df = df.dropna()

features = [f'lag_{l}' for l in lags]
X, y = df[features], df['y']
split = int(0.8 * len(df))
X_tr, X_te = X[:split], X[split:]
y_tr, y_te = y[:split], y[split:]

model = xgb.XGBRegressor(
  n_estimators=500, learning_rate=0.05,
  max_depth=5, subsample=0.8)
model.fit(X_tr, y_tr,
  eval_set=[(X_te, y_te)],
  early_stopping_rounds=50,
  verbose=False)
mae = mean_absolute_error(
  y_te, model.predict(X_te))

Library	Best For
`statsmodels`	ARIMA, SARIMA, ETS
`pmdarima`	auto_arima, order selection
`prophet`	Business forecasting, holidays
`sktime`	Unified sklearn-style TS API
`darts`	LSTM, TFT, N-BEATS, ensembles
`neuralforecast`	NHITS, PatchTST, TimesNet

Time Series Mastery Checklist

Self-Assessment

Foundations & Stationarity

Decompose a series into trend, seasonality, and residual using STL
Distinguish additive vs multiplicative decomposition
Run ADF and KPSS tests and interpret combined results
Apply first or seasonal differencing to achieve stationarity
Read an ACF plot to identify MA order
Read a PACF plot to identify AR order

ARIMA & Feature Engineering

Fit an ARIMA(p,d,q) and interpret the summary table
Extend to SARIMA with seasonal orders and period m
Use auto_arima to select orders automatically
Diagnose residuals with Ljung-Box test and ACF plot
Engineer lag, rolling mean, rolling std, and EWM features
Add calendar features (day of week, month, is_weekend)

ML Forecasting & Evaluation

Implement walk-forward cross-validation without data leakage
Build and train an LSTM forecaster with sliding window input
Set up a Prophet model with seasonality and holidays
Train XGBoost on lag features for time series regression
Calculate and compare MAE, RMSE, MAPE against naïve baseline
Choose the right library: statsmodels vs Prophet vs darts

Next up in Specialized ML → Sheet 4 covers Reinforcement Learning: reward functions, policies, Q-learning, Deep Q-Networks, policy gradients, and PPO.

4 · Reinforcement Learning Cheat Sheet →

← Back