Chapter 05

Medium

Forecasting, Information Criteria, and Diagnostics

00 · Symbol Glossary

$\hat X_{t+h\mid t}$h-step forecast

The minimum-MSE linear predictor of $X_{t+h}$ given $\{X_s:s\leq t\}$ , which for Gaussian ARMA equals $\mathbb{E}[X_{t+h}\mid X_t,X_{t-1},\ldots]$ .

$e_t(h)$h-step forecast error

$e_t(h)=X_{t+h}-\hat X_{t+h\mid t}$ . For causal ARMA, $e_t(1)=\varepsilon_{t+1}$ (the innovation) if the model is correctly specified and parameters are known.

$\mathrm{AIC},\;\mathrm{BIC}$information criteria

Penalized log-likelihood for lag order: AIC $= -2\ell/T+2k/T$ (common time-series scaling), BIC $= -2\ell/T+k\log(T)/T$ . Smaller is better. BIC charges more per parameter as $T$ grows.

$Q$Ljung–Box statistic

Portmanteau test using residual ACF $\hat\rho_\varepsilon(1),\ldots,\hat\rho_\varepsilon(H)$ jointly. Under WN residuals (and extra conditions), $Q$ is approximately $\chi^2$ with degrees of freedom $H-k$ after fitting $k$ ARMA parameters.

01 · Minimum MSE Forecasts

Definition — Linear MMSE Forecast

Among predictors of the form $c+\sum_{j=0}^{\infty} a_j X_{t-j}$ , the MMSE forecast $\hat X_{t+h\mid t}$ minimizes $\mathbb{E}[(X_{t+h}-\hat X_{t+h\mid t})^2]$ . For a causal ARMA with known coefficients it is obtained by writing $X_{t+h}=\sum_{j=0}^{\infty}\psi_j\varepsilon_{t+h-j}$ and replacing future $\varepsilon$ by $0$ and past $\varepsilon$ by the invertible filter of observed $X$ .

Plain language: split the future value into the part you can reconstruct from data through $t$ and the part that still depends on unknown future shocks. The optimal forecast keeps the first part and sets the innovation mean — zero — for the second.

For AR(1), $\hat X_{t+h\mid t}=\mu+\phi^h(X_t-\mu)$ . As $h\to\infty$ , the forecast converges to the mean $\mu$ and the error variance converges to $\gamma(0)$ : mean reversion. For a random walk, $\hat P_{t+h\mid t}=P_t$ (plus drift $h\mu$ if present) and error variance grows like $h\sigma^2$ (Chapter 04).

Example — AR(1) one- and two-step

$X_t=0.6 X_{t-1}+\varepsilon_t$ , $X_t=2$ . Then $\hat X_{t+1\mid t}=1.2$ , $\hat X_{t+2\mid t}=0.72$ . Known-parameter MSEs: $\sigma^2$ and $\sigma^2(1+\phi^2)=\sigma^2(1.36)$ .

Estimated parameters

Replacing $\phi$ by $\hat\phi$ adds parameter uncertainty. For large $T$ this is second order relative to innovation variance; for small $T$ and long $h$ , interval forecasts that ignore it are too tight. Analytic corrections exist; a practical substitute is to resample residuals (bootstrap the path).

02 · Choosing $(p,q)$ : AIC and BIC

ACF/PACF propose candidates. Criteria pick among fitted models on the same sample.

Definition — AIC and BIC (Gaussian ARMA)

Let $\ell$ be the maximized Gaussian log-likelihood and $k$ the number of estimated parameters (AR+MA+variance, and mean if fitted). Then

\mathrm{AIC} = -2\frac{\ell}{T}+\frac{2k}{T}, \qquad \mathrm{BIC} = -2\frac{\ell}{T}+\frac{k\log T}{T}

(Some packages report $T\cdot\mathrm{AIC}$ without dividing by $T$ ; only differences within one package are comparable.)

AIC is asymptotically efficient for forecasting when the true process is not a finite ARMA (it lets $k$ grow). BIC is consistent for $(p,q)$ when a true finite ARMA exists. In finance samples, AIC often selects more lags; BIC is the more conservative default unless the goal is explicitly short-horizon RMSE.

❌ Mixing AIC across software and across $T$

A model with AIC $=-1200$ is not “better” than one with AIC $=3$ from another package or a different sample length.

Why it breaks: additive constants, $T$ -scaling, and whether $\sigma^2$ is counted in $k$ differ.

Consequence: compare AIC only among models estimated the same way on the same $T$ . Prefer ranking, not the raw number.

03 · Residual Diagnostics

A fitted ARMA is credible only if $\hat\varepsilon_t$ looks like the WN it claims.

Definition — Ljung–Box $Q$

Q = T(T+2)\sum_{h=1}^{H}\frac{\hat\rho_{\hat\varepsilon}(h)^2}{T-h}

Under a correctly specified invertible ARMA and $H$ large but $H/T\to 0$ , $Q\approx\chi^2_{H-p-q}$ (mean subtracted separately). A small $p$ -value means residual autocorrelation remains — the ARMA is too small or misspecified (e.g. GARCH in squares, Chapter 06, which $Q$ on levels can miss).

Step-by-step — Residual checklist

Residual ACF/PACF: no systematic spikes; WN bands are now more appropriate than on the raw series.

Ljung–Box at

H=10,20

(or

H\approx\sqrt{T}

): joint test, not one lag.

Squares: ACF of

\hat\varepsilon_t^2

. Persistence here is volatility clustering, not a reason to inflate

(p,q)

of the mean.

Outliers / level shifts: one spike can fake MA structure. Inspect the residual time plot, not only

\hat\rho

Overfitting: extra MA/AR terms that cancel (common factors) improve in-sample

\ell

slightly and fail out of sample.

04 · Evaluation: In-Sample vs Out-of-Sample

Information criteria are in-sample. For forecasting products, hold out the last $R$ points, re-estimate or roll, and compare RMSE / MAE of $h$ -step errors. A model that wins AIC can lose a holdout if it overfit a break — for example extra AR lags that chase a pre-break sample, or a level shift the in-sample fit never saw. In production, rolling or expanding re-estimation on a trailing window is the safer default than one static fit scored only in sample.

Practice — Two losses

Explain why RMSE and MAE can rank two forecast methods differently when errors are heavy-tailed (as financial returns are). Which loss matches a quadratic utility story, and which is more robust to one crash day?

05 · Exercises

EXERCISE 5.1

Iterate the AR(1) $h$ times; take $\mathbb{E}[\,\cdot\mid \mathcal{F}_t]$ .

$X_{t+h}=\phi^h X_t + \sum_{j=0}^{h-1}\phi^j\varepsilon_{t+h-j}$ . Thus $\hat X_{t+h\mid t}=\phi^h X_t$ (zero mean). MSE $=\sigma^2(1-\phi^{2h})/(1-\phi^2)$ for $\lvert\phi\rvert<1$ , which increases with $h$ toward $\gamma(0)$ .

For causal AR(1) $X_t=\phi X_{t-1}+\varepsilon_t$ , derive $\hat X_{t+h\mid t}$ and $\mathrm{MSE}(h)$ when parameters are known.

EXERCISE 5.2

BIC penalty $k\log T$ vs AIC’s $2k$ .

For $T>e^2\approx 7.4$ , $\log T>2$ , so BIC penalizes each extra parameter more than AIC. BIC therefore tends to select smaller $(p,q)$ . That is intentional: BIC is built to find a true finite model; AIC is built to forecast well when the model is an approximation.

Why does BIC typically choose a smaller ARMA than AIC in the same sample when $T$ is large?

EXERCISE 5.3

Ljung–Box on levels vs on squares.

Yes. GARCH (Chapter 06) can leave $\hat\varepsilon_t$ uncorrelated (so $Q$ on levels is fine) while $\hat\varepsilon_t^2$ is strongly autocorrelated. Mean-equation diagnostics do not certify a constant-variance WN. Always inspect residual squares for financial returns.

Can Ljung–Box on ARMA residuals fail to reject, while the model is still a bad description of daily equity returns? What extra plot or test is required?

06 · Chapter Summary

Tool	Use
$\hat X_{t+h\mid t}$	MMSE forecast; AR(1) $\to$ mean as $h$ grows
Random-walk forecast	last observation (+ drift)
AIC	forecast-oriented order selection
BIC	consistent order if a true finite ARMA exists
Ljung–Box	joint residual ACF test; df $H-p-q$
Residual squares	GARCH, not extra ARMA lags

Next: Chapter 06 — Volatility Clustering and GARCH, when the mean is roughly WN but the variance is not.