Chapter 04

Hard

Unit Roots, Differencing, and ARIMA

00 · Symbol Glossary

$\Delta$Delta — difference operator

$\Delta X_t = X_t-X_{t-1}=(1-B)X_t$ . $\Delta^d = (1-B)^d$ . The operator that can turn an $I(d)$ series into a weakly stationary one.

$I(d)$integrated of order d

$X_t$ is $I(0)$ if it is (already) weakly stationary with absolutely summable ACF, in the usual ARMA sense. $X_t$ is $I(d)$ if $\Delta^d X_t$ is $I(0)$ and $\Delta^{d-1}X_t$ is not.

$\mathrm{ARIMA}(p,d,q)$ARIMA

$\phi(B)\Delta^d (X_t-\mu_d)=\theta(B)\varepsilon_t$ with causal $\phi$ and invertible $\theta$ . The mean $\mu_d$ sits on the differenced series (often $0$ for prices after log-differencing).

$\mathrm{ADF}$Augmented Dickey–Fuller

A regression test of $H_0$ : unit root against $H_1$ : causal AR (possibly with intercept / trend). The $t$ -statistic has a nonstandard limit — not $N(0,1)$ .

01 · Random Walk and the Unit Root

Definition — Random Walk

$P_t=P_{t-1}+\varepsilon_t$ with $\varepsilon_t\sim\mathrm{WN}(0,\sigma^2)$ is a random walk. Equivalently $\phi(B)=1-B$ , a root at $z=1$ . Then $P_t=P_0+\sum_{j=1}^t\varepsilon_j$ , so $\mathrm{Var}(P_t)=t\sigma^2$ grows without bound: not weakly stationary.

A shock $\varepsilon_t$ is permanent in the level: it is still fully present in $P_{t+h}$ for every $h$ . In a causal AR(1) with $\lvert\phi\rvert<1$ , the same shock dies as $\phi^h$ . That is the economic content of “unit root vs mean reversion.”

Example — Log price as $I(1)$

If $\log P_t = \log P_{t-1}+\mu+\varepsilon_t$ , then $r_t=\Delta\log P_t$ is WN plus drift $\mu$ . The log price is $I(1)$ ; returns are $I(0)$ under this hypothesis. Equity indexes empirically sit near this description (with GARCH volatility, Chapter 06), which is why Chapter 01 insisted on differencing logs.

02 · ARIMA $(p,d,q)$

Definition — ARIMA

$\{X_t\}$ is ARIMA $(p,d,q)$ if $\Delta^d X_t$ (after removing a mean or deterministic trend as specified) is causal invertible ARMA $(p,q)$ :

\phi(B)(1-B)^d X_t = c + \theta(B)\varepsilon_t

Standard cases: $d=0$ (ARMA), $d=1$ (first difference), $d=2$ (rare in finance; sometimes used for linear-trend-plus-I(1), better modeled as trend + $I(1)$ ). Choose $d$ from unit-root tests and the economic object, not from eyeballing a single ACF plot. The constant $c$ lives on the differenced series — it is the mean of $\Delta^d X_t$ when you include an intercept, distinct from a deterministic trend in the original levels.

Deterministic trend vs unit root

$X_t=a+bt+\varepsilon_t$ is trend-stationary: subtract $a+bt$ and you have $I(0)$ . $X_t=X_{t-1}+\mu+\varepsilon_t$ is difference-stationary (stochastic trend). Both look “trending.” Wrong choice: differencing a trend-stationary series overdifferences (introduces a unit MA root); detrending a random walk leaves a unit root in residuals. ADF with the right deterministic terms is the diagnostic, not the plot.

❌ Differencing until the ACF ‘looks white’

Second-differencing a series that was already $I(0)$ produces an MA factor $(1-B)$ in the noise.

Why it breaks: $\Delta X_t=\varepsilon_t-\varepsilon_{t-1}$ when $X_t$ was already WN. ACF of $\Delta X_t$ has $\rho(1)=-1/2$ (MA(1) with $\theta=-1$ ), which is a unit-root MA — noninvertible.

Consequence: difference at most until ADF (or a prior economic reason) supports $I(0)$ . Do not treat extra differences as harmless.

03 · Dickey–Fuller Tests

The AR(1) OLS $t$ -test of $H_0:\phi=1$ is not $N(0,1)$ under the null. The regressor $X_{t-1}$ is $I(1)$ , so the usual Student- $t$ table rejects too often if you treat it as stationary-theory $t$ .

Definition — Dickey–Fuller Regression

The basic DF regression is

\Delta X_t = \alpha + \beta t + \pi X_{t-1} + e_t

(with $\alpha$ , $\beta$ included or excluded according to the maintained deterministic model). $H_0:\pi=0$ is a unit root; $H_1:\pi<0$ is causal AR. The ADF adds lags $\sum_{j=1}^k\gamma_j\Delta X_{t-j}$ so that $e_t$ is approximately WN when the short-run dynamics are AR.

Under $H_0$ the statistic converges not to a normal but to a ratio of functionals of Brownian motion — the Dickey–Fuller distribution. It is left-skewed and shifted well below zero, and it is a different distribution for each deterministic specification.

Asymptotic 5% critical values (left tail; reject $H_0$ when the statistic is more negative):

Deterministic terms in the regression	5% critical value
none ( $\alpha=\beta=0$ )	$-1.94$
intercept only ( $\beta=0$ )	$-2.86$
intercept and trend	$-3.41$

❌ Reading the no-deterministics value as ‘basically $-1.96$’

The no-constant case has a 5% value of about $-1.94$ , which sits almost on top of the normal two-sided $1.96$ .

Why it breaks: that is a numerical coincidence between two unrelated distributions, and it applies only to the regression with no intercept and no trend — the specification you almost never want for a price or a macro series. Adding an intercept moves the cutoff to $-2.86$ ; adding a trend moves it to $-3.41$ .

Consequence: always read the critical value off the row that matches the deterministic terms you actually estimated. Software reports MacKinnon $p$ -values that already account for this; use them rather than a remembered number.

Step-by-step — A responsible ADF

Choose the economic object: log price (likely

I(1)

) vs return (likely

I(0)

) vs inflation (could be either).

Choose deterministics: no constant (rare), constant (nonzero mean under

I(0)

), constant + trend (trend-stationarity alternative).

Choose lag $k$ : AIC/BIC on the ADF regression, or a conservative rule, so serial correlation in

e_t

does not pollute

\pi

Read $H_0$ correctly: failure to reject does not prove a unit root; it fails to reject one. With

T=80

the test is weak.

Complement: KPSS reverses the null (stationarity as

H_0

). Agreement (ADF non-reject + KPSS reject) is stronger than ADF alone.

KPSS runs the other way, in both senses

KPSS is an LM statistic built from partial sums of the residuals, and it is an upper-tail test: you reject stationarity when the statistic is large. That is the opposite tail from ADF, and mixing them up inverts your conclusion. Its 5% critical values are $0.463$ against level stationarity and $0.146$ against trend stationarity — again a different cutoff per deterministic case. The statistic also depends on the long-run variance estimator, so the bandwidth (Newey–West lag truncation) is a real choice, not a default to ignore: too small a bandwidth over-rejects.

04 · Seasonal Differences (SARIMA, briefly)

Monthly series often need $\Delta_{12}X_t=X_t-X_{t-12}$ in addition to $\Delta$ . When a fixed calendar period drives the level (monthly retail, quarterly GDP), apply the seasonal difference first — it removes the repeating seasonal level — then fit ARMA/ARIMA tools from Chapters 03–05 on the seasonally differenced series. The airline model $\Delta\Delta_{12}X_t=(1+\theta B)(1+\Theta B^{12})\varepsilon_t$ is the textbook SARIMA $(0,1,1)\times(0,1,1)_{12}$ pattern; this library does not develop the full seasonal taxonomy, but the seasonal difference is not optional when seasonality dominates the plot.

05 · Exercises

EXERCISE 4.1

Write $P_t$ as a sum of shocks. Compare $\mathrm{Var}(P_{t+h}-P_t)$ for the walk vs causal AR(1).

Random walk: $P_{t+h}-P_t=\sum_{j=1}^h\varepsilon_{t+j}$ , so $\mathrm{Var}(P_{t+h}-P_t)=h\sigma^2$ , linear in the horizon. Causal AR(1): $\mathrm{Var}(X_{t+h}-X_t)=2\gamma(0)(1-\phi^h)\to 2\gamma(0)$ , which stays bounded. Increment variance that grows like $h$ is the random-walk signature (and coincides with the $h$ -step forecast MSE from $P_t$ ).

Compare $\mathrm{Var}(P_{t+h}-P_t)$ as $h$ grows for (i) a random walk and (ii) a causal AR(1). What does this say about long-horizon forecast uncertainty?

EXERCISE 4.2

If $X_t$ is WN, compute $\rho(1)$ of $\Delta X_t$ .

$\Delta X_t=\varepsilon_t-\varepsilon_{t-1}$ is MA(1) with $\theta=-1$ , $\gamma(0)=2\sigma^2$ , $\gamma(1)=-\sigma^2$ , so $\rho(1)=-1/2$ . The MA root is on the unit circle (noninvertible). Overdifferencing manufactures a noninvertible MA.

Show that the first difference of white noise is MA(1) with $\rho(1)=-1/2$ . Why is that a warning against overdifferencing?

EXERCISE 4.3

The DF $t$ is not asymptotically $N(0,1)$ under $H_0$ .

No. Under a unit root the statistic has the Dickey–Fuller distribution, which depends on the deterministic terms. Both normal cutoffs — $-1.96$ two-sided, $-1.65$ one-sided — are less negative than the intercept-case value $-2.86$ . Concretely, $t=-2.1$ clears $-1.96$ but not $-2.86$ : the normal table says “reject the unit root, the series is stationary,” the correct table says “fail to reject.” The mistake runs one way, toward over-rejecting $H_0$ and claiming stationarity that the data do not support.

A software printout shows ADF $t=-2.1$ and a user compares it to $-1.96$ . Why is that comparison wrong, and in which direction is the mistake?

06 · Chapter Summary

Idea	Content
Unit root	AR polynomial has $z=1$ ; shocks persist in the level
$I(d)$	$d$ differences to $I(0)$
ARIMA $(p,d,q)$	ARMA after $\Delta^d$
Trend vs difference stationarity	Detrend vs $\Delta$ — not interchangeable
ADF	$t$ on $\pi$ ; DF distribution, $-2.86$ with intercept, $-3.41$ with trend
KPSS	null is stationarity; upper tail; $0.463$ level, $0.146$ trend
Overdifferencing	Extra $\Delta$ $\Rightarrow$ noninvertible MA

Next: Chapter 05 — Forecasting, Information Criteria, and Diagnostics, using a stationary (or already differenced) ARMA.