Chapter 02

Medium

Stationarity, ACF, and PACF

00 · Symbol Glossary

$\gamma(h)$gamma of h — lag-h autocovariance

For a weakly stationary series, $\gamma(h)=\mathrm{Cov}(X_t,X_{t-h})$ , independent of $t$ . Even function: $\gamma(-h)=\gamma(h)$ . $\gamma(0)=\mathrm{Var}(X_t)$ .

$\rho(h)$rho of h — autocorrelation

$\rho(h)=\gamma(h)/\gamma(0)$ . Scale-free measure of linear dependence at lag $h$ . $\rho(0)=1$ and $\lvert\rho(h)\rvert\leq 1$ .

$\phi_{hh}$phi sub hh — PACF at lag h

The partial autocorrelation: correlation of $X_t$ and $X_{t-h}$ after linearly removing lags $1,\ldots,h-1$ . The AR order diagnostic in Chapter 03.

$\hat\rho(h)$sample ACF

$\hat\rho(h) = \hat\gamma(h)/\hat\gamma(0)$ with $\hat\gamma(h)=T^{-1}\sum_{t=h+1}^{T}(x_t-\bar x)(x_{t-h}-\bar x)$ . The $T^{-1}$ divisor (not $T-h$ ) is the usual convention; it keeps the sample covariance matrix positive semidefinite.

01 · Strict and Weak Stationarity

If the mechanism generating $X_t$ does not change date, then shifting the window should not change the joint law.

Definition — Strict Stationarity

$\{X_t\}$ is strictly stationary if for every $k$ , every $t$ , and every lag $h$ ,

(X_t,X_{t+1},\ldots,X_{t+k}) \;\stackrel{d}{=}\; (X_{t+h},X_{t+1+h},\ldots,X_{t+k+h})

The joint distribution of any block depends only on the spacing of the dates, not on their location on the calendar.

Definition — Weak (Second-Order, Covariance) Stationarity

$\{X_t\}$ is weakly stationary if $\mathbb{E}[X_t^2]<\infty$ , $\mathbb{E}[X_t]=\mu$ (constant), and $\mathrm{Cov}(X_t,X_{t-h})=\gamma(h)$ depends only on $h$ .

Strict stationarity plus finite variance implies weak stationarity. The converse is false: weak stationarity constrains only first and second moments. Gaussian processes are the convenient exception — a Gaussian process is strictly stationary iff it is weakly stationary (finite-dimensional laws are determined by mean and covariance).

Why weak stationarity is the working definition

ACF, ARMA, and spectral densities are second-order objects. They require $\gamma(h)$ , not the whole law. GARCH (Chapter 06) can be strictly stationary while failing weak stationarity if $\alpha+\beta$ is too large — a reminder that the two notions part company for nonlinear models.

❌ Confusing a visual trend with nonstationarity of returns

A price chart that “goes up” is not evidence that returns are nonstationary.

Why it breaks: a random walk in $P_t$ has trending levels and (under constant volatility) stationary differences. Stationarity is a statement about the object you put into $\gamma(h)$ .

Consequence: difference or detrend first, then test the transformed series (Chapter 04), rather than declaring “the market is nonstationary” from a price plot.

02 · Autocovariance and Autocorrelation

Definition — ACF

For weakly stationary $\{X_t\}$ with $\gamma(0)>0$ ,

\rho(h) = \frac{\gamma(h)}{\gamma(0)}, \qquad h\in\mathbb{Z}

Properties: $\rho(0)=1$ , $\rho(-h)=\rho(h)$ , $\lvert\rho(h)\rvert\leq 1$ . The sequence $\{\gamma(h)\}$ is positive semidefinite: for all $n$ and all $a_1,\ldots,a_n$ , $\sum_{i,j}a_i a_j\gamma(i-j)\geq 0$ . That is exactly $\mathrm{Var}(\sum_i a_i X_i)\geq 0$ .

Example — MA(1) autocovariance

Let $X_t=\varepsilon_t+\theta\varepsilon_{t-1}$ with $\varepsilon_t\sim\mathrm{WN}(0,\sigma^2)$ . Then $\gamma(0)=\sigma^2(1+\theta^2)$ , $\gamma(1)=\theta\sigma^2$ , and $\gamma(h)=0$ for $\lvert h\rvert\geq 2$ . So $\rho(1)=\theta/(1+\theta^2)$ and $\rho(h)=0$ for $\lvert h\rvert\geq 2$ : the ACF of an MA(1) cuts off after lag 1.

03 · Partial Autocorrelation

The ACF of an AR(1) decays geometrically, so many lags look “significant.” The PACF isolates the extra linear contribution of lag $h$ .

Definition — PACF

Let $\phi_{h1},\ldots,\phi_{hh}$ be the coefficients in the $h$ -th linear projection

X_t = \phi_{h1} X_{t-1}+\cdots+\phi_{hh} X_{t-h}+\eta_t

chosen to minimize $\mathbb{E}[\eta_t^2]$ (population OLS on the stationary process). The PACF at lag $h$ is $\phi_{hh}$ . Equivalently, $\phi_{hh}=\mathrm{Corr}(X_t^\star, X_{t-h}^\star)$ where $^\star$ denotes residual after projecting on $X_{t-1},\ldots,X_{t-h+1}$ . In practice, $\phi_{hh}$ is the last coefficient in the lag- $h$ Yule–Walker / OLS projection on $X_{t-1},\ldots,X_{t-h}$ .

For a causal AR $(p)$ , $\phi_{hh}=0$ for all $h>p$ : the PACF cuts off after lag $p$ . That is the identification counterpart of the MA cutoff in the ACF (Chapter 03).

04 · Sample ACF and Bartlett Bands

Definition — Sample Autocovariance

\hat\gamma(h) = \frac{1}{T}\sum_{t=h+1}^{T}(x_t-\bar x)(x_{t-h}-\bar x), \qquad \hat\rho(h)=\frac{\hat\gamma(h)}{\hat\gamma(0)}

with $\bar x = T^{-1}\sum_{t=1}^{T}x_t$ . Using $T$ rather than $T-h$ is deliberate: the Toeplitz matrix of $\hat\gamma$ stays positive semidefinite.

Under iid $(0,\sigma^2)$ with finite fourth moment, $\sqrt{T}\hat\rho(h)\Rightarrow N(0,1)$ for each fixed $h\geq 1$ . The rule of thumb Bartlett band $\pm 1.96/\sqrt{T}$ is a 95% interval for a white-noise null. It is not valid under dependence: if the true process is AR(1) with $\rho(1)=0.8$ , the sampling variance of $\hat\rho(h)$ is larger, and “spikes” at large $h$ are not comparable to the iid bands.

Ergodicity, in one sentence

Weak stationarity makes $\gamma(h)$ well-defined. Ergodicity makes $\hat\gamma(h)\to\gamma(h)$ in probability as $T\to\infty$ . Mixing conditions (dependence dying out) are the usual sufficient story. Without something like that, a single path need not reveal $\gamma$ .

❌ Reading every spike outside ±1.96/√T as ‘significant AR structure’

With $T=100$ , twenty plotted lags, and a true WN series, several $\hat\rho(h)$ will wander outside the bands by chance (multiple testing).

Why it breaks: each band is a marginal 5% test. Twenty lags are twenty tests.

Consequence: use the bands as a screen, then confirm with a portmanteau test (Ljung–Box, Chapter 05) and with PACF/ACF patterns (cutoff vs decay), not isolated spikes.

05 · Exercises

EXERCISE 2.1

Check mean and $\mathrm{Cov}(X_t,X_{t-h})$ for $X_t = a + bt +\varepsilon_t$ .

$\mathbb{E}[X_t]=a+bt$ depends on $t$ , so $\{X_t\}$ is not weakly stationary. The residual $\varepsilon_t$ may be; the observed series is not. Detrend first: $X_t-\hat a-\hat b t$ , then inspect ACF of residuals.

Let $X_t=5+0.2t+\varepsilon_t$ with $\varepsilon_t\sim\mathrm{WN}(0,1)$ . Is $\{X_t\}$ weakly stationary? What transformation would you apply before computing $\hat\rho(h)$ ?

EXERCISE 2.2

For MA(1), $\rho(1)=\theta/(1+\theta^2)$ . Maximize as a function of $\theta$ .

Let $f(\theta)=\theta/(1+\theta^2)$ . Then $f'(\theta)=(1-\theta^2)/(1+\theta^2)^2=0$ at $\theta=\pm 1$ , and $\lvert f(\pm 1)\rvert=1/2$ . So $\lvert\rho(1)\rvert\leq 1/2$ for every MA(1). A sample $\hat\rho(1)=0.7$ is incompatible with an invertible (or any) MA(1) as the population ACF — look at AR or ARMA instead.

Show that every MA(1) satisfies $\lvert\rho(1)\rvert\leq 1/2$ . Why does a sample ACF with $\hat\rho(1)=0.7$ argue against a pure MA(1)?

EXERCISE 2.3

AR(1) $X_t=\phi X_{t-1}+\varepsilon_t$ , $\lvert\phi\rvert<1$ , has $\rho(h)=\phi^{\lvert h\rvert}$ and PACF $\phi_{11}=\phi$ , $\phi_{hh}=0$ for $h>1$ .

ACF: geometric decay $\rho(h)=\phi^h$ for $h\geq 0$ (here $0.7^{h}$ ). PACF: spike at lag 1 equal to $0.7$ , then zeros. That pair is the textbook AR(1) signature, complementary to the MA(1) signature (ACF cutoff, PACF decay).

For causal AR(1) with $\phi=0.7$ , write $\rho(h)$ and describe the PACF pattern.

06 · Chapter Summary

Object	Role
Strict stationarity	Joint laws shift-invariant
Weak stationarity	Constant mean, $\gamma(h)$ only
ACF $\rho(h)$	Linear dependence at lag $h$
PACF $\phi_{hh}$	Extra linear dependence at lag $h$
Sample bands $\pm 1.96/\sqrt{T}$	WN null, not a general significance law
MA(1) ACF	Cuts off after lag 1; $\lvert\rho(1)\rvert\leq 1/2$
AR(1) ACF / PACF	Geometric decay / cutoff after lag 1

Next: Chapter 03 — Linear Processes and ARMA Models, which turn these patterns into parametric filters of white noise.