Chapter 01

Easy

Discrete Time and White Noise

00 · Symbol Glossary

$\{X_t\}$X sub t — a time series

A sequence of random variables indexed by time $t = 1,2,3,\ldots$ (or $t\in\mathbb{Z}$ ). One observed path $x_1,x_2,\ldots,x_T$ is a realization; the object being modeled is the whole process.

$\mu_t$mu sub t — mean function

$\mu_t = \mathbb{E}[X_t]$ , the expected value at date $t$ . If this depends on $t$ , the series has a time-varying level (trend, break, or seasonality).

$\gamma(t,s)$gamma — autocovariance

$\gamma(t,s) = \mathrm{Cov}(X_t,X_s) = \mathbb{E}[(X_t-\mu_t)(X_s-\mu_s)]$ . Dependence between two dates, not just contemporaneous variance.

$\varepsilon_t$epsilon sub t — innovation / shock

The unpredictable piece at time $t$ . In this chapter, white noise is the baseline shock process; later chapters build ARMA and GARCH on top of it.

$r_t$r sub t — log return

$r_t = \log P_t - \log P_{t-1}$ for a price $P_t$ . The usual modeling object in finance — closer to stationary than the raw price.

01 · A Time Series Is a Stochastic Process

A time series is not a list of numbers. It is a stochastic process $\{X_t\}_{t\in\mathbb{T}}$ : for each date $t$ there is a random variable $X_t$ , and the family is jointly distributed. What you download from a market data vendor is one sample path.

Definition — Discrete-Time Process

A discrete-time stochastic process is a collection $\{X_t : t\in\mathbb{Z}\}$ (or $t=1,2,\ldots$ ) of random variables on a common probability space. A realization of length $T$ is a sequence $x_1,\ldots,x_T$ produced by one draw of that collection.

The index $t$ is ordered. That is the whole point: $X_{t}$ can depend on $X_{t-1}$ in a way $X_{t+5}$ cannot reverse. Cross-sectional data (many firms at one date) does not have that arrow of time.

One path is not the process

Estimators in later chapters pretend that time averages along the observed path recover ensemble averages $\mathbb{E}[X_t]$ . That leap is ergodicity (Chapter 02). Until then, keep the distinction: formulas are about the process; data are one path.

02 · Mean, Variance, and Autocovariance

Two dates can be dependent even if each margin is ordinary.

Definition — Second-Order Structure

For a process with $\mathbb{E}[X_t^2]<\infty$ ,

\mu_t = \mathbb{E}[X_t], \qquad \gamma(t,s) = \mathrm{Cov}(X_t,X_s)

The variance function is $\sigma_t^2 = \gamma(t,t)$ . If $\gamma(t,s)$ depends only on the lag $h=t-s$ , write $\gamma(h)$ — that is the stationary case of Chapter 02.

Example — Independent versus merely uncorrelated

Let $Z_t$ be i.i.d. $N(0,1)$ and set $X_t = Z_t Z_{t-1}$ . Then $\mathbb{E}[X_t]=0$ and $\mathrm{Cov}(X_t,X_{t-h})=0$ for every $h\neq 0$ , so the second-order structure is that of pure noise. But $X_t$ and $X_{t-1}$ both contain $Z_{t-1}$ , so they are not independent: $\mathbb{E}[X_t^2 X_{t-1}^2]\neq\mathbb{E}[X_t^2]\mathbb{E}[X_{t-1}^2]$ . Zero autocovariance is a statement about linear dependence only.

❌ Treating a price as if it were a return

Daily close $P_t$ of an equity index wanders. Sample variance of $P_t$ grows with the sample window; $\mathrm{Cov}(P_t,P_{t-h})$ stays large at long lags.

Why it breaks: integrated levels (Chapter 04) are the wrong object for second-order tools that assume a constant variance. Log returns $r_t=\log(P_t/P_{t-1})$ are the standard transform precisely so that $\mu_t$ and $\gamma$ can be approximately stable.

Consequence: ACF plots of raw prices look “highly persistent” even when returns are nearly white. Always state whether the series is a level or a difference.

03 · White Noise, iid, and Gaussian White Noise

Definition — White Noise

A process $\{\varepsilon_t\}$ is white noise, written $\varepsilon_t\sim\mathrm{WN}(0,\sigma^2)$ , if

\mathbb{E}[\varepsilon_t]=0, \qquad \mathrm{Var}(\varepsilon_t)=\sigma^2<\infty, \qquad \mathrm{Cov}(\varepsilon_t,\varepsilon_{t-h})=0 \text{ for all } h\neq 0

It is iid noise if the $\varepsilon_t$ are independent and identically distributed. It is Gaussian white noise if additionally $\varepsilon_t\sim N(0,\sigma^2)$ iid.

White noise is uncorrelated, not necessarily independent. GARCH (Chapter 06) builds a process whose levels are white noise (or martingale differences) while squares are strongly dependent. That distinction is why the WN definition uses covariances, not independence.

Martingale difference

In finance one often weakens WN to $\mathbb{E}[\varepsilon_t\mid \varepsilon_{t-1},\varepsilon_{t-2},\ldots]=0$ . Then $\varepsilon_t$ is a martingale difference: unpredictable in mean given the past, which is the efficient-market statement for a return innovation. Uncorrelatedness follows; identical variance need not.

04 · Building Blocks: Trend, Season, Cycle, Irregular

Observed series are often written as a sum of components, even before a formal model.

Definition — Additive Decomposition

X_t = T_t + S_t + C_t + I_t

$T_t$ is a slowly moving trend, $S_t$ a seasonal pattern with known period (e.g. $12$ for months), $C_t$ a cycle of unspecified period (business-cycle band, not the fixed calendar season), and $I_t$ an irregular remainder. Multiplicative form $X_t = T_t S_t C_t I_t$ is used when seasonal swings scale with the level — take logs first, then add in log space.

Step-by-step — Why log prices, then returns

Level:

P_t

is typically close to a random walk (Chapter 04). Variance of

P_t

is not stable.

Log:

p_t=\log P_t

turns multiplicative shocks into additive ones:

P_t = P_{t-1}e^{r_t}

becomes

p_t = p_{t-1}+r_t

Difference once:

r_t=\Delta p_t

is the object whose mean and autocovariance later chapters treat as (approximately) time-invariant. One difference after the log is the usual finance transform; a second difference is for confirmed

I(2)

levels, not a default step.

Do not difference twice by habit. Overdifferencing a series that was already stationary inflates MA structure (Chapter 04).

05 · What This Subject Is For

Time series in this library is the toolkit for dependent observations in calendar time: forecasting a yield, measuring persistence of inflation, modeling clustered volatility, testing whether two prices share an equilibrium, filtering a latent beta.

It is not a substitute for probability (distributions, likelihood) or stochastic calculus (continuous-time limits). Those subjects supply the measure-theoretic and continuous-time language; here the time index is discrete and the emphasis is second-order structure plus a handful of nonlinear models (GARCH, state-space) that markets actually use.

Where this sits next to Stochastic Calculus

Both subjects open with a random walk, and they mean the same discrete object $P_t=P_{t-1}+\varepsilon_t$ . The paths diverge immediately after. Stochastic Calculus rescales the walk and takes a limit, arriving at Brownian motion, Itô integrals, and SDEs in continuous time. Time Series keeps $t$ on the calendar and asks what you can estimate from one finite path: autocovariances, ARMA coefficients, unit-root tests. If the question involves $dW_t$ , it belongs to the other subject; if it involves $\hat\rho(h)$ from $T$ observations, it belongs here.

Practice — Classify the object

For each series, say whether you would model the level, a log, or a difference, and why: (a) daily VIX close; (b) quarterly real GDP; (c) a default count per month; (d) the EURUSD spot rate.

06 · Exercises

EXERCISE 1.1

White noise requires uncorrelatedness at every nonzero lag, not identical distribution of the observed path.

No. WN $(0,\sigma^2)$ requires $\mathrm{Cov}(\varepsilon_t,\varepsilon_{t-h})=0$ for $h\neq 0$ . A deterministic sine wave is perfectly forecastable from its past, so covariances at the seasonal lag are not zero. A sinusoidal mean is a time-varying $\mu_t$ , which already violates a constant-mean WN definition.

Is a deterministic sine wave $x_t=\sin(2\pi t/12)$ white noise? Explain using the definition, not a plot.

EXERCISE 1.2

Write $\mathrm{Cov}(P_t,P_{t-1})$ for $P_t=P_{t-1}+\varepsilon_t$ with $\varepsilon_t$ WN and $P_0$ fixed.

$P_t=P_0+\sum_{j=1}^t\varepsilon_j$ , so $\mathrm{Var}(P_t)=t\sigma^2$ and $\mathrm{Cov}(P_t,P_{t-1})=(t-1)\sigma^2$ . Both grow with $t$ . The pair $(P_t,P_{t-1})$ is not second-order stationary. The difference $\Delta P_t=\varepsilon_t$ is WN.

Let $P_t=P_{t-1}+\varepsilon_t$ with $\varepsilon_t\sim\mathrm{WN}(0,\sigma^2)$ and $P_0=0$ . Compute $\mathrm{Var}(P_t)$ and $\mathrm{Cov}(P_t,P_{t-1})$ . What does this say about modeling $P_t$ with tools that assume constant variance?

EXERCISE 1.3

Independence implies uncorrelatedness if second moments exist; the converse needs extra structure (e.g. joint Gaussianity).

iid $\Rightarrow$ independent $\Rightarrow$ uncorrelated (given finite variance), so iid noise with mean $0$ is WN. Conversely, WN only forces uncorrelatedness. The product $X_t=Z_tZ_{t-1}$ of Section 02 is one counterexample. The finance counterexample is GARCH (Chapter 06): $\varepsilon_t=\sigma_t z_t$ has $\mathrm{Cov}(\varepsilon_t,\varepsilon_{t-h})=0$ , so the levels are WN, while $\varepsilon_t^2$ is strongly autocorrelated because $\sigma_t^2$ depends on $\varepsilon_{t-1}^2$ . Uncorrelated returns and predictable volatility coexist.

Show that iid $(0,\sigma^2)$ implies WN $(0,\sigma^2)$ . Give a process that is WN but not iid.

07 · Chapter Summary

Concept	Meaning
Process vs path	$\{X_t\}$ is random at every $t$ ; data are one realization
Mean function	$\mu_t=\mathbb{E}[X_t]$
Autocovariance	$\gamma(t,s)=\mathrm{Cov}(X_t,X_s)$
White noise	mean $0$ , finite variance, uncorrelated across lags
iid / Gaussian WN	stronger: independence, and normality if specified
Finance default	model $r_t=\Delta\log P_t$ , not raw $P_t$

Next: Chapter 02 — Stationarity, ACF, and PACF, which turns $\gamma(t,s)$ into a function of lag alone and makes sample autocorrelations meaningful.