Chapter 10

Rigorous

State-Space Models and the Kalman Filter

00 · Symbol Glossary

$\alpha_t$alpha sub t — state vector

The latent Markov state at date $t$ . What you care about but do not observe directly: a local level, a time-varying beta, an ARMA companion vector.

$a_{t\mid t-1},\ P_{t\mid t-1}$predicted state and its covariance

$a_{t\mid t-1}=\mathbb{E}[\alpha_t\mid Y_{1:t-1}]$ and $P_{t\mid t-1}=\mathrm{Var}(\alpha_t\mid Y_{1:t-1})$ , the Kalman prediction. After the observation $y_t$ arrives they become the updated $a_t$ , $P_t$ .

$v_t,\ F_t$innovation and its variance

$v_t=y_t-Z a_{t\mid t-1}$ is the one-step prediction error of the observation; $F_t=\mathrm{Var}(v_t\mid Y_{1:t-1})$ . The Gaussian likelihood is built from $(v_t,F_t)$ .

$K_t$K sub t — Kalman gain

$K_t=P_{t\mid t-1}Z^\top F_t^{-1}$ . How much to move the predicted state toward the new observation. Small $H$ (precise data) or large $P$ (uncertain state) means a large gain.

$A,\ Z,\ Q,\ H$the system matrices

$A$ moves the state forward, $Z$ maps state to observation, $Q=\mathrm{Var}(\eta_t)$ is state noise, $H=\mathrm{Var}(\varepsilon_t)$ is observation noise. Many texts call the transition matrix $T$ ; this library reserves $T$ for the sample length, as in every other chapter.

01 · Why State Space

ARMA, GARCH, and VARs write the observable as a finite lag of itself. Many finance objects are latent: a slowly moving mean, a time-varying regression coefficient, a factor that several yields load on. A state-space model splits the problem in two: a Markov equation for the hidden state, and an observation equation that sees a linear image of that state plus noise.

Every causal ARMA has a state-space form (companion state). The converse is broader: the observation matrix $Z$ can be time-varying (a regressor $x_t$ ), and the state can be a random walk. The Kalman filter is the recursive MMSE estimator of $\alpha_t$ given data through $t$ , when the system is linear and the noises are uncorrelated through time. Under Gaussian noises it is also the conditional mean, and it delivers the Gaussian likelihood as a byproduct.

02 · The Linear Gaussian System

Definition — Time-Invariant State Space

\alpha_t = A\alpha_{t-1} + R\eta_t, \qquad y_t = Z\alpha_t + \varepsilon_t

with $\eta_t\sim\mathrm{WN}(0,Q)$ , $\varepsilon_t\sim\mathrm{WN}(0,H)$ , and $\eta$ , $\varepsilon$ uncorrelated with each other and with $\alpha_0$ . Here $y_t$ may be a vector; the scalar case is the one to compute by hand. Time-varying $Z_t$ , $A_t$ , $H_t$ are allowed — the filter formulas below still hold with $t$ subscripts.

Which timing convention this is

Two conventions circulate and they are not interchangeable. Here the state is dated contemporaneously: $\alpha_t$ is driven by $\eta_t$ , and $y_t$ observes that same $\alpha_t$ (Harvey’s form). Durbin and Koopman instead write $\alpha_{t+1}=A\alpha_t+R\eta_t$ , so their state equation looks one period ahead and their “predicted state” is indexed differently. The filters agree on the same model; the subscripts do not. When you copy a recursion out of a book, copy its timing convention with it, or the prediction step will be off by one period.

Example — Local level (random walk plus noise)

\mu_t=\mu_{t-1}+\eta_t, \qquad y_t=\mu_t+\varepsilon_t

So $\alpha_t=\mu_t$ , $A=1$ , $Z=1$ , $R=1$ . The series $y_t$ is $I(1)$ plus measurement noise — a useful model for a slowly moving mean (or a noisy integrated price). Signal-to-noise is $Q/H$ : large $Q$ makes the extracted level jumpy; $Q=0$ recovers a constant mean.

Example — Time-varying beta

\beta_t=\beta_{t-1}+\eta_t, \qquad r_t=x_t\beta_t+\varepsilon_t

Now $Z_t=x_t$ (the observable factor or market return). The Kalman filter tracks $\beta_t$ without a rolling-window OLS. This is the workhorse “dynamic CAPM / rolling beta” story, stated as a model rather than as an ad hoc window.

An AR $(1)$ $X_t=\phi X_{t-1}+\varepsilon_t$ is already in state-space form with $\alpha_t=X_t$ , $Z=1$ , $H=0$ (perfect observation of the state). An ARMA $(p,q)$ uses a companion state $\alpha_t$ of dimension $\max(p,q+1)$ that stacks the current value and its lags so the observation equation picks off the first component; $H=0$ then means the observed series is a noiseless linear readout of that state while the MA part lives in the state noise matrix $R$ .

03 · Predict and Update

Initialize with a prior $(a_0,P_0)$ (diffuse if you are agnostic). Then for $t=1,2,\ldots$ :

Step-by-step — Kalman filter (one pass)

Predict the state:

a_{t\mid t-1}=A a_{t-1}

P_{t\mid t-1}=A P_{t-1}A^\top + RQR^\top

Predict the observation:

\hat y_{t\mid t-1}=Z a_{t\mid t-1}

. Innovation

v_t=y_t-Z a_{t\mid t-1}

, with

F_t=ZP_{t\mid t-1}Z^\top+H

Gain:

K_t=P_{t\mid t-1}Z^\top F_t^{-1}

Update:

a_t=a_{t\mid t-1}+K_t v_t

P_t=P_{t\mid t-1}-K_t F_t K_t^\top

The gain $K_t$ is large when the state is uncertain relative to the observation noise. If $H=0$ and $Z$ is invertible, the update pins $\alpha_t$ to the observation. If $Q=0$ and $A=I$ , the filter is essentially recursive least squares on a constant state.

The update is often written $P_t=(I-K_tZ)P_{t\mid t-1}$ . That is the same line: substituting $K_t=P_{t\mid t-1}Z^\top F_t^{-1}$ gives $K_tF_tK_t^\top=K_tZP_{t\mid t-1}$ , so the two forms agree.

Under Gaussian noises, the log-likelihood (up to a constant, for scalar $y$ ) is

\ell = -\frac{1}{2}\sum_{t=1}^{T}\left(\log F_t + \frac{v_t^2}{F_t}\right)

where $T$ here is the sample length. Maximum likelihood of $(A,Z,Q,H)$ is then numerical optimization wrapped around the filter. That is why software estimates a local-level or a time-varying-parameter regression by “running Kalman.”

Smoothing, not derived

The filter uses data through $t$ . Smoothing runs a backward pass to get $a_{t\mid T}=\mathbb{E}[\alpha_t\mid Y_{1:T}]$ — the right object for a historical extracted trend. The formulas (RTS / Durbin–Koopman disturbance smoothing) are a sequel. Use a smoother for ex-post charts; use the filter for real-time tracking and for the likelihood.

❌ Treating the Kalman path as ‘the true beta’

A plotted $a_t$ is an MMSE estimate under this $Q$ , $H$ , and linear Gaussian model.

Why it breaks: if $\beta_t$ jumps, or $\varepsilon_t$ is heavy-tailed, or $Z_t$ is misspecified, $a_t$ is a compromise, not an oracle. $Q$ in particular is weakly identified in short samples and controls how much the path wiggles.

Consequence: report $Q$ (or a signal-to-noise ratio), show $P_{t\mid t}$ bands, and do not confuse a smooth extracted path with a discovered law of motion.

04 · What We Skip

Information-form and square-root Kalman filters (numerical stability when $P$ is ill-conditioned). Nonlinear filters (EKF, unscented, particle) for models that are not linear in $\alpha_t$ — including stochastic volatility with $\log\sigma_t^2$ as state and a nonlinear observation. Those need a stronger computational / filtering course. The linear Gaussian filter above is the one that sits under ARMA likelihoods, local-level trends, and time-varying betas.

05 · Exercises

EXERCISE 10.1

Match $A,Z,R,Q,H$ to the local-level equations.

$\alpha_t=\mu_t$ , $A=1$ , $Z=1$ , $R=1$ , $Q=\mathrm{Var}(\eta_t)$ , $H=\mathrm{Var}(\varepsilon_t)$ . The prediction is $a_{t\mid t-1}=a_{t-1}$ , $P_{t\mid t-1}=P_{t-1}+Q$ .

Write $A,Z,R,Q,H$ for the local-level model $y_t=\mu_t+\varepsilon_t$ , $\mu_t=\mu_{t-1}+\eta_t$ . What is $P_{t\mid t-1}$ in terms of $P_{t-1}$ and $Q$ ?

EXERCISE 10.2

$K_t=P_{t\mid t-1}Z^\top F_t^{-1}$ with $Z=1$ and $F_t=P_{t\mid t-1}+H$ .

$K_t=P_{t\mid t-1}/(P_{t\mid t-1}+H)$ . If $H\to 0$ , $K_t\to 1$ : trust the observation completely. If $P_{t\mid t-1}\to 0$ , $K_t\to 0$ : trust the predicted state; the observation is ignored.

In the scalar local-level filter, simplify $K_t$ . What happens as $H\to 0$ ? As $P_{t\mid t-1}\to 0$ ?

EXERCISE 10.3

The likelihood is a product of $N(0,F_t)$ densities of the innovations $v_t$ .

Yes — that is the prediction-error decomposition. Once $(v_t,F_t)$ are computed by the filter, Gaussian MLE is a numerical maximization of $\ell=-\frac12\sum(\log F_t+v_t^2/F_t)$ . No batch Kalman matrix of size $T$ is required.

True or false: the Kalman filter computes the Gaussian likelihood of a linear state-space model as it runs. Why does that matter for estimating $Q$ and $H$ ?

06 · Chapter Summary

Object	Role
State $\alpha_t$	latent Markov factor, $\alpha_t=A\alpha_{t-1}+R\eta_t$
Observation	$y_t=Z\alpha_t+\varepsilon_t$
Predict / update	MMSE recursion for $a_t$ , $P_t$
Gain $K_t$	trust data vs trust the model
$(v_t,F_t)$	innovations; Gaussian log-likelihood
Smoothing	$a_{t\mid T}$ , for historical extraction

This is the last Time Series chapter. The ten-chapter arc is: discrete time and white noise $\to$ stationarity $\to$ ARMA $\to$ unit roots $\to$ forecasting $\to$ GARCH $\to$ VAR $\to$ cointegration $\to$ spectra $\to$ state space. What was left out (long memory, Markov-switching, full Johansen derivations, particle filters) is a sequel, not a hole in the core.