Chapter 01

Medium

Random Walks and the Continuous-Time Limit

00 · Symbol Glossary

$S_n$S sub n — partial sum

$S_n = \xi_1+\xi_2+\cdots+\xi_n$ , the position of a discrete random walk after $n$ steps. Each $\xi_i$ is an i.i.d. step with $\mathbb{E}[\xi_i]=0$ , $\mathrm{Var}(\xi_i)=1$ .

$\Delta t$Delta t — time increment

The calendar length of one step once the walk is mapped onto a clock: $n$ steps cover $[0,t]$ when $\Delta t = t/n$ . Letting $\Delta t\to 0$ (equivalently $n\to\infty$ ) is the continuous-time limit this chapter builds.

$X_t^{(n)}$X sub t superscript n — rescaled walk

The random walk after it has been given a clock and a step size: $X_t^{(n)} = \sigma\sqrt{\Delta t}\,S_{\lfloor t/\Delta t\rfloor}$ . The superscript tracks the number of steps used to approximate $[0,t]$ ; the object of interest is what happens as $n\to\infty$ .

$\sigma^2$sigma squared — variance rate

The variance the walk accumulates per unit of calendar time. Chosen so that $\mathrm{Var}(X_t^{(n)})=\sigma^2 t$ for every $n$ , holding the limit stationary as the number of steps grows.

$W_t$W sub t — Wiener process (forward reference)

The continuous-time limit object this chapter is building toward. Defined precisely with its full axioms in Chapter 02; here it appears only as "whatever $X_t^{(n)}$ converges to."

01 · The Discrete Random Walk, Revisited

Every path in this subject starts from the same object as Time Series: a sum of i.i.d. shocks. The difference is what you do with it next.

Definition — Simple Random Walk

Let $\xi_1,\xi_2,\ldots$ be i.i.d. with $\mathbb{E}[\xi_i]=0$ and $\mathrm{Var}(\xi_i)=1$ . The random walk is the partial-sum process

S_0 = 0, \qquad S_n = \xi_1+\xi_2+\cdots+\xi_n = S_{n-1}+\xi_n

By independence and linearity, $\mathbb{E}[S_n]=0$ and $\mathrm{Var}(S_n)=n$ : variance grows linearly in the number of steps, not the step size.

Same object, different subject

This is exactly $P_t = P_{t-1}+\varepsilon_t$ from ts-01 with $P_0=0$ and $\varepsilon_t=\xi_t$ mean-zero white noise — Time Series' opening random walk. That chapter keeps $t$ on the calendar and studies one finite path with estimators. This chapter takes the same object and asks what it becomes as the step size shrinks to zero and the number of steps grows without bound.

Example — Symmetric coin-flip walk

Take $\xi_i=+1$ or $-1$ with probability $\tfrac12$ each. Then $S_{100}$ has mean $0$ and variance $100$ , so a typical displacement after $100$ steps is on the order of $\sqrt{100}=10$ — an order of magnitude smaller than the $100$ steps taken. That $\sqrt{n}$ scaling of typical displacement, not $n$ , is the single fact this chapter is built around.

02 · Scaling as $\Delta t \to 0$

To turn a step-counted walk into a process on a clock, fix a horizon $t$ , split $[0,t]$ into $n$ steps of length $\Delta t = t/n$ , and ask what step size keeps the walk from degenerating as $n\to\infty$ .

Definition — Rescaled Random Walk

For a fixed variance rate $\sigma^2$ , define

X_t^{(n)} = \sigma\sqrt{\Delta t}\;S_{\lfloor t/\Delta t\rfloor}, \qquad \Delta t = t/n

Since $S_{\lfloor t/\Delta t\rfloor}$ has variance $\lfloor t/\Delta t\rfloor \approx t/\Delta t$ , the rescaled process has

\mathrm{Var}\big(X_t^{(n)}\big) = \sigma^2\Delta t \cdot \frac{t}{\Delta t} = \sigma^2 t

for every $n$ . The $\sqrt{\Delta t}$ step size is exactly the one that makes variance depend on calendar time $t$ alone, not on how finely the interval is chopped up.

Plain language: shrink each step, but shrink it by the square root of the time it represents, not by the time itself. That mismatch between "time shrinks like $\Delta t$ " and "typical step shrinks like $\sqrt{\Delta t}$ " is the entire reason continuous-time paths look the way they do — rough, not smooth.

❌ Scaling steps the wrong way

Suppose instead each step has fixed size $\pm 1$ regardless of $\Delta t$ (no rescaling), or size $\pm\Delta t$ (linear scaling).

Why it breaks: with fixed step size $\pm 1$ , $\mathrm{Var}(S_{t/\Delta t}) = t/\Delta t \to \infty$ as $\Delta t\to 0$ — the walk explodes. With linear step size $\pm\Delta t$ , $\mathrm{Var} = (\Delta t)^2\cdot(t/\Delta t) = t\Delta t \to 0$ — the walk collapses to the deterministic constant $0$ .

Consequence: only the $\sqrt{\Delta t}$ scaling produces a nondegenerate random process with finite, positive variance in the limit. This is not a modeling choice; it is the unique scaling for which a continuous-time limit exists at all.

03 · The Invariance Principle, Informally

The coin-flip walk of Section 01 is one choice of step distribution among many. The remarkable fact is that the limit does not care which one you picked.

Definition — Invariance Principle (Informal)

Let $\xi_1,\xi_2,\ldots$ be any i.i.d. sequence with $\mathbb{E}[\xi_i]=0$ and $\mathrm{Var}(\xi_i)=1$ — not necessarily $\pm 1$ coin flips. As $\Delta t\to 0$ , the rescaled walk $X_t^{(n)}$ converges in distribution (as a whole path, not just at one time $t$ ) to the same limit process, a process with variance rate $\sigma^2$ . The limiting law does not depend on the shape of the step distribution — only on its mean $0$ and variance $1$ .

This is Donsker's theorem, and it is a functional version of the ordinary Central Limit Theorem: the CLT says one rescaled sum $S_n/\sqrt{n}$ converges to $N(0,1)$ ; the invariance principle says the entire path of partial sums, rescaled, converges to a single universal continuous-time process.

Example — Two different coins, one limit

Walk A uses $\xi_i=\pm1$ with probability $\tfrac12$ each. Walk B uses $\xi_i$ uniform on $[-\sqrt3,\sqrt3]$ (chosen so $\mathrm{Var}(\xi_i)=1$ matches Walk A). The two walks have completely different step distributions, but both rescaled walks $X_t^{(n)}$ converge to the same limit process as $\Delta t\to 0$ . Only the first two moments of the step survive to the limit.

04 · What Carries Over — and What Does Not

Some properties of the discrete walk pass to the limit unchanged; one does not, and that exception drives the rest of this subject.

Properties that carry over: increments over disjoint time intervals stay independent; increments over intervals of equal length stay identically distributed; variance still grows linearly in elapsed time, $\mathrm{Var}(X_t-X_s)=\sigma^2(t-s)$ ; and the process stays Markov — the distribution of the future given the whole past depends only on the current value, exactly as $S_{n+1}$ depends on $S_n,\ldots,S_1$ only through $S_n$ .

One property does not carry over cleanly: the discrete walk's path, plotted step to step, is a sequence of straight line segments — differentiable almost everywhere it is defined. The limit process is continuous, but Chapter 02 shows it is nowhere differentiable. Smoothing $\Delta t$ down to $0$ does not make the path smoother; it makes it rougher, because the number of "kinks" per unit time grows without bound even as each individual kink shrinks.

05 · Why Stochastic Calculus Exists

Refresher — Expectation, Variance, i.i.d.

For a random variable $X$ , $\mathbb{E}[X]$ is its probability-weighted average and $\mathrm{Var}(X)=\mathbb{E}[(X-\mathbb{E}[X])^2]$ measures spread around that average. A sequence is i.i.d. (independent and identically distributed) if every term has the same distribution and knowing any subset of the terms gives no information about the others. These three notions are the only probability machinery this chapter needs; a full treatment belongs to a dedicated probability course, not repeated here.

Ordinary calculus — derivatives, the chain rule, Riemann integrals — is built for functions whose graphs are smooth enough to have well-defined slopes at (almost) every point. Section 04 already flags the problem: the continuous-time limit of a random walk is not that kind of function. Its increments have variance $\sigma^2 h$ over a time step $h$ , so a typical increment has size $\sqrt{h}$ , not $h$ — and $\sqrt{h}/h = 1/\sqrt h \to \infty$ as $h\to 0$ . A slope computed as (change in value)/(change in time) diverges instead of settling down.

That single scaling fact — increments of order $\sqrt{h}$ rather than $h$ — is why stochastic calculus needs its own integral (the Itô integral, Chapter 05), its own chain rule (Itô's lemma, Chapter 06), and its own notion of a differential equation (an SDE, Chapter 07). None of those tools are optional refinements; they exist because the object built in this chapter genuinely breaks the tools from ordinary calculus.

06 · Exercises

EXERCISE 1.1

Compute $\mathrm{Var}(S_n)$ directly from independence, then substitute $n=t/\Delta t$ before rescaling.

$\mathrm{Var}(S_n)=n$ by independence of the $\xi_i$ . With $n=t/\Delta t$ steps and rescaled process $X_t^{(n)}=\sigma\sqrt{\Delta t}\,S_n$ , $\mathrm{Var}(X_t^{(n)}) = \sigma^2\Delta t\cdot n = \sigma^2\Delta t\cdot(t/\Delta t)=\sigma^2 t$ . The $\Delta t$ in the scaling factor and the $1/\Delta t$ in the step count cancel exactly, leaving a variance that depends only on $t$ .

For $X_t^{(n)}=\sigma\sqrt{\Delta t}\,S_{\lfloor t/\Delta t\rfloor}$ with $\Delta t=t/n$ , show directly that $\mathrm{Var}(X_t^{(n)})=\sigma^2 t$ for every $n$ , and identify which two factors cancel to make the result independent of $n$ .

EXERCISE 1.2

Try step size $\Delta t^{1/4}$ and recompute the variance of $X_t^{(n)}$ as $\Delta t\to 0$ .

With step size $\Delta t^{1/4}$ , $\mathrm{Var}(X_t^{(n)}) = (\Delta t^{1/4})^2\cdot(t/\Delta t) = \Delta t^{1/2}\cdot t \to 0$ as $\Delta t\to 0$ . The walk collapses to $0$ , the same failure mode as the linear scaling in Section 02's FailBlock, just slower. Any exponent on $\Delta t$ strictly greater than $\tfrac12$ collapses the limit; any exponent strictly less than $\tfrac12$ explodes it. Exponent $\tfrac12$ is the unique value that survives.

Suppose the step size is $\Delta t^{1/4}$ instead of $\sqrt{\Delta t}=\Delta t^{1/2}$ . Show what happens to $\mathrm{Var}(X_t^{(n)})$ as $\Delta t\to 0$ , and state why this confirms $\tfrac12$ is not an arbitrary choice of exponent.

EXERCISE 1.3

The Markov property of $S_n$ follows from the fact that $S_{n+1}-S_n=\xi_{n+1}$ is independent of $S_1,\ldots,S_n$ .

$S_{n+1}=S_n+\xi_{n+1}$ with $\xi_{n+1}$ independent of $(S_1,\ldots,S_n)$ , so the conditional distribution of $S_{n+1}$ given the whole history $(S_1,\ldots,S_n)$ equals the conditional distribution given $S_n$ alone — the extra history carries no information once $S_n$ is known. The same argument applies to any process built as (current value) + (independent increment), which is exactly how $X_t^{(n)}$ and its limit are constructed. Hence the Markov property of the discrete walk is inherited by the limit for the same structural reason, not by coincidence.

Explain, using the definition of the random walk (not the limiting process), why $S_n$ is Markov. Then explain why the same argument suggests the continuous-time limit should also be Markov.

07 · Chapter Summary

Concept	Meaning
$S_n$	discrete random walk, $\mathrm{Var}(S_n)=n$
$\sqrt{\Delta t}$ scaling	unique step-size scaling with a nondegenerate limit
Invariance principle	limit law depends only on step mean/variance, not shape
Carries over	independent increments, linear variance growth, Markov property
Does not carry over	differentiable paths — the limit is rough
Why new calculus	increments of order $\sqrt{h}$ break the ordinary derivative

Next: Chapter 02 — Brownian Motion: Definition and Path Properties, which states the limiting process from this chapter as a precise set of axioms and studies its paths directly.