Chapter 03

Medium

Conditional Probability & Bayes

00 · Symbol Glossary

$\mathbb{P}(A\mid B)$P of A given B — conditional probability

The probability of $A$ once you know $B$ occurred. Defined only when $\mathbb{P}(B)>0$ , by $\mathbb{P}(A\cap B)/\mathbb{P}(B)$ . Read the bar as "given", never as division.

$B_1,\ldots,B_n$B sub i — a partition

Events that are pairwise disjoint and together cover $\Omega$ : exactly one $B_i$ occurs. In finance the cells are usually scenarios or regimes (calm, normal, stress).

$\mathbb{P}(H)$P of H — prior

Probability of a hypothesis $H$ (a firm is fragile, the market is in stress) before looking at the evidence. In Bayes' theorem this is the base rate.

$\mathbb{P}(D\mid H)$P of D given H — likelihood

Probability of observing the data $D$ if the hypothesis $H$ is true. A test's hit rate and false-positive rate are both likelihoods.

$\mathbb{P}(H\mid D)$P of H given D — posterior

Probability of the hypothesis after seeing the data. Bayes' theorem converts prior and likelihood into posterior. Confusing $\mathbb{P}(H\mid D)$ with $\mathbb{P}(D\mid H)$ is the most expensive error in this chapter.

$h_k$h sub k — conditional default probability

$h_k=\mathbb{P}(\text{default during year }k\mid\text{alive at the start of year }k)$ . Each $h_k$ is a probability in a different conditional world. They are chained by the multiplication rule; they are not added.

$\Lambda$Likelihood ratio

For evidence $D$ and hypothesis $H$ , $\Lambda=\mathbb{P}(D\mid H)/\mathbb{P}(D\mid H^c)$ . Posterior odds equal prior odds times $\Lambda$ , so a second independent signal (Chapter 04) multiplies by a second ratio.

01 · Information Shrinks the Sample Space

Chapter 02 assigned every event a single number. In practice you rarely evaluate a probability in a vacuum: you already know something — yesterday's close, a rating action, the output of a risk model. Learning that an event $B$ occurred rules out every outcome outside $B$ . Probabilities of other events must be recomputed inside that smaller world.

On a finite space with the uniform measure the recomputation is just counting. Among the outcomes still possible (those in $B$ ), what fraction also lie in $A$ ?

\mathbb{P}(A\mid B)=\frac{|A\cap B|}{|B|}

Example — Three trading days, at least two up

Model three days of up/down moves as the $8$ equally likely strings in $\Omega=\{UUU,UUD,\ldots,DDD\}$ . Let $A$ be "day 1 is up", so $\mathbb{P}(A)=4/8=1/2$ .

Now you are told $B$ = "at least two of the three days were up". Then $B=\{UUU,UUD,UDU,DUU\}$ has $4$ outcomes, and $A\cap B=\{UUU,UUD,UDU\}$ has $3$ . Inside the reduced world,

\mathbb{P}(A\mid B)=\frac{3}{4}\neq\frac12=\mathbb{P}(A)

The information "mostly up" is evidence about each individual day, even though the days were generated with no link between them. Conditioning is about what you know, not about what caused what.

02 · Definition

Counting only works for uniform measures. The general definition replaces counts with probabilities.

Definition — Conditional Probability

For events $A,B$ with $\mathbb{P}(B)>0$ ,

\mathbb{P}(A\mid B) := \frac{\mathbb{P}(A\cap B)}{\mathbb{P}(B)}

If $\mathbb{P}(B)=0$ , the expression $\mathbb{P}(A\mid B)$ is left undefined.

Plain language: take the probability mass sitting on $B$ , keep the part that also sits on $A$ , and rescale so that $B$ itself has total mass $1$ .

A conditional probability is a probability

Fix $B$ with $\mathbb{P}(B)>0$ . The map $A\mapsto\mathbb{P}(A\mid B)$ satisfies every axiom of Chapter 02: it is non-negative, $\mathbb{P}(\Omega\mid B)=\mathbb{P}(B)/\mathbb{P}(B)=1$ , and it is additive over disjoint events because $(A_1\cup A_2)\cap B$ splits into the disjoint pieces $A_1\cap B$ and $A_2\cap B$ . So every rule you already know still holds after conditioning — for example $\mathbb{P}(A^c\mid B)=1-\mathbb{P}(A\mid B)$ . The two numbers $\mathbb{P}(A\mid B)$ and $\mathbb{P}(A\mid B^c)$ live in different conditional worlds, so they do not add to $1$ . Section 04 combines them correctly, as a weighted average with weights $\mathbb{P}(B)$ and $\mathbb{P}(B^c)$ .

❌ Conditioning on a scenario the model rules out

A historical-simulation VaR model is built from ten years of data in which a particular rate never went below zero. A risk manager asks for "the loss distribution given negative rates".

Why it breaks: under the empirical measure, the event $B$ = "rate below zero" has $\mathbb{P}(B)=0$ . The ratio $\mathbb{P}(A\cap B)/\mathbb{P}(B)$ is $0/0$ — not small, not large, undefined. No arithmetic on the existing model produces an answer.

Consequence: to condition on a scenario you must first build a model that gives it positive probability (a stressed scenario, an expanded sample, a parametric tail). A conditional number quoted for a zero-probability scenario was produced by something other than the model it claims to come from.

03 · The Multiplication Rule

Rearranging the definition gives the most-used identity in the chapter.

Definition — Multiplication Rule

If $\mathbb{P}(B)>0$ ,

\mathbb{P}(A\cap B)=\mathbb{P}(A\mid B)\,\mathbb{P}(B)

More generally, for events with $\mathbb{P}(A_1\cap\cdots\cap A_{n-1})>0$ , the chain rule is

\mathbb{P}(A_1\cap A_2\cap\cdots\cap A_n)=\mathbb{P}(A_1)\,\mathbb{P}(A_2\mid A_1)\,\mathbb{P}(A_3\mid A_1\cap A_2)\cdots\mathbb{P}(A_n\mid A_1\cap\cdots\cap A_{n-1})

The chain rule is how you compute the probability of a sequence of events: multiply the probability of the first step by the probability of each next step given everything before it.

Example — Picking defaulters without replacement

A portfolio holds $10$ bonds, of which exactly $3$ will default this year. An auditor samples $2$ bonds at random without replacement. Let $A_1$ = "first sampled bond defaults", $A_2$ = "second sampled bond defaults". Then $\mathbb{P}(A_1)=3/10$ , and given $A_1$ only $2$ defaulters remain among $9$ bonds:

\mathbb{P}(A_1\cap A_2)=\frac{3}{10}\cdot\frac{2}{9}=\frac{1}{15}\approx 0.067

Multiplying $3/10\cdot 3/10=0.09$ instead would ignore that the first draw changed what is left in the pool.

04 · The Law of Total Probability

Often $\mathbb{P}(A)$ is hard to compute directly but easy to compute scenario by scenario. Split $\Omega$ into scenarios, solve each one, and recombine.

Definition — Law of Total Probability

If $B_1,\ldots,B_n$ partition $\Omega$ and $\mathbb{P}(B_i)>0$ for each $i$ , then for any event $A$ ,

\mathbb{P}(A)=\sum_{i=1}^n \mathbb{P}(A\mid B_i)\,\mathbb{P}(B_i)

Proof in one line: $A$ is the disjoint union of the pieces $A\cap B_i$ , so $\mathbb{P}(A)=\sum_i\mathbb{P}(A\cap B_i)$ by additivity, and each piece equals $\mathbb{P}(A\mid B_i)\mathbb{P}(B_i)$ by the multiplication rule. The unconditional probability is a weighted average of conditional probabilities, with the scenario probabilities as weights.

Step-by-step — A three-regime loss probability

Partition. Each day the market is in exactly one regime: calm (

\mathbb{P}=0.6

), normal (

\mathbb{P}=0.3

), or stress (

\mathbb{P}=0.1

). The weights sum to

1

Conditionals. Let

L

= "portfolio loses more than

2\%

today". The desk estimates

\mathbb{P}(L\mid\text{calm})=0.01

\mathbb{P}(L\mid\text{normal})=0.04

\mathbb{P}(L\mid\text{stress})=0.20

Weight and add.

\mathbb{P}(L)=0.01\cdot 0.6+0.04\cdot 0.3+0.20\cdot 0.1=0.006+0.012+0.020=0.038

Read the decomposition. The stress regime is only

10\%

of days but contributes

0.020/0.038\approx 53\%

of all large losses. A model fitted to calm data alone would quote

1\%

— understating the true frequency almost fourfold.

Mixtures make tails heavy

Each regime above could have perfectly well-behaved losses. Mixing them produces an unconditional loss frequency that no single regime exhibits, and the rare regime dominates the tail. This is one of the standard explanations for why return distributions look fatter-tailed than any one calm-period model suggests.

Practice — Build the tree

Draw the three-regime example as a tree: first branch on regime, then on $L$ versus $L^c$ . Label each edge with its conditional probability, multiply along each root-to-leaf path, and check that the six leaf probabilities sum to $1$ .

05 · Bayes' Theorem

The law of total probability goes forward: from scenario to outcome. Bayes' theorem goes backward: having seen the outcome, how likely is each scenario?

Definition — Bayes' Theorem

If $\mathbb{P}(A)>0$ and $\mathbb{P}(B)>0$ ,

\mathbb{P}(B\mid A)=\frac{\mathbb{P}(A\mid B)\,\mathbb{P}(B)}{\mathbb{P}(A)}

If $B_1,\ldots,B_n$ partition $\Omega$ with each $\mathbb{P}(B_i)>0$ , expanding the denominator by total probability gives

\mathbb{P}(B_j\mid A)=\frac{\mathbb{P}(A\mid B_j)\,\mathbb{P}(B_j)}{\sum_{i=1}^n \mathbb{P}(A\mid B_i)\,\mathbb{P}(B_i)}

In the language of the glossary: posterior $\propto$ likelihood $\times$ prior, and the denominator just rescales so the posteriors over all hypotheses sum to $1$ . Applied to the three-regime example, seeing a large loss moves the probability of stress from a prior of $0.1$ to a posterior of $0.020/0.038\approx 0.53$ .

Step-by-step — A stress-test flag with a low base rate

Prior. Of the banks a supervisor monitors,

2\%

are genuinely fragile:

\mathbb{P}(F)=0.02

Likelihoods. The stress test flags a fragile bank with probability

\mathbb{P}(\text{flag}\mid F)=0.90

, and wrongly flags a sound bank with probability

\mathbb{P}(\text{flag}\mid F^c)=0.05

Evidence. By total probability,

\mathbb{P}(\text{flag})=0.90\cdot 0.02+0.05\cdot 0.98=0.018+0.049=0.067

Posterior.

\mathbb{P}(F\mid\text{flag})=0.018/0.067\approx 0.27

. A test that is "

90\%

accurate" is wrong about roughly three flagged banks in four.

Sanity check in counts. Out of

10{,}000

banks,

200

are fragile and

180

of them are flagged;

9{,}800

are sound and

490

of them are flagged. Among

670

flags, only

180

are real:

180/670\approx 27\%

The false positives win because they are drawn from a population $49$ times larger. When the base rate is small, even a modest false-positive rate swamps the true signal.

❌ Quoting the joint and calling it the posterior

In the same arithmetic, $0.90\times 0.02=0.018$ is the probability of the path "fragile and flagged". A note then says a flagged bank has a $1.8\%$ chance of being fragile.

Why it breaks: $0.018=\mathbb{P}(\text{flag}\cap F)$ is a joint probability. The posterior divides by the probability of the evidence, $\mathbb{P}(\text{flag})=0.067$ , and only then equals about $0.27$ . Stopping before that division leaves the answer in the original world, where flags are uncommon, instead of in the world where a flag was seen.

Consequence: the $1.8\%$ figure is small because the prior is small, which is the opposite of what the flag is trying to tell you. Joint, likelihood, and posterior are three different numbers; a risk limit or a supervisory action has to say which one it is using.

Odds form — updating twice

Dividing Bayes for $F$ by Bayes for $F^c$ cancels the denominator: posterior odds $=$ likelihood ratio $\times$ prior odds. Here the likelihood ratio is $0.90/0.05=18$ and the prior odds are $2/98$ , so the posterior odds are $36/98$ — probability $36/134\approx 0.27$ , as before. If a second, different test also flags the bank, and the two tests err independently given the bank's true state (an assumption Chapter 04 makes precise), multiply by the second likelihood ratio. With another ratio of $18$ the odds become $648/98$ , a posterior of about $0.87$ .

❌ The prosecutor's fallacy

Surveillance software flags a trading pattern. Honest accounts produce the pattern with probability $1/1000$ ; rogue accounts produce it almost surely. Someone argues: "the chance of this pattern from an honest trader is $0.1\%$ , so the trader is $99.9\%$ likely to be rogue."

Why it breaks: the argument reports $\mathbb{P}(\text{pattern}\mid\text{honest})$ and calls it $\mathbb{P}(\text{honest}\mid\text{pattern})$ . Bayes needs the base rate. If $1$ account in $10{,}000$ is rogue, then among $100{,}000$ accounts about $10$ rogue accounts are flagged and about $100$ honest accounts are flagged. So $\mathbb{P}(\text{rogue}\mid\text{flag})\approx 10/110\approx 9\%$ , and an honest trader is the likely explanation.

Consequence: the same swap appears whenever someone says a model "had only a $5\%$ chance of producing this backtest if it had no skill, so it has skill with $95\%$ probability". A small likelihood under one hypothesis is not a small posterior for that hypothesis; the answer depends on the prior and on how likely the evidence is under the alternatives. Chapter 14 returns to this distinction when a hypothesis test is read as a probability that the hypothesis is true.

06 · Conditional Survival Along a Credit Curve

A cumulative default probability answers "does the name default at some point before this date?" A trading desk usually inputs something narrower: the chance of defaulting this year if the name is still alive at the start of it. Those inputs sit in different conditional worlds. The chain rule is what stitches them into one probability on the original space.

Definition — Conditional Default Probabilities

Let $S_{k-1}$ be "alive at the start of year $k$ " and $D_k$ be "defaults during year $k$ ", with $S_0=\Omega$ . When $\mathbb{P}(S_{k-1})>0$ ,

h_k=\mathbb{P}(D_k\mid S_{k-1})

The multiplication rule gives $\mathbb{P}(D_k)=\mathbb{P}(S_{k-1})\,h_k$ and $\mathbb{P}(S_k)=\mathbb{P}(S_{k-1})\,(1-h_k)$ .

Unrolling the product, survival through $n$ years and default during a specific year are

\mathbb{P}(S_n)=\prod_{k=1}^{n}(1-h_k), \qquad \mathbb{P}(D_k)=h_k\prod_{j=1}^{k-1}(1-h_j)

with the empty product (when $k=1$ ) equal to $1$ . The events $D_1,D_2,\ldots$ are disjoint — a name defaults in at most one year — so the probability of default within $n$ years is the sum of those pieces, which is also $1-\mathbb{P}(S_n)$ .

Example — Three conditional inputs, one cumulative

An investment-grade name has $h_1=0.02$ , $h_2=0.03$ , $h_3=0.04$ .

\mathbb{P}(S_3)=0.98\times 0.97\times 0.96=0.912576

Default during year 2, which requires surviving year 1, has probability $0.98\times 0.03=0.0294$ . Default within three years has probability $1-0.912576=0.087424$ . The sum of the inputs, $0.02+0.03+0.04=0.09$ , is close and is still a different number: each $h_k$ was computed in a smaller world than the one before it.

❌ Adding conditional default probabilities

A distressed name is assigned conditional default probabilities $0.40$ , $0.50$ , and $0.60$ for the next three years. A summary adds them and reports a $150\%$ chance of default within three years.

Why it breaks: $h_2=0.50$ is the probability of default in year 2 given survival through year 1, not the probability of that event in the original sample space. The three numbers are not the probabilities of three disjoint events, so the additivity axiom does not apply to them. The chain rule gives

\mathbb{P}(S_3)=0.60\times 0.50\times 0.40=0.12, \qquad \mathbb{P}(D_1\cup D_2\cup D_3)=1-0.12=0.88

Consequence: a sum above $1$ is easy to catch, and it is not the only failure. In the investment-grade example the sum $0.09$ sits quietly above the true cumulative $0.087$ and would pass a "does it exceed one?" check. Convert each conditional probability into an unconditional piece with the product of survivals, then add those pieces.

Where the tail of the curve goes

Pushing $n$ out, $\mathbb{P}(S_n)$ is the probability of surviving every year through $n$ , and $1-\mathbb{P}(S_n)$ increases to the probability of eventual default (Chapter 02's nested limit). Chapter 15 builds tail risk — value at risk and expected shortfall — by restricting attention to bad outcomes and rescaling. That is the same habit: recompute inside the event you conditioned on. A default curve is not itself one of those tail risk numbers.

07 · Exercises

EXERCISE 3.1

Write $\mathbb{P}(A\cap B_j)$ two ways using the multiplication rule, then replace $\mathbb{P}(A)$ with its law-of-total-probability expansion.

By the multiplication rule, $\mathbb{P}(A\cap B_j)=\mathbb{P}(B_j\mid A)\mathbb{P}(A)=\mathbb{P}(A\mid B_j)\mathbb{P}(B_j)$ . Dividing by $\mathbb{P}(A)>0$ gives $\mathbb{P}(B_j\mid A)=\mathbb{P}(A\mid B_j)\mathbb{P}(B_j)/\mathbb{P}(A)$ . Since the $B_i$ partition $\Omega$ with positive probabilities, $\mathbb{P}(A)=\sum_i\mathbb{P}(A\mid B_i)\mathbb{P}(B_i)$ , which is the denominator of the partition form. Summing the result over $j$ gives $1$ , confirming that the posteriors form a probability over the hypotheses.

Derive the partition form of Bayes' theorem from the multiplication rule and the law of total probability, and check that the posteriors $\mathbb{P}(B_1\mid A),\ldots,\mathbb{P}(B_n\mid A)$ sum to $1$ .

EXERCISE 3.2

Compute $\mathbb{P}(\text{flag})$ by total probability first. For the second part, condition on "no flag" and use $\mathbb{P}(\text{no flag}\mid D)=1-0.80$ .

$\mathbb{P}(\text{flag})=0.80\cdot 0.05+0.10\cdot 0.95=0.040+0.095=0.135$ , so $\mathbb{P}(D\mid\text{flag})=0.040/0.135\approx 0.296$ . For unflagged borrowers, $\mathbb{P}(\text{no flag})=0.865$ and $\mathbb{P}(D\cap\text{no flag})=0.20\cdot 0.05=0.010$ , so $\mathbb{P}(D\mid\text{no flag})=0.010/0.865\approx 0.0116$ . The flag raises default risk from $5\%$ to about $30\%$ and an absent flag lowers it to about $1.2\%$ : the model is useful for ranking, but declining every flagged applicant would turn away roughly seven good borrowers for every three defaulters.

Five percent of a lender's borrowers default within a year. A scoring model flags $80\%$ of eventual defaulters and $10\%$ of borrowers who do not default. Let $D$ be "borrower defaults". Find $\mathbb{P}(D\mid\text{flag})$ and $\mathbb{P}(D\mid\text{no flag})$ , and interpret both numbers for a credit officer.

EXERCISE 3.3

The hypotheses are the two regimes. Compute the likelihood of "two down days" in each regime, multiply by the prior, and normalize.

Likelihoods: $\mathbb{P}(DD\mid\text{bull})=0.45^2=0.2025$ and $\mathbb{P}(DD\mid\text{bear})=0.60^2=0.36$ . Unnormalized posteriors: $0.7\cdot 0.2025=0.14175$ and $0.3\cdot 0.36=0.108$ , summing to $0.24975$ . Hence $\mathbb{P}(\text{bear}\mid DD)=0.108/0.24975\approx 0.432$ , up from a prior of $0.30$ . Two down days are only mildly more likely under bear than bull (likelihood ratio $0.36/0.2025\approx 1.78$ ), so the update is real but modest; a single noisy observation rarely settles which regime you are in.

A market is in a bull regime with probability $0.7$ and a bear regime with probability $0.3$ . A down day has probability $0.45$ in the bull regime and $0.60$ in the bear regime, and within a regime the model treats the two days as unrelated, so the probability of two down days is the square of the one-day probability. After observing two consecutive down days, find the posterior probability of the bear regime. (Chapter 04 will name "unrelated" as independence; the product used here is part of the question's assumptions.)

EXERCISE 3.4

Survival through year $k$ multiplies one more factor $(1-h_k)$ . Default during year 2 equals survival through year 1 times $h_2$ . The three conditional inputs are not the probabilities of disjoint events.

(a) $\mathbb{P}(S_3)=0.90\times 0.80\times 0.75=0.54$ . (b) Default during year 2 requires survival through year 1, so $\mathbb{P}(D_2)=0.90\times 0.20=0.18$ . (c) $\mathbb{P}(D_1\cup D_2\cup D_3)=1-0.54=0.46$ . As a check, the disjoint pieces are $\mathbb{P}(D_1)=0.10$ , $\mathbb{P}(D_2)=0.18$ , and $\mathbb{P}(D_3)=0.90\times 0.80\times 0.25=0.18$ , which sum to $0.46$ . (d) The sum $h_1+h_2+h_3=0.55$ adds probabilities from three different conditional worlds. It exceeds the cumulative default probability $0.46$ and is not itself the probability of any one event on the original space.

A name has conditional default probabilities $h_1=0.10$ , $h_2=0.20$ , $h_3=0.25$ , each given survival to the start of that year. (a) Find the probability of surviving all three years. (b) Find the probability of defaulting during year 2. (c) Find the probability of defaulting within three years. (d) Explain why $h_1+h_2+h_3$ is not the answer to (c).

08 · Chapter Summary

Concept	Meaning
Conditional probability	$\mathbb{P}(A\mid B)=\mathbb{P}(A\cap B)/\mathbb{P}(B)$ , defined only for $\mathbb{P}(B)>0$
Conditioning keeps the axioms	$\mathbb{P}(\cdot\mid B)$ is itself a probability on $\Omega$
Multiplication / chain rule	probability of a sequence = product of step-by-step conditionals
Total probability	unconditional probability = scenario-weighted average of conditionals
Bayes' theorem	posterior $\propto$ likelihood $\times$ prior
Base-rate effect	rare hypotheses stay unlikely after a positive signal unless the false-positive rate is tiny
Prosecutor's fallacy	$\mathbb{P}(D\mid H)$ is not $\mathbb{P}(H\mid D)$
Joint versus posterior	likelihood times prior is $\mathbb{P}(D\cap H)$ ; divide by $\mathbb{P}(D)$ before calling it a conditional probability
Conditional hazard $h_k$	default this year given survival so far; chain the survivals, then add the disjoint year-by-year pieces

Next: Chapter 04 — Independence of Events, which studies the special case where conditioning on $B$ leaves the probability of $A$ unchanged, and shows why that assumption is so often wrong for defaults.