Chapter 14

Rigorous

Positive Definite Matrices

00 · Symbol Glossary

$A \succ 0$A succ zero — positive definite

Shorthand for " $A$ is positive definite." The $\succ$ symbol is the matrix analogue of the scalar inequality $>0$ — but it means $\mathbf{x}^TA\mathbf{x}>0$ for all nonzero $\mathbf{x}$ , not that every entry of $A$ is positive. Only defined for symmetric matrices.

$A \succeq 0$A succeq zero — positive semi-definite

" $A$ is positive semi-definite." Requires $\mathbf{x}^TA\mathbf{x}\geq0$ for all $\mathbf{x}$ — equality is permitted. A PSD matrix may have zero eigenvalues, meaning it is singular. Valid covariance matrices are PSD (not necessarily PD) because a portfolio with zero variance is possible (perfectly correlated assets).

$L$L — Cholesky factor

The lower-triangular matrix in the Cholesky decomposition $A=LL^T$ . Exists (and is unique with positive diagonal entries) whenever $A$ is symmetric positive definite. The Cholesky factor is the matrix analogue of the square root: $L$ "is the square root" of $A$ in the sense that $A = LL^T$ .

$\lambda_{\min}(A)$lambda min A — smallest eigenvalue

The smallest eigenvalue of a symmetric matrix $A$ . $A\succ0$ $\iff$ $\lambda_{\min}(A)>0$ . Provides a quantitative measure of "how positive definite" $A$ is — the further $\lambda_{\min}$ is from zero, the better conditioned $A$ is.

01 · What Is Positive Definiteness?

A matrix is called positive definite when multiplying it by any nonzero vector and then dotting with that same vector always gives a positive number. The condition $\mathbf{x}^TA\mathbf{x}>0$ captures a geometric idea: $A$ never "collapses" a direction — it stretches every nonzero vector to have some component in its own direction.

This matters in finance because a covariance matrix $\Sigma$ must satisfy $\mathbf{w}^T\Sigma\mathbf{w}\geq0$ — portfolio variance cannot be negative. If $\Sigma$ is estimated from data and fails this condition (e.g. due to numerical errors), the "covariance matrix" is mathematically invalid.

Definition — Positive Definite Matrix

A symmetric matrix $A \in \mathbb{R}^{n\times n}$ is positive definite ( $A \succ 0$ ) if:

\mathbf{x}^T A \mathbf{x} > 0 \quad \text{for all } \mathbf{x} \in \mathbb{R}^n,\ \mathbf{x} \neq \mathbf{0}

$A$ is positive semi-definite ( $A \succeq 0$ ) if the inequality is $\geq 0$ (zero is allowed).

$A$ is negative definite if $-A\succ0$ , and indefinite if neither $A$ nor $-A$ is PSD.

✓ Example — Testing Positive Definiteness by Definition

$A=\begin{pmatrix}2&1\\1&3\end{pmatrix}$ . For $\mathbf{x}=\begin{pmatrix}x_1\\x_2\end{pmatrix}$ :

$\mathbf{x}^TA\mathbf{x} = \begin{pmatrix}x_1&x_2\end{pmatrix}\begin{pmatrix}2&1\\1&3\end{pmatrix}\begin{pmatrix}x_1\\x_2\end{pmatrix} = 2x_1^2+2x_1x_2+3x_2^2$ .

Complete the square: $2x_1^2+2x_1x_2+3x_2^2 = 2(x_1+\tfrac{1}{2}x_2)^2+\tfrac{5}{2}x_2^2$ .

Both terms are non-negative; both are zero only when $x_2=0$ and then $x_1=0$ . Therefore $\mathbf{x}^TA\mathbf{x}>0$ for all $\mathbf{x}\neq\mathbf{0}$ , so $A\succ0$ .

❌ What Breaks — Positive Entries Do Not Imply Positive Definite

$A=\begin{pmatrix}1&2\\2&1\end{pmatrix}$ has all positive entries, but $\mathbf{x}^TA\mathbf{x}=x_1^2+4x_1x_2+x_2^2$ . Try $\mathbf{x}=\begin{pmatrix}1\\-1\end{pmatrix}$ : $1-4+1=-2<0$ . $A$ is indefinite — not even PSD. Positive entries are completely unrelated to positive definiteness.

02 · Four Equivalent Characterisations

Positive definiteness can be detected four ways. Each characterisation is useful in a different situation.

Definition — Four Equivalent Conditions for PD

For a symmetric $A \in \mathbb{R}^{n\times n}$ , the following are equivalent:

PD1 (Quadratic form): $\mathbf{x}^TA\mathbf{x}>0$ for all $\mathbf{x}\neq\mathbf{0}$ .

PD2 (Eigenvalues): All eigenvalues of $A$ are strictly positive: $\lambda_1,\ldots,\lambda_n>0$ .

PD3 (Leading minors): All $k$ leading principal minors are positive: $\det(A_k)>0$ for $k=1,\ldots,n$ , where $A_k$ is the top-left $k\times k$ submatrix.

PD4 (Cholesky): $A$ has a Cholesky decomposition $A=LL^T$ with $L$ lower-triangular and positive diagonal entries.

Which Condition to Use When

PD2 (eigenvalues) is the most intuitive but requires computing eigenvalues. PD3 (leading minors) gives a fast check for small matrices — just compute up to $n$ determinants. PD4 (Cholesky) is the numerical standard: if the Cholesky algorithm completes without a zero or negative diagonal, the matrix is PD.

Step-by-step — Verifying all four PD conditions for $A=\begin{pmatrix}4&2\\2&3\end{pmatrix}$

PD1 — Quadratic form: $\mathbf{x}^TA\mathbf{x}=4x_1^2+4x_1x_2+3x_2^2 = 4(x_1+\frac{1}{2}x_2)^2+2x_2^2$ .

Both squared terms non-negative; zero only at $\mathbf{x}=\mathbf{0}$ . $A\succ0$ ✓.

PD2 — Eigenvalues: $p(\lambda)=(4-\lambda)(3-\lambda)-4=\lambda^2-7\lambda+8$ .

$\lambda = \frac{7\pm\sqrt{49-32}}{2} = \frac{7\pm\sqrt{17}}{2}$ .

$\lambda_1=\frac{7+\sqrt{17}}{2}\approx5.56>0$ ✓. $\lambda_2=\frac{7-\sqrt{17}}{2}\approx1.44>0$ ✓. Both positive.

PD3 — Leading minors:

$\det(A_1) = \det(4) = 4 > 0$ ✓ — the top-left $1\times1$ minor.

$\det(A_2) = \det\begin{pmatrix}4&2\\2&3\end{pmatrix} = 12-4 = 8 > 0$ ✓ — the full $2\times2$ determinant.

Both leading minors positive confirms $A\succ0$ .

PD4 — Cholesky: find lower-triangular $L=\begin{pmatrix}\ell_{11}&0\\\ell_{21}&\ell_{22}\end{pmatrix}$ with $LL^T=A$ .

$\ell_{11}^2=4 \Rightarrow \ell_{11}=2$ . $\ell_{21}\ell_{11}=2 \Rightarrow \ell_{21}=1$ . $\ell_{21}^2+\ell_{22}^2=3 \Rightarrow \ell_{22}^2=3-1=2 \Rightarrow \ell_{22}=\sqrt{2}$ .

L = \begin{pmatrix}2&0\\1&\sqrt{2}\end{pmatrix}, \quad LL^T = \begin{pmatrix}2&0\\1&\sqrt{2}\end{pmatrix}\begin{pmatrix}2&1\\0&\sqrt{2}\end{pmatrix} = \begin{pmatrix}4&2\\2&3\end{pmatrix} = A \checkmark

Positive diagonal entries $\ell_{11}=2>0$ , $\ell_{22}=\sqrt{2}>0$ confirm $A\succ0$ .

03 · The Cholesky Decomposition

The Cholesky decomposition $A=LL^T$ is the matrix analogue of taking a square root. It exists for every PD matrix and is unique when diagonal entries of $L$ are required positive.

Step-by-step — Cholesky decomposition of $A=\begin{pmatrix}9&3&3\\3&5&1\\3&1&6\end{pmatrix}$

Column 1 of $L$ : $\ell_{11}=\sqrt{a_{11}}=\sqrt{9}=3$ . $\ell_{21}=a_{21}/\ell_{11}=3/3=1$ . $\ell_{31}=a_{31}/\ell_{11}=3/3=1$ .

Column 2 of $L$ : $\ell_{22}=\sqrt{a_{22}-\ell_{21}^2}=\sqrt{5-1}=2$ . $\ell_{32}=(a_{32}-\ell_{31}\ell_{21})/\ell_{22}=(1-1\cdot1)/2=0$ .

Column 3 of $L$ : $\ell_{33}=\sqrt{a_{33}-\ell_{31}^2-\ell_{32}^2}=\sqrt{6-1-0}=\sqrt{5}$ .

Assemble and verify:

L = \begin{pmatrix}3&0&0\\1&2&0\\1&0&\sqrt{5}\end{pmatrix}

$LL^T = \begin{pmatrix}3&0&0\\1&2&0\\1&0&\sqrt{5}\end{pmatrix}\begin{pmatrix}3&1&1\\0&2&0\\0&0&\sqrt{5}\end{pmatrix} = \begin{pmatrix}9&3&3\\3&5&1\\3&1&6\end{pmatrix} = A$ ✓.

04 · Connection Between the Four Conditions

The equivalences follow from the spectral theorem.

PD1 $\iff$ PD2 via spectral theorem: Every symmetric $A$ has real eigenvalues and orthogonal eigenvectors: $A=Q\Lambda Q^T$ . Then $\mathbf{x}^TA\mathbf{x}=\mathbf{x}^TQ\Lambda Q^T\mathbf{x}=\mathbf{y}^T\Lambda\mathbf{y}=\sum_i\lambda_i y_i^2$ where $\mathbf{y}=Q^T\mathbf{x}$ . Since $Q$ is invertible, $\mathbf{y}\neq\mathbf{0}$ iff $\mathbf{x}\neq\mathbf{0}$ . Therefore $\mathbf{x}^TA\mathbf{x}>0$ for all $\mathbf{x}\neq\mathbf{0}$ iff $\lambda_i>0$ for all $i$ .

PD1 $\iff$ PD3 (Sylvester's criterion): The leading minors capture the "partial" positive definiteness condition — if any $k\times k$ leading submatrix is indefinite, the full matrix is not PD.

PD1 $\iff$ PD4: If $A=LL^T$ , then $\mathbf{x}^TA\mathbf{x}=\mathbf{x}^TLL^T\mathbf{x}=\|L^T\mathbf{x}\|^2\geq0$ , with equality iff $L^T\mathbf{x}=\mathbf{0}$ iff $\mathbf{x}=\mathbf{0}$ (since $L$ has positive diagonal, hence $L$ is invertible). Conversely, the Cholesky algorithm succeeds without encountering a square root of a non-positive number iff $A\succ0$ .

❌ What Breaks — PSD Is Not PD

A covariance matrix estimated from $n$ observations on $p$ assets with $n < p$ has rank at most $n$ , so it has at least $p-n$ zero eigenvalues — it is PSD but not PD. Cholesky fails (encounters $\sqrt{0}$ ), and the matrix is singular: no unique mean-variance-optimal portfolio exists. The fix is regularisation: add $\lambda I$ (ridge shrinkage) to make $\hat{\Sigma}+\lambda I$ positive definite.

05 · Quant Application — Valid Covariance Matrices

Portfolio variance is $\sigma_p^2=\mathbf{w}^T\Sigma\mathbf{w}$ . For this to be a valid variance:

It must be non-negative for all weight vectors $\mathbf{w}$ — so $\Sigma\succeq0$ (PSD).
If every non-trivial portfolio has positive variance, $\Sigma\succ0$ (PD).

When does $\Sigma$ fail to be PD?

Two perfectly correlated assets: $\rho=1$ , so $\Sigma=\begin{pmatrix}\sigma_1^2&\sigma_1\sigma_2\\\sigma_1\sigma_2&\sigma_2^2\end{pmatrix}$ has $\det=0$ . The hedge portfolio $\mathbf{w}=(w,-w)^T$ with $w\sigma_1=w\sigma_2$ achieves zero variance.
Sample estimate with too few observations: $n<p$ means rank deficiency.
Numerical errors in estimation: slightly negative eigenvalues from floating-point arithmetic.

Nearest PD matrix: Given a symmetric indefinite matrix $\tilde{\Sigma}$ (from numerical errors), the nearest PD matrix (in Frobenius norm) is obtained by replacing negative eigenvalues with a small $\epsilon>0$ : $\hat{\Sigma}=Q\Lambda_+Q^T$ where $\Lambda_+ = \text{diag}(\max(\lambda_i,\epsilon))$ .

06 · Practice Exercises

EXERCISE 14.1

Compute $\mathbf{x}^TA\mathbf{x}$ symbolically. Then try to complete the square — if you can write it as a sum of squared terms with positive coefficients, it is PD. Alternatively, check eigenvalues.

$A=\begin{pmatrix}3&1\\1&2\end{pmatrix}$ .

$\mathbf{x}^TA\mathbf{x}=3x_1^2+2x_1x_2+2x_2^2$ .

Complete the square: $3(x_1+\frac{1}{3}x_2)^2+(2-\frac{1}{3})x_2^2=3(x_1+\frac{1}{3}x_2)^2+\frac{5}{3}x_2^2$ .

Both terms $\geq0$ ; equal to zero only when $x_2=0$ and $x_1+\frac{1}{3}(0)=0$ , i.e. $x_1=x_2=0$ . Therefore $A\succ0$ ✓.

Eigenvalue check: $p(\lambda)=(3-\lambda)(2-\lambda)-1=\lambda^2-5\lambda+5$ . $\lambda=\frac{5\pm\sqrt{5}}{2}$ . Both $\approx3.618$ and $\approx1.382$ — both positive ✓.

Determine whether $A=\begin{pmatrix}3&1\\1&2\end{pmatrix}$ is positive definite. Use the quadratic form (complete the square) and verify using eigenvalues.

EXERCISE 14.2

Check the three leading minors: $\det(a_{11})>0$ , $\det\begin{pmatrix}a_{11}&a_{12}\\a_{21}&a_{22}\end{pmatrix}>0$ , $\det(A)>0$ . All three must be positive.

$A=\begin{pmatrix}4&2&0\\2&3&1\\0&1&2\end{pmatrix}$ .

Minor 1: $\det(4)=4>0$ ✓.

Minor 2: $\det\begin{pmatrix}4&2\\2&3\end{pmatrix}=12-4=8>0$ ✓.

Minor 3 ( $\det(A)$ ): expanding along row 3: $0\cdot(\ldots)-1\cdot\det\begin{pmatrix}4&0\\2&1\end{pmatrix}+2\cdot\det\begin{pmatrix}4&2\\2&3\end{pmatrix}=0-1(4)+2(8)=-4+16=12>0$ ✓.

All three leading minors are positive by Sylvester's criterion $\Rightarrow$ $A\succ0$ .

Apply Sylvester's criterion (PD3 — check all leading principal minors) to determine if $A=\begin{pmatrix}4&2&0\\2&3&1\\0&1&2\end{pmatrix}$ is positive definite.

EXERCISE 14.3

Follow the Cholesky algorithm: $\ell_{11}=\sqrt{a_{11}}$ , $\ell_{j1}=a_{j1}/\ell_{11}$ , then $\ell_{22}=\sqrt{a_{22}-\ell_{21}^2}$ , and so on.

$A=\begin{pmatrix}4&2\\2&2\end{pmatrix}$ .

$\ell_{11}=\sqrt{4}=2$ .

$\ell_{21}=a_{21}/\ell_{11}=2/2=1$ .

$\ell_{22}=\sqrt{a_{22}-\ell_{21}^2}=\sqrt{2-1}=1$ .

$L=\begin{pmatrix}2&0\\1&1\end{pmatrix}$ .

Verify: $LL^T=\begin{pmatrix}2&0\\1&1\end{pmatrix}\begin{pmatrix}2&1\\0&1\end{pmatrix}=\begin{pmatrix}4&2\\2&2\end{pmatrix}=A$ ✓.

Both diagonal entries $\ell_{11}=2>0$ , $\ell_{22}=1>0$ confirm $A\succ0$ .

Find the Cholesky decomposition $A=LL^T$ for $A=\begin{pmatrix}4&2\\2&2\end{pmatrix}$ . Verify $LL^T=A$ and confirm the diagonal entries of $L$ are positive.

EXERCISE 14.4

$A=Q\Lambda Q^T$ (spectral theorem). Use $\mathbf{y}=Q^T\mathbf{x}$ to change variables. Then $\mathbf{x}^TA\mathbf{x}=\mathbf{y}^T\Lambda\mathbf{y}=\sum_i\lambda_iy_i^2$ . What sign does this have when all $\lambda_i>0$ ?

Step 1: By the spectral theorem (A symmetric), $A=Q\Lambda Q^T$ where $Q$ is orthogonal (eigenvector matrix) and $\Lambda=\text{diag}(\lambda_1,\ldots,\lambda_n)$ .

Step 2: Substitute into the quadratic form. Let $\mathbf{y}=Q^T\mathbf{x}$ . Since $Q$ is invertible, $\mathbf{y}=\mathbf{0}\iff\mathbf{x}=\mathbf{0}$ .

$\mathbf{x}^TA\mathbf{x}=\mathbf{x}^TQ\Lambda Q^T\mathbf{x}=(Q^T\mathbf{x})^T\Lambda(Q^T\mathbf{x})=\mathbf{y}^T\Lambda\mathbf{y}=\sum_{i=1}^n\lambda_i y_i^2$ .

Step 3 ( $\Rightarrow$ ): If all $\lambda_i>0$ , then for $\mathbf{x}\neq\mathbf{0}$ (so $\mathbf{y}\neq\mathbf{0}$ ): $\sum\lambda_iy_i^2>0$ since all terms $\lambda_iy_i^2\geq0$ and at least one $y_j^2>0$ multiplied by $\lambda_j>0$ .

Step 4 ( $\Leftarrow$ ): If some $\lambda_k\leq0$ , pick $\mathbf{y}=\mathbf{e}_k$ (standard basis), so $\mathbf{x}=Q\mathbf{e}_k$ (the $k$ -th eigenvector). Then $\mathbf{x}^TA\mathbf{x}=\lambda_k\leq0$ — not positive definite.

Prove that a symmetric matrix $A\succ0$ (PD1) if and only if all eigenvalues of $A$ are strictly positive (PD2). Use the spectral decomposition $A=Q\Lambda Q^T$ and the substitution $\mathbf{y}=Q^T\mathbf{x}$ .

EXERCISE 14.5

Show that for any $B$ , the matrix $B^TB$ is always PSD. It is PD iff $B$ has full column rank (no null vector). Write the quadratic form $\mathbf{x}^TB^TB\mathbf{x}=\|B\mathbf{x}\|^2$ .

$B^TB$ is always PSD: For any $\mathbf{x}$ : $\mathbf{x}^TB^TB\mathbf{x}=(B\mathbf{x})^T(B\mathbf{x})=\|B\mathbf{x}\|^2\geq0$ . Equality when $B\mathbf{x}=\mathbf{0}$ , i.e. $\mathbf{x}\in\ker(B)$ .

$B^TB$ is PD iff $B$ has full column rank: $B^TB\succ0\iff\|B\mathbf{x}\|^2>0$ for all $\mathbf{x}\neq\mathbf{0}\iff B\mathbf{x}\neq\mathbf{0}$ for all $\mathbf{x}\neq\mathbf{0}\iff\ker(B)=\{\mathbf{0}\}\iff B$ has full column rank.

Application: the sample covariance matrix $\hat{\Sigma}=\frac{1}{n-1}X^TX$ (where $X$ is the mean-centered data matrix) is PD iff the $n$ observations are not confined to a proper subspace — equivalently, iff there are no perfectly collinear assets and $n\geq p+1$ .

Prove that $B^TB$ is always positive semi-definite for any real matrix $B$ . State the condition under which $B^TB$ is strictly positive definite. Apply this to explain when the sample covariance matrix $\hat{\Sigma}=\frac{1}{n-1}X^TX$ is PD.

EXERCISE 14.6

Compute the eigenvalues of $\tilde{\Sigma}$ . Replace any negative eigenvalues with a small $\epsilon>0$ to get $\Lambda_+$ . Reconstruct $\hat{\Sigma}=Q\Lambda_+Q^T$ .

$\tilde{\Sigma}=\begin{pmatrix}1&0.9\\0.9&0.8\end{pmatrix}$ (estimated, possibly from numerical error).

$\det(\tilde{\Sigma})=0.8-0.81=-0.01<0$ — $\tilde{\Sigma}$ is indefinite (negative determinant $\Rightarrow$ negative eigenvalue).

Eigenvalues: $p(\lambda)=\lambda^2-1.8\lambda+(0.8-0.81)=\lambda^2-1.8\lambda-0.01$ .

$\lambda=\frac{1.8\pm\sqrt{3.24+0.04}}{2}=\frac{1.8\pm\sqrt{3.28}}{2}\approx\frac{1.8\pm1.811}{2}$ .

$\lambda_1\approx1.806>0$ , $\lambda_2\approx-0.006<0$ — one negative eigenvalue confirms the matrix is not PSD.

Fix with $\epsilon=0.01$ : Replace $\lambda_2=-0.006$ with $\epsilon=0.01$ .

$\Lambda_+=\begin{pmatrix}1.806&0\\0&0.01\end{pmatrix}$ .

The eigenvector for $\lambda_1\approx1.806$ : $(Σ-1.806I)\mathbf{v}=0$ gives approximate direction $\begin{pmatrix}0.83\\0.56\end{pmatrix}$ ; for $\lambda_2$ : $\begin{pmatrix}-0.56\\0.83\end{pmatrix}$ .

$\hat{\Sigma}=Q\Lambda_+Q^T$ — the nearest PD matrix (in Frobenius norm) to $\tilde{\Sigma}$ . In practice, the repaired matrix has slightly higher variance in the near-zero direction, ensuring all portfolios have positive variance.

A numerical estimation procedure yields $\tilde{\Sigma}=\begin{pmatrix}1&0.9\\0.9&0.8\end{pmatrix}$ . Verify that $\tilde{\Sigma}$ is not positive semi-definite by computing its determinant and eigenvalues. Describe the procedure for finding the nearest valid (PSD) covariance matrix by thresholding eigenvalues.

07 · Chapter Summary

Concept	Key Fact
Positive definite (PD1)	$\mathbf{x}^TA\mathbf{x}>0$ for all $\mathbf{x}\neq\mathbf{0}$ ; only for symmetric $A$
PD via eigenvalues (PD2)	All eigenvalues strictly positive
PD via minors (PD3)	All leading principal minors positive (Sylvester)
PD via Cholesky (PD4)	$A=LL^T$ ; $L$ lower-triangular, positive diagonal
Positive semi-definite	$\mathbf{x}^TA\mathbf{x}\geq0$ ; zero eigenvalues allowed
$B^TB$ always PSD	$\mathbf{x}^TB^TB\mathbf{x}=\\|B\mathbf{x}\\|^2\geq0$ ; PD iff $B$ has full col rank
Positive entries $\not\Rightarrow$ PD	Need all eigenvalues positive, not all entries positive
Covariance matrix	Always PSD; PD iff full rank and $n\geq p+1$
Repair indefinite matrix	Replace negative eigenvalues: $\hat{\Sigma}=Q\Lambda_+Q^T$
Cholesky fails when	Matrix is not PD — square root of negative number encountered

Next: Chapter 15 — Covariance Matrices & Quadratic Forms builds on positive definiteness to derive sample covariance matrices from data, compute portfolio variance $\mathbf{w}^T\Sigma\mathbf{w}$ as a quadratic form, and connect the spectral decomposition to principal component analysis.