Chapter 06

Hard

Eigenvalues & Eigenvectors

00 — Symbol Glossary

01 — Eigenvalue–Eigenvector Definition

Definition

Let $A\in\mathbb{R}^{n\times n}$ . A scalar $\lambda\in\mathbb{C}$ is an eigenvalue of $A$ if there exists a nonzero vector $\mathbf{v}\in\mathbb{R}^n$ (or $\mathbb{C}^n$ ) such that

$A\mathbf{v}=\lambda\mathbf{v}$

Such a vector $\mathbf{v}\neq\mathbf{0}$ is called an eigenvector corresponding to $\lambda$ .

Rearranging: $(A-\lambda I)\mathbf{v}=\mathbf{0}$ . This system has a nonzero solution if and only if $A-\lambda I$ is singular, i.e.

$\det(A-\lambda I)=0$

Note

Every scalar multiple of an eigenvector is also an eigenvector for the same eigenvalue. The set of all eigenvectors for $\lambda$ (together with $\mathbf{0}$ ) forms a subspace called the eigenspace $E_\lambda$ .

Example

If $A$ represents a stretch by factor 3 along the $x$ -axis and factor 1 along the $y$ -axis, then the $x$ -axis direction is an eigenvector with $\lambda=3$ and the $y$ -axis direction is an eigenvector with $\lambda=1$ .

Common mistake

Wrong: $\mathbf{v}=\mathbf{0}$ satisfies $A\mathbf{0}=\lambda\mathbf{0}$ for any $\lambda$ , so $\mathbf{0}$ is an eigenvector.
Why it happens: The equation is satisfied, so it feels valid.
Correct: Eigenvectors must be nonzero by definition. The zero vector is excluded because it carries no directional information.
Check: Any definition of eigenvector explicitly states $\mathbf{v}\neq\mathbf{0}$ .

02 — The Characteristic Equation

Definition

The characteristic polynomial of $A\in\mathbb{R}^{n\times n}$ is

$p(\lambda)=\det(A-\lambda I)$

It is a degree- $n$ polynomial in $\lambda$ . The eigenvalues of $A$ are the roots of $p(\lambda)=0$ (the characteristic equation).

For a $2\times2$ matrix $A=\begin{pmatrix}a&b\\c&d\end{pmatrix}$ :

p(\lambda)=\det\begin{pmatrix}a-\lambda&b\\c&d-\lambda\end{pmatrix}=(a-\lambda)(d-\lambda)-bc=\lambda^2-\text{tr}(A)\lambda+\det(A)

Find eigenvalues of $A=\begin{pmatrix}4&1\\2&3\end{pmatrix}$

$A-\lambda I=\begin{pmatrix}4-\lambda&1\\2&3-\lambda\end{pmatrix}$ — subtract $\lambda$ from each diagonal entry $4$ and $3$ .

$p(\lambda)=\det\begin{pmatrix}4-\lambda&1\\2&3-\lambda\end{pmatrix}=(4-\lambda)(3-\lambda)-1\cdot2$ $=(12-7\lambda+\lambda^2)-2=\lambda^2-7\lambda+10$ — the $12$ comes from $4\cdot3$ ; the $7\lambda$ from $-(4\lambda+3\lambda)$ ; the $-2$ from $-(1\cdot2)$ .

$\lambda^2-7\lambda+10=0 \implies (\lambda-5)(\lambda-2)=0$ — factor or use the quadratic formula; roots are $\lambda=5$ and $\lambda=2$ .

$\text{tr}(A)=4+3=7=5+2\,\checkmark$ ; $\det(A)=4\cdot3-1\cdot2=10=5\cdot2\,\checkmark$ .

Note

Trace = sum of eigenvalues and determinant = product of eigenvalues (both with algebraic multiplicity). These are fast sanity checks.

03 — Finding Eigenvectors

Once you have an eigenvalue $\lambda_k$ , find $E_{\lambda_k}=\ker(A-\lambda_k I)$ by row-reducing $(A-\lambda_k I)\mathbf{v}=\mathbf{0}$ .

Find eigenvectors of $A=\begin{pmatrix}4&1\\2&3\end{pmatrix}$ for $\lambda=5$ and $\lambda=2$

$A-5I=\begin{pmatrix}-1&1\\2&-2\end{pmatrix}$ — subtract $5$ from each diagonal: $4-5=-1$ and $3-5=-2$ .

$R_2\leftarrow R_2+2R_1$ : $\begin{pmatrix}-1&1\\0&0\end{pmatrix}$ — $2+2(-1)=0$ ; $-2+2(1)=0$ . One free variable ( $v_2=t$ ).

From $-v_1+v_2=0$ : $v_1=v_2=t$ . Eigenvector: $t\begin{pmatrix}1\\1\end{pmatrix}$ , $t\neq0$ .

$A-2I=\begin{pmatrix}2&1\\2&1\end{pmatrix}$ — $4-2=2$ ; $3-2=1$ .

$R_2\leftarrow R_2-R_1$ : $\begin{pmatrix}2&1\\0&0\end{pmatrix}$ — one free variable ( $v_2=s$ ).

From $2v_1+v_2=0$ : $v_1=-v_2/2=-s/2$ . Eigenvector: $s\begin{pmatrix}-1\\2\end{pmatrix}$ (set $s=2$ for integer entries), $s\neq0$ .

Common mistake

Wrong: "the eigenvector for $\lambda=5$ is $\begin{pmatrix}1\\1\end{pmatrix}$ ."
Why it happens: We computed $t=1$ for convenience.
Correct: the eigenspace is $E_5=\text{span}\!\left\{\begin{pmatrix}1\\1\end{pmatrix}\right\}$ — any nonzero scalar multiple is equally valid.
Check: verify $A\cdot(2)\begin{pmatrix}1\\1\end{pmatrix}=5\cdot(2)\begin{pmatrix}1\\1\end{pmatrix}$ also holds.

04 — Algebraic and Geometric Multiplicity

Definition

Let $\lambda_k$ be an eigenvalue of $A$ .

Algebraic multiplicity $m_a(\lambda_k)$ : the multiplicity of $\lambda_k$ as a root of $p(\lambda)=\det(A-\lambda I)$ .
Geometric multiplicity $m_g(\lambda_k)$ : the dimension of the eigenspace $E_{\lambda_k}=\ker(A-\lambda_k I)$ .

It always holds that $1\leq m_g(\lambda_k)\leq m_a(\lambda_k)$ .

If $m_g(\lambda_k)<m_a(\lambda_k)$ for any $\lambda_k$ , the matrix is defective — it cannot be diagonalised.

Example

$A=\begin{pmatrix}2&1\\0&2\end{pmatrix}$ has characteristic polynomial $(\lambda-2)^2$ , so $m_a(2)=2$ . But $A-2I=\begin{pmatrix}0&1\\0&0\end{pmatrix}$ has a 1-dimensional null space, so $m_g(2)=1<2$ . The matrix is defective.

05 — Eigenvalues of Special Matrices

Matrix type	Eigenvalues
Diagonal $D=\text{diag}(d_1,\ldots,d_n)$	$d_1,\ldots,d_n$ (the diagonal entries)
Triangular	Diagonal entries
Symmetric ( $A=A^\top$ )	All real
Orthogonal ( $A^\top A=I$ )	All have $\lvert\lambda\rvert=1$
Projection ( $A^2=A$ )	Only $0$ and $1$
Positive definite	All $\lambda>0$

Note

Symmetric matrices always have real eigenvalues and orthogonal eigenvectors — a key reason covariance matrices (which are symmetric positive semi-definite) are so tractable in multivariate statistics.

06 — Quant Application — PCA and Covariance Spectra

Principal Component Analysis (PCA) is purely the eigendecomposition of the covariance matrix $\Sigma$ .

Given $p$ assets with covariance matrix $\Sigma$ (symmetric, positive semi-definite):

Find eigenvalues $\lambda_1\geq\lambda_2\geq\cdots\geq\lambda_p\geq0$ and corresponding orthonormal eigenvectors $\mathbf{v}_1,\ldots,\mathbf{v}_p$ .
The $k$ -th principal component is the portfolio $\mathbf{w}=\mathbf{v}_k$ ; its variance is $\lambda_k$ .
The first PC explains $\lambda_1/\sum_i\lambda_i$ of total variance.

In fixed-income PCA, the first three PCs of yield-curve moves are almost universally interpreted as level ( $\lambda_1\approx80\%$ ), slope ( $\lambda_2$ ), and curvature ( $\lambda_3$ ).

A near-zero eigenvalue of $\Sigma$ signals that a linear combination of assets is nearly riskless — useful for detecting near-arbitrage or near-multicollinear factors.

Exercises

EXERCISE 6.1

Form $A-\lambda I$ , compute $\det(A-\lambda I)=0$ , solve the resulting quadratic. Verify using $\text{tr}=\lambda_1+\lambda_2$ and $\det=\lambda_1\lambda_2$ .

$A=\begin{pmatrix}6&2\\1&5\end{pmatrix}$ .

$p(\lambda)=\det\begin{pmatrix}6-\lambda&2\\1&5-\lambda\end{pmatrix}=(6-\lambda)(5-\lambda)-2=(30-11\lambda+\lambda^2)-2=\lambda^2-11\lambda+28$ .

$\lambda^2-11\lambda+28=(\lambda-7)(\lambda-4)=0 \Rightarrow \lambda_1=7,\,\lambda_2=4$ .

Check: $\text{tr}(A)=11=7+4\,\checkmark$ ; $\det(A)=30-2=28=7\cdot4\,\checkmark$ .

Find the eigenvalues of $\begin{pmatrix}6&2\\1&5\end{pmatrix}$ .

EXERCISE 6.2

For each eigenvalue $\lambda_k$ found in 6.1, solve $(A-\lambda_k I)\mathbf{v}=\mathbf{0}$ by row reduction. Express the eigenspace as a span.

Eigenvalue $\lambda=7$ : $A-7I=\begin{pmatrix}-1&2\\1&-2\end{pmatrix}$ . $R_2\leftarrow R_2+R_1$ : $\begin{pmatrix}-1&2\\0&0\end{pmatrix}$ . So $-v_1+2v_2=0 \Rightarrow v_1=2v_2$ . $E_7=\text{span}\!\left\{\begin{pmatrix}2\\1\end{pmatrix}\right\}$ .

Eigenvalue $\lambda=4$ : $A-4I=\begin{pmatrix}2&2\\1&1\end{pmatrix}$ . $R_2\leftarrow R_2-\tfrac{1}{2}R_1$ : $\begin{pmatrix}2&2\\0&0\end{pmatrix}$ . So $2v_1+2v_2=0 \Rightarrow v_1=-v_2$ . $E_4=\text{span}\!\left\{\begin{pmatrix}-1\\1\end{pmatrix}\right\}$ .

Verify: $A\begin{pmatrix}2\\1\end{pmatrix}=\begin{pmatrix}12+2\\2+5\end{pmatrix}=\begin{pmatrix}14\\7\end{pmatrix}=7\begin{pmatrix}2\\1\end{pmatrix}\,\checkmark$ .

Find the eigenvectors of $\begin{pmatrix}6&2\\1&5\end{pmatrix}$ for each eigenvalue found in Exercise 6.1.

EXERCISE 6.3

A triangular matrix has eigenvalues equal to its diagonal entries. No computation needed — just read them off.

$T=\begin{pmatrix}3&7&2\\0&-1&5\\0&0&4\end{pmatrix}$ is upper triangular.

Eigenvalues: $\lambda_1=3$ , $\lambda_2=-1$ , $\lambda_3=4$ .

Check: $\text{tr}(T)=6=3+(-1)+4\,\checkmark$ ; $\det(T)=3\cdot(-1)\cdot4=-12=(3)(-1)(4)\,\checkmark$ .

State the eigenvalues of $\begin{pmatrix}3&7&2\\0&-1&5\\0&0&4\end{pmatrix}$ and justify without full computation.

EXERCISE 6.4

Use the trace and determinant relations: $\lambda_1+\lambda_2=\text{tr}(A)$ and $\lambda_1\lambda_2=\det(A)$ . Also recall that for positive definite matrices all eigenvalues are positive.

$\Sigma=\begin{pmatrix}4&2\\2&3\end{pmatrix}$ .

$\text{tr}(\Sigma)=7=\lambda_1+\lambda_2$ ; $\det(\Sigma)=12-4=8=\lambda_1\lambda_2$ .

Characteristic equation: $\lambda^2-7\lambda+8=0$ . $\lambda=\frac{7\pm\sqrt{49-32}}{2}=\frac{7\pm\sqrt{17}}{2}$ .

$\lambda_1=\frac{7+\sqrt{17}}{2}\approx5.56$ ; $\lambda_2=\frac{7-\sqrt{17}}{2}\approx1.44$ .

Both positive $\Rightarrow$ $\Sigma$ is positive definite. $\lambda_1/(\lambda_1+\lambda_2)\approx79\%$ of variance is explained by the first PC.

A covariance matrix for two assets is $\Sigma=\begin{pmatrix}4&2\\2&3\end{pmatrix}$ . Find the eigenvalues and determine the percentage of total variance explained by the first principal component.

EXERCISE 6.5

A projection satisfies $A^2=A$ . If $A\mathbf{v}=\lambda\mathbf{v}$ , apply $A$ again: $A^2\mathbf{v}=A(\lambda\mathbf{v})=\lambda^2\mathbf{v}$ . But $A^2\mathbf{v}=A\mathbf{v}=\lambda\mathbf{v}$ . What does that tell you about $\lambda$ ?

If $A^2=A$ and $A\mathbf{v}=\lambda\mathbf{v}$ , then $A^2\mathbf{v}=\lambda^2\mathbf{v}$ (apply $A$ once more).

But $A^2\mathbf{v}=A\mathbf{v}=\lambda\mathbf{v}$ .

So $\lambda^2\mathbf{v}=\lambda\mathbf{v}$ . Since $\mathbf{v}\neq\mathbf{0}$ : $\lambda^2=\lambda \Rightarrow \lambda(\lambda-1)=0 \Rightarrow \lambda\in\{0,1\}$ .

Geometrically: vectors in the image of the projection are fixed ( $\lambda=1$ ); vectors in the kernel are mapped to zero ( $\lambda=0$ ).

Prove that the only eigenvalues of a projection matrix ( $A^2=A$ ) are $0$ and $1$ .

EXERCISE 6.6

The covariance matrix of returns $\Sigma$ has eigenvalues equal to the variances of the principal components. The condition number $\kappa=\lambda_{\max}/\lambda_{\min}$ measures near-singularity. When $\lambda_{\min}\approx0$ , a linear combination of assets has near-zero variance.

Given eigenvalues $\lambda_1=12$ , $\lambda_2=3$ , $\lambda_3=0.1$ .

Total variance: $12+3+0.1=15.1$ .

PC1 explains $12/15.1\approx79.5\%$ ; PC2 explains $3/15.1\approx19.9\%$ ; PC3 explains $0.1/15.1\approx0.66\%$ .

Condition number: $\kappa=\lambda_1/\lambda_3=12/0.1=120$ . A condition number of $120$ means the portfolio of assets corresponding to $\mathbf{v}_3$ has variance $0.1$ — near-riskless relative to the dominant risk factor.

Quant implication: the factor $\mathbf{v}_3$ (the third PC) is a near-arbitrage combination. A long-short portfolio along $\mathbf{v}_3$ has very low residual risk and could be a mean-reversion candidate.

A three-asset covariance matrix has eigenvalues $12,\,3,\,0.1$ . Compute the percentage of variance explained by each principal component and interpret the smallest eigenvalue in the context of statistical arbitrage.

Chapter Summary

Concept	Formula / Rule
Eigenvalue equation	$A\mathbf{v}=\lambda\mathbf{v}$ , $\mathbf{v}\neq\mathbf{0}$
Characteristic polynomial	$p(\lambda)=\det(A-\lambda I)$
Eigenvalues	Roots of $p(\lambda)=0$
Eigenspace	$E_\lambda=\ker(A-\lambda I)$
Trace = sum of eigenvalues	$\text{tr}(A)=\sum_i\lambda_i$
Determinant = product of eigenvalues	$\det(A)=\prod_i\lambda_i$
Algebraic multiplicity	Multiplicity as root of $p(\lambda)$
Geometric multiplicity	$\dim\ker(A-\lambda I)$
Defective matrix	$m_g<m_a$ for some $\lambda$
PCA connection	Eigenvectors of $\Sigma$ = principal components; eigenvalues = component variances

Next chapter: Chapter 07 — Diagonalization, where we factor $A=PDP^{-1}$ using eigenvectors as columns of $P$ and eigenvalues on the diagonal of $D$ .