by, Subhajit Gorai
Date: 20/07/2026

Linear Algebra

Preface

This treatise is written for a reader who already possesses fluency with abstract algebraic structures — groups, rings, fields — and with the grammar of proof-writing (direct proof, contradiction, induction, contrapositive). It does not re-derive what a field is, nor does it pause to motivate why linear algebra matters; the reader is assumed to already believe that. Instead, the emphasis throughout is on:

  1. Structural completeness — every definition is stated in full generality (over R\mathbb{R} or C\mathbb{C}, or an arbitrary field F\mathbb{F} where the distinction is immaterial), and every theorem is stated with its precise hypotheses.
  2. Exam-grade rigor — the proofs given are the ones you would be expected to reproduce, or at least sketch cogently, in an ISI/JAM/GATE written examination.
  3. The statistical undercurrent — wherever a piece of linear algebra has a direct statistical avatar (idempotents as projections in OLS, positive-definiteness as the admissibility condition for a covariance matrix, the pseudoinverse as the tool of generalized least squares), this is flagged as a Remark.

Where a fully general proof is genuinely outside the scope of an entrance-exam treatise (e.g. the existence of Jordan Canonical Form over C\mathbb{C}), the theorem is stated precisely, its proof is outlined with the key lemmas named, and the reader is pointed to the standard references (Hoffman–Kunze, Horn & Johnson, Axler).

Work through the Challenge Problems seriously. They are calibrated to ISI M.Stat subjective-paper difficulty, not to routine JAM objective-paper difficulty — if a Challenge Problem takes you forty minutes and two false starts, that is exactly the intended experience.


Table of Contents

  • Conventions and Notation
  • Part 1 — Vector Spaces and Linear Transformations
    • 1.1 Vector Spaces and Subspaces
    • 1.2 Span, Independence, Basis, Dimension
    • 1.3 Linear Transformations and Matrix Representation
    • 1.4 The Four Fundamental Subspaces and Rank–Nullity
    • 1.5 Challenge Problems
  • Part 2 — Determinant, Trace, Rank, and Inverse
    • 2.1 Rank Revisited: Sylvester's Inequality
    • 2.2 Trace
    • 2.3 Determinants
    • 2.4 The Inverse, Adjugate, and Matrix Identities
    • 2.5 Problem-Solving Tricks
    • 2.6 Challenge Problems
  • Part 3 — Systems of Linear Equations
    • 3.1 Matrix Formulation and Consistency
    • 3.2 The Rouché–Capelli Theorem
    • 3.3 Structure of the Solution Set
    • 3.4 The Moore–Penrose Pseudoinverse
    • 3.5 Challenge Problems
  • Part 4 — Eigenvalues, Eigenvectors, and Diagonalization
    • 4.1 Eigenvalues, Eigenvectors, Characteristic Polynomial
    • 4.2 Multiplicities and Diagonalizability
    • 4.3 Cayley–Hamilton and the Minimal Polynomial
    • 4.4 Similarity Invariants
    • 4.5 Challenge Problems
  • Part 5 — Spectral Theory and Special Matrices
    • 5.1 Symmetric, Orthogonal, Idempotent, Nilpotent
    • 5.2 The Spectral Theorem
    • 5.3 Positive Definite and Semi-Definite Matrices
    • 5.4 Singular Value Decomposition
    • 5.5 Challenge Problems
  • Appendix: Master Formula Sheet
  • Closing Remark: The Unifying Theme

Conventions and Notation

  • F\mathbb{F} denotes a general field; where a statement requires R\mathbb{R} or C\mathbb{C} specifically, this is stated.
  • V,W,UV, W, U denote vector spaces over F\mathbb{F}; dimV\dim V denotes dimension.
  • Mm×n(F)M_{m\times n}(\mathbb{F}) is the space of m×nm\times n matrices over F\mathbb{F}; Mn(F):=Mn×n(F)M_n(\mathbb{F}) := M_{n\times n}(\mathbb{F}).
  • ATA^T denotes transpose; A=AˉTA^{*} = \bar{A}^T denotes conjugate transpose (Hermitian adjoint). Over R\mathbb{R}, A=ATA^* = A^T.
  • InI_n (or II) is the identity matrix; 00 is the zero matrix/vector, disambiguated by context.
  • rank(A)\text{rank}(A), tr(A)\text{tr}(A), det(A)\det(A), null(A)\text{null}(A) denote rank, trace, determinant, nullity.
  • C(A)\mathcal{C}(A): column space; R(A)\mathcal{R}(A): row space; N(A)\mathcal{N}(A): null space (kernel); N(AT)\mathcal{N}(A^T): left null space.
  • ,\langle \cdot,\cdot\rangle denotes an inner product; on Rn\mathbb{R}^n this is x,y=xTy\langle x,y\rangle = x^Ty unless otherwise stated.
  • A0A \succeq 0 means AA is positive semi-definite; A0A \succ 0 means positive definite. For symmetric A,BA,B, ABA\succeq B means AB0A-B\succeq0 (the Loewner order).
  • σ(A)\sigma(A) denotes the spectrum (multiset of eigenvalues) of AA.
  • We use span(S)\text{span}(S), dim(V)\dim(V), and VWV \oplus W (direct sum) in the standard sense.
  • All vector spaces considered are finite-dimensional unless explicitly stated otherwise.
  • \blacksquare closes a proof; \square closes a worked example.

Part 1 — Vector Spaces and Linear Transformations

1.1 Vector Spaces and Subspaces

Definition 1.1.1 (Vector Space).
A vector space over a field F\mathbb{F} is a set VV equipped with two operations, vector addition +:V×VV+: V\times V \to V and scalar multiplication :F×VV\cdot: \mathbb{F}\times V \to V, satisfying the following axioms for all u,v,wVu,v,w \in V and α,βF\alpha,\beta \in \mathbb{F}:

  1. (V,+)(V,+) is an abelian group: associativity, commutativity, existence of additive identity 0V0_V, existence of additive inverses.
  2. α(u+v)=αu+αv\alpha(u+v) = \alpha u + \alpha v (distributivity over vector addition).
  3. (α+β)v=αv+βv(\alpha+\beta)v = \alpha v + \beta v (distributivity over scalar addition).
  4. (αβ)v=α(βv)(\alpha\beta)v = \alpha(\beta v) (compatibility of scalar multiplication with field multiplication).
  5. 1v=v1\cdot v = v for the multiplicative identity 1F1 \in \mathbb{F}.

Remark. For F=R\mathbb{F} = \mathbb{R}, VV is called a real vector space; for F=C\mathbb{F} = \mathbb{C}, a complex vector space. In statistics, real vector spaces dominate (data lives in Rn\mathbb{R}^n), but complex vector spaces are indispensable in spectral theory, since a real matrix may have complex eigenvalues.

Definition 1.1.2 (Subspace).
A subset WVW \subseteq V is a subspace if WW is itself a vector space under the operations inherited from VV. Equivalently (and this is the operational criterion), WW is a subspace iff:

(i) 0VW0_V \in W;
(ii) WW is closed under addition: u,vW    u+vWu,v \in W \implies u+v \in W;
(iii) WW is closed under scalar multiplication: αF,vW    αvW\alpha \in \mathbb{F}, v\in W \implies \alpha v \in W.

(Conditions (ii) and (iii) can be compressed into a single closure-under-linear-combination condition.)

Theorem 1.1.3. If U,WU, W are subspaces of VV, then UWU \cap W is a subspace of VV. In general, UWU \cup W is not a subspace unless UWU \subseteq W or WUW \subseteq U.

Proof. 0VUW0_V \in U \cap W since 0VU0_V \in U and 0VW0_V \in W. Closure under addition and scalar multiplication in UWU \cap W follows immediately from closure in each of UU and WW separately. For the second claim: suppose neither UWU \subseteq W nor WUW \subseteq U. Pick uUWu \in U \setminus W and wWUw \in W \setminus U. If u+wUWu + w \in U \cup W, then either u+wUu+w \in U, forcing w=(u+w)uUw = (u+w) - u \in U — contradiction — or u+wWu + w \in W, forcing uWu \in W by the symmetric argument — contradiction. Hence u+wUWu+w \notin U\cup W, so UWU \cup W fails closure. \blacksquare

Definition 1.1.4 (Sum of Subspaces). For subspaces U,WVU, W \subseteq V, the sum is
U+W:={u+w:uU, wW}.U + W := \{u + w : u \in U,\ w \in W\}.
This is the smallest subspace of VV containing UWU \cup W. If UW={0}U \cap W = \{0\}, the sum is called a direct sum, written UWU \oplus W, and every vUWv \in U \oplus W has a unique decomposition v=u+wv = u + w.


1.2 Span, Independence, Basis, Dimension

Definition 1.2.1 (Linear Span). For S={v1,,vk}VS = \{v_1,\dots,v_k\} \subseteq V,
span(S):={i=1kαivi:αiF}.\text{span}(S) := \left\{\sum_{i=1}^k \alpha_i v_i : \alpha_i \in \mathbb{F}\right\}.
This is a subspace of VV — indeed, the intersection of all subspaces containing SS.

Definition 1.2.2 (Linear Independence). S={v1,,vk}S = \{v_1,\dots,v_k\} is linearly independent if
i=1kαivi=0    α1==αk=0.\sum_{i=1}^k \alpha_i v_i = 0 \implies \alpha_1 = \cdots = \alpha_k = 0.
Otherwise SS is linearly dependent: some vjv_j is expressible as a linear combination of the others.

Definition 1.2.3 (Basis). BV\mathcal{B} \subseteq V is a basis if B\mathcal{B} is linearly independent and span(B)=V\text{span}(\mathcal{B}) = V.

Theorem 1.2.4 (Steinitz Exchange Lemma). Let {v1,,vm}\{v_1,\dots,v_m\} be linearly independent in VV and let {w1,,wn}\{w_1,\dots,w_n\} span VV. Then mnm \le n, and mm of the wiw_i can be replaced by the viv_i to yield a spanning set.

Proof sketch. Induct on mm. Since {wi}\{w_i\} spans VV, write v1=ciwiv_1 = \sum c_i w_i; some cj0c_j \ne 0 (as v10v_1 \ne 0 by independence), so we may swap wjw_j out for v1v_1, and the resulting set {v1,w1,,wj^,,wn}\{v_1, w_1,\dots,\widehat{w_j},\dots,w_n\} still spans VV. Repeat this exchange for v2,,vmv_2,\dots,v_m; independence of the viv_i guarantees at each stage that some ww-vector remains available to eject (if none did, the current vkv_k would already lie in the span of the previously-inserted v1,,vk1v_1,\dots,v_{k-1}, contradicting independence), forcing mnm \le n. \blacksquare

Corollary 1.2.5 (Well-Definedness of Dimension). Any two bases of VV have the same cardinality. This common cardinality is called dim(V)\dim(V).

Theorem 1.2.6 (Dimension Formula for Sums).
dim(U+W)=dim(U)+dim(W)dim(UW).\dim(U+W) = \dim(U) + \dim(W) - \dim(U\cap W).

(Full proof given in §1.5, Challenge Problem 1 — this is the flagship result of Part 1.)

Trick 1.2.7. To check linear independence of kk vectors in Fn\mathbb{F}^n quickly, stack them as rows/columns of a matrix and row-reduce; independence     \iff rank equals kk. For abstract vector spaces (e.g. polynomial spaces), evaluate at kk well-chosen points or differentiate, turning the question into a Wronskian/determinant computation.

Example 1.2.8. In P3(R)P_3(\mathbb{R}) (polynomials of degree 3\le 3), are 1,x,x21,x3+x1, x, x^2 - 1, x^3 + x linearly independent? Express in the monomial basis coordinates (1,0,0,0),(0,1,0,0),(1,0,1,0),(0,1,0,1)(1,0,0,0), (0,1,0,0), (-1,0,1,0), (0,1,0,1), stack as rows, row reduce:
(1000010010100101)R3R3+R1, R4R4R2(1000010000100001),\begin{pmatrix} 1&0&0&0\\0&1&0&0\\-1&0&1&0\\0&1&0&1\end{pmatrix} \xrightarrow{R_3 \to R_3+R_1,\ R_4\to R_4 - R_2} \begin{pmatrix}1&0&0&0\\0&1&0&0\\0&0&1&0\\0&0&0&1\end{pmatrix},
rank 44, so the four vectors are independent (and hence a basis of the 44-dimensional space P3(R)P_3(\mathbb{R})). \square


1.3 Linear Transformations and Matrix Representation

Definition 1.3.1 (Linear Transformation). T:VWT: V \to W is linear if T(αu+βv)=αT(u)+βT(v)T(\alpha u + \beta v) = \alpha T(u) + \beta T(v) for all u,vV, α,βFu,v\in V,\ \alpha,\beta\in\mathbb{F}.

Definition 1.3.2 (Matrix Representation). Fix ordered bases B=(v1,,vn)\mathcal{B} = (v_1,\dots,v_n) of VV and C=(w1,,wm)\mathcal{C} = (w_1,\dots,w_m) of WW. The matrix representation [T]CBMm×n(F)[T]_{\mathcal{C}}^{\mathcal{B}} \in M_{m\times n}(\mathbb{F}) has jj-th column equal to the C\mathcal{C}-coordinate vector of T(vj)T(v_j). Then for any vVv \in V,
[T(v)]C=[T]CB[v]B.[T(v)]_{\mathcal{C}} = [T]_{\mathcal{C}}^{\mathcal{B}} [v]_{\mathcal{B}}.

Theorem 1.3.3 (Change of Basis). Let B,B\mathcal{B}, \mathcal{B}' be two bases of VV and let PP be the change-of-basis matrix whose columns are the B\mathcal{B}-coordinates of the B\mathcal{B}'-basis vectors (so [v]B=P[v]B[v]_{\mathcal{B}} = P[v]_{\mathcal{B}'} for every vVv \in V). If T:VVT: V \to V is linear with matrix A=[T]BBA = [T]_{\mathcal{B}}^{\mathcal{B}}, then
[T]BB=P1AP.[T]_{\mathcal{B}'}^{\mathcal{B}'} = P^{-1} A P.
That is, matrices representing the same linear operator in different bases are similar.

Proof. Since [w]B=P[w]B[w]_{\mathcal{B}} = P[w]_{\mathcal{B}'} holds for every wVw \in V, equivalently [w]B=P1[w]B[w]_{\mathcal{B}'} = P^{-1}[w]_{\mathcal{B}} for every ww. Apply this to w=T(v)w = T(v):
[T(v)]B=P1[T(v)]B=P1A[v]B=P1AP[v]B.[T(v)]_{\mathcal{B}'} = P^{-1}[T(v)]_{\mathcal{B}} = P^{-1}A[v]_{\mathcal{B}} = P^{-1}AP[v]_{\mathcal{B}'}.
Since this holds for all vv, the matrix P1APP^{-1}AP represents TT in basis B\mathcal{B}'. \blacksquare

Remark. Similarity is the correct notion of "sameness" for linear operators — it will recur throughout Part 4, since eigenvalues, trace, determinant, rank, and the characteristic polynomial are all similarity invariants. It is also the abstract statement behind principal component analysis: choosing a new orthonormal basis (the eigenbasis of a covariance matrix Σ\Sigma) is nothing but a change of basis of exactly this kind, engineered so that Σ\Sigma's representation in the new basis is diagonal (§5.2).


1.4 The Four Fundamental Subspaces and Rank–Nullity

For AMm×n(F)A \in M_{m\times n}(\mathbb{F}), viewed as a linear map FnFm\mathbb{F}^n \to \mathbb{F}^m:

Definition 1.4.1.

  • Column space C(A)={Ax:xFn}Fm\mathcal{C}(A) = \{Ax : x \in \mathbb{F}^n\} \subseteq \mathbb{F}^m — the range of AA.
  • Null space N(A)={xFn:Ax=0}Fn\mathcal{N}(A) = \{x \in \mathbb{F}^n : Ax = 0\} \subseteq \mathbb{F}^n — the kernel of AA.
  • Row space R(A)=C(AT)Fn\mathcal{R}(A) = \mathcal{C}(A^T) \subseteq \mathbb{F}^n.
  • Left null space N(AT)={yFm:ATy=0}Fm\mathcal{N}(A^T) = \{y \in \mathbb{F}^m : A^Ty = 0\} \subseteq \mathbb{F}^m.

Theorem 1.4.2 (Row Rank = Column Rank). For any AMm×n(F)A \in M_{m\times n}(\mathbb{F}), dimC(A)=dimR(A)\dim\mathcal{C}(A) = \dim\mathcal{R}(A). This common value is rank(A)\text{rank}(A).

Proof. Row-reduce AA to row echelon form R=EAR = EA (EE invertible, a product of elementary matrices). Row operations preserve the row space (each row of RR is a linear combination of rows of AA and vice versa via E1E^{-1}), so dimR(A)=dimR(R)=\dim\mathcal{R}(A) = \dim\mathcal{R}(R) = number of nonzero (pivot) rows of RR. Also, the pivot columns of RR correspond exactly to a maximal linearly independent subset of columns of AA (since xExx \mapsto Ex preserves linear dependence relations among columns, EE being invertible), giving dimC(A)=\dim\mathcal{C}(A) = number of pivot columns == number of pivot rows. \blacksquare

Theorem 1.4.3 (Rank–Nullity Theorem). For a linear map T:VWT: V \to W with dimV=n<\dim V = n < \infty,
dim(kerT)+dim(im T)=n.\dim(\ker T) + \dim(\text{im } T) = n.
Equivalently, for AMm×n(F)A \in M_{m\times n}(\mathbb{F}): rank(A)+dimN(A)=n\text{rank}(A) + \dim\mathcal{N}(A) = n.

Proof. Let {u1,,uk}\{u_1,\dots,u_k\} be a basis of kerT\ker T (k=dimkerTk = \dim\ker T). Extend to a basis {u1,,uk,v1,,vnk}\{u_1,\dots,u_k,v_1,\dots,v_{n-k}\} of VV. Claim: {T(v1),,T(vnk)}\{T(v_1),\dots,T(v_{n-k})\} is a basis of im(T)\text{im}(T).

Spanning: any T(v)=T(aiui+bjvj)=bjT(vj)T(v) = T(\sum a_iu_i + \sum b_j v_j) = \sum b_j T(v_j) since T(ui)=0T(u_i)=0.

Independence: if bjT(vj)=0\sum b_j T(v_j) = 0, then bjvjkerT\sum b_j v_j \in \ker T, so bjvj=aiui\sum b_j v_j = \sum a_i u_i for some aia_i, i.e. bjvjaiui=0\sum b_jv_j - \sum a_iu_i = 0; independence of the extended basis forces all ai=bj=0a_i = b_j = 0.

Hence dim(im T)=nk\dim(\text{im }T) = n-k, giving k+(nk)=nk + (n-k) = n. \blacksquare

Theorem 1.4.4 (Orthogonality of the Fundamental Subspaces, over R\mathbb{R}).
R(A)N(A)withR(A)N(A)=Rn,\mathcal{R}(A) \perp \mathcal{N}(A) \quad\text{with}\quad \mathcal{R}(A) \oplus \mathcal{N}(A) = \mathbb{R}^n,
C(A)N(AT)withC(A)N(AT)=Rm.\mathcal{C}(A) \perp \mathcal{N}(A^T) \quad\text{with}\quad \mathcal{C}(A) \oplus \mathcal{N}(A^T) = \mathbb{R}^m.

Proof. If xN(A)x \in \mathcal{N}(A) and y=ATzR(A)y = A^Tz \in \mathcal{R}(A), then x,y=xTATz=(Ax)Tz=0Tz=0\langle x, y\rangle = x^TA^Tz = (Ax)^Tz = 0^Tz = 0. So R(A)N(A)\mathcal{R}(A) \perp \mathcal{N}(A). By Theorem 1.4.3 applied to A:RnRmA: \mathbb R^n \to \mathbb R^m, rank(A)+dimN(A)=n\text{rank}(A) + \dim\mathcal N(A) = n; by Theorem 1.4.2, dimR(A)=rank(A)\dim\mathcal R(A) = \text{rank}(A), so dimR(A)+dimN(A)=n\dim\mathcal{R}(A)+\dim\mathcal N(A) = n. Orthogonality forces R(A)N(A)={0}\mathcal R(A) \cap \mathcal N(A) = \{0\} (any vector in the intersection is orthogonal to itself, hence zero), so this dimension count forces R(A)N(A)=Rn\mathcal{R}(A) \oplus \mathcal{N}(A) = \mathbb{R}^n. The second pair follows by applying the first pair to ATA^T (noting R(AT)=C(A)\mathcal{R}(A^T) = \mathcal{C}(A) by definition). \blacksquare

Remark (Statistical Avatar). This theorem is the abstract source of a fact every regression student memorizes without always seeing why it is true. In ordinary least squares with design matrix XX, the residual vector e=yXβ^e = y - X\hat\beta satisfies the normal equations XTe=0X^Te = 0, i.e. eN(XT)e \in \mathcal{N}(X^T) — the left null space of XX — while the fitted values y^=Xβ^\hat y = X\hat\beta lie in C(X)\mathcal{C}(X) by construction. Theorem 1.4.4, applied with XX in place of AA, says precisely C(X)N(XT)\mathcal{C}(X)\perp\mathcal N(X^T): the residual vector is orthogonal to the fitted values, and to every regressor, automatically — not a special virtue of least squares to be separately verified, but a direct instance of an orthogonality that holds for any matrix XX. Least squares merely chooses β^\hat\beta so that y^\hat y is the point of C(X)\mathcal C(X) closest to yy; orthogonality of the resulting residual is then forced by this theorem.

Trick 1.4.5. Rank–Nullity is the single most exam-tested identity in this subject. Any question of the form "given constraints on kerT\ker T or im T\text{im }T, find the other" collapses to bookkeeping via this theorem. Combined with rank(A)min(m,n)\text{rank}(A) \le \min(m,n), it immediately yields: if n>mn > m (more unknowns than equations), N(A){0}\mathcal{N}(A) \ne \{0\}, i.e. Ax=0Ax=0 has a nontrivial solution.


1.5 Challenge Problems

Challenge Problem 1. (Flagship result — full proof required.) Let U,WU, W be subspaces of a finite-dimensional vector space VV. Prove that
dim(U+W)=dim(U)+dim(W)dim(UW).\dim(U+W) = \dim(U) + \dim(W) - \dim(U\cap W).

Proof. Let dim(UW)=k\dim(U \cap W) = k, dimU=p\dim U = p, dimW=q\dim W = q. Choose a basis {x1,,xk}\{x_1,\dots,x_k\} of UWU \cap W. Extend it to a basis {x1,,xk,u1,,upk}\{x_1,\dots,x_k,u_1,\dots,u_{p-k}\} of UU, and separately extend it to a basis {x1,,xk,w1,,wqk}\{x_1,\dots,x_k,w_1,\dots,w_{q-k}\} of WW.

Claim: S={x1,,xk,u1,,upk,w1,,wqk}\mathcal{S} = \{x_1,\dots,x_k,u_1,\dots,u_{p-k},w_1,\dots,w_{q-k}\} is a basis of U+WU+W.

Spanning: Immediate, since every element of UU is a combination of {xi,uj}\{x_i,u_j\} and every element of WW is a combination of {xi,wl}\{x_i, w_l\}, and U+WU+W is spanned by UWU \cup W.

Independence: Suppose
i=1kaixi+j=1pkbjuj+l=1qkclwl=0.()\sum_{i=1}^k a_ix_i + \sum_{j=1}^{p-k} b_ju_j + \sum_{l=1}^{q-k} c_lw_l = 0. \quad (\star)
Rearranging, clwl=aixibjujU\sum c_l w_l = -\sum a_ix_i - \sum b_ju_j \in U (right side is a combination of UU-basis vectors). But clwlW\sum c_lw_l \in W too, so clwlUW\sum c_l w_l \in U \cap W. Hence clwl=dixi\sum c_lw_l = \sum d_ix_i for some scalars did_i, i.e.
clwldixi=0.\sum c_lw_l - \sum d_ix_i = 0.
Since {xi,wl}\{x_i, w_l\} is a basis of WW (hence independent), all cl=0c_l = 0 and all di=0d_i = 0. Substituting cl=0c_l=0 back into ()(\star) gives aixi+bjuj=0\sum a_ix_i + \sum b_ju_j = 0; independence of the basis of UU forces all ai=bj=0a_i = b_j = 0. Hence ()(\star) has only the trivial solution, proving independence.

Therefore dim(U+W)=S=k+(pk)+(qk)=p+qk=dimU+dimWdim(UW)\dim(U+W) = |\mathcal{S}| = k + (p-k) + (q-k) = p+q-k = \dim U + \dim W - \dim(U\cap W). \blacksquare

Challenge Problem 2. Let T:VVT: V \to V be a linear operator on a finite-dimensional space with T2=TT^2 = T (idempotent, i.e. a projection — see also §5.1). Prove that V=kerTim(T)V = \ker T \oplus \text{im}(T).

Proof outline. For vVv \in V, write v=(vTv)+Tvv = (v - Tv) + Tv. Check Tvim(T)Tv \in \text{im}(T) trivially, and T(vTv)=TvT2v=TvTv=0T(v-Tv) = Tv - T^2v = Tv - Tv = 0 so vTvkerTv - Tv \in \ker T; this gives V=kerT+im(T)V = \ker T + \text{im}(T). For directness, suppose xkerTim(T)x \in \ker T \cap \text{im}(T); then x=Tyx = Ty for some yy, and Tx=0Tx = 0, so Tx=T2y=Ty=xTx = T^2y = Ty = x, forcing x=0x = 0. Hence the sum is direct. \blacksquare (This result reappears with statistical content in §5.1 and the Closing Remark.)

Challenge Problem 3 (ISI-style, abstract operator). Let VV be a finite-dimensional vector space and T:VVT: V \to V linear with rank(T)=rank(T2)\text{rank}(T) = \text{rank}(T^2). Prove that kerTim(T)={0}\ker T \cap \text{im}(T) = \{0\}, and hence V=kerTim(T)V = \ker T \oplus \text{im}(T).

Proof outline. By Rank–Nullity applied to TT restricted to im(T)\text{im}(T): consider Tim(T):im(T)im(T)T|_{\text{im}(T)}: \text{im}(T) \to \text{im}(T), well-defined since T(im T)im(T2)im(T)T(\text{im }T) \subseteq \text{im}(T^2)\subseteq \text{im}(T). Its image is im(T2)\text{im}(T^2), which by hypothesis has the same dimension as im(T)\text{im}(T) (the domain of this restricted map), forcing the restricted map to be injective (Rank–Nullity on the restriction: dimker(TimT)=dim(im T)dim(im T2)=0\dim\ker(T|_{\text{im}T}) = \dim(\text{im }T) - \dim(\text{im }T^2) = 0). But ker(TimT)=kerTim(T)\ker(T|_{\text{im}T}) = \ker T \cap \text{im}(T), giving the result. Directness of the sum then follows from dimension counting via Rank–Nullity, as in Challenge Problem 2. \blacksquare

Challenge Problem 4 (A trap for the unwary — conceptual, not computational). Students who have just proved dim(U+W)=dimU+dimWdim(UW)\dim(U+W)=\dim U+\dim W-\dim(U\cap W) for two subspaces frequently conjecture the analogous inclusion–exclusion formula for three subspaces U1,U2,U3VU_1,U_2,U_3\subseteq V:
dim(U1+U2+U3)=?idimUii<jdim(UiUj)+dim(U1U2U3).()\dim(U_1+U_2+U_3) \stackrel{?}{=} \sum_i \dim U_i - \sum_{i<j}\dim(U_i\cap U_j) + \dim(U_1\cap U_2\cap U_3). \quad (\dagger)
Show by an explicit low-dimensional counterexample that ()(\dagger) is false in general, and identify precisely which step of the two-subspace proof fails to generalize.

Solution. Take V=R2V=\mathbb R^2 and
U1=span{(1,0)},U2=span{(0,1)},U3=span{(1,1)}.U_1 = \text{span}\{(1,0)\},\qquad U_2=\text{span}\{(0,1)\},\qquad U_3=\text{span}\{(1,1)\}.
Each UiU_i is 11-dimensional, and U1+U2+U3=R2U_1+U_2+U_3=\mathbb R^2 (already U1+U2=R2U_1+U_2=\mathbb R^2), so dim(U1+U2+U3)=2\dim(U_1+U_2+U_3)=2.

Every pairwise intersection is trivial: U1U2={0}U_1\cap U_2=\{0\} since (1,0),(0,1)(1,0),(0,1) are independent; U1U3={0}U_1\cap U_3=\{0\} since (1,0),(1,1)(1,0),(1,1) are independent; U2U3={0}U_2\cap U_3=\{0\} since (0,1),(1,1)(0,1),(1,1) are independent. Consequently U1U2U3={0}U_1\cap U_2\cap U_3=\{0\} as well. Substituting into the right side of ()(\dagger):
idimUii<jdim(UiUj)+dim(U1U2U3)=(1+1+1)(0+0+0)+0=32=dim(U1+U2+U3).\sum_i\dim U_i - \sum_{i<j}\dim(U_i\cap U_j)+\dim(U_1\cap U_2\cap U_3) = (1+1+1)-(0+0+0)+0 = 3 \ne 2 = \dim(U_1+U_2+U_3).
So ()(\dagger) overcounts by exactly 11, and is false.

Where the two-subspace proof breaks down. The proof of Challenge Problem 1 rests on the modular law dim(U+W)+dim(UW)=dimU+dimW\dim(U+W)+\dim(U\cap W)=\dim U+\dim W, which holds for any two subspaces. Iterating it naively for three subspaces implicitly assumes the subspace lattice is distributive — that \cap distributes over ++, i.e. U1(U2+U3)=(U1U2)+(U1U3)U_1\cap(U_2+U_3)=(U_1\cap U_2)+(U_1\cap U_3) — because that is what licenses expanding a three-term inclusion–exclusion the way it expands for sets (where \cap genuinely does distribute over \cup). The lattice of subspaces of a vector space of dimension 2\ge 2 is modular but not distributive, and the counterexample above exhibits the failure directly: U1(U2+U3)=U1R2=U1U_1\cap(U_2+U_3)=U_1\cap\mathbb R^2=U_1 (dimension 11), while (U1U2)+(U1U3)={0}+{0}={0}(U_1\cap U_2)+(U_1\cap U_3)=\{0\}+\{0\}=\{0\} (dimension 00) — the two sides disagree. This is the precise mechanism by which ()(\dagger) fails: pairwise (and triple) intersection dimensions alone do not determine dim(U1+U2+U3)\dim(U_1+U_2+U_3), because they do not encode how the intersections sit relative to one another inside the sum. What does survive to any number of subspaces is the weaker subadditivity inequality
dim(U1++Uk)i=1kdimUi,\dim(U_1+\cdots+U_k) \le \sum_{i=1}^k \dim U_i,
obtained by inducting on the two-subspace formula and discarding the (nonnegative) intersection term at each stage — this is the fact used, for instance, in Challenge Problem 2 of §2.6. \blacksquare


Part 2 — Determinant, Trace, Rank, and Inverse

2.1 Rank Revisited: Sylvester's Inequality

Theorem 2.1.1 (Sub-additivity/Sub-multiplicativity of Rank under Products). For AMm×n(F)A \in M_{m\times n}(\mathbb{F}), BMn×p(F)B\in M_{n\times p}(\mathbb{F}),
rank(A)+rank(B)nrank(AB)min(rank(A),rank(B)).\text{rank}(A) + \text{rank}(B) - n \le \text{rank}(AB) \le \min(\text{rank}(A), \text{rank}(B)).
The left inequality is Sylvester's Rank Inequality.

Proof.
Upper bound: C(AB)C(A)\mathcal{C}(AB) \subseteq \mathcal{C}(A) since every column of ABAB is AA times something, hence a combination of columns of AA; so rank(AB)rank(A)\text{rank}(AB) \le \text{rank}(A). Also rank(AB)=rank((AB)T)=rank(BTAT)rank(BT)=rank(B)\text{rank}(AB) = \text{rank}((AB)^T) = \text{rank}(B^TA^T) \le \text{rank}(B^T) = \text{rank}(B) by the same column-space argument applied to BTATB^TA^T.

Lower bound (Sylvester): Consider the linear map AC(B):C(B)FmA|_{\mathcal{C}(B)}: \mathcal{C}(B)\to \mathbb{F}^m, the restriction of xAxx \mapsto Ax to the subspace C(B)Fn\mathcal{C}(B) \subseteq \mathbb{F}^n. We have im(AC(B))=C(AB)\text{im}(A|_{\mathcal{C}(B)}) = \mathcal{C}(AB) and ker(AC(B))=C(B)N(A)\ker(A|_{\mathcal{C}(B)}) = \mathcal{C}(B)\cap \mathcal{N}(A), so by Rank–Nullity applied to this restricted map,
dimC(B)=dim(C(B)N(A))+rank(AB).\dim\mathcal{C}(B) = \dim(\mathcal{C}(B)\cap\mathcal{N}(A)) + \text{rank}(AB).
Now C(B)N(A)N(A)\mathcal{C}(B) \cap \mathcal{N}(A) \subseteq \mathcal{N}(A), so dim(C(B)N(A))dimN(A)=nrank(A)\dim(\mathcal{C}(B)\cap \mathcal{N}(A)) \le \dim\mathcal{N}(A) = n - \text{rank}(A) (Rank–Nullity for A:FnFmA: \mathbb{F}^n \to \mathbb{F}^m). Substituting,
rank(B)(nrank(A))+rank(AB)    rank(A)+rank(B)nrank(AB).\text{rank}(B) \le (n - \text{rank}(A)) + \text{rank}(AB) \implies \text{rank}(A)+\text{rank}(B) - n \le \text{rank}(AB). \blacksquare

Remark. Sylvester's inequality is the standard tool for bounding rank(Ak)\text{rank}(A^k) and for showing that products of "sufficiently full rank" matrices retain rank.


2.2 Trace

Definition 2.2.1. For AMn(F)A \in M_n(\mathbb{F}), tr(A):=i=1nAii\text{tr}(A) := \sum_{i=1}^n A_{ii}.

Theorem 2.2.2 (Cyclic Property). For AMm×n(F)A \in M_{m\times n}(\mathbb{F}), BMn×m(F)B \in M_{n\times m}(\mathbb{F}): tr(AB)=tr(BA)\text{tr}(AB) = \text{tr}(BA).

Proof. tr(AB)=i=1m(AB)ii=i=1mj=1nAijBji=j=1ni=1mBjiAij=j=1n(BA)jj=tr(BA)\text{tr}(AB) = \sum_{i=1}^m (AB)_{ii} = \sum_{i=1}^m\sum_{j=1}^n A_{ij}B_{ji} = \sum_{j=1}^n \sum_{i=1}^m B_{ji}A_{ij} = \sum_{j=1}^n (BA)_{jj} = \text{tr}(BA). \blacksquare

Corollary 2.2.3 (Invariance under Similarity). If A=P1APA' = P^{-1}AP for invertible PP, then tr(A)=tr(A)\text{tr}(A') = \text{tr}(A).

Proof. Apply the cyclic property with X=P1X=P^{-1}, Y=APY=AP: tr(P1AP)=tr(APP1)=tr(A)\text{tr}(P^{-1}\cdot AP) = \text{tr}(AP\cdot P^{-1}) = \text{tr}(A). \blacksquare

Remark. Trace is therefore a similarity invariant, and (Part 4) equals the sum of eigenvalues (with algebraic multiplicity) — this is often the fastest route to eigenvalue sum without computing the characteristic polynomial explicitly. It also underlies a fact used implicitly in PCA: since a change to an orthonormal eigenbasis is a similarity transformation (indeed an orthogonal one), tr(Σ)\text{tr}(\Sigma) — the total variance iVar(Xi)\sum_i\text{Var}(X_i) — is exactly preserved by rotating into principal components; PCA redistributes variance across new coordinate axes without creating or destroying any of it.

Trick 2.2.4. tr(AAT)=i,jAij2=AF20\text{tr}(AA^T) = \sum_{i,j}A_{ij}^2 = \|A\|_F^2 \ge 0, with equality iff A=0A = 0. This is the standard trick for proving a matrix is zero: show tr(AAT)=0\text{tr}(AA^T) = 0; equivalently, for a vector, Ax2=xTATAx=0    Ax=0\|Ax\|^2 = x^TA^TAx = 0 \iff Ax=0.


2.3 Determinants

Definition 2.3.1 (Axiomatic/Multilinear Definition). det:Mn(F)F\det: M_n(\mathbb{F}) \to \mathbb{F} is the unique function that is (i) multilinear in the rows, (ii) alternating (equal rows     \implies determinant 00), and (iii) normalized so det(I)=1\det(I) = 1. Equivalently, via the Leibniz formula,
det(A)=σSnsgn(σ)i=1nAi,σ(i).\det(A) = \sum_{\sigma \in S_n} \text{sgn}(\sigma)\prod_{i=1}^n A_{i,\sigma(i)}.

Theorem 2.3.2 (Core Properties).
(a) det(AB)=det(A)det(B)\det(AB) = \det(A)\det(B) (multiplicativity).
(b) det(AT)=det(A)\det(A^T) = \det(A).
(c) AA is invertible     det(A)0\iff \det(A) \ne 0.
(d) Row/column swap negates det\det; scaling a row by cc scales det\det by cc; adding a multiple of one row to another leaves det\det unchanged.
(e) det(cA)=cndet(A)\det(cA) = c^n\det(A) for AMn(F)A \in M_n(\mathbb{F}).

(These follow directly from the multilinear-alternating axioms; multiplicativity (a) is most cleanly proved via the fact that both sides are, as functions of AA for fixed invertible BB, alternating multilinear functions of the rows of AA agreeing at A=IA=I, hence equal by uniqueness of the alternating multilinear form up to scalar — or via elementary matrix factorization of AA.)

Theorem 2.3.3 (Cauchy–Binet Formula). For AMm×n(F)A \in M_{m\times n}(\mathbb{F}), BMn×m(F)B \in M_{n\times m}(\mathbb{F}) with mnm \le n,
det(AB)=S([n]m)det(A:,S)det(BS,:),\det(AB) = \sum_{S \in \binom{[n]}{m}} \det(A_{:,S})\det(B_{S,:}),
where the sum is over all mm-element subsets SS of {1,,n}\{1,\dots,n\}, A:,SA_{:,S} is the m×mm\times m submatrix of AA formed by columns indexed by SS, and BS,:B_{S,:} is the m×mm\times m submatrix of BB formed by rows indexed by SS.

Remark. Cauchy–Binet generalizes multiplicativity of determinants to non-square factors, and specializes (taking B=ATB = A^T, so BS,:=(A:,S)TB_{S,:}=(A_{:,S})^T) to det(AAT)=Sdet(A:,S)20\det(AA^T) = \sum_S \det(A_{:,S})^2 \ge 0 — an important positivity fact used repeatedly in Part 5.

Theorem 2.3.4 (Block Matrix Determinants). For a block matrix M=(ABCD)M = \begin{pmatrix}A & B\\ C& D\end{pmatrix} with AA square and invertible,
det(M)=det(A)det(DCA1B).\det(M) = \det(A)\det(D - CA^{-1}B).
If instead DD is square and invertible,
det(M)=det(D)det(ABD1C).\det(M) = \det(D)\det(A - BD^{-1}C).
In particular, if B=0B=0 or C=0C=0 (block-triangular), det(M)=det(A)det(D)\det(M) = \det(A)\det(D).

Proof. Use the block factorization
(ABCD)=(I0CA1I)(A00DCA1B)(IA1B0I).\begin{pmatrix}A&B\\C&D\end{pmatrix} = \begin{pmatrix}I&0\\CA^{-1}&I\end{pmatrix}\begin{pmatrix}A&0\\0&D-CA^{-1}B\end{pmatrix}\begin{pmatrix}I&A^{-1}B\\0&I\end{pmatrix}.
The two outer factors are unit lower/upper block-triangular, hence have determinant 11 (triangular with 11's on the diagonal blocks). Taking determinants of both sides and using multiplicativity and the block-triangular case gives the result. The second formula is symmetric (pivot on DD instead). \blacksquare

The matrix DCA1BD - CA^{-1}B (or ABD1CA - BD^{-1}C) is called the Schur complement of AA (resp. DD) in MM; it reappears throughout §2.4–§2.6 and again in §5.3.

Corollary 2.3.5 (Matrix Determinant Lemma). For AMn(F)A\in M_n(\mathbb F) invertible and u,vFnu,v\in\mathbb F^n,
det(A+uvT)=det(A)(1+vTA1u).\det(A+uv^T) = \det(A)\left(1+v^TA^{-1}u\right).
In particular, det(In+uvT)=1+vTu\det(I_n+uv^T)=1+v^Tu.

Proof. Consider the block matrix N=(AuvT1)Mn+1(F)N = \begin{pmatrix}A & u \\ -v^T & 1\end{pmatrix} \in M_{n+1}(\mathbb{F}). Applying Theorem 2.3.4 with the invertible top-left block AA:
det(N)=det(A)det ⁣(1(vT)A1u)=det(A)(1+vTA1u).\det(N) = \det(A)\cdot\det\!\left(1-(-v^T)A^{-1}u\right) = \det(A)(1+v^TA^{-1}u).
Applying Theorem 2.3.4 instead with the invertible bottom-right 1×11\times1 block:
det(N)=det(1)det ⁣(Au11(vT))=det(A+uvT).\det(N) = \det(1)\cdot\det\!\left(A-u\cdot1^{-1}\cdot(-v^T)\right) = \det(A+uv^T).
Equating the two evaluations of det(N)\det(N) gives the claim; setting A=InA=I_n gives the stated special case. \blacksquare

Remark. This is the determinant-side counterpart of the Sherman–Morrison formula (Theorem 2.4.3 below): both arise from viewing a rank-one perturbation through the same bordered (n+1)×(n+1)(n+1)\times(n+1) matrix NN, evaluated via its two Schur complements. It also explains where Sherman–Morrison's denominator comes from: 1+vTA1u1+v^TA^{-1}u is not an arbitrary normalizing constant, it is literally det(A+uvT)/det(A)\det(A+uv^T)/\det(A) — so it vanishes exactly when A+uvTA+uv^T is singular, consistent with Sherman–Morrison's invertibility hypothesis.


2.4 The Inverse, Adjugate, and Matrix Identities

Definition 2.4.1 (Adjugate). For AMn(F)A \in M_n(\mathbb{F}), the adjugate (classical adjoint) adj(A)\text{adj}(A) is the transpose of the cofactor matrix: adj(A)ij=Cji=(1)i+jMji\text{adj}(A)_{ij} = C_{ji} = (-1)^{i+j}M_{ji}, where MjiM_{ji} is the (j,i)(j,i) minor of AA.

Theorem 2.4.2. Aadj(A)=adj(A)A=det(A)IA\cdot\text{adj}(A) = \text{adj}(A)\cdot A = \det(A)I. Consequently, if det(A)0\det(A)\ne 0,
A1=1det(A)adj(A).A^{-1} = \frac{1}{\det(A)}\text{adj}(A).

Proof. (Aadj(A))ii=jAij(adj A)ji=jAijCij=det(A)(A\cdot\text{adj}(A))_{ii} = \sum_j A_{ij}(\text{adj }A)_{ji} = \sum_j A_{ij}C_{ij} = \det(A) by cofactor expansion along row ii. For iki \ne k, (Aadj(A))ik=jAijCkj(A\cdot\text{adj}(A))_{ik} = \sum_j A_{ij}C_{kj}, which is the cofactor expansion of the determinant of the matrix obtained from AA by replacing row kk with a copy of row ii — a matrix with a repeated row, hence determinant 00. \blacksquare

Theorem 2.4.3 (Sherman–Morrison Formula). For invertible AMn(F)A \in M_n(\mathbb{F}) and u,vFnu,v \in \mathbb{F}^n with 1+vTA1u01 + v^TA^{-1}u \ne 0,
(A+uvT)1=A1A1uvTA11+vTA1u.(A + uv^T)^{-1} = A^{-1} - \frac{A^{-1}uv^TA^{-1}}{1+v^TA^{-1}u}.

Proof. Direct verification: multiply the RHS by (A+uvT)(A+uv^T) and simplify using the scalar s:=vTA1us := v^TA^{-1}u:
(A1A1uvTA11+s)(A+uvT)=I+A1uvTA1uvTA1A+A1u(vTA1u)vT1+s=I+A1uvT(1+s)A1uvT1+s=I.\left(A^{-1} - \tfrac{A^{-1}uv^TA^{-1}}{1+s}\right)(A+uv^T) = I + A^{-1}uv^T - \tfrac{A^{-1}uv^TA^{-1}A + A^{-1}u(v^TA^{-1}u)v^T}{1+s} = I + A^{-1}uv^T - \tfrac{(1+s)A^{-1}uv^T}{1+s} = I. \blacksquare

Theorem 2.4.4 (Woodbury Identity — matrix generalization). For conformable A,U,C,VA, U, C, V with A,CA, C invertible:
(A+UCV)1=A1A1U(C1+VA1U)1VA1.(A + UCV)^{-1} = A^{-1} - A^{-1}U(C^{-1}+VA^{-1}U)^{-1}VA^{-1}.

Remark. Sherman–Morrison/Woodbury is the workhorse for rank-one/low-rank updates: recomputing a regression's inverse-covariance after adding one observation (an O(n2)O(n^2) update rather than an O(n3)O(n^3) re-inversion) is a direct application. Running this in reverse — removing rather than adding an observation — is exactly how closed-form leave-one-out diagnostics in regression (the "hat value" hiih_{ii}, the PRESS statistic, Cook's distance) are derived without literally refitting the model nn times: deleting row ii of XX is a rank-one downdate of XTXX^TX, and Sherman–Morrison converts an O(n3)O(n^3) refit into an O(p2)O(p^2) correction.


2.5 Problem-Solving Tricks

Trick 2.5.1 (Rank Factorization). Any AMm×n(F)A \in M_{m\times n}(\mathbb{F}) with rank(A)=r\text{rank}(A) = r can be written A=BCA = BC with BMm×rB \in M_{m\times r}, CMr×nC \in M_{r\times n}, both of rank rr (take BB = a basis of C(A)\mathcal{C}(A) as columns, CC = the coefficients expressing each column of AA in that basis). This turns rank questions about AA into questions about smaller full-rank factors, and is the standard route to prove rank(A)=rank(ATA)=rank(AAT)\text{rank}(A) = \text{rank}(A^TA) = \text{rank}(AA^T) (over R\mathbb{R}), via N(ATA)=N(A)\mathcal N(A^TA)=\mathcal N(A) (Trick 2.2.4).

Trick 2.5.2 (Trace/Determinant for Nilpotency). AA is nilpotent (Ak=0A^k=0 for some kk)     \iff all eigenvalues of AA are 00     \iff tr(Aj)=0\text{tr}(A^j) = 0 for all j=1,,nj=1,\dots,n (Newton's identities relate power sums of eigenvalues to elementary symmetric polynomials, i.e. to coefficients of the characteristic polynomial). A quick necessary condition often sufficient for exam purposes: if A2=0A^2 = 0 then C(A)N(A)\mathcal{C}(A)\subseteq\mathcal{N}(A), hence rank(A)nrank(A)\text{rank}(A) \le n - \text{rank}(A), i.e. rank(A)n/2\text{rank}(A) \le n/2.

Trick 2.5.3 (Schur Complement for Block Inversion). To invert M=(ABCD)M = \begin{pmatrix}A&B\\C&D\end{pmatrix} with AA invertible, use the block-LDU factorization from Theorem 2.3.4 to get
M1=(A1+A1BS1CA1A1BS1S1CA1S1),S:=DCA1B.M^{-1} = \begin{pmatrix}A^{-1}+A^{-1}BS^{-1}CA^{-1} & -A^{-1}BS^{-1}\\ -S^{-1}CA^{-1} & S^{-1}\end{pmatrix}, \qquad S := D-CA^{-1}B.
This is the standard route for inverting a covariance matrix partitioned into blocks — central to conditional distributions of the multivariate normal.


2.6 Challenge Problems

Challenge Problem 1 (Block Determinant Identity). Let A,BMn(R)A,B \in M_n(\mathbb{R}) with AB=BAAB = BA (commuting). Prove that
det(ABBA)=det(A2+B2).\det\begin{pmatrix}A & B\\ -B& A\end{pmatrix} = \det(A^2+B^2).

Proof. Work in M2n(C)M_{2n}(\mathbb C) and set M=(ABBA)M=\begin{pmatrix}A&B\\-B&A\end{pmatrix}. Consider the 2n×2n2n\times2n matrix, in n×nn\times n blocks,
P=(IIiIiI).P = \begin{pmatrix}I&I\\iI&-iI\end{pmatrix}.
Direct block multiplication — using only that iIiI is a scalar matrix, so it commutes with every block; commutativity of A,BA,B is not needed for this step — gives
MP=(A+iBAiBiABiAB),P(A+iB00AiB)=(A+iBAiBiABiAB),MP = \begin{pmatrix}A+iB&A-iB\\iA-B&-iA-B\end{pmatrix}, \qquad P\begin{pmatrix}A+iB&0\\0&A-iB\end{pmatrix} = \begin{pmatrix}A+iB&A-iB\\iA-B&-iA-B\end{pmatrix},
and the two right-hand sides agree entry by entry, so
MP=P(A+iB00AiB).()MP = P\begin{pmatrix}A+iB&0\\0&A-iB\end{pmatrix}. \quad(\ast)
Now PP is invertible: by Theorem 2.3.4, pivoting on the invertible top-left block II,
detP=det(I)det(iIiII1I)=det(2iI)=(2i)n0.\det P = \det(I)\cdot\det\big(-iI-iI\cdot I^{-1}\cdot I\big) = \det(-2iI) = (-2i)^n \ne 0.
So ()(\ast) rearranges to M=P(A+iB00AiB)P1M = P\begin{pmatrix}A+iB&0\\0&A-iB\end{pmatrix}P^{-1}: MM is similar over C\mathbb C to diag(A+iB,AiB)\text{diag}(A+iB,A-iB). Similar matrices have equal determinants (Theorem 4.4.1, or directly from multiplicativity), so
det(M)=det(A+iB)det(AiB).()\det(M) = \det(A+iB)\det(A-iB). \quad(\ast\ast)
This identity holds for any square real A,BA,B — commutativity has not yet been used. It enters only in the final step, where AB=BAAB=BA permits
(A+iB)(AiB)=A2iAB+iBA+B2=A2+B2+i(BAAB)=A2+B2,(A+iB)(A-iB) = A^2-iAB+iBA+B^2 = A^2+B^2+i(BA-AB) = A^2+B^2,
so that det(A+iB)det(AiB)=det((A+iB)(AiB))=det(A2+B2)\det(A+iB)\det(A-iB)=\det\big((A+iB)(A-iB)\big)=\det(A^2+B^2) by multiplicativity once more. Combining with ()(\ast\ast):
det(M)=det(A2+B2).\det(M) = \det(A^2+B^2). \qquad\blacksquare

Remark. The intermediate identity ()(\ast\ast) is worth isolating: it holds unconditionally and is the determinant of the real representation of the complex matrix Z=A+iBZ=A+iB acting on CnR2n\mathbb C^n\cong\mathbb R^{2n}. When A,BA,B commute, AiB=A+iBA-iB=\overline{A+iB} (entrywise conjugate, as A,BA,B are real), so det(M)=det(Z)det(Z)=detZ20\det(M)=\det(Z)\overline{\det(Z)}=|\det Z|^2\ge0 — recovering positivity of det(A2+B2)\det(A^2+B^2) directly, without needing to know A2+B20A^2+B^2\succeq0 (which in fact requires more than mere commutativity to guarantee).

Challenge Problem 2 (Rank Bound for Sums). For A,BMm×n(F)A, B \in M_{m\times n}(\mathbb{F}), prove
rank(A)rank(B)rank(A+B)rank(A)+rank(B).|\text{rank}(A) - \text{rank}(B)| \le \text{rank}(A+B) \le \text{rank}(A)+\text{rank}(B).

Proof. Upper bound: C(A+B)C(A)+C(B)\mathcal{C}(A+B) \subseteq \mathcal{C}(A)+\mathcal{C}(B), and dim(C(A)+C(B))dimC(A)+dimC(B)\dim(\mathcal{C}(A)+\mathcal{C}(B)) \le \dim\mathcal{C}(A)+\dim\mathcal{C}(B) by Theorem 1.2.6 (dropping the intersection term). Lower bound: apply the upper bound to A=(A+B)+(B)A = (A+B) + (-B): rank(A)rank(A+B)+rank(B)\text{rank}(A) \le \text{rank}(A+B)+\text{rank}(B), i.e. rank(A)rank(B)rank(A+B)\text{rank}(A)-\text{rank}(B)\le\text{rank}(A+B); symmetrically for rank(B)rank(A)\text{rank}(B)-\text{rank}(A). \blacksquare

Challenge Problem 3 (Schur Complement Positivity — bridges to Part 5). Let M=(ABBTD)M = \begin{pmatrix}A&B\\B^T&D\end{pmatrix} be symmetric with A0A \succ 0. Prove that M0M \succeq 0 iff the Schur complement S=DBTA1B0S = D - B^TA^{-1}B \succeq 0.

Proof. By Theorem 2.3.4's factorization (pivoting on the invertible top-left block AA),
M=(I0BTA1I)(A00S)(IA1B0I).M = \begin{pmatrix}I&0\\B^TA^{-1}&I\end{pmatrix}\begin{pmatrix}A&0\\0&S\end{pmatrix}\begin{pmatrix}I&A^{-1}B\\0&I\end{pmatrix}.
Writing L=(I0BTA1I)L=\begin{pmatrix}I&0\\B^TA^{-1}&I\end{pmatrix}, the third factor is exactly LTL^T (since (BTA1)T=A1B(B^TA^{-1})^T=A^{-1}B, using A1A^{-1} symmetric). So
M=L(A00S)LT,()M = L\begin{pmatrix}A&0\\0&S\end{pmatrix}L^T, \quad(\star)
a congruence: MM and diag(A,S)\text{diag}(A,S) are congruent via the invertible matrix LL (unit lower-triangular, detL=1\det L=1).

For x=(x1x2)x=\binom{x_1}{x_2}, set y=LTx=(y1y2)y=L^Tx=\binom{y_1}{y_2}; since LTL^T is invertible, xyx\mapsto y is a bijection. By ()(\star),
xTMx=yT(A00S)y=y1TAy1+y2TSy2.x^TMx = y^T\begin{pmatrix}A&0\\0&S\end{pmatrix}y = y_1^TAy_1+y_2^TSy_2.

(\Leftarrow) If S0S\succeq0: since A0A\succ0, y1TAy10y_1^TAy_1\ge0 always and y2TSy20y_2^TSy_2\ge0 by hypothesis, so xTMx0x^TMx\ge0 for every xx, i.e. M0M\succeq0.

(\Rightarrow) If M0M\succeq0: given any y2Rmy_2\in\mathbb R^m, since LT=(IA1B0I)L^T=\begin{pmatrix}I&A^{-1}B\\0&I\end{pmatrix} is invertible, there is a unique xx with LTx=(0y2)L^Tx=\binom{0}{y_2} — namely x2=y2x_2=y_2, x1=A1By2x_1=-A^{-1}By_2. For this xx, xTMx=0TA0+y2TSy2=y2TSy20x^TMx=0^TA0+y_2^TSy_2=y_2^TSy_2\ge0 (since M0M\succeq0). As y2y_2 was arbitrary, S0S\succeq0. \blacksquare

Remark (Statistical Payoff). This is precisely the tool for checking that a partitioned covariance matrix is admissible: if Σ=(Σ11Σ12Σ21Σ22)\Sigma=\begin{pmatrix}\Sigma_{11}&\Sigma_{12}\\\Sigma_{21}&\Sigma_{22}\end{pmatrix} is the covariance matrix of (X1,X2)(X_1,X_2) jointly, with Σ110\Sigma_{11}\succ0, then M0M\succeq0 automatically (it is a genuine covariance matrix), and this theorem tells you the conditional covariance matrix of X2X1X_2\mid X_1 in the multivariate normal model — which is exactly the Schur complement Σ22Σ21Σ111Σ12\Sigma_{22}-\Sigma_{21}\Sigma_{11}^{-1}\Sigma_{12} — is itself a bona fide (PSD) covariance matrix. Conditioning can only shrink uncertainty (in the Loewner order — see Challenge Problem 4 of §5.5), never produce a nonsensical negative-variance direction, and the Schur complement is the precise algebraic reason why.

Challenge Problem 4 (Rank Additivity via the Schur Complement). Let M=(ABBTD)Mn+m(R)M=\begin{pmatrix}A&B\\B^T&D\end{pmatrix}\in M_{n+m}(\mathbb R) be symmetric with DMm(R)D\in M_m(\mathbb R) invertible. Prove that
rank(M)=rank(D)+rank ⁣(ABD1BT).\text{rank}(M) = \text{rank}(D) + \text{rank}\!\left(A-BD^{-1}B^T\right).

Proof. Mirror the factorization of Theorem 2.3.4, pivoting instead on the invertible bottom-right block DD:
M=(IBD10I)(ABD1BT00D)(I0D1BTI).M = \begin{pmatrix}I&BD^{-1}\\0&I\end{pmatrix}\begin{pmatrix}A-BD^{-1}B^T&0\\0&D\end{pmatrix}\begin{pmatrix}I&0\\D^{-1}B^T&I\end{pmatrix}.
(Verified by the identical block-multiplication check used to prove Theorem 2.3.4, with the two pivot blocks' roles exchanged; expanding the right side reproduces MM entry by entry, using D1D=ID^{-1}D=I to cancel the off-diagonal blocks.) Both outer factors are unit-triangular, hence invertible. Left- or right-multiplying by an invertible matrix does not change rank (it carries a maximal independent set of columns to another maximal independent set), so
rank(M)=rank(ABD1BT00D).\text{rank}(M) = \text{rank}\begin{pmatrix}A-BD^{-1}B^T&0\\0&D\end{pmatrix}.
For a block-diagonal matrix diag(X,Y)\text{diag}(X,Y), columns supported in the "XX-rows" and those supported in the "YY-rows" cannot participate in a common linear dependency (any dependency forces the XX-part and YY-part to vanish separately), so a maximal independent set of columns is the concatenation of maximal independent sets from each block: rankdiag(X,Y)=rank(X)+rank(Y)\text{rank}\,\text{diag}(X,Y)=\text{rank}(X)+\text{rank}(Y). Hence
rank(M)=rank(ABD1BT)+rank(D).\text{rank}(M) = \text{rank}(A-BD^{-1}B^T)+\text{rank}(D). \qquad\blacksquare

Remark. This is a rank-additivity identity for partitioned symmetric matrices, and it is the rank-theoretic sibling of the determinant and definiteness statements already proved for the same Schur complement (Theorem 2.3.4, Challenge Problem 3). It has a clean statistical use: for a design X=[1X1]X=[\mathbf 1\mid X_1] (intercept plus covariates), partition XTX=(n1TX1X1T1X1TX1)X^TX=\begin{pmatrix}n&\mathbf1^TX_1\\X_1^T\mathbf1&X_1^TX_1\end{pmatrix} this way and pivot on the invertible 1×11\times1 top-left block. The rank — and hence the identifiability — of the full regression reduces to the rank of the Schur complement X1TX11n(X1T1)(1TX1)X_1^TX_1-\tfrac1n(X_1^T\mathbf1)(\mathbf1^TX_1), which is exactly nn times the sample covariance matrix of the columns of X1X_1. This is the algebraic seed of the Frisch–Waugh–Lovell theorem: the covariates' contribution to the model, after partialling out the intercept, is governed entirely by their own centered cross-product matrix.


Part 3 — Systems of Linear Equations

3.1 Matrix Formulation and Consistency

Definition 3.1.1. A linear system in nn unknowns and mm equations is written Ax=bAx = b, AMm×n(F)A \in M_{m\times n}(\mathbb{F}), xFnx \in \mathbb{F}^n, bFmb \in \mathbb{F}^m. It is homogeneous if b=0b=0, else non-homogeneous. The system is consistent if a solution exists.

Remark. Ax=bAx=b is consistent     bC(A)\iff b \in \mathcal{C}(A) — a restatement of the definition of column space, but the operational move of the whole section is to turn this membership question into a rank comparison.


3.2 The Rouché–Capelli Theorem

Definition 3.2.1 (Augmented Matrix). [Ab]Mm×(n+1)(F)[A\,|\,b] \in M_{m\times(n+1)}(\mathbb{F}) is AA with bb appended as an extra column.

Theorem 3.2.2 (Rouché–Capelli / Kronecker–Capelli). The system Ax=bAx=b is consistent iff
rank(A)=rank([Ab]).\text{rank}(A) = \text{rank}([A\,|\,b]).
Moreover, if consistent:

  • If rank(A)=n\text{rank}(A) = n (number of unknowns), the solution is unique.
  • If rank(A)=r<n\text{rank}(A) = r < n, the solution set is an affine subspace of dimension nrn-r (a (nr)(n-r)-parameter family).

Proof. (\Rightarrow) If Ax=bAx=b has a solution, then bC(A)b \in \mathcal{C}(A), so appending bb as a column adds no new dimension to the column space: rank([Ab])=dimC(A)=rank(A)\text{rank}([A|b]) = \dim\mathcal{C}(A) = \text{rank}(A).

(\Leftarrow) If rank(A)=rank([Ab])=r\text{rank}(A) = \text{rank}([A|b]) = r, then since C(A)C([Ab])\mathcal{C}(A) \subseteq \mathcal{C}([A|b]) and both have dimension rr, we get C(A)=C([Ab])\mathcal{C}(A) = \mathcal{C}([A|b]), so bC([Ab])=C(A)b \in \mathcal{C}([A|b]) = \mathcal{C}(A), i.e. the system is consistent.

Uniqueness/parametrization: If x0x_0 is one solution, the general solution is x0+N(A)x_0 + \mathcal{N}(A) (verify: A(x0+z)=b    Az=0A(x_0+z) = b \iff Az=0). By Rank–Nullity, dimN(A)=nr\dim\mathcal{N}(A) = n - r. If r=nr=n, N(A)={0}\mathcal{N}(A)=\{0\}, giving uniqueness; else the solution set is a coset of the (nr)(n-r)-dimensional subspace N(A)\mathcal{N}(A). \blacksquare


3.3 Structure of the Solution Set

Theorem 3.3.1 (Superposition/Affine Structure). The general solution of a consistent system Ax=bAx=b is
x=xp+xh,xp any particular solution, xhN(A).x = x_p + x_h, \qquad x_p \text{ any particular solution},\ x_h \in \mathcal{N}(A).

Remark (Geometry). The solution set is a translate of the null space, i.e. an affine subspace parallel to N(A)\mathcal{N}(A). The "degrees of freedom" of the system is exactly dimN(A)=nrank(A)\dim\mathcal{N}(A) = n - \text{rank}(A) — the number of free variables remaining after row-reduction. This is precisely the notion of degrees of freedom that resurfaces in ANOVA decompositions and the residual degrees of freedom in linear regression.

Trick 3.3.2. To parametrize N(A)\mathcal{N}(A) practically: row-reduce AA to reduced row echelon form, identify free columns (non-pivot), and for each free variable set it to 11 and the rest to 00, solving for the pivot variables — this generates a basis for N(A)\mathcal{N}(A) directly.


3.4 The Moore–Penrose Pseudoinverse

Definition 3.4.1. For AMm×n(R)A \in M_{m\times n}(\mathbb{R}) (possibly non-square, possibly rank-deficient), the Moore–Penrose pseudoinverse A+Mn×m(R)A^+ \in M_{n\times m}(\mathbb{R}) is the unique matrix satisfying the four Penrose conditions:

  1. AA+A=AAA^+A = A
  2. A+AA+=A+A^+AA^+ = A^+
  3. (AA+)T=AA+(AA^+)^T = AA^+
  4. (A+A)T=A+A(A^+A)^T = A^+A

Theorem 3.4.2 (Existence and Uniqueness). A+A^+ exists and is unique for every AMm×n(R)A \in M_{m\times n}(\mathbb{R}).

(Proof outline: existence is constructive via the SVD, A=UΣVT    A+=VΣ+UTA = U\Sigma V^T \implies A^+ = V\Sigma^+U^T where Σ+\Sigma^+ inverts the nonzero singular values in place — see §5.4. Uniqueness follows by a direct algebraic argument from the four Penrose conditions.)

Theorem 3.4.3 (Closed Forms in Full-Rank Cases).

  • If AA has full column rank (rank(A)=nm\text{rank}(A)=n \le m): A+=(ATA)1ATA^+ = (A^TA)^{-1}A^T (the "left inverse"; A+A=InA^+A = I_n).
  • If AA has full row rank (rank(A)=mn\text{rank}(A)=m\le n): A+=AT(AAT)1A^+ = A^T(AA^T)^{-1} (the "right inverse"; AA+=ImAA^+ = I_m).

Theorem 3.4.4 (Pseudoinverse Solves Least Squares). For possibly-inconsistent Ax=bAx=b, x=A+bx^* = A^+b is the minimum-norm least-squares solution: it minimizes Axb2\|Ax-b\|_2, and among all minimizers, it has the smallest x2\|x\|_2.

Remark. This is the crucial statistical avatar of this section: when AA (the design matrix) has full column rank, A+b=(ATA)1ATbA^+b = (A^TA)^{-1}A^Tb is exactly the OLS estimator β^\hat\beta. When ATAA^TA is singular (rank-deficient design, e.g. collinear regressors), the pseudoinverse still delivers a well-defined "solution" — the minimum-norm one — which is precisely what many software implementations of regression fall back to (see Challenge Problem 4 below for exactly what remains well-defined in that case).


3.5 Challenge Problems

Challenge Problem 1 (Parameterized Consistency). For which values of λR\lambda \in \mathbb{R} does the system
x+y+λz=1,x+λy+z=1,λx+y+z=1x + y + \lambda z = 1,\qquad x+\lambda y + z = 1,\qquad \lambda x + y + z = 1
have (a) a unique solution, (b) infinitely many solutions, (c) no solution?

Solution. Write the system as Ax=bAx=b with
A=(11λ1λ1λ11),b=(111).A=\begin{pmatrix}1&1&\lambda\\1&\lambda&1\\\lambda&1&1\end{pmatrix}, \qquad b=\begin{pmatrix}1\\1\\1\end{pmatrix}.

Step 1: Compute detA\det A. Every row of AA sums to λ+2\lambda+2, so adding columns 22 and 33 into column 11 (C1C1+C2+C3C_1\to C_1+C_2+C_3) produces a common factor:
detA=det(λ+21λλ+2λ1λ+211)=(λ+2)det(11λ1λ1111).\det A = \det\begin{pmatrix}\lambda+2&1&\lambda\\\lambda+2&\lambda&1\\\lambda+2&1&1\end{pmatrix} = (\lambda+2)\det\begin{pmatrix}1&1&\lambda\\1&\lambda&1\\1&1&1\end{pmatrix}.
In the remaining determinant, subtract row 33 from rows 11 and 22 (R1R1R3R_1\to R_1-R_3, R2R2R3R_2\to R_2-R_3):
det(00λ10λ10111)=1(1)3+1det(0λ1λ10)=(λ1)2\det\begin{pmatrix}0&0&\lambda-1\\0&\lambda-1&0\\1&1&1\end{pmatrix} = 1\cdot(-1)^{3+1}\det\begin{pmatrix}0&\lambda-1\\\lambda-1&0\end{pmatrix} = -(\lambda-1)^2
(cofactor expansion along column 11, which has a single nonzero entry). Hence
detA=(λ+2)(λ1)2.\det A = -(\lambda+2)(\lambda-1)^2.
This vanishes exactly at λ=1\lambda=1 (a double root) and λ=2\lambda=-2 (a simple root); the leading minus sign does not affect the case analysis below (only the roots matter for that), but it is worth being explicit about — a bare detA=(λ+2)(λ1)2\det A = (\lambda+2)(\lambda-1)^2, missing the sign, is a common arithmetic slip and gives the wrong value if the determinant itself is ever needed (e.g. via Cramer's rule).

Step 2: Case analysis (Rouché–Capelli, Theorem 3.2.2).

Case λ1,2\lambda\ne1,-2: detA0\det A\ne0, so rank(A)=3=\text{rank}(A)=3= number of unknowns. Unique solution.

Case λ=1\lambda=1: AA becomes the all-ones matrix J3J_3, of rank 11. Every equation of the system reduces to x+y+z=1x+y+z=1, so [Ab][A\mid b] also has rank 11; the system is consistent with rank(A)=1<3\text{rank}(A)=1<3, giving a (31)=2(3-1)=2-parameter family. Infinitely many solutions.

Case λ=2\lambda=-2: Each row of AA sums to λ+2=0\lambda+2=0, i.e. A1=0A\mathbf 1=0 where 1=(1,1,1)T\mathbf1=(1,1,1)^T; since AA is symmetric, this simultaneously says 1\mathbf1 lies in the left null space, AT1=0A^T\mathbf1=0. A direct check of the leading 2×22\times2 minor, det(1112)=30\det\begin{pmatrix}1&1\\1&-2\end{pmatrix}=-3\ne0, confirms rank(A)=2\text{rank}(A)=2 exactly (not less). By the Fredholm alternative — Ax=bAx=b is consistent iff bb is orthogonal to every vector in N(AT)\mathcal N(A^T), which is precisely Theorem 1.4.4 (C(A)N(AT)\mathcal{C}(A)\perp\mathcal{N}(A^T), so bC(A)    bN(AT)b\in\mathcal C(A)\iff b\perp\mathcal N(A^T)) — consistency here requires 1Tb=0\mathbf1^Tb=0. But 1Tb=1+1+1=30\mathbf1^Tb=1+1+1=3\ne0. No solution. \blacksquare

Remark. The identification of 1\mathbf 1 as spanning N(AT)\mathcal N(A^T) at the critical value λ=2\lambda=-2, followed by the one-line check 1Tb0\mathbf1^Tb\ne0, is the Rouché–Capelli theorem in its most efficient operational form: rather than row-reducing the full augmented matrix, one only needs a spanning set for the left null space of AA (often visible by inspection, as here from the zero row-sum) and a single inner product against bb.

Challenge Problem 2 (Pseudoinverse Identity). Prove that for any AMm×n(R)A\in M_{m\times n}(\mathbb R),
A+=(ATA)+AT,A^+ = (A^TA)^+A^T,
and that this reduces to Theorem 3.4.3 when AA has full column rank.

Proof. Let A=UΣVTA=U\Sigma V^T be an SVD of AA (Theorem 5.4.1 — logically this problem sits most naturally after §5.4, but is placed here for thematic continuity with the pseudoinverse). Then
ATA=VΣTUTUΣVT=V(ΣTΣ)VT,A^TA = V\Sigma^TU^TU\Sigma V^T = V(\Sigma^T\Sigma)V^T,
using UTU=IU^TU=I. This exhibits ATAA^TA in exactly the spectral form of Theorem 5.2.1: an orthogonal matrix times a diagonal PSD matrix times its transpose. The pseudoinverse of a symmetric PSD matrix given in this form is obtained by inverting the nonzero diagonal entries in place:
(ATA)+=V(ΣTΣ)+VT.(A^TA)^+ = V(\Sigma^T\Sigma)^+V^T.
Hence
X:=(ATA)+AT=V(ΣTΣ)+VTVΣTUT=V(ΣTΣ)+ΣTUT,X:=(A^TA)^+A^T = V(\Sigma^T\Sigma)^+V^T\cdot V\Sigma^TU^T = V(\Sigma^T\Sigma)^+\Sigma^TU^T,
using VTV=IV^TV=I. It remains to check (ΣTΣ)+ΣT=Σ+(\Sigma^T\Sigma)^+\Sigma^T=\Sigma^+. Both sides are n×mn\times m and zero off the "diagonal," so compare the ii-th diagonal entry, 1imin(m,n)1\le i\le\min(m,n): if σi0\sigma_i\ne0, the left side gives 1σi2σi=1σi\frac{1}{\sigma_i^2}\cdot\sigma_i=\frac1{\sigma_i}, matching Σ+\Sigma^+'s convention; if σi=0\sigma_i=0, both sides give 00. Hence (ΣTΣ)+ΣT=Σ+(\Sigma^T\Sigma)^+\Sigma^T=\Sigma^+ exactly, and
X=VΣ+UT=A+.X = V\Sigma^+U^T = A^+.
This proves the identity for every AA, with no rank hypothesis.

When AA has full column rank (rank(A)=n\text{rank}(A)=n), all nn singular values are nonzero, so ATAA^TA is invertible and (ATA)+=(ATA)1(A^TA)^+=(A^TA)^{-1}, recovering A+=(ATA)1ATA^+=(A^TA)^{-1}A^T, i.e. Theorem 3.4.3. \blacksquare

Remark. This identity is the theoretical justification for computing OLS coefficients as β^=(XTX)1XTy\hat\beta=(X^TX)^{-1}X^Ty even when one thinks of the conceptually "right" object as X+yX^+y: the two expressions provably coincide whenever XX has full column rank, and the pseudoinverse route degrades gracefully — delivering the minimum-norm solution — exactly when (XTX)1(X^TX)^{-1} stops existing.

Challenge Problem 3 (Structural). Let AMm×n(R)A \in M_{m\times n}(\mathbb{R}). Prove that Ax=bAx=b has a solution for every bRmb \in \mathbb{R}^m iff rank(A)=m\text{rank}(A) = m (i.e. AA has full row rank), and relate this to the surjectivity of the linear map xAxx \mapsto Ax.

Proof. Immediate from Rouché–Capelli: consistency for all bb means rank(A)=rank([Ab])\text{rank}(A) = \text{rank}([A|b]) for every bb; taking bC(A)b \notin \mathcal{C}(A) would strictly increase the rank on the right unless C(A)=Rm\mathcal{C}(A) = \mathbb{R}^m already, i.e. dimC(A)=m\dim\mathcal{C}(A)=m, i.e. rank(A)=m\text{rank}(A)=m. This is exactly the statement that xAxx\mapsto Ax is onto Rm\mathbb{R}^m. \blacksquare

Challenge Problem 4 (Well-Posedness of Fitted Values Under Collinearity). Let XMn×p(R)X\in M_{n\times p}(\mathbb{R}) be a design matrix with rank(X)=r<p\text{rank}(X)=r<p (so the regressors are collinear and the normal equations XTXβ=XTyX^TX\beta=X^Ty do not pin down a unique β\beta). Prove that nevertheless every solution β^\hat\beta of the normal equations produces the same fitted-value vector Xβ^X\hat\beta, and that this common value equals HyHy for the hat matrix H=XX+H=XX^+.

Proof. Suppose β1,β2Rp\beta_1,\beta_2\in\mathbb R^p both satisfy the normal equations: XTXβ1=XTy=XTXβ2X^TX\beta_1=X^Ty=X^TX\beta_2. Then XTX(β1β2)=0X^TX(\beta_1-\beta_2)=0. Writing δ:=β1β2\delta:=\beta_1-\beta_2,
δTXTXδ=0    Xδ22=0    Xδ=0\delta^TX^TX\delta = 0 \iff \|X\delta\|_2^2=0 \iff X\delta = 0
(Trick 2.2.4, with XδX\delta in place of AA), so Xβ1=Xβ2X\beta_1=X\beta_2. The fitted-value vector is identical for every solution of the normal equations, regardless of the rank deficiency of XX.

To identify the common value: the minimum-norm solution β^0:=X+y\hat\beta_0:=X^+y (Theorem 3.4.4) satisfies the normal equations, since transposing the Penrose identity XX+X=XXX^+X=X and using symmetry of XX+XX^+ (Penrose condition 3) gives XTXX+=XTX^TXX^+=X^T, hence XTX(X+y)=XTyX^TX(X^+y)=X^Ty for every yy. Since every solution gives the same fitted value as β^0\hat\beta_0 does, and Xβ^0=XX+y=HyX\hat\beta_0=XX^+y=Hy, every solution β^\hat\beta satisfies
Xβ^=Hy.X\hat\beta = Hy. \qquad\blacksquare

Remark. This is the precise sense in which "OLS is well-defined even with collinear regressors": the coefficients β^\hat\beta are not identifiable (any two solutions differ by a vector in N(X)\mathcal N(X), itself nontrivial exactly when r<pr<p), but every quantity actually observable from the fit — fitted values, residuals, R2R^2, the hat matrix, predictions at points within C(X)\mathcal C(X) — is identical no matter which generalized-inverse solution one's software happens to return. This is why collinearity is diagnosed as a problem for interpreting individual coefficients, not as a problem for prediction.


Part 4 — Eigenvalues, Eigenvectors, and Diagonalization

4.1 Eigenvalues, Eigenvectors, Characteristic Polynomial

Definition 4.1.1. For AMn(F)A \in M_n(\mathbb{F}), λF\lambda \in \mathbb{F} (or an algebraic closure thereof) is an eigenvalue if there exists v0v \ne 0 with Av=λvAv = \lambda v; vv is the corresponding eigenvector. The eigenspace Eλ:=N(AλI)E_\lambda := \mathcal{N}(A-\lambda I).

Definition 4.1.2. The characteristic polynomial is pA(λ):=det(λIA)p_A(\lambda) := \det(\lambda I - A) (or det(AλI)\det(A-\lambda I), differing by sign (1)n(-1)^n by convention — both conventions appear across texts; we use det(λIA)\det(\lambda I - A), monic of degree nn). Eigenvalues are precisely the roots of pAp_A.

Theorem 4.1.3. pA(λ)=λntr(A)λn1++(1)ndet(A)p_A(\lambda) = \lambda^n - \text{tr}(A)\lambda^{n-1} + \cdots + (-1)^n\det(A). In particular, the sum of the eigenvalues (with algebraic multiplicity, over C\mathbb{C}) equals tr(A)\text{tr}(A), and their product equals det(A)\det(A).

(Follows from Vieta's formulas applied to the roots of pAp_A, using the expansion of the determinant det(λIA)\det(\lambda I - A); the coefficient of λnk\lambda^{n-k} is (1)kek(λ1,,λn)(-1)^ke_k(\lambda_1,\ldots,\lambda_n), where eke_k is the kk-th elementary symmetric polynomial in the eigenvalues.)


4.2 Multiplicities and Diagonalizability

Definition 4.2.1. The algebraic multiplicity a(λ)a(\lambda) of eigenvalue λ\lambda is its multiplicity as a root of pAp_A. The geometric multiplicity g(λ):=dimEλg(\lambda) := \dim E_\lambda.

Theorem 4.2.2. For every eigenvalue λ\lambda, 1g(λ)a(λ)1 \le g(\lambda) \le a(\lambda).

Proof. g(λ)1g(\lambda)\ge 1 trivially (eigenspace is nontrivial by definition of eigenvalue). For g(λ)a(λ)g(\lambda)\le a(\lambda): let {v1,,vg}\{v_1,\dots,v_g\} be a basis of EλE_\lambda, extend to a basis of Fn\mathbb{F}^n. In this basis, AA is similar to a block matrix (λIgB0C)\begin{pmatrix}\lambda I_g & B\\0&C\end{pmatrix} (since Avi=λviAv_i=\lambda v_i for igi\le g). Then pA(t)=det(tIA)=(tλ)gdet(tIngC)p_A(t) = \det(tI-A) = (t-\lambda)^g \det(tI_{n-g}-C) (block-triangular determinant), so (tλ)g(t-\lambda)^g divides pA(t)p_A(t), i.e. a(λ)ga(\lambda) \ge g. \blacksquare

Theorem 4.2.3 (Diagonalizability Criterion). AMn(F)A \in M_n(\mathbb{F}) is diagonalizable (similar to a diagonal matrix, over the splitting field of pAp_A) iff g(λ)=a(λ)g(\lambda) = a(\lambda) for every eigenvalue λ\lambda, iff Fn\mathbb{F}^n (or its algebraic closure) admits a basis of eigenvectors of AA.

Theorem 4.2.4 (Sufficient Condition). If AA has nn distinct eigenvalues, AA is diagonalizable. (Eigenvectors for distinct eigenvalues are automatically linearly independent — a standard induction argument: suppose a minimal dependent combination civi=0\sum c_iv_i=0 (all ci0c_i\ne0); apply AA, and separately multiply the original relation by λ1\lambda_1, then subtract, to get i2ci(λiλ1)vi=0\sum_{i\ge2}c_i(\lambda_i-\lambda_1)v_i=0, a strictly shorter nontrivial dependency (since λiλ1\lambda_i\ne\lambda_1, ci0c_i\ne0 for i2i\ge2), contradicting minimality unless no such dependency existed at all.)


4.3 Cayley–Hamilton and the Minimal Polynomial

Theorem 4.3.1 (Cayley–Hamilton). Every AMn(F)A \in M_n(\mathbb{F}) satisfies its own characteristic polynomial: pA(A)=0p_A(A) = 0.

Proof sketch (adjugate method). Let B(λ)=adj(λIA)B(\lambda) = \text{adj}(\lambda I - A), a matrix whose entries are polynomials in λ\lambda of degree n1\le n-1; write B(λ)=k=0n1BkλkB(\lambda) = \sum_{k=0}^{n-1}B_k\lambda^k for constant matrices BkB_k. From Theorem 2.4.2, (λIA)B(λ)=det(λIA)I=pA(λ)I(\lambda I-A)B(\lambda) = \det(\lambda I - A)I = p_A(\lambda)I. Expanding both sides as polynomials in λ\lambda with matrix coefficients and equating coefficients of like powers of λ\lambda gives a system relating each BkB_k to Bk1B_{k-1} and to the coefficients ckc_k of pAp_A; multiplying the λk\lambda^k-coefficient equation on the left by AkA^k and summing over k=0,,nk=0,\dots,n causes every BkB_k term to appear once with a ++ sign and once with a - sign, telescoping the entire left side to 00 while the right side sums to exactly kckAk=pA(A)\sum_kc_kA^k=p_A(A), yielding pA(A)=0p_A(A) = 0. \blacksquare (A fully rigorous write-up requires care in tracking the boundary terms of the telescoping sum — standard in Hoffman–Kunze §6.4 or Horn & Johnson §2.4; the outline above captures the mechanism expected on an exam.)

Definition 4.3.2. The minimal polynomial mA(λ)m_A(\lambda) is the monic polynomial of least degree with mA(A)=0m_A(A) = 0.

Theorem 4.3.3. mAm_A divides every polynomial qq with q(A)=0q(A)=0 (in particular mApAm_A\mid p_A, by Cayley–Hamilton), and mAm_A and pAp_A have exactly the same roots (possibly with different multiplicities).

Proof. Divisibility: if q(A)=0q(A)=0, polynomial division gives q=hmA+rq=hm_A+r with degr<degmA\deg r<\deg m_A (or r=0r=0). Then r(A)=q(A)h(A)mA(A)=0r(A)=q(A)-h(A)m_A(A)=0. If r0r\ne0, this contradicts minimality of degmA\deg m_A among nonzero annihilating polynomials; hence r=0r=0 and mAqm_A\mid q.

Eigenvalue \Rightarrow root of mAm_A: if Av=λvAv=\lambda v, v0v\ne0, then Akv=λkvA^kv=\lambda^kv for every k0k\ge0, so q(A)v=q(λ)vq(A)v=q(\lambda)v for every polynomial qq. In particular 0=mA(A)v=mA(λ)v0=m_A(A)v=m_A(\lambda)v; since v0v\ne0, mA(λ)=0m_A(\lambda)=0.

Root of mAm_A \Rightarrow eigenvalue: let μ\mu be a root of mAm_A, and write mA(λ)=(λμ)h(λ)m_A(\lambda)=(\lambda-\mu)h(\lambda), degh=degmA1\deg h=\deg m_A-1. Minimality forces h(A)0h(A)\ne0; pick ww with u:=h(A)w0u:=h(A)w\ne0. Then (AμI)u=(AμI)h(A)w=mA(A)w=0(A-\mu I)u=(A-\mu I)h(A)w=m_A(A)w=0, so u0u\ne0 is an eigenvector with eigenvalue μ\mu.

The roots of mAm_A are thus exactly the eigenvalues of AA, which are exactly the roots of pAp_A. \blacksquare

Theorem 4.3.4. AA is diagonalizable iff mAm_A has no repeated roots (i.e. mAm_A splits into distinct linear factors over the algebraic closure).

Proof. (\Rightarrow) Suppose AA is diagonalizable with distinct eigenvalues λ1,,λr\lambda_1,\ldots,\lambda_r. The polynomial q(λ):=i=1r(λλi)q(\lambda):=\prod_{i=1}^r(\lambda-\lambda_i) annihilates AA: in an eigenbasis, q(A)q(A) is diagonal, and every diagonal entry is q(λi)=0q(\lambda_i)=0 for the relevant eigenvalue λi\lambda_i; hence q(A)=0q(A)=0. By Theorem 4.3.3, mAqm_A\mid q, and a divisor of a polynomial with only simple roots has only simple roots; so mAm_A has no repeated roots (and, since every λi\lambda_i is also a root of mAm_A by Theorem 4.3.3, in fact mA=qm_A=q exactly).

(\Leftarrow) Suppose mA(λ)=i=1r(λλi)m_A(\lambda)=\prod_{i=1}^r(\lambda-\lambda_i) with λ1,,λr\lambda_1,\ldots,\lambda_r distinct. Define the Lagrange basis polynomials Pi(λ):=jiλλjλiλjP_i(\lambda):=\prod_{j\ne i}\frac{\lambda-\lambda_j}{\lambda_i-\lambda_j}. The polynomial iPi(λ)1\sum_iP_i(\lambda)-1 has degree r1\le r-1 but rr distinct roots λ1,,λr\lambda_1,\ldots,\lambda_r, forcing iPi1\sum_iP_i\equiv1 identically. Setting Ei:=Pi(A)E_i:=P_i(A) gives iEi=I\sum_iE_i=I. Using mA(A)=0m_A(A)=0 together with the partial-fraction structure of the PiP_i, one further checks Ei2=EiE_i^2=E_i, EiEj=0E_iE_j=0 for iji\ne j, and (AλiI)Ei=0(A-\lambda_iI)E_i=0, so each EiE_i is a projection onto (a subspace of) the λi\lambda_i-eigenspace; iEi=I\sum_iE_i=I then exhibits Fn\mathbb F^n (or its closure) as a direct sum of eigenspaces of AA, i.e. AA has a full eigenbasis. \blacksquare (The verification that each EiE_i is idempotent and mutually annihilating is a short computation from the partial-fraction identity for 1/mA(λ)1/m_A(\lambda); see Hoffman–Kunze §7.1 or Axler Ch. 8 for the complete construction.)

Remark. This gives a slick alternative diagonalizability test that avoids computing individual eigenspace dimensions: compute mAm_A (often via minimal-degree relations found by inspection, e.g. for idempotents/involutions/nilpotents where mAm_A is forced to divide λ(λ1)\lambda(\lambda-1), λ21\lambda^2-1, λk\lambda^k respectively) and check for repeated roots.


4.4 Similarity Invariants

Theorem 4.4.1. If ABA \sim B (similar, i.e. B=P1APB=P^{-1}AP), then A,BA,B share: characteristic polynomial, eigenvalues (with algebraic and geometric multiplicities), minimal polynomial, trace, determinant, rank.

(Each follows from direct substitution: e.g. pB(λ)=det(λIP1AP)=det(P1(λIA)P)=det(P)1pA(λ)det(P)=pA(λ)p_B(\lambda) = \det(\lambda I - P^{-1}AP) = \det(P^{-1}(\lambda I - A)P) = \det(P)^{-1}p_A(\lambda)\det(P) = p_A(\lambda), using λIP1AP=P1(λIA)P\lambda I - P^{-1}AP = P^{-1}(\lambda I - A)P.)

Remark (Jordan Canonical Form — stated without full proof). Over an algebraically closed field (e.g. C\mathbb{C}), every AMn(C)A \in M_n(\mathbb{C}) is similar to a Jordan canonical form — a block-diagonal matrix of Jordan blocks Jk(λ)=λIk+NkJ_k(\lambda) = \lambda I_k + N_k (NkN_k the nilpotent shift matrix with 11's on the superdiagonal). This is the finest similarity invariant: two matrices are similar iff they have the same Jordan form up to block reordering. The full existence/uniqueness proof (via the structure theorem for finitely generated modules over a PID, applied to F[λ]\mathbb{F}[\lambda] acting on Fn\mathbb{F}^n via AA) is beyond entrance-exam scope; what is exam-relevant is: (i) the number of Jordan blocks for eigenvalue λ\lambda equals g(λ)g(\lambda); (ii) the sizes of the blocks are determined by the ranks of powers of (AλI)(A-\lambda I); (iii) AA is diagonalizable iff every Jordan block has size 11, consistent with Theorem 4.3.4.


4.5 Challenge Problems

Challenge Problem 1 (Minimal Polynomial Trick). Let AMn(R)A \in M_n(\mathbb{R}) satisfy A3=AA^3 = A. Prove AA is diagonalizable (over R\mathbb{R}, in the sense of being similar to a diagonal matrix with real entries).

Proof. mA(λ)m_A(\lambda) divides λ3λ=λ(λ1)(λ+1)\lambda^3-\lambda = \lambda(\lambda-1)(\lambda+1), a product of distinct linear factors over R\mathbb{R}. Hence mAm_A itself is a product of a subset of these distinct linear factors, so mAm_A has no repeated roots; by Theorem 4.3.4, AA is diagonalizable, and since all roots of mAm_A lie in {1,0,1}R\{-1,0,1\}\subset\mathbb{R}, the diagonalization is over R\mathbb{R}. \blacksquare

Challenge Problem 2 (Patterned Matrix Diagonalizability). Let JnJ_n be the n×nn\times n all-ones matrix. Show that A=(ab)In+bJnA = (a-b)I_n + bJ_n (i.e. aa on the diagonal, bb off-diagonal) is diagonalizable for all a,bRa,b\in\mathbb{R}, and find its eigenvalues explicitly.

Proof. JnJ_n itself is symmetric, hence diagonalizable (Spectral Theorem, §5.2) with eigenvalues nn (multiplicity 1, eigenvector 1\mathbf{1}) and 00 (multiplicity n1n-1, eigenvectors spanning 1\mathbf{1}^\perp). Since AA is an affine function of JnJ_n (i.e. A=(ab)I+bJnA = (a-b)I + bJ_n), AA shares the same eigenvectors as JnJ_n, with eigenvalues (ab)+bn=a+(n1)b(a-b)+bn = a+(n-1)b (multiplicity 1) and (ab)+0=ab(a-b)+0 = a-b (multiplicity n1n-1). Since AA has an eigenbasis, AA is diagonalizable. \blacksquare (This matrix is exactly the equicorrelated covariance structure σ2[(1ρ)I+ρJ]\sigma^2[(1-\rho)I+\rho J] up to scaling — ubiquitous in exchangeable-random-variable models; see Challenge Problem 2 of §5.5.)

Challenge Problem 3 (Matrix Polynomial via Minimal Polynomial). Let AMn(C)A \in M_n(\mathbb{C}) with minimal polynomial mA(λ)=(λ1)(λ2)m_A(\lambda) = (\lambda-1)(\lambda-2). Compute A100A^{100} in terms of AA and II.

Proof. Since mAm_A has distinct roots 1,21,2, AA is diagonalizable with eigenvalues only 1,21,2. Perform polynomial division: λ100=q(λ)mA(λ)+(aλ+b)\lambda^{100} = q(\lambda)m_A(\lambda) + (a\lambda+b) for some remainder aλ+ba\lambda+b (degree <2<2). Evaluate at the roots: 1=a+b1 = a+b (from λ=1\lambda=1), 2100=2a+b2^{100}=2a+b (from λ=2\lambda=2). Solve: a=21001a = 2^{100}-1, b=22100b = 2-2^{100}. Since mA(A)=0m_A(A)=0, A100=aA+bI=(21001)A+(22100)IA^{100} = aA+bI = (2^{100}-1)A + (2-2^{100})I. \blacksquare (This "polynomial remainder via minimal polynomial" trick generalizes to any matrix function once mAm_A's roots are known, and is the fast route to computing high matrix powers without diagonalizing explicitly.)

Challenge Problem 4 (Simultaneous Diagonalization). Let A,BMn(F)A,B\in M_n(\mathbb{F}) satisfy AB=BAAB=BA, and suppose both AA and BB are diagonalizable. Prove that AA and BB are simultaneously diagonalizable: there is a single basis of Fn\mathbb{F}^n consisting of eigenvectors of both AA and BB.

Proof. Step 1: every eigenspace of AA is BB-invariant. Let λ\lambda be an eigenvalue of AA and vEλ(A)v\in E_\lambda(A), i.e. Av=λvAv=\lambda v. Then
A(Bv)=(AB)v=(BA)v=B(Av)=B(λv)=λ(Bv),A(Bv) = (AB)v = (BA)v = B(Av) = B(\lambda v) = \lambda(Bv),
so BvEλ(A)Bv\in E_\lambda(A) too. Hence B(Eλ(A))Eλ(A)B\big(E_\lambda(A)\big)\subseteq E_\lambda(A).

Step 2: BB restricted to each eigenspace of AA is diagonalizable. Since BB is diagonalizable, its minimal polynomial mBm_B has distinct roots (Theorem 4.3.4). Let Bλ:=BEλ(A)B_\lambda:=B|_{E_\lambda(A)}, a well-defined operator on Eλ(A)E_\lambda(A) by Step 1. Since mB(B)=0m_B(B)=0 on all of Fn\mathbb F^n, restricting to Eλ(A)E_\lambda(A) gives mB(Bλ)=0m_B(B_\lambda)=0; so the minimal polynomial of BλB_\lambda divides mBm_B (Theorem 4.3.3), and a divisor of a squarefree polynomial is squarefree. By Theorem 4.3.4 again (applied to BλB_\lambda on Eλ(A)E_\lambda(A)), BλB_\lambda is diagonalizable.

Step 3: assemble the common eigenbasis. Since AA is diagonalizable, Fn=λEλ(A)\mathbb F^n=\bigoplus_\lambda E_\lambda(A) (sum over the distinct eigenvalues of AA). By Step 2, choose within each Eλ(A)E_\lambda(A) a basis of eigenvectors of BλB_\lambda; every such vector is automatically an eigenvector of BB itself (since BB agrees with BλB_\lambda on Eλ(A)E_\lambda(A)), and it is already an eigenvector of AA with eigenvalue λ\lambda (by membership in Eλ(A)E_\lambda(A)). The union of these bases over all eigenvalues λ\lambda of AA is a basis of Fn\mathbb F^n — the concatenation of bases of the summands in a direct-sum decomposition — consisting entirely of common eigenvectors of AA and BB. \blacksquare

Remark (Statistical Payoff). This is the linear-algebraic core of several multivariate methods that use two covariance-type matrices at once — canonical correlation analysis, MANOVA, and Fisher's linear discriminant analysis all hinge on a generalized eigenvalue problem Σ1v=μΣ2v\Sigma_1v=\mu\Sigma_2v for a pair of matrices that need not literally commute. The standard resolution "whitens" by Σ21/2\Sigma_2^{-1/2} (when Σ20\Sigma_2\succ0) to convert the pencil into an ordinary eigenvalue problem for the single symmetric matrix Σ21/2Σ1Σ21/2\Sigma_2^{-1/2}\Sigma_1\Sigma_2^{-1/2}; the present theorem is the special case that already applies without whitening whenever the two matrices happen to commute outright — for instance, whenever both are diagonal in the same natural basis by construction, as with covariance matrices possessing a shared block or circulant structure.


Part 5 — Spectral Theory and Special Matrices

5.1 Symmetric, Orthogonal, Idempotent, Nilpotent

Definition 5.1.1. AMn(R)A \in M_n(\mathbb{R}) is:

  • Symmetric if A=ATA = A^T.
  • Orthogonal if ATA=AAT=IA^TA = AA^T = I (equivalently, columns/rows form an orthonormal basis).
  • Idempotent (a projection matrix) if A2=AA^2 = A.
  • Nilpotent if Ak=0A^k = 0 for some k1k \ge 1.

Theorem 5.1.2 (Eigenvalue Constraints from Structure).

  • If AA is orthogonal, every eigenvalue λ\lambda (possibly complex) satisfies λ=1|\lambda|=1. (Proof: Av=λv    v=Av=λv    λ=1Av=\lambda v \implies \|v\|=\|Av\|=|\lambda|\,\|v\| \implies |\lambda|=1, using Av2=vAAv=vv=v2\|Av\|^2=v^*A^*Av = v^*v=\|v\|^2 since ATA=IA^TA=I.)
  • If AA is idempotent, every eigenvalue is 00 or 11. (Proof: Av=λv    A2v=λ2v=Av=λv    λ2=λ    λ{0,1}Av=\lambda v \implies A^2v = \lambda^2v = Av=\lambda v \implies \lambda^2=\lambda \implies \lambda\in\{0,1\} for v0v \ne 0.)
  • If AA is nilpotent, every eigenvalue is 00. (Proof: Av=λv    Akv=λkv=0    λk=0    λ=0Av=\lambda v \implies A^kv = \lambda^kv = 0 \implies \lambda^k=0 \implies \lambda=0.)

Theorem 5.1.3 (Trace and Rank of an Idempotent). If AA is idempotent, tr(A)=rank(A)\text{tr}(A) = \text{rank}(A).

Proof. By Challenge Problem 2 of §1.5, Rn=kerAim(A)\mathbb{R}^n = \ker A \oplus \text{im}(A), and AA acts as the identity on im(A)\text{im}(A) (since for y=AximAy=Ax\in\text{im}A, Ay=A2x=Ax=yAy=A^2x=Ax=y) and as 00 on kerA\ker A. Choosing a basis adapted to this direct sum (basis of kerA\ker A followed by basis of imA\text{im}A), AA is similar to (000Ir)\begin{pmatrix}0&0\\0&I_r\end{pmatrix} where r=dimim(A)=rank(A)r=\dim\text{im}(A)=\text{rank}(A). Trace is a similarity invariant (Corollary 2.2.3), so tr(A)=tr(000Ir)=r=rank(A)\text{tr}(A) = \text{tr}\begin{pmatrix}0&0\\0&I_r\end{pmatrix} = r = \text{rank}(A). \blacksquare

Remark. This is the identity behind "trace of the hat matrix = number of estimated parameters" in OLS — see the Closing Remark.


5.2 The Spectral Theorem

Theorem 5.2.1 (Spectral Theorem for Real Symmetric Matrices). If AMn(R)A \in M_n(\mathbb{R}) is symmetric, then:
(a) All eigenvalues of AA are real.
(b) AA is orthogonally diagonalizable: there exists an orthogonal matrix QQ and diagonal Λ\Lambda (real entries) with A=QΛQTA = Q\Lambda Q^T.
(c) Eigenvectors corresponding to distinct eigenvalues are orthogonal.

Proof.
(a) Let Av=λvAv=\lambda v, v0v\ne 0 (possibly complex). Then vAv=λvvv^*Av = \lambda v^*v. Taking conjugate transpose of the scalar vAvv^*Av: (vAv)=vAv=vAv(v^*Av)^* = v^*A^*v = v^*Av (since AA real symmetric, A=AT=AA^*=A^T=A), so vAvv^*Av is real. Since vv=v2>0v^*v = \|v\|^2 >0 is real and positive, λ=(vAv)/(vv)\lambda = (v^*Av)/(v^*v) is real.

(c) Let Av1=λ1v1Av_1=\lambda_1v_1, Av2=λ2v2Av_2=\lambda_2v_2, λ1λ2\lambda_1\ne\lambda_2. Then λ1v1,v2=Av1,v2=v1,ATv2=v1,Av2=λ2v1,v2\lambda_1\langle v_1,v_2\rangle = \langle Av_1,v_2\rangle = \langle v_1,A^Tv_2\rangle=\langle v_1,Av_2\rangle=\lambda_2\langle v_1,v_2\rangle (using symmetry A=ATA=A^T), so (λ1λ2)v1,v2=0(\lambda_1-\lambda_2)\langle v_1,v_2\rangle=0, forcing v1,v2=0\langle v_1,v_2\rangle=0.

(b) By induction on nn. Take an eigenvalue λ1\lambda_1 (exists, real, by (a) and the Fundamental Theorem of Algebra applied to pAp_A) with unit eigenvector v1v_1. Extend {v1}\{v_1\} to an orthonormal basis of Rn\mathbb{R}^n; in this basis AA is similar (via an orthogonal change of basis matrix, since orthonormal-to-orthonormal change of basis is orthogonal) to (λ100A)\begin{pmatrix}\lambda_1&0\\0&A'\end{pmatrix} where AMn1(R)A' \in M_{n-1}(\mathbb{R}) is symmetric — symmetry is preserved since the conjugating matrix is orthogonal (QTAQQ^TAQ is symmetric whenever AA is and QQ is orthogonal, as (QTAQ)T=QTATQ=QTAQ(Q^TAQ)^T=Q^TA^TQ=Q^TAQ). Apply the inductive hypothesis to AA' and combine the orthogonal matrices block-diagonally. \blacksquare

Remark. This theorem is the load-bearing wall of Part 5: every subsequent statistical fact — PCA as eigendecomposition of a covariance matrix, positive-definiteness as admissibility for a covariance, SVD via ATAA^TA — descends directly from Theorem 5.2.1.


5.3 Positive Definite and Semi-Definite Matrices

Definition 5.3.1. Symmetric AMn(R)A\in M_n(\mathbb{R}) is:

  • Positive semi-definite (A0A\succeq0) if xTAx0 xRnx^TAx \ge 0\ \forall x\in\mathbb{R}^n.
  • Positive definite (A0A\succ0) if xTAx>0 x0x^TAx>0\ \forall x\ne0.

Theorem 5.3.2 (Characterizations). For symmetric AA, TFAE:
(i) A0A \succ 0 (resp. 0\succeq 0).
(ii) All eigenvalues of AA are >0>0 (resp. 0\ge0).
(iii) All leading principal minors of AA are >0>0 (Sylvester's Criterion — for strict positive-definiteness only; the semi-definite analogue requires all principal minors, not just leading ones, to be 0\ge 0).
(iv) A=BTBA = B^TB for some invertible BB (resp. A=BTBA=B^TB for some BB, possibly singular/rectangular).

Proof sketch. (i)    \iff(ii): By Theorem 5.2.1, A=QΛQTA=Q\Lambda Q^T; set y=QTxy=Q^Tx (a bijective change of variables since QQ orthogonal), so xTAx=yTΛy=λiyi2x^TAx = y^T\Lambda y = \sum\lambda_iy_i^2, which is positive for all x0x\ne0 (equivalently all y0y\ne0) iff every λi>0\lambda_i>0.
(ii)\Rightarrow(iv): with A=QΛQTA=Q\Lambda Q^T, Λ=Λ1/2Λ1/2\Lambda=\Lambda^{1/2}\Lambda^{1/2} (entrywise square root, valid since eigenvalues 0\ge0), so A=(Λ1/2QT)T(Λ1/2QT)=BTBA = (\Lambda^{1/2}Q^T)^T(\Lambda^{1/2}Q^T) = B^TB with B=Λ1/2QTB=\Lambda^{1/2}Q^T.
(iv)\Rightarrow(ii)/(i): xTAx=xTBTBx=Bx20x^TAx = x^TB^TBx = \|Bx\|^2\ge0 always; =0=0 only if Bx=0Bx=0, which (for BB invertible) forces x=0x=0, giving strict positivity.
(iii) is the classical Sylvester determinantal criterion; proof proceeds by induction using the Schur complement (Theorem 2.3.4) applied to leading blocks — the sign of each successive Schur complement is controlled by the ratio of consecutive leading principal minors. \blacksquare

Remark (A Classic Trap). The caveat in (iii) is not pedantry: the tempting guess that "all leading principal minors 0\ge0" characterizes PSD-ness is false. Take
A=(0001).A=\begin{pmatrix}0&0\\0&-1\end{pmatrix}.
Both leading principal minors are M1=00M_1=0\ge0 and M2=detA=00M_2=\det A=0\ge0, yet AA has eigenvalue 1<0-1<0, so A⪰̸0A\not\succeq0. The correct semi-definite criterion genuinely requires checking all 2n12^n-1 principal minors (not just the nn leading ones) — considerably more expensive, and the reason Sylvester's criterion is usually quoted only for the strict (0\succ0) case in practice.

Corollary 5.3.3 (Positive Definite Square Root). If A0A\succ0 (resp. A0A\succeq0), there is a symmetric B0B\succ0 (resp. B0B\succeq0) with B2=AB^2=A; write B=A1/2B=A^{1/2}. It satisfies (A1/2)1=(A1)1/2=:A1/2(A^{1/2})^{-1}=(A^{-1})^{1/2}=:A^{-1/2} when A0A\succ0.

Proof. From A=QΛQTA=Q\Lambda Q^T (Theorem 5.2.1) with Λ=diag(λ1,,λn)\Lambda=\text{diag}(\lambda_1,\ldots,\lambda_n), λi>0\lambda_i>0 (resp. 0\ge0), set B:=QΛ1/2QTB:=Q\Lambda^{1/2}Q^T, Λ1/2:=diag(λ1,,λn)\Lambda^{1/2}:=\text{diag}(\sqrt{\lambda_1},\ldots,\sqrt{\lambda_n}). Then B=BTB=B^T, B0B\succ0 (resp. 0\succeq0) by Theorem 5.3.2(ii) since its eigenvalues λi\sqrt{\lambda_i} inherit the sign of λi\lambda_i, and B2=QΛ1/2QTQΛ1/2QT=QΛQT=AB^2=Q\Lambda^{1/2}Q^TQ\Lambda^{1/2}Q^T=Q\Lambda Q^T=A (using QTQ=IQ^TQ=I). The identity (A1/2)1=(A1)1/2(A^{1/2})^{-1}=(A^{-1})^{1/2} follows by the same computation applied to Λ1=diag(1/λi)\Lambda^{-1}=\text{diag}(1/\lambda_i): both sides equal Qdiag(1/λi)QTQ\,\text{diag}(1/\sqrt{\lambda_i})\,Q^T. \blacksquare (Uniqueness of the PSD square root is true but not needed below, so we omit it.)

Remark (Statistical Payoff). A covariance matrix Σ\Sigma of a random vector must be 0\succeq 0 (since Var(aTX)=aTΣa0\text{Var}(a^TX) = a^T\Sigma a \ge0 for all aa), and 0\succ0 is required for a non-degenerate multivariate normal density (so that Σ1\Sigma^{-1} exists and the density's quadratic form in the exponent is a genuine "bowl"). This connection is elaborated in the Closing Remark.


5.4 Singular Value Decomposition

Theorem 5.4.1 (SVD). Every AMm×n(R)A \in M_{m\times n}(\mathbb{R}) can be written
A=UΣVT,A = U\Sigma V^T,
where UMm(R)U\in M_m(\mathbb{R}), VMn(R)V\in M_n(\mathbb{R}) are orthogonal, and ΣMm×n(R)\Sigma \in M_{m\times n}(\mathbb{R}) is diagonal (in the rectangular sense: Σii=σi0\Sigma_{ii}=\sigma_i\ge0, off-diagonal entries 00) with σ1σ20\sigma_1\ge\sigma_2\ge\cdots\ge0 — the singular values of AA.

Proof outline. ATAMn(R)A^TA \in M_n(\mathbb{R}) is symmetric and positive semi-definite (it is of the form BTBB^TB with B=AB=A, Theorem 5.3.2(iv)), so by the Spectral Theorem, ATA=VΛVTA^TA = V\Lambda V^T with VV orthogonal, Λ=diag(λ1λn0)\Lambda=\text{diag}(\lambda_1\ge\cdots\ge\lambda_n\ge0). Define σi:=λi\sigma_i := \sqrt{\lambda_i} and, for σi>0\sigma_i>0, ui:=1σiAviu_i := \tfrac{1}{\sigma_i}Av_i; one checks {ui}\{u_i\} are orthonormal, since uiTuj=1σiσjviTATAvj=λjσiσjviTvj=δiju_i^Tu_j = \tfrac{1}{\sigma_i\sigma_j}v_i^TA^TAv_j = \tfrac{\lambda_j}{\sigma_i\sigma_j}v_i^Tv_j = \delta_{ij} (using orthonormality of the viv_i, and λi/σi2=1\lambda_i/\sigma_i^2=1 when i=ji=j). For indices with σi=0\sigma_i=0: Avi2=viTATAvi=λi=0\|Av_i\|^2=v_i^TA^TAv_i=\lambda_i=0, so Avi=0Av_i=0 automatically — no separate uiu_i is forced by AA, and the construction extends {ui:σi>0}\{u_i:\sigma_i>0\} to a full orthonormal basis UU of Rm\mathbb{R}^m arbitrarily on the remaining coordinates. In either case Avi=σiuiAv_i=\sigma_iu_i holds for every ii (trivially when σi=0\sigma_i=0), i.e. AV=UΣAV=U\Sigma, giving A=UΣVTA=U\Sigma V^T. \blacksquare

Remark. So: the singular values of AA are the square roots of the eigenvalues of ATAA^TA (equivalently of AATAA^T, which shares the same nonzero eigenvalues); the right singular vectors VV are eigenvectors of ATAA^TA; the left singular vectors UU are eigenvectors of AATAA^T. SVD is thus "the Spectral Theorem applied to ATAA^TA, transported back to AA" — it exists for every matrix, not just symmetric/square ones, which is precisely why it underlies PCA (eigendecomposition of the empirical covariance 1nXTX\tfrac1nX^TX is literally an SVD of the centered data matrix XX).


5.5 Challenge Problems

Challenge Problem 1 (Idempotent \Rightarrow Diagonalizable, trace = rank, full proof). Let AA be idempotent (not assumed symmetric). Prove AA is diagonalizable and tr(A)=rank(A)\text{tr}(A)=\text{rank}(A).

Proof. Diagonalizability: A2A=0A^2-A=0, so the minimal polynomial mA(λ)m_A(\lambda) divides λ2λ=λ(λ1)\lambda^2-\lambda=\lambda(\lambda-1), a product of distinct linear factors; by Theorem 4.3.4, AA is diagonalizable. Trace = rank: this was proved directly in Theorem 5.1.3 via the direct-sum decomposition Rn=kerAim(A)\mathbb{R}^n=\ker A\oplus\text{im}(A); alternatively, having established diagonalizability, AA is similar to diag(1,,1r,0,,0nr)\text{diag}(\underbrace{1,\dots,1}_{r},\underbrace{0,\dots,0}_{n-r}) where r=r= number of eigenvalue-11's =dimE1=dimim(A)=rank(A)=\dim E_1 = \dim\text{im}(A)=\text{rank}(A) (using E1=im(A)E_1=\text{im}(A) for idempotents: Av=vvimAAv=v\Rightarrow v\in\text{im}A trivially, and conversely y=AxAy=A2x=Ax=yy=Ax\Rightarrow Ay=A^2x=Ax=y), and tr(A)=\text{tr}(A)= sum of the diagonal entries of the similar diagonal matrix =r=r. \blacksquare

Challenge Problem 2 (Structured Covariance Eigenvalues). Let Σ=σ2[(1ρ)In+ρJn]\Sigma = \sigma^2[(1-\rho)I_n+\rho J_n] be an equicorrelated covariance matrix (JnJ_n = all-ones matrix), with 1n1<ρ<1-\tfrac{1}{n-1}<\rho<1 to ensure Σ0\Sigma\succ0. Find the eigenvalues of Σ\Sigma and verify positive-definiteness reduces to the stated constraint on ρ\rho.

Proof. Write Σ\Sigma in the form of Challenge Problem 2 of §4.5, Σ=(ab)In+bJn\Sigma=(a-b)I_n+bJ_n, by matching ab=σ2(1ρ)a-b=\sigma^2(1-\rho) and b=σ2ρb=\sigma^2\rho (so a=σ2a=\sigma^2). That problem's conclusion transfers directly: eigenvalues of Σ\Sigma are
a+(n1)b=σ2[1+(n1)ρ](multiplicity 1),ab=σ2(1ρ)(multiplicity n1).a+(n-1)b = \sigma^2[1+(n-1)\rho] \quad(\text{multiplicity }1), \qquad a-b = \sigma^2(1-\rho) \quad(\text{multiplicity } n-1).
By Theorem 5.3.2(ii), Σ0\Sigma\succ0 iff both are strictly positive: 1ρ>0ρ<11-\rho>0 \Rightarrow \rho<1, and 1+(n1)ρ>0ρ>1n11+(n-1)\rho>0 \Rightarrow \rho > -\tfrac{1}{n-1}. This recovers exactly the stated admissible range — the classical bound on intraclass correlation. \blacksquare

Challenge Problem 3 (Advanced, SVD/rank interplay). Let AMm×n(R)A\in M_{m\times n}(\mathbb{R}) with SVD A=UΣVTA=U\Sigma V^T and rank(A)=r\text{rank}(A)=r. Prove that the best rank-kk approximation to AA in Frobenius norm (k<rk<r) is Ak=i=1kσiuiviTA_k = \sum_{i=1}^k\sigma_iu_iv_i^T, and that the approximation error is AAkF2=i=k+1rσi2\|A-A_k\|_F^2 = \sum_{i=k+1}^r\sigma_i^2.

Proof outline (Eckart–Young–Mirsky Theorem). Orthogonal invariance of the Frobenius norm — ABF=UT(AB)VF\|A-B\|_F = \|U^T(A-B)V\|_F for orthogonal U,VU,V, since XF2=tr(XTX)\|X\|_F^2=\text{tr}(X^TX) is invariant under XUTXVX\mapsto U^TXV by cyclicity of trace (Theorem 2.2.2) — reduces the problem to approximating the diagonal matrix Σ\Sigma by a rank-k\le k matrix UTBV=:YU^TBV=:Y. Among all rank-k\le k matrices YY, one shows (via a Ky Fan norm / interlacing argument, beyond entrance-exam scope to derive from scratch, but standard — see Horn & Johnson §3.4 or Golub–Van Loan) that ΣYF\|\Sigma-Y\|_F is minimized by zeroing out the smallest rkr-k diagonal entries of Σ\Sigma, i.e. by Y=diag(σ1,,σk,0,,0)Y=\text{diag}(\sigma_1,\ldots,\sigma_k,0,\ldots,0), giving ΣYF2=i=k+1rσi2\|\Sigma-Y\|_F^2=\sum_{i=k+1}^r\sigma_i^2 and, transporting back via B=UYVT=i=1kσiuiviT=AkB=UYV^T=\sum_{i=1}^k\sigma_iu_iv_i^T=A_k, the stated result. \blacksquare (The reduction to the diagonal case above is fully rigorous and exam-fair; the optimality of truncation among diagonal competitors is the part cited rather than re-derived.)

Challenge Problem 4 (Order-Reversal of Matrix Inversion). Let A,BA,B be symmetric n×nn\times n matrices with B0B\succ0 and ABA\succeq B (that is, AB0A-B\succeq0, the Loewner order). Prove that A0A\succ0 and
B1A1.B^{-1}\succeq A^{-1}.

Proof. A0A\succ0: for any x0x\ne0, xTAx=xTBx+xT(AB)xxTBx>0x^TAx = x^TBx+x^T(A-B)x \ge x^TBx>0 (using B0B\succ0 and AB0A-B\succeq0). So A0A\succ0, and both A1/2,B1/2A^{-1/2},B^{-1/2} exist (Corollary 5.3.3).

Reduction to a normalized case. Let C:=A1/2BA1/2C:=A^{-1/2}BA^{-1/2}, symmetric since CT=A1/2BTA1/2=CC^T=A^{-1/2}B^TA^{-1/2}=C. Congruence preserves the Loewner order: for symmetric XYX\succeq Y and any matrix PP, zTPT(XY)Pz=(Pz)T(XY)(Pz)0z^TP^T(X-Y)Pz=(Pz)^T(X-Y)(Pz)\ge0 for all zz, i.e. PTXPPTYPP^TXP\succeq P^TYP. So from BAB\preceq A, congruence by A1/2A^{-1/2} gives
C=A1/2BA1/2A1/2AA1/2=I,C = A^{-1/2}BA^{-1/2} \preceq A^{-1/2}AA^{-1/2} = I,
and from B0B\succ0, the same congruence (strict version) gives C0C\succ0. So CC is symmetric with 0CI0\prec C\preceq I: every eigenvalue of CC lies in (0,1](0,1].

Inverting. Diagonalize C=RΓRTC=R\Gamma R^T (Γ=diag(γ1,,γn)\Gamma=\text{diag}(\gamma_1,\ldots,\gamma_n), 0<γi10<\gamma_i\le1). Then C1=RΓ1RTC^{-1}=R\Gamma^{-1}R^T has eigenvalues 1/γi11/\gamma_i\ge1, so for any zz, writing y=RTzy=R^Tz: zT(C1I)z=yT(Γ1I)y=i(1/γi1)yi20z^T(C^{-1}-I)z=y^T(\Gamma^{-1}-I)y=\sum_i(1/\gamma_i-1)y_i^2\ge0. Hence C1IC^{-1}\succeq I.

Undoing the reduction. C1=(A1/2BA1/2)1=A1/2B1A1/2C^{-1}=(A^{-1/2}BA^{-1/2})^{-1}=A^{1/2}B^{-1}A^{1/2} (using (XYZ)1=Z1Y1X1(XYZ)^{-1}=Z^{-1}Y^{-1}X^{-1} with X=Z=A1/2X=Z=A^{-1/2}). So A1/2B1A1/2IA^{1/2}B^{-1}A^{1/2}\succeq I. Congruence by A1/2A^{-1/2} once more:
A1/2(A1/2B1A1/2)A1/2A1/2IA1/2,A^{-1/2}\big(A^{1/2}B^{-1}A^{1/2}\big)A^{-1/2} \succeq A^{-1/2}IA^{-1/2},
and the left side collapses to B1B^{-1} while the right side is A1A^{-1}, giving
B1A1.B^{-1}\succeq A^{-1}. \qquad\blacksquare

Remark (Statistical Payoff). This is the linear-algebraic engine behind every "more information can only help" statement in estimation theory. If θ^1,θ^2\hat\theta_1,\hat\theta_2 are two unbiased estimators with Cov(θ^1)Cov(θ^2)\text{Cov}(\hat\theta_1)\preceq\text{Cov}(\hat\theta_2) (the Gauss–Markov/Loewner sense in which θ^1\hat\theta_1 is more efficient), this theorem says the corresponding precision matrices (inverse covariances) satisfy the reverse inequality Cov(θ^1)1Cov(θ^2)1\text{Cov}(\hat\theta_1)^{-1}\succeq\text{Cov}(\hat\theta_2)^{-1} — more efficient estimation corresponds to more precision, exactly as intuition demands, but the direction-reversal on inverting is precisely this theorem, and it is easy to get backwards under exam pressure.


Appendix: Master Formula Sheet

Trace

IdentityCondition
tr(A+B)=tr(A)+tr(B)\text{tr}(A+B)=\text{tr}(A)+\text{tr}(B)always
tr(AB)=tr(BA)\text{tr}(AB)=\text{tr}(BA)conformable
tr(ABC)=tr(BCA)=tr(CAB)\text{tr}(ABC)=\text{tr}(BCA)=\text{tr}(CAB)cyclic, NOT tr(ACB)\text{tr}(ACB) in general
tr(A)=iλi\text{tr}(A) = \sum_i\lambda_isum of eigenvalues (with algebraic multiplicity)
tr(P1AP)=tr(A)\text{tr}(P^{-1}AP)=\text{tr}(A)similarity invariance
tr(A)=rank(A)\text{tr}(A)=\text{rank}(A)AA idempotent
tr(AAT)=AF20\text{tr}(AA^T)=\|A\|_F^2\ge0equality iff A=0A=0

Determinant

IdentityCondition
det(AB)=det(A)det(B)\det(AB)=\det(A)\det(B)square, conformable
det(AT)=det(A)\det(A^T)=\det(A)always
det(A1)=1/det(A)\det(A^{-1})=1/\det(A)AA invertible
det(cA)=cndet(A)\det(cA)=c^n\det(A)AMnA\in M_n
det(A)=iλi\det(A)=\prod_i\lambda_iproduct of eigenvalues
det(AB0D)=det(A)det(D)\det\begin{pmatrix}A&B\\0&D\end{pmatrix}=\det(A)\det(D)block-triangular
det(ABCD)=det(A)det(DCA1B)\det\begin{pmatrix}A&B\\C&D\end{pmatrix}=\det(A)\det(D-CA^{-1}B)AA invertible; Schur complement
det(AB)=Sdet(A:,S)det(BS,:)\det(AB)=\sum_S\det(A_{:,S})\det(B_{S,:})Cauchy–Binet, AMm×n,BMn×m,mnA\in M_{m\times n}, B\in M_{n\times m}, m\le n
det(A+uvT)=det(A)(1+vTA1u)\det(A+uv^T)=\det(A)(1+v^TA^{-1}u)matrix determinant lemma (Cor. 2.3.5); =1+vTu=1+v^Tu when A=IA=I

Rank

IdentityCondition
rank(A)+dimN(A)=n\text{rank}(A)+\dim\mathcal{N}(A)=nRank–Nullity
rank(A)=rank(AT)\text{rank}(A)=\text{rank}(A^T)row rank = column rank
rank(AB)min(rankA,rankB)\text{rank}(AB)\le\min(\text{rank}A,\text{rank}B)always
rank(A)+rank(B)nrank(AB)\text{rank}(A)+\text{rank}(B)-n\le\text{rank}(AB)Sylvester's inequality
$\text{rank}(A)-\text{rank}(B)
rank(A)=rank(ATA)=rank(AAT)\text{rank}(A)=\text{rank}(A^TA)=\text{rank}(AA^T)over R\mathbb{R}
rank(ABBTD)=rank(D)+rank(ABD1BT)\text{rank}\begin{pmatrix}A&B\\B^T&D\end{pmatrix}=\text{rank}(D)+\text{rank}(A-BD^{-1}B^T)DD invertible; Guttman-type rank additivity (§2.6 CP4)

Inverse / Block Inversion

IdentityCondition
A1=1detAadj(A)A^{-1}=\dfrac{1}{\det A}\text{adj}(A)detA0\det A\ne0
(A+uvT)1=A1A1uvTA11+vTA1u(A+uv^T)^{-1}=A^{-1}-\dfrac{A^{-1}uv^TA^{-1}}{1+v^TA^{-1}u}Sherman–Morrison
(A+UCV)1=A1A1U(C1+VA1U)1VA1(A+UCV)^{-1}=A^{-1}-A^{-1}U(C^{-1}+VA^{-1}U)^{-1}VA^{-1}Woodbury
M1=(A1+A1BS1CA1A1BS1S1CA1S1)M^{-1}=\begin{pmatrix}A^{-1}+A^{-1}BS^{-1}CA^{-1}&-A^{-1}BS^{-1}\\-S^{-1}CA^{-1}&S^{-1}\end{pmatrix}, S=DCA1BS=D-CA^{-1}Bblock inversion via Schur complement

Matrix Calculus (brief)

DerivativeResult
(aTx)x\dfrac{\partial (a^Tx)}{\partial x}aa
(xTAx)x\dfrac{\partial (x^TAx)}{\partial x}(A+AT)x(A+A^T)x; =2Ax=2Ax if AA symmetric
(xTAx)xxT\dfrac{\partial (x^TAx)}{\partial x \partial x^T} (Hessian)A+ATA+A^T; =2A=2A if AA symmetric
Axb22x\dfrac{\partial \|Ax-b\|_2^2}{\partial x}2AT(Axb)2A^T(Ax-b) — setting this to 00 is the normal equations ATAx=ATbA^TAx=A^Tb
tr(AX)X\dfrac{\partial \,\text{tr}(AX)}{\partial X}ATA^T
tr(XTAX)X\dfrac{\partial \,\text{tr}(X^TAX)}{\partial X}(A+AT)X(A+A^T)X
logdet(X)X\dfrac{\partial \log\det(X)}{\partial X}(X1)T(X^{-1})^T; =X1=X^{-1} if XX symmetric
det(X)X\dfrac{\partial \det(X)}{\partial X}det(X)(X1)T\det(X)(X^{-1})^T

Closing Remark: The Unifying Theme

Strip away the exam-paper packaging and this treatise has told one story from five angles: linear algebra is the geometry that makes multivariate statistics legible.

A random vector XX has a covariance matrix Σ=Cov(X)\Sigma = \text{Cov}(X), and the very first thing you learn to check about Σ\Sigma — that it is symmetric and 0\succeq 0 — is not an incidental algebraic nicety but a theorem (§5.3, Theorem 5.3.2) in disguise: aTΣa=Var(aTX)0a^T\Sigma a = \text{Var}(a^TX) \ge 0 for every direction aa is exactly the defining inequality of positive semi-definiteness. When Σ0\Sigma \succ 0, the Spectral Theorem (§5.2) hands you an orthogonal basis of principal axes — eigenvectors of Σ\Sigma — along which the components of XX decorrelate, and the multivariate normal density exp(12(xμ)TΣ1(xμ))\propto \exp(-\tfrac12(x-\mu)^T\Sigma^{-1}(x-\mu)) is literally the equation of an ellipsoid whose principal axes are Σ\Sigma's eigenvectors and whose semi-axis lengths are proportional to the square roots of Σ\Sigma's eigenvalues — equivalently, inversely proportional to the square roots of Σ1\Sigma^{-1}'s eigenvalues, so that directions of large variance become the long axes of the density's contours, exactly as intuition demands. Positive-definiteness is not a technical side-condition on this picture; it is the condition that the ellipsoid is a genuine, non-degenerate bowl rather than a flattened, singular one — and Challenge Problem 4 of §5.5 is the statement that shrinking this ellipsoid (in the Loewner order) is the same thing as growing its inverse, the precise sense in which more precise estimation means a tighter concentration ellipsoid.

And the idempotent matrices of §5.1 are the other half of the same coin. In ordinary least squares, β^=(XTX)1XTy\hat\beta = (X^TX)^{-1}X^Ty (an instance of the pseudoinverse of §3.4, since X+=(XTX)1XTX^+ = (X^TX)^{-1}X^T when XX has full column rank) produces fitted values y^=Xβ^=X(XTX)1XTH, the hat matrixy\hat y = X\hat\beta = \underbrace{X(X^TX)^{-1}X^T}_{H,\text{ the hat matrix}}y. The hat matrix HH is symmetric and idempotent — H2=X(XTX)1XTX(XTX)1XT=X(XTX)1XT=HH^2 = X(X^TX)^{-1}X^TX(X^TX)^{-1}X^T = X(X^TX)^{-1}X^T = H — which is precisely the algebraic signature of an orthogonal projection (Challenge Problem 2 of §1.5: Rn=kerHim(H)\mathbb{R}^n = \ker H\oplus\text{im}(H), and here that decomposition is the decomposition of Rn\mathbb{R}^n into the column space of XX and its orthogonal complement, the residual space — the very orthogonality proved abstractly in Theorem 1.4.4). Theorem 5.1.3 — tr(H)=rank(H)\text{tr}(H) = \text{rank}(H) — combined with the fact that im(H)=C(X)\text{im}(H)=\mathcal C(X) (so rank(H)=rank(X)=p\text{rank}(H)=\text{rank}(X)=p when XX has full column rank pp) tells you the trace of the hat matrix equals pp, the number of estimated regression coefficients — exactly the "degrees of freedom used up" that appears in every FF-test and every unbiasedness calculation for σ^2\hat\sigma^2 in a linear model.

So when you next diagonalize a symmetric matrix, or check that a Schur complement is positive, or compute tr(H)\text{tr}(H) for a hat matrix — you are not doing algebra and then applying it to statistics. You are doing the same piece of mathematics twice, once in the language of vector spaces and once in the language of random variables, and linear algebra is simply the more honest of the two languages about what is actually going on.

+
+
+
+