Singular Value Decomposition (SVD) & Low-Rank Matrix Approximation (LoRA)
Factorize any rectangular matrix into rotation-stretch-rotation (A = U Sigma V^T) and prove the Eckart-Young theorem.
Every real matrix A in R^{m x n} admits a Singular Value Decomposition A = U Sigma V^T, where U in R^{m x m} and V in R^{n x n} are orthogonal matrices and Sigma in R^{m x n} is diagonal with non-negative singular values sigma_1 >= sigma_2 >= ... >= sigma_r > 0.
Any linear transformation—even between spaces of different dimensions—maps the unit sphere into a hyper-ellipse. V^T rotates the input orthogonal axes to align with the principal axes, Sigma stretches each axis by sigma_i, and U rotates the stretched ellipse into the output space.
SVD & Low-Rank Approximation (SVDUnitCircleDecompositionScene)
Step-by-Step Symbolic LaTeX Derivation
Follow each algebraic transformation from first principles to computational implementation.
Connection to Eigendecomposition of Gram Matrix A^T A
Right singular vectors V are the eigenvectors of A^T A, and singular values are the square roots of its eigenvalues.
By the Eckart-Young-Mirsky theorem, how do you construct the best rank-k approximation to matrix A in Frobenius norm?
Why is SVD always defined even when Eigendecomposition does not exist?
Eigendecomposition requires a square matrix and may fail if the matrix is defective; SVD exists for ANY m x n matrix because A^T A is always symmetric positive semi-definite.