SVD Calculator (Singular Value Decomposition)

What the SVD tells you

Geometrically, any matrix transformation can be broken into three steps: a rotation or reflection (Vᵀ), a stretch along perpendicular axes by the singular values (Σ), and another rotation or reflection (U). The singular values measure how much the matrix stretches space in each direction. Unlike eigen-decomposition, the SVD exists for every matrix – square or rectangular, invertible or not.

How to compute the SVD by hand

1. Compute AᵀA (an n × n symmetric matrix) – or AAᵀ if that is smaller.
2. Find its eigenvalues λ₁ ≥ λ₂ ≥ … ≥ 0. The singular values are σᵢ = √λᵢ.
3. Find unit eigenvectors of AᵀA; these are the columns of V.
4. Compute each column of U as uᵢ = Avᵢ ÷ σᵢ for every non-zero σᵢ.
5. For a full SVD, complete U and V with extra orthonormal vectors.

Worked example

Take A = [[3, 2, 2], [2, 3, −2]]. Here the 2 × 2 matrix AAᵀ is easier: AAᵀ = [[17, 8], [8, 17]]. Its eigenvalues solve (17 − λ)² − 64 = 0, giving λ = 25 and λ = 9, so the singular values are σ₁ = 5 and σ₂ = 3. The eigenvectors of AAᵀ give U = (1/√2) [[1, 1], [1, −1]]. Then vᵢ = Aᵀuᵢ ÷ σᵢ: v₁ = (1/√2, 1/√2, 0) and v₂ = (1/√18)(1, −1, 4). A third vector, v₃ = (2/3, −2/3, −1/3), completes V for the full SVD. Multiplying UΣVᵀ gives back A.

Full vs reduced SVD

Rank, norm and condition number

The number of non-zero singular values is the rank of A. The largest singular value σ₁ is the matrix 2-norm – the most any unit vector can be stretched. The ratio σ₁ ÷ σₙ is the condition number: a large value means the matrix is close to singular and that solving Ax = b will amplify rounding errors. In the example, the condition number of the 2 × 3 matrix's non-zero part is 5 ÷ 3 ≈ 1.67, which is very well conditioned.

Low-rank approximation and applications

Keeping only the largest k singular values gives the best possible rank-k approximation of A, in both the 2-norm and Frobenius norm (the Eckart–Young theorem). The error in the 2-norm is exactly σₖ₊₁. This is the idea behind many applications:

• Principal component analysis (PCA): the SVD of a centred data matrix gives the principal directions (V) and the variance explained by each (σᵢ² ÷ (n − 1)).
• Image compression: keeping the top few dozen singular values of an image matrix keeps most of its visible detail with far less data.
• Recommender systems and latent semantic analysis: low-rank approximations reveal hidden patterns in user–item or term–document matrices.
• Least squares and the pseudoinverse: A⁺ = VΣ⁺Uᵀ, where Σ⁺ inverts the non-zero singular values, solves least-squares problems even when A is not invertible.
• Signal processing and noise reduction: small singular values often correspond to noise and can be discarded.

Numerical notes

Software does not compute the SVD by forming AᵀA, because squaring the matrix squares the condition number and loses accuracy. Libraries such as LAPACK (used by NumPy, MATLAB and R) use orthogonal transformations – typically Golub–Kahan bidiagonalisation followed by an iterative step. Results are unique except for signs: you may see u₁ and v₁ both negated compared with a textbook answer, which is equally correct.

A square example

For A = [[3, 0], [4, 5]], AᵀA = [[25, 20], [20, 25]], whose eigenvalues are 45 and 5. The singular values are therefore √45 ≈ 6.708 and √5 ≈ 2.236. The determinant of A is 15, which equals the product of the singular values (√45 × √5 = 15) – a useful check for square matrices, since |det A| = σ₁σ₂…σₙ.

Frobenius norm and energy

The sum of the squared singular values equals the sum of the squares of every entry of the matrix: ‖A‖F² = σ₁² + σ₂² + …. For the square example, 3² + 0² + 4² + 5² = 50 = 45 + 5. The share σ₁² ÷ Σσᵢ² is the fraction of the matrix's 'energy' captured by the first component – here 90% – which tells you how well a rank-1 approximation will do.

Sources

1. Strang G. – Introduction to Linear Algebra (singular value decomposition) – textbook
2. Golub G.H., Van Loan C.F. – Matrix Computations (SVD algorithms) – textbook