Singular Value Decomposition (SVD) Explained Simply

What does a matrix actually do to space? Not "what does it compute," but what does it look like, geometrically, when it acts on every point around it at once?

Here's the answer, and it's surprisingly simple: any matrix takes a circle and turns it into an ellipse. Every single one. Square, rectangular, symmetric, not symmetric, doesn't matter.

Singular value decomposition is just the precise description of that ellipse: which direction it points, how stretched it is, and how to get there from the original circle in three clean moves. Let's build it from that one picture.

Every matrix is secretly a circle squisher

Picture the unit circle, every point exactly distance 1 from the origin. Now pick a matrix $A$ and apply it to every point on that circle at once.

The result is never a random blob. It's always an ellipse.

Drag the point v around the circle and watch Av trace the ellipse out. Notice the four rose-colored marks. Those aren't decoration, they're the two special directions where the ellipse reaches its widest and narrowest points.

Here's the part that makes this genuinely useful: the matrix we're using here, $A = \begin{bmatrix} 2 & 1 \\ -1 & 1 \end{bmatrix}$, doesn't even have real eigenvectors. Its eigenvalues are complex numbers. If you tried to find "the direction A stretches without rotating," in the eigenvalue sense, you'd come up empty. But it still squishes a circle into a perfectly well-defined ellipse. SVD works when eigenvalues don't.

Where exactly does the stretching happen?

If every matrix produces an ellipse, the ellipse must have a long axis and a short axis. Those correspond to some input direction that gets stretched the most, and another that gets stretched the least.

We write the amount of stretch as $|A\mathbf{v}|$, the length of the output vector when the input is a unit vector $\mathbf{v}$. So the question becomes: which $\mathbf{v}$ makes $|A\mathbf{v}|$ as big as possible? And which makes it as small as possible?

Drag along the curve. You'll notice it has exactly one peak and one valley as the angle sweeps a full circle. But here's the detail that's easy to miss: look at the angle between Av and the image of its perpendicular partner. For almost every angle, that angle isn't 90 degrees. The transform tilts things around. Except right at the peak and the valley. There, and only there, two perpendicular inputs still land as two perpendicular outputs.

That's not a coincidence. It's the defining property of the directions we're looking for. We call the input directions $\mathbf{v}_1$ and $\mathbf{v}_2$, the right singular vectors. They're perpendicular to each other. Their images, scaled down to unit length, are called $\mathbf{u}_1$ and $\mathbf{u}_2$, the left singular vectors, and they're perpendicular too. The stretch factors themselves, $\sigma_1$ and $\sigma_2$, are the singular values, and they're exactly the semi-axis lengths of that ellipse from before.

So in one line:

An orthogonal pair goes in, a (possibly different) orthogonal pair comes out, just scaled. That's the entire geometric content of SVD.

Writing the whole thing as three simple moves

Once you know $\mathbf{v}_1, \mathbf{v}_2$ (a rotated pair of input axes), $\sigma_1, \sigma_2$ (the stretch amounts), and $\mathbf{u}_1, \mathbf{u}_2$ (a rotated pair of output axes), you can describe what $A$ does to any vector, not just the special ones, as three moves in sequence:

1. Rotate the input so the $\mathbf{v}$-directions line up with the plain x and y axes.
2. Stretch along those axes by $\sigma_1$ and $\sigma_2$.
3. Rotate again so the result lines up with the $\mathbf{u}$-directions.

Hit play. Watch the arrow rotate, then stretch, then rotate again, and land in exactly the same place it would if you'd just applied $A$ directly in one shot. That's not a trick. Three simple, well understood operations (rotate, scale, rotate) chain together to reproduce any linear map at all.

Now for the notation. We collect $\mathbf{v}_1, \mathbf{v}_2$ as columns of a matrix $V$, we collect $\mathbf{u}_1, \mathbf{u}_2$ as columns of a matrix $U$, and we put the stretch factors on the diagonal of a matrix $\Sigma$. The three-step process above is exactly the matrix product:

$V^\top$ does the first rotation, $\Sigma$ does the stretching, $U$ does the second rotation. Every matrix factors this way. That's the theorem. Everything else is bookkeeping.

Where do σ and v actually come from?

You don't have to guess $\mathbf{v}_1, \mathbf{v}_2$ by trial and error. There's a shortcut, and it connects straight back to eigenvectors.

Look at $A^\top A$. This matrix is always symmetric, and it's always well-behaved (positive semi-definite, if you want the formal term), which means it always has real eigenvectors. It turns out those eigenvectors are exactly $\mathbf{v}_1, \mathbf{v}_2$, and the eigenvalues are exactly $\sigma_1^2, \sigma_2^2$:

So even when $A$ itself has no real eigenvectors, like our example above, $A^\top A$ always does. That's the trick that makes SVD universal: it borrows the eigen-machinery, but applies it to a matrix that's guaranteed to cooperate.

Once you have $\mathbf{v}_i$ and $\sigma_i$, getting $\mathbf{u}_i$ is just one more step: $\mathbf{u}_i = A\mathbf{v}_i / \sigma_i$, the direction $A$ actually sends $\mathbf{v}_i$ to, rescaled back down to a unit vector.

Throwing away the small stretches on purpose

Here's where SVD earns its keep in the real world. A matrix full of numbers, a spreadsheet, a grayscale image, a table of user ratings, can be rebuilt as a sum of simple pieces, largest stretch first:

If you keep only the first $k$ terms, you get $A_k$, the best possible rank-$k$ approximation of $A$. Not just a good approximation. The provably best one, out of every matrix of that rank.

Slide k up from zero. Notice the rebuild starts blurry and sharpens fast, then barely changes near the end. That's because the singular values are sorted biggest to smallest, so the first few terms already carry most of the "energy" (the technical term is exactly that: energy, meaning $\sigma_i^2$ as a share of the total). The last few terms are mostly fine detail and noise, and you can often throw them away and barely notice.

This is the same idea behind image compression, recommendation systems guessing what you'll rate a movie, and noise reduction in signal processing. Different names, same three-step decomposition underneath.

The short version

Every matrix squishes the unit circle into an ellipse. The ellipse's axes point along the singular vectors, and its semi-axis lengths are the singular values. Writing $A = U\Sigma V^\top$ just names those three ingredients: rotate into the right input axes, stretch by the singular values, rotate into the right output axes.

The singular vectors and values come from the eigen-decomposition of $A^\top A$, a matrix that's always well-behaved even when $A$ itself is strange. And because the singular values are sorted from biggest to smallest, keeping only the first few gives you the best possible simplified version of $A$, which is exactly why SVD shows up everywhere from compression to recommendation engines.

A circle, an ellipse, and three simple moves to get from one to the other. That's the whole idea.

All visualizations are interactive React components running entirely in your browser. The 2x2 decomposition uses a closed-form eigen-solution of $A^\top A$; the rank-k image reconstruction uses power iteration with deflation. No libraries beyond React.