Problem: Singular Value Decomposition

Linear Algebra

This week we will tackle the derivation of Singular Value Decomposition (SVD). SVD is a popular linear algebra technique used in Data Science for different purposes. Its also closely related to the Principal Component Analysis (PCA) method used for dimensional reduction. Here we will focus on SVD for real matrices. Let’s go into details.

Singular Value Decomposition asserts that any real matrix with non-vanishing rank can be casted as the product of three matrices:

\(\begin{align} \mathcal{A} &= U \, \Sigma \, V^{T} \\[0.6cm] \mathcal{A}:&~ m \times n ~ \text{real matrix of rank}~r \\ U:&~ m \times r ~ \text{real orthogonal matrix} \\ \Sigma:&~ r \times r ~ \text{diagonal real matrix} \\ V:&~ n \times r ~ \text{real orthogonal matrix} \end{align} \)

The problem of this week is to derive the SVD relation above. If you are stuck, feel free to consult literature and do some research about this, the important thing is that you understand the full derivation of this relation. This way you will get a deeper understanding of the SVD.

In the next post we will be sharing a solution to this problem. Ideally you can use it to compare with yours. Additionally we will be sharing some additional material for your learning.