Learn you a machine for great good!

Many techniques exist to use statistics to learn about problems that are too complex for straightforward analysis. These problems include facial recognition, finding patterns in DNA sequences, or even predicting what Brilliant problems a given user is most likely to enjoy.

One class of problem is known as matrix factorization. In this kind of problem one hopes to simplify a task, such as learning how to reconstruct a face, down to finding a set of things that can be recombined linearly in the right proportion in order to reconstruct the thing of interest. For example, if one tried to learn the color purple, these kinds of techniques would say "mix equal parts blue and red", or "mix equal parts magenta and cyan."

Three techniques for matrix factorization are Principal Components Analysis (PCA), Vector Quantization (VQ), and Nonnegative Matrix Factorization (NMF). All three try to find objects that can be recombined to approximate the thing of interest, i.e. faces, sounds, etc. VQ tries to find objects that can approximate the answer on their own. PCA tries to find objects that are maximally different from one another, but can reconstruct the answer by adding and subtracting many of them. Importantly, PCA can recombine pieces through addition OR subtraction. Finally, NMF finds objects that can combine in a purely constructive manner, i.e. only by adding, to approximate the answer.

To illustrate the difference, if we set out to learn the matrix $A = \left(\begin{array}{cc}1&0\\0&1\end{array}\right)$ PCA might attempt to approximate it by the difference $\left(\begin{array}{cc}2.3&-1\\0&1\end{array}\right) - \left(\begin{array}{cc}1.3&-1\\0&0\end{array}\right)$

neither piece of which are particularly intuitive choices. VQ might find that $A$ is roughly $\left(\begin{array}{cc}1.12&0\\0&0.91\end{array}\right)$

not a great estimate for $A$ , but simple.

Finally, NMF might say $A$ is approximately given by $\left(\begin{array}{cc}1.04&0\\0&0\end{array}\right) + \left(\begin{array}{cc}0&0\\0&0.9\end{array}\right)$

Suppose we apply these three techniques to learn how to reconstruct a song. Which of the three is most likely to identify a recognizable element of a composition, such as the guitar part?