Matrices in machine learning

Matrices are used as mathematical objects to represent images, datasets for real world machine learning applications like a face or text recognition, medical imaging, principal component analysis, numerical accuracy, and so on.

As an example, eigen decomposition is explained here. Many mathematical objects can be understood better by breaking them into constituent parts, or by finding properties which are universal.

Like when integers are decomposed into prime factors, matrix decomposition is called eigen decomposition, where we decompose a matrix into eigenvectors and eigenvalues.

Eigenvector v of a matrix A is such that multiplication by A alters only the scale of v, as shown next:

Av = λv

The scalar λ is known as the eigenvalue corresponding to this eigenvector. The eigen decomposition of A is then given by the following:

A = V diag(λ)V −1

The eigen decomposition of a matrix shares many facts about the matrix. The matrix is singular if and only if any of the eigenvalues is 0. The eigen decomposition of a real symmetric matrix can also be used to optimize quadratic expressions and much more. Eigenvectors and eigenvalues are used for Principal Component Analysis.

The following example shows how a DenseMatrix is used to get eigenvalues and eigen vectors:

// The data 
val msData = DenseMatrix( 
  (2.5,2.4), (0.5,0.7), (2.2,2.9), (1.9,2.2), (3.1,3.0), 
  (2.3,2.7), (2.0,1.6), (1.0,1.1), (1.5,1.6), (1.1,0.9)) 



def main(args: Array[String]): Unit = { 
       val pca = breeze.linalg.princomp(msData) 

       print("Center" , msData(*,::) - pca.center) 

       //the covariance matrix of the data 

       print("covariance matrix", pca.covmat) 

       // the eigenvalues of the covariance matrix, IN SORTED ORDER 
       print("eigen values",pca.eigenvalues) 

       // eigenvectors 
       print("eigen vectors",pca.loadings) 
       print(pca.scores) 
} 

This gives us the following result:

eigen values = DenseVector(1.2840277121727839, 0.04908339893832732)
eigen vectors = -0.6778733985280118  -0.735178655544408