Differences

This shows you the differences between two versions of the page.

--- private:writeups:svd [2020/10/18 19:44]
admin
+++ private:writeups:svd [2020/10/18 21:25] (current)
admin
@@ Line 1: / Line 1: @@
 ====== Math ======
-$$X=\begin{bmatrix}
+$$X_{(m,n)}=\begin{bmatrix}
 | & | &  & | \\
 x_1 & x_2 & \dots & x_m  \\
@@ Line 39: / Line 39: @@
-U and V are the eigenvectors of the correlation matrices,$X X^T$ and $X^T X$, respectively.  Which is the inner product of $x_i ^T x_j$ where $i$ and $j$ are the $i$th and $j$th column of $X$.  If the inner product is large, it means that $x_i$ and $x_j$ are similar. If small, it means that  $x_i$ and $x_j$  are different.
+U and V are the eigenvectors of the correlation matrices,$X X^T$ and $X^T X$, respectively.  Because calculating $U$ and $V$ this way is slow, more optimized methods are available.    Which is the inner product of $x_i ^T x_j$ where $i$ and $j$ are the $i$th and $j$th column of $X$.  If the inner product is large, it means that $x_i$ and $x_j$ are similar. If small, it means that  $x_i$ and $x_j$  are different.
+====== Principal component analysis (PCA) ======
+This is the statistical interpretation of SVD.
+PCA finds elipsoids (depending on dimensionality) that account for the largest, second largest, etc. variance in your data.
+Note that X is defined differently as a matrix of horizontally aligned matrix (instead of vertical).
+https://www.youtube.com/watch?v=fkf4IBRSeEc
+Steps.
+.  Compute mean row, X'
+.  Subtract mean from X.  $B=X-X'$
+.  Calc covariance matrix, $C=B^T B$
+.  Computer eigs of C.  Eigenvectors, V, and eigvenvalues, D.
+.  Principal components, T=BV.  (V also called "loadings")
+If $B=U \Sigma V^T$, then $T=U \Sigma$.
+Takeaway, PCA can be found by saving the SVD of (X minus its row-mean).
+====== Least squares, regression, and pseudo-inverse ======
+SVD allows us to generalized Ax=b to non-square A matrices.  (pseudo-inverse)

My DokuWiki

User Tools

Site Tools

Differences

Page Tools