My DokuWiki

User Tools

Site Tools

Trace:
private:writeups:svd

Differences

This shows you the differences between two versions of the page.

Link to this comparison view

Both sides previous revision Previous revision
Next revision 		Previous revision
private:writeups:svd [2020/10/18 19:24]
admin 		private:writeups:svd [2020/10/18 21:25] (current)
admin
Line 1: Line 1:
 +====== Math ======
  
-$$X=\begin{bmatrix}+$$X_{(m,n)}=\begin{bmatrix}
 | & | &  & | \\  | & | &  & | \\ 
 x_1 & x_2 & \dots & x_m  \\  x_1 & x_2 & \dots & x_m  \\ 
Line 18: Line 19:
 \end{bmatrix} \end{bmatrix}
 \begin{bmatrix} \begin{bmatrix}
-\text{---}  & v_1^T & \text{---}  \\  +| & | &  & | \\  
-\text{---}  & v_2^T & \text{---}  \\  +v_1 & v_2 & \dots & v_m  \\  
-\text{---}  & \vdots & \text{---}  \\  +| & | &  & | 
-\text{---}  & v_m^T & \text{---}  +\end{bmatrix}^T 
-\end{bmatrix}$$+$$
  
 where $n$ is the number of points in each $x$, and $m$ is the number of $x$ values.  where $n$ is the number of points in each $x$, and $m$ is the number of $x$ values. 
Line 32: Line 33:
 We can therefore select the first $r$ terms of this expansion to get a reduced-order approximation of X.  I.e. We can therefore select the first $r$ terms of this expansion to get a reduced-order approximation of X.  I.e.
 $$ X \approx \tilde{U}\tilde{\Sigma}\tilde{V}^T$$. $$ X \approx \tilde{U}\tilde{\Sigma}\tilde{V}^T$$.
 +
 +====== Interpretation ======
 +
 +Correlation between the rows and columns of X.  
 +
 +
 +U and V are the eigenvectors of the correlation matrices,$X X^T$ and $X^T X$, respectively.  Because calculating $U$ and $V$ this way is slow, more optimized methods are available.    Which is the inner product of $x_i ^T x_j$ where $i$ and $j$ are the $i$th and $j$th column of $X$.  If the inner product is large, it means that $x_i$ and $x_j$ are similar. If small, it means that  $x_i$ and $x_j$  are different.  
 +
 +
 +====== Principal component analysis (PCA) ======
 +This is the statistical interpretation of SVD.
 +
 +PCA finds elipsoids (depending on dimensionality) that account for the largest, second largest, etc. variance in your data.  
 +
 +Note that X is defined differently as a matrix of horizontally aligned matrix (instead of vertical).
 +
 +https://www.youtube.com/watch?v=fkf4IBRSeEc
 +
 +Steps.
 +1.  Compute mean row, X'
 +2.  Subtract mean from X.  $B=X-X'$ 
 +3.  Calc covariance matrix, $C=B^T B$
 +4.  Computer eigs of C.  Eigenvectors, V, and eigvenvalues, D.
 +5.  Principal components, T=BV.  (V also called "loadings")
 +If $B=U \Sigma V^T$, then $T=U \Sigma$.  
 +
 +Takeaway, PCA can be found by saving the SVD of (X minus its row-mean).  
 +
 +
 +====== Least squares, regression, and pseudo-inverse ======
 +
 +SVD allows us to generalized Ax=b to non-square A matrices.  (pseudo-inverse)
private/writeups/svd.1603049086.txt.gz · Last modified: 2020/10/18 19:24 by admin

Page Tools

Donate Powered by PHP Valid HTML5 Valid CSS Driven by DokuWiki