我觉得线性代数中最主要的概念是基变换和矩阵分解。矩阵分解的本质就是基变换。选择不同的基，可以将矩阵分解为不同的形式。几种不同的线性变换 A A A m ∗ n m*n m ∗ n A A A A T A^T A T N ( A T ) N(A^T) N ( A T ) v 1 , v 2 , . . . , v r v_1, v_2,...,v_r v 1 , v 2 , . . . , v r u 1 , u 2 , . . . , u r u_1, u_2,...,u_r u 1 , u 2 , . . . , u r r r r A v i = σ i u i , i = 1 , 2 , . . . , r Av_i=\sigma_i u_i, i=1,2,...,r A v i = σ i u i , i = 1 , 2 , . . . , r V = [ v 1 , v 2 , . . . , v r ] , U = [ u 1 , u 2 , . . . , u r ] V=[v_1,v_2, ...,v_r], U=[u_1,u_2, ..., u_r] V = [ v 1 , v 2 , . . . , v r ] , U = [ u 1 , u 2 , . . . , u r ] A V = U Σ , A = U Σ V T AV=U\Sigma, A=U\Sigma V^T A V = U Σ , A = U Σ V T A = U Σ V T , A T = V Σ U T A=U\Sigma V^T, A^T=V\Sigma U^T A = U Σ V T , A T = V Σ U T A A T = U Σ 2 U T , A T A = V Σ 2 V T AA^T=U\Sigma^2U^T, A^TA=V\Sigma^2 V^T A A T = U Σ 2 U T , A T A = V Σ 2 V T U , V , Σ U,V,\Sigma U , V , Σ A A T , A T A AA^T, A^TA A A T , A T A
主成分分析这里有详细的推导，也是Ali教授讲的，很好
latent semantic analysis (LSA)
用一个矩阵来描述词term和文章doc的关联性。在这个矩阵中，每一行对应一个词，每一列对应一个文档。设有m m m n n n a i j a_{ij} a i j A = [ a 11 . . . a 1 n . . . a i 1 . . . a i n . . . a m 1 . . . a m n ]
A= \left[
\begin{matrix}
a_{11} & ... & a_{1n} \\
... \\
a_{i1} & ... & a_{in} \\
... \\
a_{m1} & ... & a_{mn} \\
\end{matrix}
\right]
A = ⎣ ⎢ ⎢ ⎢ ⎢ ⎡ a 1 1 . . . a i 1 . . . a m 1 . . . . . . . . . a 1 n a i n a m n ⎦ ⎥ ⎥ ⎥ ⎥ ⎤ A A A A A A A t = d At=d A t = d t t t d d d
对它进行奇异值分解A = U Σ V T A=U\Sigma V^T A = U Σ V T
import numpy as np
A = np. matrix(
[ [ 1 , 1 , 1 ] , [ 0 , 1 , 1 ] , [ 1 , 0 , 0 ] , [ 0 , 1 , 0 ] , [ 1 , 0 , 0 ] , [ 1 , 0 , 1 ] , [ 1 , 1 , 1 ] , [ 1 , 1 , 1 ] , [ 1 , 0 , 1 ] , [ 0 , 2 , 0 ] ,
[ 0 , 1 , 1 ] ] )
U, S, VT = np. linalg. svd( A, full_matrices= False )
B = U * np. diag( S) * VT
print ( np. isclose( A, B) )
[ − 0.42012157 − 0.07479925 − 0.04597244 − 0.29948676 0.20009226 0.40782766 − 0.12063481 − 0.27489151 − 0.4538001 − 0.157561 0.30464762 − 0.2006467 − 0.12063481 − 0.27489151 − 0.4538001 − 0.26256057 − 0.37944687 0.15467426 − 0.42012157 − 0.07479925 − 0.04597244 − 0.42012157 − 0.07479925 − 0.04597244 − 0.26256057 − 0.37944687 0.15467426 − 0.315122 0.60929523 − 0.40129339 − 0.29948676 0.20009226 0.40782766 ] [ 4.09887197 0. 0. 0. 2.3615708 0. 0. 0. 1.27366868 ] [ − 0.49446664 − 0.64582238 − 0.58173551 − 0.64917576 0.71944692 n − 0.24691489 − 0.57799098 − 0.25555741 0.77499473 ]
\left[\begin{matrix}
-0.42012157 & -0.07479925 & -0.04597244 \\
-0.29948676 & 0.20009226 & 0.40782766 \\
-0.12063481 &-0.27489151& -0.4538001 \\
-0.157561 & 0.30464762 & -0.2006467 \\
-0.12063481 & -0.27489151 & -0.4538001 \\
-0.26256057 & -0.37944687 & 0.15467426 \\
-0.42012157 & -0.07479925 & -0.04597244 \\
-0.42012157 & -0.07479925 & -0.04597244 \\
-0.26256057 & -0.37944687 & 0.15467426 \\
-0.315122 & 0.60929523 & -0.40129339 \\
-0.29948676 & 0.20009226 & 0.40782766
\end{matrix}\right]
\left[\begin{matrix} 4.09887197 & 0. & 0. \\
0. & 2.3615708 & 0. \\
0. & 0. & 1.27366868
\end{matrix}\right]
\left[\begin{matrix} -0.49446664 & -0.64582238 & -0.58173551\\
-0.64917576 & 0.71944692 & n-0.24691489 \\
-0.57799098 & -0.25555741 & 0.77499473 \end{matrix}\right]
⎣ ⎢ ⎢ ⎢ ⎢ ⎢ ⎢ ⎢ ⎢ ⎢ ⎢ ⎢ ⎢ ⎢ ⎢ ⎢ ⎢ ⎡ − 0 . 4 2 0 1 2 1 5 7 − 0 . 2 9 9 4 8 6 7 6 − 0 . 1 2 0 6 3 4 8 1 − 0 . 1 5 7 5 6 1 − 0 . 1 2 0 6 3 4 8 1 − 0 . 2 6 2 5 6 0 5 7 − 0 . 4 2 0 1 2 1 5 7 − 0 . 4 2 0 1 2 1 5 7 − 0 . 2 6 2 5 6 0 5 7 − 0 . 3 1 5 1 2 2 − 0 . 2 9 9 4 8 6 7 6 − 0 . 0 7 4 7 9 9 2 5 0 . 2 0 0 0 9 2 2 6 − 0 . 2 7 4 8 9 1 5 1 0 . 3 0 4 6 4 7 6 2 − 0 . 2 7 4 8 9 1 5 1 − 0 . 3 7 9 4 4 6 8 7 − 0 . 0 7 4 7 9 9 2 5 − 0 . 0 7 4 7 9 9 2 5 − 0 . 3 7 9 4 4 6 8 7 0 . 6 0 9 2 9 5 2 3 0 . 2 0 0 0 9 2 2 6 − 0 . 0 4 5 9 7 2 4 4 0 . 4 0 7 8 2 7 6 6 − 0 . 4 5 3 8 0 0 1 − 0 . 2 0 0 6 4 6 7 − 0 . 4 5 3 8 0 0 1 0 . 1 5 4 6 7 4 2 6 − 0 . 0 4 5 9 7 2 4 4 − 0 . 0 4 5 9 7 2 4 4 0 . 1 5 4 6 7 4 2 6 − 0 . 4 0 1 2 9 3 3 9 0 . 4 0 7 8 2 7 6 6 ⎦ ⎥ ⎥ ⎥ ⎥ ⎥ ⎥ ⎥ ⎥ ⎥ ⎥ ⎥ ⎥ ⎥ ⎥ ⎥ ⎥ ⎤ ⎣ ⎡ 4 . 0 9 8 8 7 1 9 7 0 . 0 . 0 . 2 . 3 6 1 5 7 0 8 0 . 0 . 0 . 1 . 2 7 3 6 6 8 6 8 ⎦ ⎤ ⎣ ⎡ − 0 . 4 9 4 4 6 6 6 4 − 0 . 6 4 9 1 7 5 7 6 − 0 . 5 7 7 9 9 0 9 8 − 0 . 6 4 5 8 2 2 3 8 0 . 7 1 9 4 4 6 9 2 − 0 . 2 5 5 5 5 7 4 1 − 0 . 5 8 1 7 3 5 5 1 n − 0 . 2 4 6 9 1 4 8 9 0 . 7 7 4 9 9 4 7 3 ⎦ ⎤
上面对例子中：d 1 = A [ 1 0 0 ] d_1=A\left[\begin{matrix} 1 \\ 0 \\ 0 \end{matrix}\right] d 1 = A ⎣ ⎡ 1 0 0 ⎦ ⎤ [ 1 , 0 , 0 ] [1, 0, 0] [ 1 , 0 , 0 ]
以doc为例，原来的基是I I I V = [ v 1 , v 2 , . . . , v r , v r + 1 , . . . , v n ] V=[v_1,v_2, ...,v_r,v_{r+1},...,v_{n}] V = [ v 1 , v 2 , . . . , v r , v r + 1 , . . . , v n ] n − r n-r n − r x x x x ′ x' x ′ I x = V x ′ x ′ = V T x x ′ = [ v 1 T ⋮ v r T v r + 1 T ⋮ v n T ] x = [ v 1 T x 1 ⋮ v r T x r v r + 1 T x r + 1 ⋮ v n T x n ] = [ v 1 T x 1 ⋮ v r T x r 0 ⋮ 0 ]
Ix=Vx' \\ x'=V^Tx \\
x'=\left[ \begin{matrix}
v_1^T \\ \vdots\\v_r^T\\v_{r+1}^T \\ \vdots \\ v_n^T \\
\end{matrix}
\right]x=\left[ \begin{matrix}
v_1^Tx_1 \\ \vdots \\ v_r^Tx_r \\ v_{r+1}^Tx_{r+1} \\ \vdots \\ v_n^Tx_n \\
\end{matrix}
\right] =\left[ \begin{matrix}
v_1^Tx_1 \\ \vdots \\ v_r^Tx_r \\ 0 \\ \vdots \\ 0 \\
\end{matrix}
\right]
I x = V x ′ x ′ = V T x x ′ = ⎣ ⎢ ⎢ ⎢ ⎢ ⎢ ⎢ ⎢ ⎢ ⎡ v 1 T ⋮ v r T v r + 1 T ⋮ v n T ⎦ ⎥ ⎥ ⎥ ⎥ ⎥ ⎥ ⎥ ⎥ ⎤ x = ⎣ ⎢ ⎢ ⎢ ⎢ ⎢ ⎢ ⎢ ⎢ ⎡ v 1 T x 1 ⋮ v r T x r v r + 1 T x r + 1 ⋮ v n T x n ⎦ ⎥ ⎥ ⎥ ⎥ ⎥ ⎥ ⎥ ⎥ ⎤ = ⎣ ⎢ ⎢ ⎢ ⎢ ⎢ ⎢ ⎢ ⎢ ⎡ v 1 T x 1 ⋮ v r T x r 0 ⋮ 0 ⎦ ⎥ ⎥ ⎥ ⎥ ⎥ ⎥ ⎥ ⎥ ⎤ n n n x x x r r r x ′ x' x ′
下面解释这段话：A = U Σ V T A=U\Sigma V^T A = U Σ V T U U U R m R^m R m [ 1 , 0... , 0 ] T [1, 0...,0]^T [ 1 , 0 . . . , 0 ] T U U U U T [ 1 , 0 , . . . , 0 ] T U^T[1,0,...,0]^T U T [ 1 , 0 , . . . , 0 ] T U T U^T U T U U U R m R^m R m I I I U U U
