When I am trying to work with LDA from Scikit-Learn, it keeps only giving me one component, even though I am asking for more:
>>> from sklearn.lda import LDA
>>> x = np.random.randn(5,5)
>>> y = [True, False, True, False, True]
>>> for i in range(1,6):
... lda = LDA(n_components=i)
... model = lda.fit(x,y)
... model.transform(x)
Gives
/Users/orthogonal/virtualenvs/osxml/lib/python2.7/site-packages/sklearn/lda.py:161: UserWarning: Variables are collinear
warnings.warn("Variables are collinear")
array([[-0.12635305],
[-1.09293574],
[ 1.83978459],
[-0.37521856],
[-0.24527725]])
array([[-0.12635305],
[-1.09293574],
[ 1.83978459],
[-0.37521856],
[-0.24527725]])
array([[-0.12635305],
[-1.09293574],
[ 1.83978459],
[-0.37521856],
[-0.24527725]])
array([[-0.12635305],
[-1.09293574],
[ 1.83978459],
[-0.37521856],
[-0.24527725]])
array([[-0.12635305],
[-1.09293574],
[ 1.83978459],
[-0.37521856],
[-0.24527725]])
As you can see, it's only printing out one dimension each time. Why is this? Does it have anything to do with the variables being collinear?
Additionally, when I do this with Scikit-Learn's PCA, it gives me what I want.
>>> from sklearn.decomposition import PCA
>>> for i in range(1,6):
... pca = PCA(n_components=i)
... model = pca.fit(x)
... model.transform(x)
...
array([[ 0.83688322],
[ 0.79565477],
[-2.4373344 ],
[ 0.72500848],
[ 0.07978792]])
array([[ 0.83688322, -1.56459039],
[ 0.79565477, 0.84710518],
[-2.4373344 , -0.35548589],
[ 0.72500848, -0.49079647],
[ 0.07978792, 1.56376757]])
array([[ 0.83688322, -1.56459039, -0.3353066 ],
[ 0.79565477, 0.84710518, -1.21454498],
[-2.4373344 , -0.35548589, -0.16684946],
[ 0.72500848, -0.49079647, 1.09006296],
[ 0.07978792, 1.56376757, 0.62663807]])
array([[ 0.83688322, -1.56459039, -0.3353066 , 0.22196922],
[ 0.79565477, 0.84710518, -1.21454498, -0.15961993],
[-2.4373344 , -0.35548589, -0.16684946, -0.04114339],
[ 0.72500848, -0.49079647, 1.09006296, -0.2438673 ],
[ 0.07978792, 1.56376757, 0.62663807, 0.2226614 ]])
array([[ 8.36883220e-01, -1.56459039e+00, -3.35306597e-01,
2.21969223e-01, -1.66533454e-16],
[ 7.95654771e-01, 8.47105182e-01, -1.21454498e+00,
-1.59619933e-01, 3.33066907e-16],
[ -2.43733440e+00, -3.55485895e-01, -1.66849458e-01,
-4.11433949e-02, 0.00000000e+00],
[ 7.25008484e-01, -4.90796471e-01, 1.09006296e+00,
-2.43867297e-01, -1.38777878e-16],
[ 7.97879229e-02, 1.56376757e+00, 6.26638070e-01,
2.22661402e-01, 2.22044605e-16]])
This is the relevant, dimension-reducing line of LDA.transform, it uses scalings_. As described in the docstring, scalings_ has maximally n_classes - 1 columns. This is then the maximal number of columns you can hope to obtain using transform. In your case, 2 classes (True, False), yields maximally 1 column.
来源：https://stackoverflow.com/questions/26963454/lda-ignoring-n-components