ValueError: matrix must be 2-dimensional when passing two arrays to the function

问题

I have a function which is written entirely in numpy functions and takes two input values. The function consists of some matrix operations and when I pass two large arrays It gives me an ValueError: matrix must be 2-dimensional. Using loops and numpy.apply_along_axis will fix the problem but these methods with make the code very slow.

The following is the code I have written

import numpy as np
import random
data = np.random.normal(size=600*600*2)
data = data.reshape(600*600,2)

def fun(x,y):
    f1 = sin(x)*cos(y)
    f2 = cos(x)*sin(y)
    eig1 = f1*np.mat([[f1],[f2]])
    eig2 = f2*np.mat([[f2+f1],[f1]])
    return np.sum(np.linalg.eig(eig1*eig2.T)[0])


fun(data[:,0],data[:,1])

It gives me the following error

ValueError: matrix must be 2-dimensional

回答1:

And the traceback is:

     ...: fun(data[:,0],data[:,1])                                                                     
---------------------------------------------------------------------------
ValueError                                Traceback (most recent call last)
<ipython-input-262-144f145bebe5> in <module>
     10 
     11 
---> 12 fun(data[:,0],data[:,1])

<ipython-input-262-144f145bebe5> in fun(x, y)
      5     f1 = np.sin(x)*np.cos(y)
      6     f2 = np.cos(x)*np.sin(y)
----> 7     eig1 = f1*np.mat([[f1],[f2]])
      8     eig2 = f2*np.mat([[f2+f1],[f1]])
      9     return np.sum(np.linalg.eig(eig1*eig2.T)[0])

/usr/local/lib/python3.6/dist-packages/numpy/matrixlib/defmatrix.py in asmatrix(data, dtype)
     69 
     70     """
---> 71     return matrix(data, dtype=dtype, copy=False)
     72 
     73 

/usr/local/lib/python3.6/dist-packages/numpy/matrixlib/defmatrix.py in __new__(subtype, data, dtype, copy)
    149         shape = arr.shape
    150         if (ndim > 2):
--> 151             raise ValueError("matrix must be 2-dimensional")
    152         elif ndim == 0:
    153             shape = (1, 1)

ValueError: matrix must be 2-dimensional

So the problem is the input to the np.mat function. What's that?

In [263]: data.shape                                                                                   
Out[263]: (360000, 2)

The sin/cos calls don't change shape, so:

In [264]: [[data[:,0]],[data[:,1]]]                                                                                             
Out[264]: 
[[array([-0.95915424,  1.38013956,  1.26480082, ...,  1.34129623,
          1.14664781,  0.90385798])],
 [array([-0.77785621,  0.695089  ,  1.17894725, ..., -0.2891861 ,
          0.47051436, -0.22550854])]]
In [265]: np.array([[data[:,0]],[data[:,1]]])                                                          
Out[265]: 
array([[[-0.95915424,  1.38013956,  1.26480082, ...,  1.34129623,
          1.14664781,  0.90385798]],

       [[-0.77785621,  0.695089  ,  1.17894725, ..., -0.2891861 ,
          0.47051436, -0.22550854]]])
In [266]: _.shape                                                                                      
Out[266]: (2, 1, 360000)

So you are trying to give np.mat a 3d array.

np.mat works if I drop the []:

In [274]: np.mat([data[:,0],data[:,1]]).shape                                                          
Out[274]: (2, 360000)

but f1*np.mat([f1,f2]) has a problem with the matrix product:

In [275]: data[:,0]*np.mat([data[:,0],data[:,1]])                                                      
---------------------------------------------------------------------------
ValueError                                Traceback (most recent call last)
<ipython-input-275-555ec143af3f> in <module>
----> 1 data[:,0]*np.mat([data[:,0],data[:,1]])

/usr/local/lib/python3.6/dist-packages/numpy/matrixlib/defmatrix.py in __rmul__(self, other)
    224 
    225     def __rmul__(self, other):
--> 226         return N.dot(other, self)
    227 
    228     def __imul__(self, other):

ValueError: shapes (360000,) and (2,360000) not aligned: 360000 (dim 0) != 2 (dim 0)

By making a np.mat the * is now a dot product, not a element-wise product.

回答2:

I encountered this issue using XGBoost and managed to solve it as follows:

Instead of:

import numpy as np
x_trainNPArray = np.array(x_train)
x_testNPArray = np.array(x_test)
y_trainNPArray = np.array(y_train)
y_testNPArray = np.array(y_test)
model.fit(x_trainNPArray,y_trainNPArray)
...

, I used:

x_trainNPArray = np.vstack(x_trainNPArray)
x_testNPArray = np.vstack(x_testNPArray)
y_trainNPArray = np.vstack(y_trainNPArray)
y_testNPArray = np.vstack(y_testNPArray )
model.fit(x_trainNPArray,y_trainNPArray)
...

来源：https://stackoverflow.com/questions/56470894/valueerror-matrix-must-be-2-dimensional-when-passing-two-arrays-to-the-function