Save Numpy Array using Pickle

Question

I\'ve got a Numpy array that I would like to save (130,000 x 3) that I would like to save using Pickle, with the following code. However, I keep getting the error \"EOFError: Ra

Answer 1

I have no problems using pickle:

In [126]: arr = np.zeros((1000,2))
In [127]: with open('test.pkl','wb') as f:
     ...:     pickle.dump(arr, f)
     ...:     
In [128]: with open('test.pkl','rb') as f:
     ...:     x = pickle.load(f)
     ...:     print(x.shape)
     ...:     
     ...:     
(1000, 2)

pickle and np.save/load have a deep reciprocity. Like I can load this pickle with np.load:

In [129]: np.load('test.pkl').shape
Out[129]: (1000, 2)

If I open the pickle file in the wrong I do get your error:

In [130]: with open('test.pkl','wb') as f:
     ...:     x = pickle.load(f)
     ...:     print(x.shape)
     ...:    
UnsupportedOperation: read

But that shouldn't be surprising - you can't read a freshly opened write file. It will be empty.

np.save/load is the usual pair for writing numpy arrays. But pickle uses save to serialize arrays, and save uses pickle to serialize non-array objects (in the array). Resulting file sizes are similar. Curiously in timings the pickle version is faster.