Convert pandas dataframe to NumPy array

前端 未结 15 2358
别那么骄傲
别那么骄傲 2020-11-21 23:57

I am interested in knowing how to convert a pandas dataframe into a NumPy array.

dataframe:

import numpy as np
import pandas as pd

index = [1, 2, 3,         


        
15条回答
  •  醉酒成梦
    2020-11-22 00:40

    df.to_numpy() is better than df.values, here's why.

    It's time to deprecate your usage of values and as_matrix().

    pandas v0.24.0 introduced two new methods for obtaining NumPy arrays from pandas objects:

    1. to_numpy(), which is defined on Index, Series, and DataFrame objects, and
    2. array, which is defined on Index and Series objects only.

    If you visit the v0.24 docs for .values, you will see a big red warning that says:

    Warning: We recommend using DataFrame.to_numpy() instead.

    See this section of the v0.24.0 release notes, and this answer for more information.



    Towards Better Consistency: to_numpy()

    In the spirit of better consistency throughout the API, a new method to_numpy has been introduced to extract the underlying NumPy array from DataFrames.

    # Setup
    df = pd.DataFrame(data={'A': [1, 2, 3], 'B': [4, 5, 6], 'C': [7, 8, 9]}, 
                      index=['a', 'b', 'c'])
    
    # Convert the entire DataFrame
    df.to_numpy()
    # array([[1, 4, 7],
    #        [2, 5, 8],
    #        [3, 6, 9]])
    
    # Convert specific columns
    df[['A', 'C']].to_numpy()
    # array([[1, 7],
    #        [2, 8],
    #        [3, 9]])
    

    As mentioned above, this method is also defined on Index and Series objects (see here).

    df.index.to_numpy()
    # array(['a', 'b', 'c'], dtype=object)
    
    df['A'].to_numpy()
    #  array([1, 2, 3])
    

    By default, a view is returned, so any modifications made will affect the original.

    v = df.to_numpy()
    v[0, 0] = -1
     
    df
       A  B  C
    a -1  4  7
    b  2  5  8
    c  3  6  9
    

    If you need a copy instead, use to_numpy(copy=True).


    pandas >= 1.0 update for ExtensionTypes

    If you're using pandas 1.x, chances are you'll be dealing with extension types a lot more. You'll have to be a little more careful that these extension types are correctly converted.

    a = pd.array([1, 2, None], dtype="Int64")                                  
    a                                                                          
    
    
    [1, 2, ]
    Length: 3, dtype: Int64 
    
    # Wrong
    a.to_numpy()                                                               
    # array([1, 2, ], dtype=object)  # yuck, objects
    
    # Correct
    a.to_numpy(dtype='float', na_value=np.nan)                                 
    # array([ 1.,  2., nan])
    
    # Also correct
    a.to_numpy(dtype='int', na_value=-1)
    # array([ 1,  2, -1])
    

    This is called out in the docs.


    If you need the dtypes in the result...

    As shown in another answer, DataFrame.to_records is a good way to do this.

    df.to_records()
    # rec.array([('a', 1, 4, 7), ('b', 2, 5, 8), ('c', 3, 6, 9)],
    #           dtype=[('index', 'O'), ('A', '

    This cannot be done with to_numpy, unfortunately. However, as an alternative, you can use np.rec.fromrecords:

    v = df.reset_index()
    np.rec.fromrecords(v, names=v.columns.tolist())
    # rec.array([('a', 1, 4, 7), ('b', 2, 5, 8), ('c', 3, 6, 9)],
    #           dtype=[('index', '

    Performance wise, it's nearly the same (actually, using rec.fromrecords is a bit faster).

    df2 = pd.concat([df] * 10000)
    
    %timeit df2.to_records()
    %%timeit
    v = df2.reset_index()
    np.rec.fromrecords(v, names=v.columns.tolist())
    
    12.9 ms ± 511 µs per loop (mean ± std. dev. of 7 runs, 100 loops each)
    9.56 ms ± 291 µs per loop (mean ± std. dev. of 7 runs, 100 loops each)
    


    Rationale for Adding a New Method

    to_numpy() (in addition to array) was added as a result of discussions under two GitHub issues GH19954 and GH23623.

    Specifically, the docs mention the rationale:

    [...] with .values it was unclear whether the returned value would be the actual array, some transformation of it, or one of pandas custom arrays (like Categorical). For example, with PeriodIndex, .values generates a new ndarray of period objects each time. [...]

    to_numpy aim to improve the consistency of the API, which is a major step in the right direction. .values will not be deprecated in the current version, but I expect this may happen at some point in the future, so I would urge users to migrate towards the newer API, as soon as you can.



    Critique of Other Solutions

    DataFrame.values has inconsistent behaviour, as already noted.

    DataFrame.get_values() is simply a wrapper around DataFrame.values, so everything said above applies.

    DataFrame.as_matrix() is deprecated now, do NOT use!

提交回复
热议问题