Replacing masked values (--) with a Null or None value using fiil_value from ma numpy in Python

问题

Is there a way to replace a masked value in a numpy masked array as a null or None value? This is what I have tried but does not work.

for stars in range(length_masterlist_final):
    ....
    star = customSimbad.query_object(star_names[stars])
        #obtain stellar info.
        photometry_dataframe.iloc[stars,0] = star_IDs[stars]
        photometry_dataframe.iloc[stars,1] = star_names[stars]

        photometry_dataframe.iloc[stars,2] = star['FLUX_U'][0]
        #Replace "--" masked values with a Null (i.e., '') value.
        photometry_dataframe.iloc[stars,2] = ma.filled(photometry_dataframe.iloc[stars,2], fill_value=None)
.....
photometry_dataframe.to_csv(output_dir + "simbad_photometry.csv", index=False, header=True, na_rep='NaN')

specifically

(photometry_dataframe.iloc[stars,2] = ma.filled(photometry_dataframe.iloc[stars,2], fill_value=None))

produces

'MaskedConstant' object has no attribute '_fill_value'

I want to replace masked values '--' with '' when I output the dataframe as a csv file. One work around is to read the outputted csv file back into python and replace '--' with '', but this is a horrible solution. There must be a better solution. I don't want masked values printed as '--' in the csv file.

回答1:

Use Astropy:

>>> from pandas import DataFrame
>>> from astropy.table import Table
>>> import numpy as np
>>> 
>>> df = DataFrame()
>>> df['a'] = [1, np.nan, 2]
>>> df['b'] = [3, 4, np.nan]
>>> df
    a   b
0   1   3
1 NaN   4
2   2 NaN
>>> t = Table.from_pandas(df)
>>> t
<Table masked=True length=3>
   a       b   
float64 float64
------- -------
    1.0     3.0
     --     4.0
    2.0      --
>>> t.write('photometry.csv', format='ascii.csv')
>>> 
(astropy)neptune$ cat photometry.csv 
a,b
1.0,3.0
,4.0
2.0,

You can specify arbitrary transformations from table values to output values using the fill_values parameter (http://docs.astropy.org/en/stable/io/ascii/write.html#parameters-for-write).

来源：https://stackoverflow.com/questions/35026040/replacing-masked-values-with-a-null-or-none-value-using-fiil-value-from-ma

标签

python

numpy

pandas

dataframe

astropy