Astropy Fits: How to write out a table with rows sliced out?

Question

I'm currently working with some fits tables and I'm having trouble with outputting in Astropy.io.fits. Essentially, I am slicing out a bunch of rows that have data for objects I'm not interested in, but when I save the new table all of those rows have magically reappeared.

For example:

import astropy.io.fits as fits
import numpy as np

hdu = fits.open('some_fits_file.fits')[1].data

sample_slice = [True True True False False True]

hdu_sliced = hdu[sample_slice]

Now my naive mind expects that "hdu" has 6 rows and hdu_sliced has 4 rows, which is what you would get if you used np.size(). So if I save hdu_sliced, the new fits file will also have 4 rows:

new_hdu = fits.BinTableHDU.from_columns(fits.ColDefs(hdu_sliced.columns))

new_hdu.writeto('new_fits_file.fits')

np.size(hdu3)
6

So those two rows that I got rid of with the slice are for some reason not actually being removed from the table and the outputted file is just the same as the original file.

How do I delete the rows I don't want from the table and then output that new data to a new file?

Cheers, Ashley

Answer 1

If you want to stick to using FITS_rec, you can try the following, which seems to be a workaround:

new_hdu = fits.BinTableHDU.from_columns(hdu_sliced._get_raw_data())

Answer 2

Can you use astropy.table.Table instead of astropy.io.fits.BinTable?

It's a much more friendly table object.

One way to make a row selection is to index into the table object with a list (or array) of rows you want:

>>> from astropy.table import Table
>>> table = Table()
>>> table['col_a'] = [1, 2, 3]
>>> table['col_b'] = ['spam', 'ham', 'jam']
>>> print(table)
col_a col_b
----- -----
    1  spam
    2   ham
    3   jam
>>> table[[0, 2]] # Table with rows 0 and 2 only, row 1 removed (a copy)
<Table length=2>
col_a col_b
int64  str4
----- -----
    1  spam
    3   jam

You can read and write to FITS directly with Table:

table = Table.read('file.fits', hdu='mydata')
table2 = table[[2, 7, 10]]
table2.write('file2.fits')

There are potential issues, e.g. the FITS BINTABLE header isn't preserved when using Table, only the key, value info is storead in table.meta. You can consult the Astropy docs on table and FITS BINTABLE for details about the two table objects, how they represent data or how you can convert between the two, or just ask follow-up questions here or on the astropy-dev mailing list.