The short of the long is that DataFrames are spitting out endianess errors when I try to create them from functional numpy arrays. Here is a pastebin, more details below: http:/
Ok, The FITS file is in fact the issue. Turns out FITS are all big endian while pandas and scipy and stuff tend to assume little endian (I have no idea what this endian business is, just summarizing a thread) and this causes some weird issues apparently (that I've never seen until looking at pandas).
The solution I have found is:
d = fits.getdata('data.fit')
df=pd.DataFrame(np.array(d).byteswap().newbyteorder())
The solution was located here: https://github.com/astropy/astropy/issues/1156