Reading key-value pairs into Pandas

Question

Pandas makes it really easy to read a CSV file:

pd.read_table(\'data.txt\', sep=\',\')

Does Pandas having something similar for a file with

Answer 1

I'm not sure what the best way to do this is, but assuming that the delimiters aren't found in the values -- it hurts my brain to think of the corner cases -- then something like this isn't super-elegant but is straightforward:

>>> df = pd.read_csv("esm.csv", sep=",|=", header=None)
>>> df2 = df.ix[:,1::2]
>>> df2.columns = list(df.ix[0,0::2])
>>> df2
  symbol exchange         timestamp  price  quantity
0   ESM3   GLOBEX  1365428525690751   1548       551
1   ESM3   GLOBEX  1365428525697183   1548       551
2   ESM3   GLOBEX  1365428525714498   1548       551
3   ESM3   GLOBEX  1365428525734967   1548       551
4   ESM3   GLOBEX  1365428525735567   1548       555
5   ESM3   GLOBEX  1365428525735585   1548       556
6   ESM3   GLOBEX  1365428525736116   1548       556
7   ESM3   GLOBEX  1365428525740757   1548       556
8   ESM3   GLOBEX  1365428525748502   1548       556
9   ESM3   GLOBEX  1365428525748952   1548       557

Basically, read it in, and then do the pivot yourself, keeping every other element and then fixing the column names.

Answer 2

If you know the key names beforehand and if the names always appear in the same order, then you could use a converter to chop off the key names, and then use the names parameter to name the columns:

import pandas as pd

def value(item):
    return item[item.find('=')+1:]

df = pd.read_table('data.txt', header=None, delimiter=',',
                   converters={i:value for i in range(5)},
                   names='symbol exchange timestamp price quantity'.split())
print(df)

on your posted data yields

  symbol exchange         timestamp    price quantity
0   ESM3   GLOBEX  1365428525690751  1548.00      551
1   ESM3   GLOBEX  1365428525697183  1548.00      551
2   ESM3   GLOBEX  1365428525714498  1548.00      551
3   ESM3   GLOBEX  1365428525734967  1548.00      551
4   ESM3   GLOBEX  1365428525735567  1548.00      555
5   ESM3   GLOBEX  1365428525735585  1548.00      556
6   ESM3   GLOBEX  1365428525736116  1548.00      556
7   ESM3   GLOBEX  1365428525740757  1548.00      556
8   ESM3   GLOBEX  1365428525748502  1548.00      556
9   ESM3   GLOBEX  1365428525748952  1548.00      557