Parse key value pairs in a text file

Question

I am a newbie with Python and I search how to parse a .txt file. My .txt file is a namelist with computation informations like :

myfile.txt

Answer 1

As @kev suggests, the configparser module is the way to go.

However in some scenarios (a bit ugly, I admit) but very simple and effective way to do to this is to rename myfile.txt to myfile.py and do a from myfile import * (after you fix the typo var 0 -> var0)

However, this is very insecure, so if the file is from an external source or can be written by a malicious attacker, use something that validates the data instead of executing it blindly.

Answer 2

I suggest storing the values in a dictionary instead of in separate local variables:

myvars = {}
with open("namelist.txt") as myfile:
    for line in myfile:
        name, var = line.partition("=")[::2]
        myvars[name.strip()] = float(var)

Now access them as myvars["var1"]. If the names are all valid python variable names, you can put this below:

names = type("Names", [object], myvars)

and access the values as e.g. names.var1.

Answer 3

If there are multiple comma-separated values on a single line, here's code to parse that out:

    res = {}                                                                                                                                                                                             

    pairs = args.split(", ")                                                                                                                                                                             
    for p in pairs:                                                                                                                                                                                      
        var, val = p.split("=")                                                                                                                                                                          
        res[var] = val                                                                                                                                                                                   

Answer 4

Dict comprehensions (PEP 274) can be used for a shorter expression (60 characters):

d = {k:float(v) for k, v in (l.split('=') for l in open(f))}

EDIT: shortened from 72 to 60 characters thanks to @jmb suggestion (avoid .readlines()).

Answer 5

Use pandas.read_csv when the file format becomes more fancy (like comments).

val = u'''var0 = 16
var1 = 1.12434E10
var2 = -1.923E-3
var3 = 920'''
print(pandas.read_csv(StringIO(val), # or read_csv('myfile.txt',
            delimiter='\s*=\s*',
            header=None,
            names=['key','value'],
            dtype=dict(key=numpy.object,value=numpy.object), # or numpy.float64
            index_col=['key']).to_dict()['value'])
# prints {u'var1': u'1.12434E10', u'var0': u'16', u'var3': u'920', u'var2': u'-1.923E-3'}

Answer 6

I personally solved this by creating a .py file that just contains all the parameters as variables - then did:

include PARAMETERS.py

in the program modules that need the parameters.

It's a bit ugly, but VERY simple and easy to work with.