Parse key value pairs in a text file

Question

I am a newbie with Python and I search how to parse a .txt file. My .txt file is a namelist with computation informations like :

myfile.txt

var0 = 16
var1 = 1.12434E10
var2 = -1.923E-3
var3 = 920

How to read the values and put them in myvar0, myvar1, myvar2, myvar3 in python?

Answer 1

I suggest storing the values in a dictionary instead of in separate local variables:

myvars = {}
with open("namelist.txt") as myfile:
    for line in myfile:
        name, var = line.partition("=")[::2]
        myvars[name.strip()] = float(var)

Now access them as myvars["var1"]. If the names are all valid python variable names, you can put this below:

names = type("Names", [object], myvars)

and access the values as e.g. names.var1.

Answer 2

I personally solved this by creating a .py file that just contains all the parameters as variables - then did:

include PARAMETERS.py

in the program modules that need the parameters.

It's a bit ugly, but VERY simple and easy to work with.

Answer 3

Dict comprehensions (PEP 274) can be used for a shorter expression (60 characters):

d = {k:float(v) for k, v in (l.split('=') for l in open(f))}

EDIT: shortened from 72 to 60 characters thanks to @jmb suggestion (avoid .readlines()).

Answer 4

As @kev suggests, the configparser module is the way to go.

However in some scenarios (a bit ugly, I admit) but very simple and effective way to do to this is to rename myfile.txt to myfile.py and do a from myfile import * (after you fix the typo var 0 -> var0)

However, this is very insecure, so if the file is from an external source or can be written by a malicious attacker, use something that validates the data instead of executing it blindly.

Answer 5

If there are multiple comma-separated values on a single line, here's code to parse that out:

    res = {}                                                                                                                                                                                             

    pairs = args.split(", ")                                                                                                                                                                             
    for p in pairs:                                                                                                                                                                                      
        var, val = p.split("=")                                                                                                                                                                          
        res[var] = val                                                                                                                                                                                   

Answer 6

Use pandas.read_csv when the file format becomes more fancy (like comments).

val = u'''var0 = 16
var1 = 1.12434E10
var2 = -1.923E-3
var3 = 920'''
print(pandas.read_csv(StringIO(val), # or read_csv('myfile.txt',
            delimiter='\s*=\s*',
            header=None,
            names=['key','value'],
            dtype=dict(key=numpy.object,value=numpy.object), # or numpy.float64
            index_col=['key']).to_dict()['value'])
# prints {u'var1': u'1.12434E10', u'var0': u'16', u'var3': u'920', u'var2': u'-1.923E-3'}

Answer 7

Similar to @lauritz-v-thaulow but, just a line by line read into a variable.

Here is a simple Copy-Pasta so you can understand a bit more.
As the config file has to be a specific format.

import os

# Example creating an valid temp test file to get a better result. 
MY_CONFIG = os.path.expanduser('~/.test_credentials')
with open(MY_CONFIG, "w") as f:
    f.write("API_SECRET_KEY=123456789")
    f.write(os.linesep)
    f.write("API_SECRET_CONTENT=12345678")

myvars = {}
with open(MY_CONFIG, "r") as myfile:
    for line in myfile:
        line = line.strip()
        name, var = line.partition("=")[::2]
        myvars[name.strip()] = str(var)

# Iterate thru all the items created.
for k, v in myvars.items():
    print("{} | {}".format(k, v))

# API_SECRET_KEY | 123456789
# API_SECRET_CONTENT | 12345678

# Access the API_SECRET_KEY item directly
print("{}".format(myvars['API_SECRET_KEY']))

# 123456789

# Access the API_SECRET_CONTENT item directly
print("{}".format(myvars['API_SECRET_CONTENT']))

# 12345678