Lists in ConfigParser

Question

The typical ConfigParser generated file looks like:

[Section]
bar=foo
[Section 2]
bar2= baz

Now, is there a way to index lists like, for in

Answer 1

I landed here seeking to consume this...

[global]
spys = richard.sorge@cccp.gov, mata.hari@deutschland.gov

The answer is to split it on the comma and strip the spaces:

SPYS = [e.strip() for e in parser.get('global', 'spys').split(',')]

To get a list result:

['richard.sorge@cccp.gov', 'mata.hari@deutschland.gov']

It may not answer the OP's question exactly but might be the simple answer some people are looking for.

Answer 2

I faced the same problem in the past. If you need more complex lists, consider creating your own parser by inheriting from ConfigParser. Then you would overwrite the get method with that:

    def get(self, section, option):
    """ Get a parameter
    if the returning value is a list, convert string value to a python list"""
    value = SafeConfigParser.get(self, section, option)
    if (value[0] == "[") and (value[-1] == "]"):
        return eval(value)
    else:
        return value

With this solution you will also be able to define dictionaries in your config file.

But be careful! This is not as safe: this means anyone could run code through your config file. If security is not an issue in your project, I would consider using directly python classes as config files. The following is much more powerful and expendable than a ConfigParser file:

class Section
    bar = foo
class Section2
    bar2 = baz
class Section3
    barList=[ item1, item2 ]

Answer 3

One thing a lot of people don't know is that multi-line configuration-values are allowed. For example:

;test.ini
[hello]
barlist = 
    item1
    item2

The value of config.get('hello','barlist') will now be:

"\nitem1\nitem2"

Which you easily can split with the splitlines method (don't forget to filter empty items).

If we look to a big framework like Pyramid they are using this technique:

def aslist_cronly(value):
    if isinstance(value, string_types):
        value = filter(None, [x.strip() for x in value.splitlines()])
    return list(value)

def aslist(value, flatten=True):
    """ Return a list of strings, separating the input based on newlines
    and, if flatten=True (the default), also split on spaces within
    each line."""
    values = aslist_cronly(value)
    if not flatten:
        return values
    result = []
    for value in values:
        subvalues = value.split()
        result.extend(subvalues)
    return result

Source

Myself, I would maybe extend the ConfigParser if this is a common thing for you:

class MyConfigParser(ConfigParser):
    def getlist(self,section,option):
        value = self.get(section,option)
        return list(filter(None, (x.strip() for x in value.splitlines())))

    def getlistint(self,section,option):
        return [int(x) for x in self.getlist(section,option)]

Note that there are a few things to look out for when using this technique

1. New lines that are items should start with whitespace (e.g. a space or a tab)
2. All following lines that start with whitespace are considered to be part of the previous item. Also if it has an = sign or if it starts with a ; following the whitespace.

Answer 4

As mentioned by Peter Smit (https://stackoverflow.com/a/11866695/7424596) You might want to extend ConfigParser, in addition, an Interpolator can be used to automatically convert into and from the list.

For reference at the bottom you can find code which automatically converts config like:

[DEFAULT]
keys = [
    Overall cost structure, Capacity, RAW MATERIALS,
    BY-PRODUCT CREDITS, UTILITIES, PLANT GATE COST,
    PROCESS DESCRIPTION, AT 50% CAPACITY, PRODUCTION COSTS,
    INVESTMENT, US$ MILLION, PRODUCTION COSTS, US ¢/LB,
    VARIABLE COSTS, PRODUCTION COSTS, MAINTENANCE MATERIALS
  ]

So if you request keys you will get:

<class 'list'>: ['Overall cost structure', 'Capacity', 'RAW MATERIALS', 'BY-PRODUCT CREDITS', 'UTILITIES', 'PLANT GATE COST', 'PROCESS DESCRIPTION', 'AT 50% CAPACITY', 'PRODUCTION COSTS', 'INVESTMENT', 'US$ MILLION', 'PRODUCTION COSTS', 'US ¢/LB', 'VARIABLE COSTS', 'PRODUCTION COSTS', 'MAINTENANCE MATERIALS']

Code:

class AdvancedInterpolator(Interpolation):
    def before_get(self, parser, section, option, value, defaults):
        is_list = re.search(parser.LIST_MATCHER, value)
        if is_list:
            return parser.getlist(section, option, raw=True)
        return value


class AdvancedConfigParser(ConfigParser):

    _DEFAULT_INTERPOLATION = AdvancedInterpolator()

    LIST_SPLITTER = '\s*,\s*'
    LIST_MATCHER = '^\[([\s\S]*)\]$'

    def _to_list(self, str):
        is_list = re.search(self.LIST_MATCHER, str)
        if is_list:
            return re.split(self.LIST_SPLITTER, is_list.group(1))
        else:
            return re.split(self.LIST_SPLITTER, str)


    def getlist(self, section, option, conv=lambda x:x.strip(), *, raw=False, vars=None,
                  fallback=_UNSET, **kwargs):
        return self._get_conv(
                section, option,
                lambda value: [conv(x) for x in self._to_list(value)],
                raw=raw,
                vars=vars,
                fallback=fallback,
                **kwargs
        )

    def getlistint(self, section, option, *, raw=False, vars=None,
            fallback=_UNSET, **kwargs):
        return self.getlist(section, option, int, raw=raw, vars=vars,
                fallback=fallback, **kwargs)

    def getlistfloat(self, section, option, *, raw=False, vars=None,
            fallback=_UNSET, **kwargs):
        return self.getlist(section, option, float, raw=raw, vars=vars,
                fallback=fallback, **kwargs)

    def getlistboolean(self, section, option, *, raw=False, vars=None,
            fallback=_UNSET, **kwargs):
        return self.getlist(section, option, self._convert_to_boolean,
                raw=raw, vars=vars, fallback=fallback, **kwargs)

Ps keep in mind importance of indentdation. As reads in ConfigParser doc string:

Values can span multiple lines, as long as they are indented deeper than the first line of the value. Depending on the parser's mode, blank lines may be treated as parts of multiline values or ignored.

Answer 5

json.loads & ast.literal_eval seems to be working but simple list within config is treating each character as byte so returning even square bracket....

meaning if config has fieldvalue = [1,2,3,4,5]

then config.read(*.cfg) config['fieldValue'][0] returning [ in place of 1

Answer 6

Only primitive types are supported for serialization by config parser. I would use JSON or YAML for that kind of requirement.