How can I generate Hocon conf file dynamically using pyhocon in Python 3?

Question

I would like to use automation to create the hocon configuration with python 3 scripting. I read that lightbend (https://github.com/lightbend/config) recommends pyhocon (ht

Answer 1

So, I decided to look at documentation for JVM (Java/Scala) library (https://github.com/lightbend/config). After reading the documentation, there was a clear section on hocon examples (https://github.com/lightbend/config#examples-of-hocon). In this documentation, they categorized 7 valid hocon styles. I call these styles because if I was to automate the generation of these files, I would be picking one way to write out and sticking with it.

All of these are valid HOCON.

1.Start with valid JSON:

{
    "foo" : {
        "bar" : 10,
        "baz" : 12
    }
}

2.Drop root braces:

"foo" : {
    "bar" : 10,
    "baz" : 12
}

3.Drop quotes:

foo : {
    bar : 10,
    baz : 12
}

4.Use = and omit it before {:

foo {
    bar = 10,
    baz = 12
}

5.Remove commas:

foo {
    bar = 10
    baz = 12
}

6.Use dotted notation for unquoted keys:

foo.bar=10
foo.baz=12

7.Put the dotted-notation fields on a single line:

foo.bar=10, foo.baz=12

Because I will be using the pyhocon library, I needed to look for write solutions within the library. I found some help from chimpler's git (https://github.com/chimpler/pyhocon). What I found was that they have two hocon styles which can be simply written out. One is json and the other is something that wasn't on the list which was describe above by lightbend.

Style 1: pure JSON, witch can be written out in two ways:

HOCONConverter.to_json

#Using HOCONConverter.to_json
confTree = ConfigFactory.parse_string("{}")
confTree.put("Environment","Dev")
confTree.put("Test","${Environment}")

filename = "./json_coverted.conf"
print("Write to disk as {}".format(filename))
with open(filename, "w") as fd:
    fd.write(HOCONConverter.to_json(confTree))

HOCONConverter.to_json Result

{
    "Environment": "Dev",
    "Test": "${Environment}"
}

OR Using json.dump

#Using json.dump
confTree = ConfigFactory.parse_string("{}")
confTree.put("Environment","Dev")
confTree.put("Test","${Environment}")
filename = "./json_dumped.conf"
print("Write to disk as {}".format(filename))
with open(filename, "w") as fd:
    fd.write(json.dumps(confTree,indent=4))

Using json.dump Result

{
  "Environment": "Dev",
  "Test": "${Environment}"
}

Pyhocon's other Style, not listed by lightbend

# HOCONConverter.to_hocon
confTree = ConfigFactory.parse_string("{}")
confTree.put("Environment","Dev")
confTree.put("Test","${Environment}")
filename = "./hocon_coverted.txt"
print("Write to disk as {}".format(filename))
with open(filename, "w") as fd:
    fd.write(HOCONConverter.to_hocon(confTree))

Pyhocon's other Style, not listed by lightbend Result

Environment = "Dev"
Test = "${Environment}"

So, to answer my own question the only dependable way to generate a hocon conf file dynamically using pyhocon in Python 3 is by using one of the json methods (converter or dumps). But this still leaves an open question. The question being, will reading a json to a pyhocon ConfTree object be able dereference the substitutions when they are in the json?

For example if I read the file

{
    "Environment": "Dev",
    "Test": "${Environment}"
}

Will the ConfTree object get "Dev" as the value for Test?
No, it will not. Here is my test

filename = "json_coverted.conf"
print("Reading file{}".format(filename))
conf = ConfigFactory.parse_file(filename)
key="Test"
value=conf.get(key)
print("Key:{} Value:{}".format(key,value))

Test Result Out to screen

Reading filejson_coverted.conf
Key:Test Value:${Environment}

So, then how does one use pyhocon with substitutions?

Answer 2

Probably this example will answer your question

from pyhocon.converter import HOCONConverter
import pyhocon

string = '{"Environment": "Dev","Test": ${Environment}}'
factory = pyhocon.ConfigFactory.parse_string(string, resolve=True)
factory.put('somekey','somevalue')
print(HOCONConverter().to_hocon(factory))

returns

Environment = "Dev"
Test = "Dev"
somekey = "somevalue"