Manage configuration files across environments

Question

How do you (your company) manage the config-files of the apps/systems you build? Let me tell you how we do it, and what the problem is.

I\'m working at a company where w

Answer 1

Take a look at Config at http://www.configapp.com. The way it will work is you will import a base configuration file called web.config. Then from within the webapp, you will create 4 environments, dev, test, acc and prod. Go to the dev environment, create your environment variables, and set the values appropriate to the environment. You will be maintaining just one configuration file with environment variables. You can see all the environment variables, and easily view the differences between the environments.

You will make less mistakes because you only have 1 configuration file to manage. When you add a new common/static configuration, all environments will magically have that new setting. When you add a variable configuration, just go the correct environment tab and set the value there. You can view a specific configuration value for all environments so it's a quick way to check if you missed something. You can also eyeball each configuration file per environment, since they are all right in front of you. You don't need to RDP/SSH to each server.

You won't forget to check in since Config will be the master copy, and you won't edit configuration files directly anymore. If you do edit them directly, we have a diff functionality so you can see the differences between the master and the local copy. You can deploy the master copy to your local server using various deployment models that suits your team. You can push, manual pull, auto pull or export/copy.

We will support custom types, and one thing we can add in the future is a port data type. Part of port validation would be checking if the port is open. This will only work using the on-premises plan since it needs access to the internal network. Or you can just check it manually. Go to the port environment variable, and display all the ports currently configured. Check each port in the list. If the port looks good, add a comment in Config that it's open and it was checked on a certain date. Config is designed for searching and documentation.

I'm part of the Config team, by the way.

Answer 2

This may help, it talks about a central location for config files and services referencing from the one source.

http://blogs.msdn.com/b/youssefm/archive/2010/09/02/loading-wcf-client-configuration-from-different-files-with-configurationchannelfactory.aspx

Answer 3

Personally, if it is possible, I manage it the following way: All "static" environment settings (database connections, LDAP, etc) are placed in the server config file (not affected by code migrations), and the "dynamic" settings in the database itself.

So I only depend on database team to update settings (if you have access directly to to database, it's easier). And I have no risk to run on my dev PC a code pointing to prod database :D

BUT I have NO Visual Studio experience, so I am not sure you can apply something similar in your case, but I hope it can give you some ideas.

Answer 4

Have you considered using a deployment management tool to handle it? I've worked on the operation team for deploying a product that has about 10 different systems that needs to talk to each others, so I really understand your situation. Deploying the entire solution took about 4 hours of manual steps + 2 hours of testing. We decided to use Octopus Deploy (https://octopus.com) to automate the deployment process. This tool can do variable substitution in config files and you can define variables per environment (and much more...) The deployment of the entire solution now wake about 15 minutes and the end result is always good... You can also trigger release creation and deployment directly form TFS, VSTS, Jenkins or TeamCity, so that you could have a very solid CI/CD approach.

Hope this helps.

Answer 5

The Config4* project (disclaimer: I am its primary developer) does not have an out-of-the-box integration with .Net or WCF, so it is probably not useful to you. However, one of the features in Config4* is relevant to your question: it is the ability to embed if-then-else statements in a configuration file, so that the file can "adapt" itself for different environments (such as development, testing, acceptance and production).

You may be able to modify that concept to work with whatever configuration syntax you are using in your .Net/WCF-based project (I'm not familiar with those technologies, but I'm guessing they probably use XML-based configuration files). In particular, you could write a script using, say, Python, that uses if-then-else statements to set environment-specific name=value pairs in a map, and then use some print statements to generate a set of configuration files tailored for an environment. A pseudo-code outline of such a script is:

#--------
# Set up configuration variables suitable for a specified environment
#--------
cfg["variable1"] = "default value";
cfg["variable2"] = "another default value";
if (environment == "testing") {
  cfg["variable1"] = "override default value for this environment";
  cfg["variable3"] = "value suitable for this environment";
  ...
} else if (environment == "production") {
  ...
}
#--------
# Now use print statements to generate configuration files
# Alternatively, use the _name=value_ pairs in the map to
# perform global search-and-replace on template versions of
# configuration files.
#--------
...

For bonus points, the script could also generate a checklist of tests that need to be performed for the environment, for example, "Check if a firewall port needs to be opened between the following endpoints: ..."

Answer 6

We are developing a distributed storage system and we catch many problems like this with our unit and integration tests that run with every build. Also, we are using ReviewBoard, for other developers to have a look at any changes before they get committed. Next step is the continuous integration server (jenkins) that auto deploys and test the artifacts in different environments. Finally, our last line of defense is our stress test testbed that runs against any new version, before it is published. Of course it is essential to have a testbed that mimics your production environments as good as possible, but if you always deploy at the same hosting provider, that shouldn't be too hard.

• Concerning your special case, where changes in one file might have been forgotten in the others, etc.: I imagine this would be quite easy to auto check with a test script.

• Uncommitted changes should be as easy to detect as "git diff master", or whatever vcs you use.

• To know what ports need to be open for the services seems to be more like a question of proper documentation than of configuration management.

Many of the sites that use our system also use puppet to manage the configuration of their different nodes. I am not sure if that would be an option for you, but it might be worth having a look at.