Rename the app.config in the dll project to [dllname].dll.config. Change the properties of the same file to "copy always" and rebuild. Works great for me. I also tried adding another setting through the Settings dialog in VS and it updated fine even though it was not app.config anymore. BTW Using VS2013 update 3.
Another way to do a similar thing is to create a hardlink file with the [dllname].dll.config in the project folder and the change the properties on that to "copy always" that way you still have your app.config as normal but if you updated it, it will also update your [dllname].dll.config file. The only real caveat here is that it might not track the hardlink when you check into source control like TFS and you might accidentally overwrite the link. But then again it does honor links when you "add and existing item" as a link so it may or may not work for your situation. You decide.