Why is it a good idea to limit deployment of files to the user-profile or HKCU from my MSI or setup file?

Deployment is a crucial part of most development. Please give this content a chance. It is my firm belief that software quality can be dramatically improved by small changes in application design to make deployment more logical and more reliable - that is what this "answer" is all about - software development.

This is a Q/A-style question split from an answer that became too long: How do I avoid common design flaws in my WiX / MSI deployment solution?.

As stated above this section was split from an existing answer with broader scope: How do I avoid common design flaws in my WiX / MSI deployment solution? (an answer intended to help developers make better deployment decisions).

9. Overuse of per-user file and registry deployment.

Some applications won't run correctly for all users on a machine, because the user-specific data added during installation isn't correctly added to other user's profiles and registry. In other words the application just works for the user who installed the software. This is obviously a serious design error.

There are several ways to "fix" this, but the whole issue of deployment of per-user files and settings is somewhat messy for a few fundamental reasons:

How do you reference count components installed multiple times? (for each user on the machine)
What do you do with the installed data and settings on uninstall?
How do you deal with new files and settings to install that differ from the ones that are on disk and in the registry and have user-made changes? Surely you don't overwrite automatically?

There are no real clear cut answers, but there are several alternative ways to deal with the "problems". My preferred options are 2 & 3 since I don't think Windows installer should deploy, track or attempt to modify or worse yet, uninstall user data and settings at all - it is user data that shouldn't be meddled with:

9.1 Using Windows Installer Self-Repair or similar

The first option is to get settings and files and HKCU registry keys deployed properly via the setup itself or setup-like features. There are two major ways to do this: relying on Windows Installer "self-repair" generally triggered by an advertised shortcut, or using Microsoft Active Setup.

Self-repair is what happens when you launch a shortcut to start your application, and Windows Installer kicks in and you see a progress bar whilst "something" is being installed. What is typically added are HKCU registry entries and user-profile files.
There is also another alternative to achieve this, it is called Active Setup and is also a Microsoft feature. It essentially registers "something runnable" to run once per user on logon. This can be used to set up per-user data. Active Setup allows "anything runnable" to be executed - for example a copy of files to the user-profile. .
Both of these options mean that the user data and settings are copied in place once - and from then on they are not generally touched, but in the case of "self-repair" might get uninstalled for any user who actually runs the uninstall of the application (unless the setup is designed not to do so).
Although setting up user data with self-repair and Active Setup are "established" methods to get applications running properly, it seems wrong to track user data with Windows Installer components. Why? Because it is really user data that shouldn't be meddled with once initialized.
Accordingly my honest take on the whole issue is to try to avoid deploying user specific data or registry keys and values altogether, and this is what is described next as two other user-data deployment methods.

9.2 Application Initialization of User Data

The second alternative, and one that I find much cleaner, is to change your application executable to be able to initialize all per-user settings and files based on default setting and templates copied from a per-machine location or based on application internal defaults (from the source code) instead of writing them via your setup.

In this scenario Windows Installer will not track the files or settings that are copied to each user. It is treated as user data that should not be interfered with at all. This avoids all interference such as reset or overwritten user data during upgrades and self-repair (and manual uninstall and reinstall).
If there are cases where "fixes" must be made to application settings, this can be achieved by having the application executable update the settings for each users on launch, and then tag the registry that the update has been completed.
The overall "conclusion" is that your setup should prepare your application for first launch, it should not set up the user data and settings environment. All user-profile files and HKCU settings should be defaulted by the application in case they are missing on launch - this yields a much more robust application that is easier to test for QA personnel as well. This is particularly important for Terminal Servers where self-repair is not allowed to run at all. In such cases the application data will be missing if you rely on self-repair to put user data in place.

9.3 "Cloud" or Database Storage of User Settings

To take things a step further in today's "cloud environment" - and this is in my opinion the preferred option. Why should your application be restricted to files and registry keys and values? Why not store all user specific settings in the solution's database?

Full access, control and persistence for all settings without any deployment issues at all.
You do get new management issues though, and they must be shared between developers, system administrators and database administrators. But isn't the cloud pretty much the industry standard by now?
We have been struggling long enough with roaming profiles, corrupted user registry, mishandled user-profile data files, etc.... Developers, save yourself a lot of trouble, and create yourself some new database management issues instead of deployment issues - and start yelling at a whole new bunch of people! :-).
Settings in databases are:
- Not suffering from "dual source problems". There is one instance, and it is updated in real time. Not like the synchronization problems seen with user-profile and "roaming".
- Inspectable, manageable and patchable
- Revisable (version control - can revert older settings)
You could even "tweak" all the user settings from your setup still by running database scripts as part of deployment, but if you are in a corporate environment - isn't the thought of just raising a ticket and then have your database administrator run the maintenance scripts with proper transaction support and rollback much more appealing?
Even if you are delivering a large, fat-client vendor application for general distribution and third party use (in other words not a tailored, corporate client/server solution where you are guaranteed to have a back end database), one should consider cloud storage of user settings by having users log on to a cloud using their email or similar and then synchronize settings in real time.
- Such large applications generally always need to "cache" some settings files on the computer and in HKCU, but it seems more and more possible to save all settings in a single temporary file in the user profile area which is entirely "sacrificial" and even possible to delete if it is corrupted and then download the last saved settings.
- Instead of hosting the cloud yourself, it is obviously possible to use company DBOs to configure their own company-wide cloud where they have full control of all settings, and can also enforce mandatory policies and restrictions for your software's operation. Not to mention the proper backup that is possible for all user settings.